Anyone using Deltacopy for backup's?

[Bert0123]

Hi,

Im am figuring out how to get the best backup's of my music files.
I have followed the instructions from Moonbase. (http://wiki.slimdevices.com/index.ph..._up_Your_Music)

But the first problem is that all files copied by Rsync don't have the right permissions. So when the backup is ready, I can't open any directory or file.

The website say that the Deltacopy service must run under the Administrator account. But in Win7 I can't find an account with the name "Administrator" so I can't use that.

So I used my own account with "administrator" rights but each new file or directory is stll locked. So my own account doesn't work for this problem.

There is a function in Delta Copy "Fix permissions" and it works, but you have to do that for each new written file or directory after another backup.
If you don't run dis "fix" and for some reason you can't use the deltacopy service anymore you will end up with a bit perfect copy of all files, but you can't touch them or even look at them.

So first question is how to prevent all directories and files getting locked?


Then about speed.
I did some time measuring with 7 Gb of .Flac files.

All done on 1 computer between 2 folders

Windows (all) file copy: 5'35''
Delta Copy, full backup: 7'33''

Then I changed 2 tags in all files:
DeltaCopy, incremental backup: 10'52''

The time to complete the task is twice as big with DeltaCopy. The advantage is that DeltaCopy only transmits 8 Megabyte and Windows filecopy copies the whole 8 Gb again.

So what then is the benefit of DeltaCopy. The whole filecheck process takes more time than just copy the file again. And a fresh copied file is unmodified by DeltaCopy.

Aren't there windows programs without the hasle with locked files, but that keep track about checksums of the files and just copy the whole file again if there is something changed?
But without installing services and with better reporting about what exactly is copied.

But maybe I don't understand the benefit of using DeltaCopy/Rsync.

[Moonbase]

Bert, if I understand correctly, you are trying to use rsync/DeltaCopy for copying between two Windows machines?

I must admit that I have no experience yet using Win7 in this setup, and for syncing between Windows machines there might be better solutions.

Rsync is a well-known protocol in the Unix/Linux world, so using DeltaCopy as a rsync client on Windows to copy to/from a Linux server made much sense to me — and it's actually quite fast here, even on an old 900 MHz P3 Linux server.

Rsync actually does checksum the files and only transmit the differences — so if you update only a tag, only the changes in the file(s) are transmitted. Unfortunately, on Windows with DeltaCopy, this process seems to eat a lot of CPU resources — it's much faster on Linux machines.

I assume that if you are running both a rsync server and a client on Windows, this might account for the bad timings you gave.

Can anyone else give us some more info on using DeltaCopy's server module under Windows?

I'm sure it can be made working, but it might be overkill for a Windows/Windows copy, and use too many resources.

(I prefer using SqueezeBox Server on a Linux box, so I can mainly talk about syncing between a Windows a client and the Linux server. I reused an old 900 MHz Pentium 3 system as a music server, 768 MB RAM, 40 GB + 1 TB internal discs, Fedora 11 (command-line only) and "VortexBox" 0.8 software installed. Handles 3 players and auto-CD-ripping quite well.)

[pfarrell]

Anyone using Deltacopy for backup's?

Bert0123 wrote:
> So what then is the benefit of DeltaCopy. The whole filecheck process
> takes more time than just copy the file again. And a fresh copied file
> is unmodified by DeltaCopy.

I can't help you with specificis, but be warned. Copying a non-trivial
music library takes a very long time. Perhaps with a GigE network it
would be short enough. But as you approach a few hundred gigabytes, it
takes a very long time.

too long for my patience

--
Pat Farrell

http://www.pfarrell.com/

[Bert0123]

Well, I have built a small Atom CPU based pc which runs Squeezecenter and Win XP Pro. Linux takes to much time to learn so that is no option for me. The Vortex Box (software) is a nice product. I tried it, but I want more control and able to run more windows applications. My stripped Windows XP installation is very stable.

The backup I have in mind is just connecting an USB drive and start the backup. Not to a permanent attached drive. Once a week, or after getting a lot of new music I create a backup.

But I started tagging my music and it takes a lot of time so a good backup solution is top priority.

The testsituation with DeltaCopy, I do on my desktop pc which runs win 7RC 64bit. And there are the permission problems.
Meanwhile I found another 'partial filecopy' featured program, hopefully fully windows compatible which I might try in short.

I found it at http://www.superflexible.com/index.htm

But I rather want to figure out what the best settings for DeltaCopy/Rsync are

[Bert0123]

>I can't help you with specificis, but be warned. Copying a non-trivial
>music library takes a very long time. Perhaps with a GigE network it
>would be short enough. But as you approach a few hundred gigabytes, it
>takes a very long time.
>
>too long for my patience


But it is a copy between 2 windows drives in 1 computer The limit is the USB2 speed, say 25MB/s.

[pfarrell]

Anyone using Deltacopy for backup's?

Bert0123 wrote:
> But it is a copy between 2 windows drives in 1 computer The limit is
> the USB2 speed, say 25MB/s.

well, lets see what you really get.
SATA to SATA is fast enough for me. Ethernet, no way.
USB, I've never tried it

--
Pat Farrell

http://www.pfarrell.com/

[Teus de Jong]

I use the DeltaCopy (per Moonbase instructions -- thanks Moonbase) and I'm very impressed. Copying your whole library takes a long time (even on a gigabit network), but updating is very fast. I've copied the files for the first time by attaching my eSata drive to the linux server and do an old fashioned 'cp -a orig dest' (this is much faster than rsync via network). After this, I've attached the eSata drive to my XP machine again and use DeltaCopy to update.

Background: I'm slowly migrating to a Atom based server with vortexbox, but I keep my old server (XP; gives me a great backup also). So I'm copying from Win XP to the linux server. Update time when only a few albums are added or a few tags altered is amazingly short.

Teus

[Moonbase]

You're very welcome, Teus! I'm glad that someone else actually uses Windows/Linux rsync in almost the same setup :-)

I recently switched from my 100 MBit/s to a 1 GBit network and must say now it really shines! Just moving the changes within the files across the network really pays off — after the initial copy, of course. And I much agree that good old "cp" is much faster for the initial backup! ;-)

Another hint: External (backup) USB discs should be NTFS-formatted. This allows for swapping them between Windows and Linux systems without getting into any character set troubles if you use extended characters in filenames, like I do. The ntfs-3g driver for Linux seems to be stable enough not to mess up anything — at least it has never let me down.

I'm a little paranoid, so I have two backups: I usually rip and tag on my Windows box (500 GB external USB, NTFS-formatted), then rsync the music over the network to my VortexBox (1 TB internal xfs/LVM volume), and my main Linux server rsyncs the VortexBox music volume to an external NTFS-formatted 1 TB disc once a day (cron job).

So when recently my Windows external disc broke down, I could simply move the external drive from the main Linux machine to my Windows machine and continue working as if nothing had happened — all funny outlandish music filenames intact, and of course the mapping to drive "M:" (for "Music"). Attaching a clean new drive to the main Linux machine the other day brought back the new full backup (which admittedly took a while over the network).

Remember: If you use the rsync solution from the wiki, you need to install the mentioned cygwin patch for Windows — otherwise you'll end up with messed-up filenames! (For reasons lying within cygwin, the filenames with extended characters will look messed-up in DeltaCopy's rsync status window but will be fine on disc — and that's what counts.)

But I'm getting off-topic ... Back to Bert's problem.

For a pure Windows environment, you might also want to try one of the free "file syncers" for Windows. During my investigations (some time ago) I found that unfortunately none of them were as feature-fledged and stable as I had wished for — and none could do reliable partial file transfers.

Still you might want to try FullSync, for instance — this seemed at least quite adaptable and relatively robust, though it also only copies full files.

Let us know — if you will — if this runs under Windows 7 ;-)

[TomS]

Again, slighty off topic, but I use deltacopy/rsync to sync from windows to my linux server. And, i have an external USB disk on the linux server, formatted NTFS.

One thing that I've hit is that the ntfs-3g filesystem on the linux machine (which is what I am using to mount the external usb drive) loses its original date/timestamp - all files moved there directly through deltacopy/rsync get the current time/date. Of course if you don't care about time/date stamps this doesn't matter.

A workaround is to rsync to an ext3 drive and then sudo rsync to the usb drive (if I remember correctly).

Tom

[Teus de Jong]

Moonbase,

I'm a bit paranoid too, so the my music is also copied to my central backup NAS, for which I also use DeltaCopy/rsync. And the external eSATA disk is formatted with NTFS.

Teus

[verypsb]

I just use robocopy, included with Windows Vista, 7, 2008 & 2008 R2. For Windows XP/2003 you can download the Windows 2003 Resource Kit utilities.

My command line:

robocopy [Source] [Destination] /MIR /S /E /V /NP /LOG:"[LogLocation]" /ZB /R:10 /W:30 /DCOPY:T /TEE /FFT /COPYT

[Bert0123]

But I'm getting off-topic ... Back to Bert's problem.

For a pure Windows environment, you might also want to try one of the free "file syncers" for Windows. During my investigations (some time ago) I found that unfortunately none of them were as feature-fledged and stable as I had wished for — and none could do reliable partial file transfers.

Still you might want to try FullSync, for instance — this seemed at least quite adaptable and relatively robust, though it also only copies full files.

Let us know — if you will — if this runs under Windows 7 ;-)

I share your concern about basic file-sync programs which compare files based on timestamps and attributes. I won't use them for this purpose.

So it has to be a program which really compares the files itself. Like Deltacopy. But my experience with DeltaCopy is that the check process for the changed files takes more time than just copy them again. Ofcoarse this might not be the case on slower networks, but it since the effective speed with Deltacopy was about 15 MB/s and filecopy at 30 MB/s I can't see the advantage about the "Delta" function. Even if DC has to copy only 1Kb on a 25 MB file it takes more time then to copy it again.

But the really most important thing is reliability, I have to be sure I have 2 identical folders. And yes, I also will make 2 backups :-)

What scared me a bit too was the for me unlogical way Deltacopy works. DC's Restore is what I would call backup and vice versa . . .
I had to read the wiki very secure while for all normal file sync programs with far more features I just install and start to work without reading anything.

For that reason I'd rather would use a program with a normal windows interface and maybe warns me to protect my files if I would accidentely do something wrong. With DC you can very easy delete all your files. But you are warning for that several times on the Wiki.

But any program I am going to use really should compare checksums of the files.

I will have a look at FullSync.


I also will look at Robocopy, thanks for the tip. I know the 'program' exists and heard good things about it. If it works without a 'server' it shouldn't be that difficult. I think the /DCOPY switch indicates it does deltacopy too?
If it crc checks the files afterwards it might be a good solution too.

[4mula1]

One thing to consider about any backup program is that when you host the source and backup on the same machine when the backup runs the computer has to examine both the source and destination files for changes, which will eat a lot of CPU and I/O time.

Using DeltaCopy in a client/server configuration with another computer will yield better results, as each machine can examine the files they host and compare the results. Parallel processing at it's finest.

I use DeltaCopy to backup files on my XP box to my Solaris box, and the Solaris box uses rsync to mirror that to the Slug at my parent's house and all via ssh. Works quite well.