Home of the Squeezebox™ & Transporter® network music players.
Results 1 to 6 of 6
  1. #1
    Senior Member maggior's Avatar
    Join Date
    Dec 2006
    Location
    nj
    Posts
    2,123

    Working on optical disc backup script

    I currently have a work-in-progress set of Perl scripts to automate the backing up of a music library to DVD or any other removable media (BD, DVD-DL, etc.). Given that there has been some interest in Moonbase's project to do backups to hard discs (http://forums.slimdevices.com/showthread.php?t=67153), I thought perhaps there would be some interest in something like I am working on.

    My motivation:
    =============
    I've recently made huge additions to my music library as I finally got around to ripping the last half of my music collection. My habit has been to backup my FLACs to DVDs. Even though external drives have gotten rediculously cheap, I have a huge stock of blank DVDs that I should use. Though external hard drives are attractive due to ease-of-backup, I worry about the amount of data loss if one were to fail.

    I dreaded the task of sitting in front of Nero and dragging folders in, burn the discs, keep track of what I burned, etc. Time is at a bit of a premium for me now, so I can't afford to deal with all of that tedium.

    So...I thought it would be cool to automate the process somehow. I did some research on available command line tools on Linux and found something called growisofs. I played with it a bit and it works quite well. Discs that I back up are readable on my Linux machines, my Vista machine, and my XP machines. It supports burning ISO9660:1999 images, so most any filename and directory structures will work. I don't plan on trying to read my discs with DOS, Windows NT, or anything of that vintage, so I don't think this will be a problem.


    The Implementation (so far):
    ===========================
    I chose to write my script(s) in Perl, mainly because I want to become more proficient with it. This is very much a prototype at this point and everything is handled by a series of scripts that generate file outupt that is used by another scirpt. Here is what my scripts do at this point:

    - The music directory structure is scanned and total storage size is computed for each sub-directory (presumably making up an album). This information is output to a file. Date critera can be specified to go back only to a certain date. I may add other criteria as needed. Adding this capability brought about a dependency on a module from CPAN to deal with comparing dates. This can add to some complexity to general distribution.

    - The file is then parsed and "burn projects" are built based on a maximum storage capacity specified for the backup device. Currently, the script can't deal with an "album" that iself is larger than the size of a disc. The fact is flagged to the user can deal with it another way. Attempts are made to maximize the storage by searching for smaller albums that might fit in the remaining space on the disc. Another file is generated with the project information.

    - This file is then parsed and a command line to growisofs is built up and executed. The backup device is hard coded. I will implement a verification procedure using md5 sums. This will have a dependency on the auto mount point of the backup device.

    - Finally, I have completed a script that will take one of the project file used by the burn script and lay out on a page lables that can be cut out to put in your binder. 4 labels are printed on a page which can then be cut out. This is very specific to how I do things.


    Once I have this cleaned up a bit, I can make the code available to those that might be interested. At this time, I'm working mainly to my specific needs and haven't generalized much. This could change depending on interest. I do hope to bring the various scripts into a single one. I like the idea of having the data committed to files because it makes it easier to go back and pick up where you left off. For instance, I currently have about 60 DVDs worth of data to backup. I certainly don't plan on doing that in one sitting :-).

    What are your thoughts on this project?

    Thanks,
    Last edited by maggior; 2009-09-11 at 15:09.
    Rich
    ---------
    Setup: 2 SB3s, 4 Booms, 1 Duet, 1 Receiver, 1 Touch, iPeng on iPod Touch, SqueezeCommander, OrangeSqueeze, and SqueezePlayer on Xoom and Galaxy Player 4.2. CentOS 6.3 Server running LogitechMediaServer 7.7.2 and SqueezeSlave.
    Current library stats: 40,810 songs, 3,153 albums, 582 artists.
    http://www.last.fm/user/maggior

  2. #2
    Senior Member JJZolx's Avatar
    Join Date
    Apr 2005
    Location
    Colorado
    Posts
    11,531
    Quote Originally Posted by maggior View Post
    What are your thoughts on this project?
    I think you'd have to be a masochist to want to backup a large library to DVD. But I suppose if someone died and left you a giant pile of DVD-Rs, then what the hell.

    Though external hard drives are attractive due to ease-of-backup, I worry about the amount of data loss if one were to fail.
    I've had backup hard drives fail. You don't lose anything because it's a BACKUP. You get another one and you backup your data again. If you're _really_ paranoid, keep two backups.

  3. #3
    Senior Member pfarrell's Avatar
    Join Date
    Apr 2005
    Location
    Wayne, PA
    Posts
    4,251

    Working on optical disc backup script

    maggior wrote:
    > I currently have a work-in-progress set of Perl scripts to automate the
    > backing up of a music library to DVD or any other removable media (BD,
    > DVD-DL, etc.).



    I agree with JJ here, too much masochism for me.

    I have ~810 albums, all in flac. Assume that the flac compression makes
    one CD take 333 MB, or 3 CDs per GB. Thus we have 270 GB of music to
    back up.

    I would be unwilling to do that even if they were BlueRay and you could
    put 27GB on each. That would take ten disks in the
    insert/burn/remove/label shuffle.

    With a 500GB disk selling for well under $100, just use rsync

    --
    Pat Farrell
    http://www.pfarrell.com/


  4. #4
    Senior Member maggior's Avatar
    Join Date
    Dec 2006
    Location
    nj
    Posts
    2,123
    Quote Originally Posted by JJZolx View Post
    I think you'd have to be a masochist to want to backup a large library to DVD. But I suppose if someone died and left you a giant pile of DVD-Rs, then what the hell.
    Nobody left them to me - I bought them all when they were on sale throughout the years. I guess I didn't forsee <$99 1TB external drives in the near future :-).

    To do it manually, yes, you have to be a masochist. This is why I decided to write this script to provide some automation.

    Quote Originally Posted by JJZolx View Post
    I've had backup hard drives fail. You don't lose anything because it's a BACKUP. You get another one and you backup your data again. If you're _really_ paranoid, keep two backups.
    I guess what I'm looking at here is more of an archive than a backup. This is not something I would want to do a rapid restore from in case of a failed drive. I will probably get 1TB USB drives for backup purposes. I do like the idea of having all of my FLACS from my music library archived on media other than a hard drive.
    Rich
    ---------
    Setup: 2 SB3s, 4 Booms, 1 Duet, 1 Receiver, 1 Touch, iPeng on iPod Touch, SqueezeCommander, OrangeSqueeze, and SqueezePlayer on Xoom and Galaxy Player 4.2. CentOS 6.3 Server running LogitechMediaServer 7.7.2 and SqueezeSlave.
    Current library stats: 40,810 songs, 3,153 albums, 582 artists.
    http://www.last.fm/user/maggior

  5. #5
    Senior Member JJZolx's Avatar
    Join Date
    Apr 2005
    Location
    Colorado
    Posts
    11,531
    Quote Originally Posted by maggior View Post
    I guess I didn't forsee <$99 1TB external drives in the near future :-).
    I count 20 on http://www.newegg.com, starting at $85.99.

  6. #6
    Senior Member
    Join Date
    Feb 2008
    Location
    Nij Beets, The Netherlands
    Posts
    480
    Quote Originally Posted by maggior View Post
    I guess what I'm looking at here is more of an archive than a backup. This is not something I would want to do a rapid restore from in case of a failed drive. ...
    My completely paranoid backup:
    1. (mostly not used) 'start' library on an eSata disk attached to my main computer.
    2. Copied with Deltacopy/rsync to (a) my vortexbox (the real library) and (b) my NAS.

    I follow the same procedure for my main work, where I would be bankrupt immediately if I lost my data.

    As for archival purposes: also for my main work (I'm a designer and typesetter) I archive on hard disks these days. One in home, a copy in a different place. And to be sure disks don't crash of old age, I buy new ones every 5 years or so (they're getting cheaper all the time). Compared to the tape backups/archives of the old days I think this is not only much faster and easier, but also cheaper and more reliable.

    Teus

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •