PDA

View Full Version : Amazon's S3



wiseleyb
2007-03-09, 07:47
I was wondering is anyone was working on connecting SlimServer to Amazon's S3 (http://www.amazon.com/gp/browse.html?node=16427261) service?

andyg
2007-03-09, 07:51
S3 is great as an offsite backup, but if you are thinking of doing something like streaming music from it, you'll pay a fortune in bandwidth costs. There are other services designed for this such as mp3tunes.

jeffmeh
2007-03-09, 08:49
I was looking at these types of services to be able to have an online backup, but I decided that for the amount of data I have that it is impractical. For me, the bottleneck is the upload bandwidth to get the data out there in the first place. My flac library is not huge, around 85 GB, but my ISP limits upload bandwidth to 384 kbps, or 768 kbps if I pay an additional $10 per month.

85 GB ~= 9 x 10^10 bytes ~= 7 X 10^11 bits
384 kbps ~= 4 x 10^5 bits/sec

85 GB / 384 kbps ~= 2 x 10^6 sec ~= 23 days

Even at double the bandwidth, it would take over 10 days, assuming that I was able to get the maximum bandwidth, that it never timed out, got interrupted, encountered errors, etc.

To be honest, I have the storage space out there already, and I tested uploading 1 album's worth of flac files. It took so long that I decided not to continue. Now, if I had physical access to the server and could "seed" it with the 85 GB by connecting a USB drive, I might consider doing incremental uploads from there. However, even that is pretty impractical unless you have much better upload bandwidth.

peter
2007-03-10, 09:43
jeffmeh wrote:
> I was looking at these types of services to be able to have an online
> backup, but I decided that for the amount of data I have that it is
> impractical. For me, the bottleneck is the upload bandwidth to get the
> data out there in the first place. My flac library is not huge, around
> 85 GB, but my ISP limits upload bandwidth to 384 kbps, or 768 kbps if I
> pay an additional $10 per month.
>
> 85 GB ~= 9 x 10^10 bytes ~= 7 X 10^11 bits
> 384 kbps ~= 4 x 10^5 bits/sec
>
> 85 GB / 384 kbps ~= 2 x 10^6 sec ~= 23 days
>
> Even at double the bandwidth, it would take over 10 days, assuming that
> I was able to get the maximum bandwidth, that it never timed out, got
> interrupted, encountered errors, etc.
>
> To be honest, I have the storage space out there already, and I tested
> uploading 1 album's worth of flac files. It took so long that I
> decided not to continue. Now, if I had physical access to the server
> and could "seed" it with the 85 GB by connecting a USB drive, I might
> consider doing incremental uploads from there. However, even that is
> pretty impractical unless you have much better upload bandwidth.
>

I have 1024 Kbit upstream (20 Mbit down) and I backup my (100 G+) files
to a colo server (which I seeded first) and I can confirm that
even with incremental backups you have to be careful not to add too
much music at once. I like my backups to be finished by the time I wake up.

Regards,
Peter

norderney
2007-03-10, 14:03
jeffmeh wrote:

>

I have 1024 Kbit upstream (20 Mbit down) and I backup my (100 G+) files
to a colo server (which I seeded first) and I can confirm that
even with incremental backups you have to be careful not to add too
much music at once. I like my backups to be finished by the time I wake up.

Regards,
Peter


I am quite interested in this type of service. I have about 250Gb of FLAC files. How much would it cost to backup this data? How easy is it to setup? Once setup, can you opt to just backup new or amended files?

peter
2007-03-11, 00:19
norderney wrote:
> Peter;186941 Wrote:
>
>> jeffmeh wrote:
>>
>>
>>>
>>>
>> I have 1024 Kbit upstream (20 Mbit down) and I backup my (100 G+) files
>>
>> to a colo server (which I seeded first) and I can confirm that
>> even with incremental backups you have to be careful not to add too
>> much music at once. I like my backups to be finished by the time I wake
>> up.
>>
>> Regards,
>> Peter
>>
>
>
> I am quite interested in this type of service. I have about 250Gb of
> FLAC files. How much would it cost to backup this data? How easy is it
> to setup? Once setup, can you opt to just backup new or amended files?
>

That all depends. Surf the web for colo sites or even real or virtual
servers you can rent.

I happened to have a spare 1U server that I fitted with an extra 300 GB
disk for my backups. I pay 20 EURO's a month for rackspace at a company
near my place of work. Maximum data transfer 30G/mo (occasional
overdrafts are allowed).

Setup of a Linux box with rsnapshot is not that difficult, but that
depends on you mostly. Of course you just backup changed files. My
system keeps 7 weeks worth of backup snapshots (I can recover files for
any day for the last 7 days or any week for the last 7 weeks). Running
Linux on your media server is probably a requirement too. Then again if
you want real flexibility and power you're mostly stuck with Linux/Unix
anyway.

Regards,
Peter

Nikhil
2007-03-11, 00:41
S3 is great as an offsite backup, but if you are thinking of doing something like streaming music from it, you'll pay a fortune in bandwidth costs. There are other services designed for this such as mp3tunes.

Perhaps Slimdevices/Logitech should consider offering something like this for music files as a optional paid service for their customers.

SD customers could backup their files to SD servers, with the added advantage of being able to link it throught their SqueezeNetwork accounts for streaming that could scale according to available bandwidth. e.g. like mp3tunes but with a focus on backup for audiophiles.