Large Backups on a Cloud

[atrocity]

Since posting this, I've uploaded ±1TB of music to Amazon

And, just like that, Amascum has banned rclone.

https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/2314/10

[drmatt]

They don't like the word "unlimited" as much as their marketing would have you believe, apparently...

Sent from my ONEPLUS A3003 using Tapatalk

[atrocity]

They don't like the word "unlimited" as much as their marketing would have you believe, apparently...

If I understand the situation correctly (and I'm not sure that I do) rclone's developer had to work with Amazon to get the app approved, but Amazon banned it without warning or consulting him and has yet to respond to his inquiries. For now, I've told Amazon not to renew my plan when it expires in August. If they allow rclone again, I'll probably keep the service, but for people with large music libraries the inability to automate a sync makes even "unlimited" useless. I'd be OK (not thrilled, but OK) if they'd just allow it with --dry-run so I could see what needs to be done and do it manually, but absent that or a similar tool, forget it.

I'm currently testing a 10GB backup to https://hubic.com - they have a 25GB free version that works with rclone. Unfortunately, they don't have an unlimited plan, though their 10TB should work (for now!) for me. The test isn't finished yet, but it's been running for a few hours and seems to be working well:

Transferred:   7.864 GBytes (726.991 kBytes/s)
Errors:                 0
Checks:                 1
Transferred:         4942
Elapsed time:    3h9m1.9s

[drmatt]

So this is why I used "dar". Keep a local metadata copy and I can run differential backups against the metadata to upload as a tarball (um, darball) without having to keep the data local too.

Sent from my ONEPLUS A3003 using Tapatalk

[atrocity]

Testing hubiC at the moment. 25GB free and it works with rclone. Their largest plan is 10TB, which works for me for now, so I just might go for it.

rclone developer is hopeful that he can work things out with Amazon.

[norderney]

I use Dropbox for backing up my music database which is about 1Tb / 60,000 tracks. As a bonus I have installed an app called FLACBox on my iPhone which allows my to play music stored directly on Dropbox. So I have access to my complete music library when I'm away from home using either 3G or WiFi.
I already had a 2Tb Dropbox account but I had only a few documents stored on it. This way I can make use of the storage capacity.

[mandryka]

I use Dropbox for backing up my music database which is about 1Tb / 60,000 tracks. As a bonus I have installed an app called FLACBox on my iPhone which allows my to play music stored directly on Dropbox. So I have access to my complete music library when I'm away from home using either 3G or WiFi.
I already had a 2Tb Dropbox account but I had only a few documents stored on it. This way I can make use of the storage capacity.

But how did you get the data uploaded?

[norderney]

Using the Dropbox web site, I logged into my Dropbox account and then upload all my FLAC files to my Music folder within Dropbox. Quite easy actually. Takes a long time but only has to be done once. Works best if you have a fast fibre broadband connection.

[mandryka]

Using the Dropbox web site, I logged into my Dropbox account and then upload all my FLAC files to my Music folder within Dropbox. Quite easy actually. Takes a long time but only has to be done once. Works best if you have a fast fibre broadband connection.

What sort of upload speeds did you get? How long did it take to upload 1TB?

[drmatt]

Maths to the rescue! 1TB is 8000000 Mbits. This will take 5.1 days on an 18Mbit/s uplink (typical unlimited fibre package).

[mandryka]

The reason I was pushing the point is that my internet upload speed is about 17mbps, but using Backblaze, I was getting an upload speed of 1.5mbps.

Anyway I just spoke to their support people and they told me make some changes in the settings file and now I have 16mbps. So problem solved.

I have 2TB to upload so I don't think dropbox will work for me. So far, Backblaze looks fine though. Fingers crossed.

[drmatt]

Ok, so you had a specific problem. What did you change? Are they artificially limiting upload speed in the client package?

I had no issues with Amazon drive on upload speed.

Sent from my ONEPLUS A3003 using Tapatalk

[mandryka]

Ok, so you had a specific problem. What did you change? Are they artificially limiting upload speed in the client package?

I had no issues with Amazon drive on upload speed.

Sent from my ONEPLUS A3003 using Tapatalk

You have to go to performance in settings, turn the throttle to maximum upload speed, and choose the maximum number of threads.

[drmatt]

So it seems they restricted everyone by default to prevent impact to browsing experience and general connectivity whilst uploading. They seem to have somewhat overcooked the throttling if it went to less than 10% of your link speed, however..

Sent from my ONEPLUS A3003 using Tapatalk

[pablolie]

I see the appeal for backup, but unless it's automatically mirrored -which I haven't seen in cheaper storage- I don't see the appeal for everyday operations. Local NAS still seems to rule the cost-benefit analysis for me... (unfortunately, I'd love a cloud based cost effective solution, but local storage is MUCH cheaper over the mid term for me).