Scott Penrose

Cloud Backup

Scott is an expert software developer with over 30 years experience, specialising in education, automation and remote data.

The hope

Cloud backup brings hope to us all... automatic, secure, offsite backup :-) Horrah.

The price is even good. $6/month for unlimited storage from at least one provider.

Backup/Security seems reasonable (I have my doubts here, like where is the key stored to encrypt it, and therefore if my computer crashes, how do I get my key back to recover my files...).

The problem

But lets look at the practical side.

My iPad has 64GB of storage. That is pathetically small. It only holds a few movies. But lets count that as our starting point. Taking up 100% of a ADSL2+ link it will take 8 days solid to backup my iPad.

Now where is my important data? Documents, Source code? Nah, they are all in Git and backed up all over the place. Others are stored in Dropbox for sharing with other teams.

Movies. I own all my movies on DVD. I rip them relatively low resolution (about 1 to 2 GB per movie). Technically they are all safe on DVD. In reality having to rip all of those again would be a daunting task. But to back those up is about 2TB or 250 days.

Lets look at the REALLY important stuff that can't be put into Git. Home Movies and Photos. My Photo collections is growing and has hit over 1TB. Even if I only consider the additions and say that I take 20MB Raw images, lets say I don't have a big event (e.g. wedding) and therefore I might have 20GB per month. At that rate it will only take about 3 days to backup that new 20GB. Lets not bother with home movies just too big to count.

Looking at that last entry we still have two problems:

  • Backup solutions and service providers come and go. My photo collection is getting to 12 years old. And I doubt my chosen cloud backup solution will be here in another 12 years. Therefore you have to upload the whole lot on starting or changing service.
  • 3 days to upload a modest collection of files at 100% upload is just too much time. It will really slow down your internet connection.

For me, the reason to have an online backup solution is to backup my photos and home movies. Anything else is so small I can just use Git to have it on not only lots of machines but servers in other sites as well.

So for the one reasons to have online backup, is the one thing it can't do.

We have a LONG way to go before cloud backup is actually useful to large files, photos and videos.

P.S. It is useful, for those who don't do any backup, or use any version control

2TB Disks are now the norm (< $100). And yet the only solution online for backup would take almost a year.

What are you doing with your photos/videos ?

A solution

I am reviewing various backup solution that would backup my Mac, Linux and Windows machine locally. Probably Amanda community or Bacula - both a bit of work to configure but should work well. But I have further requirements that I don't have answers on yet:

  • Can I limit copies. Bacula and Amanda amanda backup to tape, can use disk, but how many copies? Can we have a One and One style - ie. One copy + last changed file, and period for deleted files. More like the cloud sync which only keep one copy of the old files.
  • S3/Glacier backups - can I define rules, such as X or Y directory to be duplicated to S3/Glacier in the background. Or even one of the cloud providers.

The above solution would then allow me to have high speed reliably local backups but still just using disk not tape, and have some data backed up onto remote cloud systems.

  • Setup
  • Storage