[PLUG] My approach to backups

[Rich Freeman via plug <plug@lists.phillylinux.org>]

I'm sure I've talked about this many years ago, but I figured I'd
quickly summarize how I'm handling my backups since the topic has come
up a few times recently.

First, I do encrypted daily duplicity incremental backups to Amazon S3
of everything I care about.  These tend to be pretty small usually - a
few MB of deltas that take about 30min to process (mostly disk
reads/etc - it uses librsync so it performs similarly to the command
line tool).

One of the things I do care about are my photos and those tend to come
in batches.  If I take a batch of photos I'd expect the next nightly
incremental backup to be on the order of 10-20GB.

I perform new full backups once a month, and if all those photos were
allowed to accumulate then that would start to become an issue.
However, the reality is that those photos themselves don't really
change much - certainly not after a month (most edits are tracked in
separate files and don't touch the actual images).

So, ancillary to the daily cloud backups, I archive large stuff in
monthly batches - mostly photos.  This involves moving their
directories to a different path and symlinking them back.  The new
path is excluded from the daily backups.  I create a tarball of the
archive, encrypt it with gpg, and then upload that to a different path
on S3.  This is a manual process but only happens about once a month,
and until I do it the data is backed up daily.

The result is that my daily backup set never really grows beyond maybe
100GB or so, and that is as big as full backups get.  The backups
themselves are automatically rotated and I only keep a few full backup
sets.  This allows me to go back in time a few months, while keeping
the storage costs manageable.

Aside from this I also maintain local copies using bacula which are
more broad.  They include all the critical stuff, but also a lot of
stuff I could re-retrieve (or not) with some inconvenience, and am not
willing to pay to store more securely.  So in the event that moosefs
blows up and eats all my data, most likely I won't actually have to
retrieve all my data from Amazon.  In the event of a fire then this
copy would be lost and Amazon would be where I'd go for my data.

Oh, I also keep a few copies of the gpg key used to encrypt my backups
in secure places offsite.  That never changes, and is small, so it
isn't really inconvenient to do so.  The key itself isn't used for
anything other than backups, so it isn't very useful to anybody who
manages to steal the key unless they can also compromise my Amazon
account.  Obviously if you lose the key to your backups then you lose
the backups, so you definitely want to give some thought to key
security.

-- 
Rich
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug