Re: [PLUG] Offline Backup Solutions

[Rich Freeman <r-plug@thefreemanclan.net>]

On Fri, Aug 16, 2013 at 12:40 PM, Rich Freeman
<r-plug@thefreemanclan.net> wrote:
> On Fri, Aug 16, 2013 at 11:52 AM, Sam Gleske <sam.mxracer@gmail.com> wrote:
>>
>> Have you tried rsnapshot?  A coworker of mine currently uses it and he likes
>> it.
>
> I stumbled on it but haven't tried it yet.  It looks like rsync, but
> with filled-out incrementals using hard links.  Main issue with that
> is a lack of compression, which I haven't given up on yet.

This should be a final update.  In the end I am in fact using
rsnapshot for the offline backups.  Duplicity is great, but it doesn't
support lzo and it isn't smart about what it compresses.  That makes
the performance absolutely terrible when you're doing large-scale
backups.

So, now I actually have three tiers of backup solutions:
1.  I do a daily incremental duplicity backup to s3 of my critical
files.  The compression makes sense in these files as most aren't
multimedia, and the multimedia doesn't change much.  I am running a
patched version of duplicity so that it is more glacier-friendly,
which means I need a patched version to restore, but I have it
published in a Gentoo overlay on github which means that I can still
install it from a Gentoo live CD or minimal distro install.  This is a
big improvement on my previous solution with dar/sarab - dar stores
entire files if anything changes, and duplicity stores binary diffs,
so my daily space use dropped by 90% (and I'm paying by the GB-month
for s3).

2.  I run daily differential/incremental online backups on bacula on a
somewhat wider set of files on a few boxes.  These backups are less
secure, but they're more convenient.  I do not plan to really rely on
them for disaster recovery.

3.  I run daily rsnapshot backups and keep several days to
semi-offline media (attached but not mounted).  Rsnapshot is FAST and
I like how it uses hard links to fill out the incrementals.  That is a
lot cleaner than other rsync-based approaches I've seen in the past.
I found that the lack of compression wasn't actually a big deal - in
terms of volume much of my space is multimedia anyway, so things grew
maybe 10-20%.

So, that is my final answer I think, at least for a while.  Thanks to
all for suggestions.

Now we'll just see how transitioning to btrfs goes.  I did start
replicating to my new root but I haven't actually swapped it out yet -
need to wait for a day I can nurse things along if they go south...

Rich
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug