Re: [PLUG] Backups vs Copies: was Avoid Arvixe at all costs!

[Rich Kulawiec <rsk@gsp.org>]

On Mon, Mar 27, 2017 at 11:25:27AM -0400, Rich Freeman wrote:
> That essentially works, but you might seriously consider using
> rsnapshot instead in such a situation.  The only thing that changes is
> how the files are organized. [snip]

Yes, this is definitely a better idea.

Let me talk briefly about two (of many) (many!) things that should be
considered when designing and implementing a backup system: (1) file
system changes and (2) expected restoration types.

With respect to (1): a solid backup system will cope gracefully with
events like these:

	A. Today, "fred" is a file.  Tomorrow, fred is a directory.
	B. Today, "fred" is a directory.  Tomorrow, fred is a symlink.
	C. Today, fred lives in /home/foo/fred.  Tomorrow, /home/bar/fred.
		(Fred didn't change, it just moved.)
	D. /home/fred is the top of a directory tree with 100 million files.
	E. /home/fred (with all 100 million files) is now /home/foo
		(None of the files changed)

And so on, for all the myriad variations I could enumerate if I wanted
to sit here all afternoon typing.  Some backup systems cope with these
well; some don't.  Notably, many of those based on rsync don't handle
these circumstances gracefully, particularly if the changes happen
*while it's running*, which is of course a much more difficult circumstance
to cope with than if the changes happen in between runs.  (And this is why
some of us invested rather a lot of time into dump(8) many years ago,
because it worked nicely on quiescent filesystems, not so well on
active ones where things like those could and did happen.)

This is one of the reasons I *strongly* recommend that all 'nix admins
learn how to use dump(8) properly.

With respect to (2), one of the other things to consider with any backup
system is how much work it'll be to do everything from one-off file
restorations to complete restoration of every partition on a completely
failed disk.  Sometimes, backup systems are great at backing things up,
and great at restoring individual items...but not very friendly when
you have to bring back everything.  Or vice versa.   And one of the
considerations in this regard is the expected number of steps required:
let me illustrate by using a typical dump(8) level scheme.

Suppose that you do level 0-6 on the corresponding days of the week,
with 0 = Sunday.  If the chances of a disk failing are uniformly
distributed, then the expected value of the most recent dump level
is 3, i.e., if you lose a disk, you should figure on having to restore
the level 0 and three partials on top of it.  You could reduce this
by doing 0-1-2-1-2-1-2 instead of 0-1-2-3-4-5-6, making the expected
value .6, but you'd pay for that by increased dump output volume, since
the second and third sets of 1's and 2's are relative to the level 0.
But...if your output volume is small, maybe it'd be worth it.

Similar calculations apply to all backup systems: you're always
trading run time and storage for ease of restoration -- and it's
just a matter of which balance works best for your system(s) and
your particular usage patterns.  Personally, I tend to optimize
slightly for individual restorations these days, because total
disk failures are rare and I can usually see them coming.  But that's
not true in every environment, and if yours is one where it isn't,
you probably should veer the other direction.

---rsk
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug