Re: [PLUG] Best Solution for Multiple Volume Backups

[Mark Bergman via plug <plug@lists.phillylinux.org>]

In the message dated: Mon, 13 Jul 2020 14:09:46 -0400,
The pithy ruminations from Rich Freeman via plug on 
[[PLUG] Best Solution for Multiple Volume Backups] were:
=> I want to backup one or more paths potentially across multiple
=> volumes.  I want support for:
=> 
=> * Include/Exclude
=> * Incremental backups
=> * Tracking deleted files
=> * Ideally not too dependent on mtime

All possible.

There will almost certainly be a database involved somewhere.

=> * Backup solution is itself easy to backup/restore (might just do this
=> by running it in a container and sticking a tarball on the backup
=> drive)

Not a good idea with most databases.

=> * If it uses indexes/etc these should be re-creatable if lost
=> * Backups must be able to span multiple devices which may or may not
=> all be mounted at once
=> * 100% CLI.  I don't mind if it has a nice X11 client but I need to be
=> able to run from the CLI, or do things like hit the start button on
=> X11, then kill the X server and the backup keeps running and I can
=> just re-connect to check on it.
=> * Reasonably atomic - full backups are going to take many hours so
=> stuff is going to change while running.  It might be able to work
=> around this with snapshots I guess...
=> 
=> Backup sizes are going to be large - 10+TB.  It wouldn't hurt if this

Is that a full, a differential, or an incremental? How often do you anticipate running a backup of that size?

=> can be interrupted/resumed but if not there needs to be some
=> reasonable way to restart a backup if interrupted (ie interrupted
=> incremental shouldn't require a new full backup to recover).
=> 
=> I'm pretty sure Bacula could do the job, but it is a bit heavyweight

I run a moderate Bacula environment (a "full" backup would be ~350TB,
if I did something insane like having a single 'fileset' for the whole
filesystem). Bacula could do everything you want. There is quite a bit
of learning curve.

=> and I suspect all the media management might get cumbersome with
=> disks.  I don't want to be rotating $200 hard drives the way you might
=> rotate $10 tapes.

Not as bad as you think. I don't backup to disk, but many bacula users do, and it's got a lot of features that allow
volumes on a disk to emulate a tape -- mount/umount additional disks to do a full backup, set retention policies to
overwrite existing backups as needed to save media, etc.

=> 
=> Are there other solutions that might make sense?  I'll probably start
=> looking at Bacula again - in the past my biggest concern was that it
=> was hard to back up, but sticking it in a container and creating a
=> tarball might solve that.

Not if you want a coherent restore.

My strategy is that the last job in a set of backups will:

	dump the bacula database (postgres, mysql, mariadb) to disk
	write that backup to tape

That's a very common solution to the 'bacula self-backup' dilema.

Mark

=> 
=> -- 
=> Rich
=> ___________________________________________________________________________
=> Philadelphia Linux Users Group         --        http://www.phillylinux.org
=> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
=> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug
=> 
-- 
Mark Bergman    Biker, Rock Climber, Unix mechanic, IATSE #1 Stagehand

http://wwwkeys.pgp.net:11371/pks/lookup?op=get&search=bergman%40merctech.com

I want a newsgroup with a infinite S/N ratio! Now taking CFV on:
rec.motorcycles.stagehands.pet-bird-owners.pinballers.unix-supporters
15+ So Far--Want to join? Check out: http://www.panix.com/~bergman 
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug