brent timothy saner via plug on 14 Jul 2020 09:17:32 -0700


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] Best Solution for Multiple Volume Backups 

On 7/13/20 18:27, Rich Freeman via plug wrote:
>> Not a good idea with most databases.
> 
> Yeah, that is why I had the next point.  Any databases must be
> disposable, but as you suggest I'm fine with having a backup of the
> database.

I'm going to suggest BorgBackup here (https://www.borgbackup.org/).

I wrote a wrapper with a plugin system that even has an example for
MySQL/MariaDB, as an example, that will take backup-appropriate (and, in
the case of MySQL anyways, diff-able) dumps:

wrapper:
https://git.square-r00t.net/BorgExtend/

plugin:
https://git.square-r00t.net/BorgExtend/tree/plugins/mysql.py

If you can do it in python (including with subprocess), you can do it.
Theoretically it should let you back up anything that has a dumping
output option. YMMV depending on whether that output is some form of
ordered plaintext though (like mysql dumps, *sometimes* LDIFs) or
delta-ing binary dump, though. Incrementals obviously work best when
they have a reliable method of diffing for obvious reasons.

> And really all backups need an index one way or another.  They might
> or might not be in separate files/databases/whatever, but to do an
> incremental without rescanning the last full the backup software needs
> to have some idea what has already been backed up.  I definitely want
> a backup solution where the full backup doesn't need to be accessed to
> perform an incremental.
> 

This bit here is what made me think of borg. It keeps a local cache that
does speed along the process of creating deltas. The only catch is it
can chew up a bit of local storage if your backup set is huge and you
have multiple repositories per client. And you'll obviously want to
explicitly exclude that cache dir if you're backing up its parent
directory (usually it's in ~/.cache/borg, if memory serves, of the user
calling it). But it certainly helps with not needing to pull down the
full backup just to compare.

Attachment: signature.asc
Description: OpenPGP digital signature

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug