Re: [PLUG] How to Store Video Files for 25 Years?

["Keith C. Perry" <kperry@daotechnologies.com>]

Keeping backups offline however doesn't inheriently address issue with data poisioning (i.e. corruption, loss, etc) unless you do this religious checking of data before it is taken offline.  If you're just doing a reference against the current online data then that is possible but rather time consuming.  A more robust way of doing this is to version all copies of your data and have a retention time.  If you do that then you have a complete history and protection against data posioning which is a function on your desired retention time, storage system capacity and dataset size.  This approach can protect from a failure of the human effort so long as your retention time is long enough to outrun what might be requested from a time passed.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 
Keith C. Perry, MS E.E. 
Managing Member, DAO Technologies LLC 
(O) +1.215.525.4165 x2033 
(M) +1.215.432.5167 
www.daotechnologies.com

----- Original Message -----
From: "Rich Freeman" <r-plug@thefreemanclan.net>
To: "Philadelphia Linux User's Group Discussion List" <plug@lists.phillylinux.org>
Sent: Monday, December 10, 2018 8:22:15 AM
Subject: Re: [PLUG] How to Store Video Files for 25 Years?

On Mon, Dec 10, 2018 at 1:43 AM Fred Stluka <fred@bristle.com> wrote:
>
> *Don't archive. *Keep it all on your current hard disk with your
> current files.  And do regular backups of all your files.

...

> *Do diffs. *Part of my standard automated daily routine is to diff
> a bunch of critical files and directories against yesterday's backup
> to spot unexpected changes or bit rot.

I just want to highlight this.  If you do the first part, and not the
second, you're exposing yourself to a huge risk by keeping these files
online.  If this is data you never look at, and you keep it online,
then there is a very real risk that something will happen to those
files, and over time all your backups will be rotated until the only
copy left is your corrupted copy.

One reason for keeping offline read-only backups is that they can't be
changed aside from the decay of the media, which is generally
understood and manageable for anything you should be using for
long-term archival.  So, you don't have to check it every single day
to see if it accidentally changed.  You may need to refresh the
backup, but that is a deliberate action taken every so many years, and
at that time you're going to be doing careful verification to ensure
the data was copied correctly.

If you are going to keep it online, and you DON'T religiously check
diffs every day, then if an issue creeps in you'll never know until
you go to retrieve the file in a decade or two.

Now, if you keep backups long-term on media that actually last
long-term (basically tape) then this is less of an issue.  But this
presents all of the issues of archiving to offline media in the first
place.  Having a 20 year old tape you can't read isn't much
consolation.

I mention this mainly because I think that most people aren't really
going to do daily diffs of their systems.  You could do it on just the
parts of your filesystem that you don't expect to change, but that
does involve the risk that your diff script breaks at some point and
you never notice because the normal output is no output.

As with anything, there are pros and cons to every approach.  Fred's
isn't a bad one in principle (IMO).

-- 
Rich
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug