Re: [PLUG] How to Store Video Files for 25 Years?

[Rich Kulawiec <rsk@gsp.org>]

I've worked on several long-term archiving projects and am about
to launch one of my own (for which I'll be recruiting lots of help
in good time).  Here are some things I've learned.

1. Diversify formats, storage, and locations.

Don't store everything in (just one) format or on (just one) kind of
media or in (just one) location.  Every time you add another one of
these you help to insulate yourself from obsolescence and catastrophes.

2. Store the software you need with the archive.

That means the operating system, the applications, the source code,
and if you have it, the documents which describe the formats used by
the data you're storing.  It also means that you need to keep this
updated, as you may now have some trouble finding working hardware
to run the copy of SunOS 4.1.4 for sparc-32 that you stashed 25 years ago.

3. Plan on migrating storage every few years.

Reliability, economics, capacity, etc. are all going to push you to
do this, so plan for it.

4. Keep catalogs/records on archival (acid-free) paper.

These are the breadcrumb trail that will help you recover from
unanticipated events.  But only if you have them.

5. Provide a nice environment.

Avoid sunlight, power fluctuations, temperature swings, humidity,
electromagnetic disturbances, dirt/dust, etc.

6. Script as much as possible.

Scripting creates a record of what you did and how you did it --
unlike ad hoc typing at the command line.  Save the scripts.
They may one day be the key to solving a puzzle that you didn't
know you were creating.

7. Compression is tempting...but don't.

The cost of storage continues to plummet.  The cost of recovering
from a compression error (if you even can) is enormous.

8. Fancy filesystem are tempting...but don't.

You're not building a real-time fault-tolerant high-volume transactional
database system here.  You'll rarely read data (and when you do, it will
most likely be sequential reads).  You'll even more rarely write (and
when you, it will be sequential writes).   Eschew the latest and greatest
in favor of something that's proven rock-solid over a long time.

9. RAID is tempting...but don't.

Same rationale as 7 and 8.

10. Test periodically.

Suppose you think you have 3 copies of a particular item.  You won't
know that you're *really* down to 2 readable copies unless you test them
all periodically.  It would be good to figure this out before you're
down to 0 copies.

---rsk
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug