Keith via plug on 3 Feb 2021 14:02:30 -0800


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] Backup solution 

On 2/3/21 4:16 PM, Rich Freeman wrote:

On Wed, Feb 3, 2021 at 12:42 PM Keith via plug
<plug@lists.phillylinux.org> wrote:

I think scope should be talked about too.  The op was talking about
imaging their system with dd so we're not talking about file backup.
Cloud services are going to be expensive for large datasets and the lack
of bandwidth doesn't make them viable for regular use.

++ - cloud is not so reasonable for imaging.


What about personal pictures and video?  Ok, so now we're probably pushing
terabytes.  If something happens, most people want to be able to recover
everything.

LOL, my duplicity daily backups currently use 780GB of space on AWS.
That includes two full backup sets (a full plus a month or two of
incrementals in each set).  My permanent backups of stuff that doesn't
change (which includes photos older than a month) is up to 1.9TB.

My AWS bill is a few dollars per month.  It really isn't that bad
considering that hard drives don't last forever and this has multiple
site redundancy (for the permanent stuff).

Now, recovering everything would definitely be an issue.  AWS does
charge substantial fees for this, but it still only adds up to
something like $200 or so.  At my scale it is actually cheaper to use
the Snowcone service and have them ship it to me physically, which is
probably faster anyway for 2TB.
Ok.  Fair enough on the price... a few dollars I'm guessing it is under $10 or maybe under $5.  You have the advantage on being on FIOS.  Not everyone is so pushing 780Gb daily might not be possible.  You could make the argument that daily is not necessary. That said, backing up 780Gb locally will always be faster and costs nothing.  If you have enough space, do both. 
Disk is cheap... buy two (or three)... mirror and encrypt them. Keep one
at the ready and get the other in a proper safe, in your everyday bag or
give the other to someone you trust.  I think we've forgot the value of
relying the local community for things like this.  Its an option that
should not be overlooked.

That is definitely a consideration, but you do need to make sure
you're rotating and verifying your media so that you know it is always
good, probably with multiple redundancy.
If you're not pulling down your data from AWS regularly to test a full rebuild, you are just assuming they are doing what they need to do to keep your data safe.  That is not something I would recommend anyone assume.  If you can't say you've ever rebuilt your systems from scratch from your backups then you don't know what you have is going to work on what would be a very bad day.  I would seriously recommend that anyone using any cloud or remote resource go through the entire process to make sure you understand it... especially the timeframe.  I can envision scenario's where waiting 24 hours might be too long.
I'm clearly not for cloud but it should go without saying that for whatever you are doing, you should know your (re)build process very well because you'll probably be doing it under some stress in sub-optimal conditions. 



My 2TB of data sounds like a lot, but I only generate maybe 100GB per
month when I'm actively taking lots of photos, so FIOS keeps up with
that fine, and there are no transfer fees into AWS.  Sure, it will
cost some money to retrieve the data, but I have redundancy and local
backups anyway, so this is for a doomsday scenario like the house
burning down.  If my house burns down forking out $200 to get my
photos back is really not a big deal - I wouldn't be surprised if I
could get insurance to cover it.  Plus it isn't like I need them back
in a day.
I'm using 10Tb mirrors now so 2Tb doesn't sound like to me :) (most of that is a 5Tb nilfs2 filesystem image that contains versions of my file system dumps).  I hear you though- your point is to get your data offsite and really that is the point. 




--
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Keith C. Perry, MS E.E.
Managing Member, DAO Technologies LLC
(O) +1.215.525.4165 x2033
(M) +1.215.432.5167
www.daotechnologies.com

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug