Re: [PLUG] backup lessons from a cloud storage disaster

[Keith via plug <plug@lists.phillylinux.org>]

On 4/27/21 1:12 PM, Rich Freeman via plug wrote:

> On Tue, Apr 27, 2021 at 11:08 AM JP Vossen via plug
> <plug@lists.phillylinux.org> wrote:
> > I like, "There is no cloud, there is only someone else's data center."
> I agree, because it actually encapsulates the pros and cons of the
> situation.  I don't think you can talk about "the cloud" in black and
> white terms and I think a lot of people in the FOSS community/etc
> don't stop and think about how normal people interact with technology.
>
> The average person keeps all their stuff on a desktop PC with a single
> hard drive, and one day it stops working right and they call their IT
> friend up and ask them to help them get their photos back.
>
> For THIS average person, the cloud is a MUCH better solution.  Sure,
> they're still putting all their eggs in one basket, but it is a
> professionally-managed basket vs the PC sitting under their desk
> buried in dust without a UPS.

This is a pretty fair statement.  The only thing I was add is that 
professional doesn't mean what people thinks it means. Professional 
often implies a superior level of skill or experience but classic 
definition of meaning you are paid for your services is most 
appropriate.  Clearly, in this case the hazards that this company 
succumbed to were a result of not having the experience to consider 
their own failure points or not having the skill to architecture a more 
resilient solution.  Ultimately this a 1st order failure by the guy with 
the other computer (or data center).  Such things are obvious risks.

I don't do residential business but for my strategic partners and 
others, I always advise to have 2 or 3 physical copies of their data 
that they can get access to in a reasonable amount of time and then if 
you want, get a 3 or 4th copy in some cloudy thingy.

> I'd defer to Keith and a few other consulting-types here, but I
> suspect most small businesses (who aren't paying Keith/etc) are in
> similar shape.  For them the cloud is also a better solution than what
> they have.

You're 100% right.  A big part of what I end up having to do is push 
back on the concept and use of cloud.  First explaining that it is more 
of a marketing term than anything else and second that you are doing 
nothing more that using resources at a distance. Third, if you do 
embrace the use of resources at a distance, consider why you aren't 
allocating your own resources in a remote data center.  That's usually 
where arguments break down and stakeholder realize that their 
motivations are not as sound as they thought they were.  Things really 
go sideways when I (and most people here) can build a solution that is 
going to make more sense.

That said, I have to admit that even with the expansion of attack 
surfaces due to the COVID-19 pandemic (i.e. remote work) and supposedly 
a heightened sensitivity to business continuity, I'm finding that too 
often organizations are not willing to invest in the resources to 
protect their operations from data losses and complete system failures.  
I continue to not understand that and this is like year 11 for me.

> Now, for those on this list who have a RAID and an offsite backup
> solution and regular testing and all that good stuff, the cloud may
> very well be a step back in a lot of ways.  We're not typical IT users
> though.

We PLUG'ers are so elite.  People just don't understand the words coming 
out of our collective mouths  LOL

Seriously, though, you are right.  The funny thing is I think we all 
have have these conversation professionally or privately and have they 
have gone over people heads as to why its important.

> I think for most the practical solution is probably some kind of
> combination of the cloud plus local storage.  Either it is local
> storage backed up by a cloud service (Backblaze or whatever), or a
> cloud service (Google Photos/Drive/etc) with local backup (Google
> Takeout/etc).  That is pretty convenient to set up and protects you if
> the cloud provider goes down, but also protects you if YOU go down.
>
> For people on this list there are a lot of other options that I think
> I've talked about.  Personally right now I use a combination of ZFS,
> lizardfs, bacula, and duplicity+gpg+AWS.  The first two are primary
> storage, the last two are backups.  Duplicity is run by cron daily so
> that ensures I always have a cloud backup as a fallback, and it is
> encrypted so privacy isn't really a concern here (obviously I have
> offsite copies of my key as well - and that key isn't used as anything
> else so basically it acts like a second factor in addition to my AWS
> credentials).

I've talked about my solution too, which really is the a bit of the same 
that I use for my clients so I won't burn bandwidth restating it but the 
most important think you said is that it is a combination.  Cloud / 
remote storage is fine and local storage is fine.  When you combine the 
two you get a solution that far more robust that either methodology on 
its own.  Circling back to the op's article-  the simple point that 
those clients that had other backup resources in place where fine though 
this failure can not be amplified enough.

--
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Keith C. Perry, MS E.E.
Managing Member, DAO Technologies LLC
(O) +1.215.525.4165 x2033
(M) +1.215.432.5167
www.daotechnologies.com

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug