Re: [PLUG] [plug-announce] TOMORROW - Tue, Apr 19, 2016: PLUG North - "Linux Containers" by Jason Plum and Rich Freeman (*6:30 pm* at CoreDial in Blue Bell)

[bergman@merctech.com]

In the message dated: Mon, 25 Apr 2016 17:45:16 -0400,
The pithy ruminations from "Keith C. Perry" on 
<Re: [PLUG] [plug-announce] TOMORROW - Tue, Apr 19, 2016: PLUG North - "Linux C
ontainers" by Jason Plum and Rich Freeman (*6:30 pm* at CoreDial in Blue Bell)>
 were:
=> "
=> The whole point of zfs/btrfs is that they DO detect these kinds of
=> corruptions, and they automatically use the redundant copy of the
=> data.  Sure, if the same block in n+2 drives gets hit by a cosmic ray
=> then you'll still lose it, but that is MUCH less likely to happen than
=> a single flip, or two flips of unrelated blocks.  The whole server
=> could get hit by an asteroid as well."
=> 
=> I think you might be missing my point that file system is only one layer of technology.  We can assume that one file system is superior to another an you would have have the problem of the hard disk failing.  Whether its do to cosmic rays or some other random event it is possible to have the physical HD fail in a way that would represent a silent failure in the file system.
=> 

I'm not clear on what you're trying to say here. By "silent failure
in the file system" do you mean that the fault in a physical drive was
handled transparently and caused no disruption to the filesystem (a good
thing), or do you mean that the fault was undetected by the filesystem,
which failed (a bad thing).

A physical HD that has a fault and is not detected by other layers in
the system represents a design (or operation) failure, much more than
a hardware failure.

If you consider the entire 'system', from user management of files,
to the filesystem, to redundant paths to storage devices, to redundant
controllers, to logical volumes in a redundant array of drives, down to
individual storage devices, then a well designed (and well-managed &
monitored) system does not have silent failures.

Data consistency at each level can be verified and some corruption
potentially recovered.

For some perspective, see the 2007 LISA talk by Andrew Hume:

	https://www.usenix.org/legacy/event/lisa07/tech/tech.html#hume

=> Likewise you can realize a silent error from a higher level- if you were to write a file with bad data to the file system, that file system, any file system, would never know.  For instance, a topic I'm constantly discussing now is ransomware.  Despite specific methods of active detection for that, the general question remains, how can you determine in a absolute manner over time that a file has been encrypted without the users consent.  Those solution are out of scope for here but I will say how you do that has nothing to do with the file system.  

You need to distinguish bad data (ie, unwanted content that has no
errors) from corrupt data (data that has errors after the content was
created). Ransomware represents bad data, but not corrupt data. Yes, from the
user's point of view the content is inaccessible, but it is not corrupt.

Now you're getting into general questions of Information Theory, not
data corruption. [Paging Dr.  Turing & Dr. Shannon to the Blue Conference
Room, please.]

=> 
=> 
=> ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 
=> Keith C. Perry, MS E.E. 
=> Owner, DAO Technologies LLC 
=> (O) +1.215.525.4165 x2033 
=> (M) +1.215.432.5167 
=> www.daotechnologies.com
=> 

-- 
Mark Bergman    Biker, Rock Climber, SCUBA Diver, Unix mechanic, IATSE #1 Stage
hand
'94 Yamaha GTS1000A^2
bergman@panix.com 				https://www.flickr.com/photos/r
msppu

http://wwwkeys.pgp.net:11371/pks/lookup?op=get&search=bergman%40panix.com

I want a newsgroup with a infinite S/N ratio! Now taking CFV on:
rec.motorcycles.stagehands.pet-bird-owners.pinballers.unix-supporters
15+ So Far--Want to join? Check out: http://www.panix.com/~bergman 
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug