Re: [PLUG] Wanted: volunteers with bandwidth, storage, coding skills to

Re: [PLUG] Wanted: volunteers with bandwidth, storage, coding skills to help save climate data

This topic needs to be dropped all together. But I'll answer Paul's questions:

Are these large data sets vulnerable? : Not the slightest. NOAA has multiples of the data not to mention that the data is open to other corporations that are doing white paper analysis of all academic findings (confirmed with many friends working for agricultural programs in Iowa, Minnesota, Missouri, and Illinois without having companies mentioned).

Deleting and skewing the data to skew the debate on climate change? : Not going to be touched at all. The same data that is used for creating more accurate system models is also used for climate data. Messing with the data imposes economic and strategic disadvantages. Also if you watch the debate, the arguments is whether or not "if" climate change exists but whether or not it is man made or at the very least "substantially human influenced".

Doesn't it make sense to have the strongest data redundancy possible in place for irreplaceable data that has been gathered over decades by scientists, satellites, the noaa, etc. Is it paranoid to think that this body of knowledge is extremely valuable and potentially vulnerable? : The nature of scientist replicating the studies means that the data is redundant already. I know where I work we are sitting on a 100 PB at least. I know there are other organizations from private companies that rely on the same data that have stores equal two or near 1 Exabyte. Given the call for help and concerning bandwidth, there is no way any effort to back up this data on personal machines in anyway is going to be 1) successful 2) get done before the day of suspected doom (which all signs point to Trump actually saying he's concerned about the environment). 3) private companies have backed up this public data and have it available upon request as they too benefit from the collaborative nature of the scientific community. 4) Everyone downloading this data is going to disrupt the science going on already since the data sets for a single years worth (pending model) is in the 10's of terabytes, the scientists need their bandwidth to do their work.

Are there technical solutions that our community could be a part of to help ensure that these collective resources are not endangered by sharp political swings or other unforseen circumstances? : Uhm... yeah, we have 3 PLUG members doing just that. I'll let you know from a technology end if I need help relaying in better words the need for better support of IT. In case nobody has noticed, the government pretty much has a policy of never throwing away data. Also there is enough storage on the dedicated network that is shared by Argonne National Labs, Oak Ridge Laboratory, and some other labs (both academic, commercial, and government) have available to them. If you really wish to help, get out your check book and prepare to store over half an exabyte from any particular source.

Source: I'm one of the three PLUG members currently on site at a NOAA facility that does most of the climate research.

-Will C

On Sun, Dec 18, 2016 at 4:54 PM, Steve Litt <slitt@troubleshooters.com> wrote:

The sequence of this thread is telling. Rich Kulawiec offered some info
and a mailing list for those who want to preserve the data in case it's
later destroyed. It didn't attempt to persuade anyone to change their
mind about anything. I presume he posted it here because people on this
list are likely to have a lot of disk space they know how to back up and
preserve.

One would think there would be two possible responses to Rich's post:
Either sign up on his mailing list and try to help, or ignore it and
move on.

But nooooooo. Person after person lined up to proclaim that there's no
need for worry, stating opinions as facts, or voicing just plain
ad-hominums. What is the make and model of their crystal ball? What is
their motivation in arguing with a call for volunteers? Why were there
any replies, *on this list*, to Rich's call for volunteers? Why argue
with him? Why not let him have his post and move on?

And finally, may I point out that history is littered with carnage that
started with people saying "it can't happen here."

SteveT

On Sat, 17 Dec 2016 11:11:26 -0500
Will <staticphantom@gmail.com> wrote:

> As someone that works a site that researches climate data, we are
> nearing a quarter exabyte of data. The data is fine and nobody should
> concern themselves. Someone can do me a huge favor however if they
> tell me the data is HDF of NetCDF formatted.
>
> -Will C
>
> On Dec 17, 2016 11:08, "Christopher Barry"
> <christopher.r.barry@gmail.com> wrote:
>
> > On Fri, 16 Dec 2016 10:32:33 -0500
> > Rich Kulawiec <rsk@gsp.org> wrote:
> >
> > >If you have an interest in preserving a lot of scientific research
> > >data:
> > >
> > > Scientists are frantically copying U.S. climate data,
> > > fearing it might vanish under Trump
> > > https://www.washingtonpost.com/news/energy-environment/
> > wp/2016/12/13/scientists-are-frantically-copying-u-s-
> > climate-data-fearing-it-might-vanish-under-trump/
> > >
> > >I've set up a mailing list to coordinate IT efforts to help out.
> > >Signup via climatedata-request@firemountain.net or, if you prefer
> > >Mailman's web interface,
> > >http://www.firemountain.net/mailman/listinfo/climatedata should
> > >work.
> > >
> > >---rsk
> >
> > While I greatly respect you Rich, this whole thing seems a bit like
> > 'the sky is falling' to me.
> >
> > Does anyone /really/ think Trump would seize or delete their data?
> > That's seriously paranoid and weird. Why can't they just throw it
> > up on github or somewhere like that?
> >
> > --
> > Regards,
> > Christopher
> > ____________________________________________________________
> > _______________
> > Philadelphia Linux Users Group --
> > http://www.phillylinux.org
> > Announcements - http://lists.phillylinux.org/
> > mailman/listinfo/plug-announce
> > General Discussion -- http://lists.phillylinux.org/
> > mailman/listinfo/plug
> >

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug