Paul L. Snyder on 26 Mar 2009 05:56:15 -0700

[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] scientific computing data formats

On Thu, 26 Mar 2009, Mag Gam wrote:

> I have been investigating data formats for the  research I have been
> conducting. At the moment, I am storing everything in csv files and
> the present directory structure is:
> species/family/order/class/phylum/kingdom/domain

What about CSV isn't working for you?

How are you going to be using the data?

What languages will you be using it from?

How much data are you talking about?

> I looked into hdf5 (,  and was wondering
> if anyone had any thoughts about it? I have managed to move some of my
> data over but I mistakenly corrupted the file. I tried to retrieve the
> data but I no luck.

I haven't had any experience with HDF5, so I can't help you there.  I'd be
disinclined to move to a complex binary format without a clearly definied

> Does anyone have an alternative? or should I purse hdf5 again?

Lately I've been using Python for this sort of thing.  It's quite
convenient to use one script to read you data into Python data structures,
then use the pickle module to save them out.  Other scripts can then work
directly with the pickled data.

Philadelphia Linux Users Group         --
Announcements -
General Discussion  --