Re: [PLUG] zfs vs btrfs vs …

[Christopher Barry <christopher.r.barry@gmail.com>]

On Mon, 2 Feb 2015 11:57:12 -0500
"K.S. Bhaskar" <bhaskar@bhaskars.com> wrote:

>Walt --
>
>The problem with open source software is that one never knows where it
>may be used.  We need to assume that it will run anywhere that it can
>run.  For example, with the product I manage (fis-gtm.com), I'm told
>that every cash register in Arnotts, the largest department store
>chain in Dublin, runs an application on GT.M on Linux.
>
>So, we need to make the software run on a PC, run in the cloud, be
>robust, and simple to use.  It may not be easy, but it doesn't seem
>impossible.
>
>Regards
>-- Bhaskar
>
>
>On Mon, Feb 2, 2015 at 11:48 AM, Walt Mankowski <waltman@pobox.com>
>wrote:
>
>> Ah, well, you never said this data was going to be in the cloud.
>>
>> You also mentioned that you wanted to minimize support since the
>> instructors might not be computer-savvy.  Copying files around might
>> waste expensive space, but it sounds far easier to support than the
>> complicated schemes that have been suggested in this thread.
>>
>> On Mon, Feb 02, 2015 at 11:32:04AM -0500, K.S. Bhaskar wrote:
>> > Walt --
>> >
>> > Yes, giving each student his/her own database is an option.  But
>> > cloud storage is not necessarily as cheap as hard drive storage,
>> > and the types
>> of
>> > colleges who would use this are always short of money.  30
>> > students times 5GB per student is 150GB - most of which is
>> > replicated 30 times.
>> >
>> > So, yes, if we can't find a better way, giving each student a
>> > separate database is what we'll do.
>> >
>> > Regards
>> > -- Bhaskar
>> >
>> >
>> > On Mon, Feb 2, 2015 at 11:26 AM, Walt Mankowski <waltman@pobox.com>
>> wrote:
>> >
>> > > On Sat, Jan 31, 2015 at 12:16:02PM -0500, K.S. Bhaskar wrote:
>> > > > I am trying to create a virtual machine to be used to teach
>> electronic
>> > > > health records for a class by giving each student a configured
>> > > application
>> > > > s/he can then use, e.g., to record treatments.  The application
>> includes
>> > > a
>> > > > database of several GB.  I would like to give each student an
>> individual
>> > > > database to work with, but would prefer not to have to give
>> > > > each
>> student
>> > > a
>> > > > separate copy of the database, when 99% of the database will
>> > > > be the
>> same
>> > > > for all of them.
>> > > >
>> > > > My current plan is to create the database on filesystem such as
>> btrfs or
>> > > > zfs with a master copy of the database.  To add a student, a
>> > > > script
>> would
>> > > > take a snapshot of the filesystem (or preferably just a
>> > > > sub-tree at directory in the file system) and mount the
>> > > > snapshot with
>> copy-on-write
>> > > in
>> > > > a different place for each student.  For example, if I had a
>> > > /home/master,
>> > > > one might snapshot the master subdirectory and make it
>> > > > available at /home/adam and /home/eve.  Now Adam and Eve can
>> > > > each have a complete database, with one master copy, but each
>> > > > time one of them updates the database, the blocks in the
>> > > > filesystem on which the modified parts
>> of the
>> > > > database live would be copied.  The additional space used is
>> > > > for modifications by each student.
>> > > >
>> > > > I am trying to decide between zfs and btrfs.  One advantage of
>> > > > zfs
>> over
>> > > > btrfs appears to be that the snapshots can be auto-mounted
>> > > > without
>> > > editing
>> > > > /etc/fstab - with btrfs, adding 30 students to the class would
>> > > > seem
>> to
>> > > > require 30 entries in /etc/fstab.  I have zero experience in
>> > > > zfs, and barely any experience with btrfs, so this will be a
>> > > > learning
>> experience
>> > > for
>> > > > me.
>> > > >
>> > > > A couple of questions, please:
>> > > >
>> > > >    - For this application, are there any considerations other
>> > > > than
>> > > mounting
>> > > >    in choosing between zfs and btrfs?
>> > > >    - Should I explore some other alternative, along the lines
>> > > > of unionfs-fuse?  A filesystem in userspace hs the advantage
>> > > > of not
>> > > requiring
>> > > >    root to create a copy, but at least unionfs-fuse does
>> copy-on-write
>> > > at the
>> > > >    file level, not the block level, so each student would end
>> > > > up
>> with a
>> > > >    complete copy of the database.  So, unionfs-fuse itself is
>> > > > out.
>> > > >
>> > > >
>> > > > Thank you very much, in advance, for advice, opinions, and
>> > > > pointers.
>> > >
>> > > Other people have commented on zfs and btrfs, so let me start the
>> > > discussion on "...".  Given your description, I don't see the
>> > > need to try to optimize the disk space being used.  Wouldn't
>> > > another option just be to give everyone a complete copy of the
>> > > database?  At several GB per student, even a fairly big class
>> > > could easily fit on an inexpensive 1-2 TB HD.  Why not just do
>> > > that instead of using filesystems that are still poorly
>> > > supported on Linux?
>> > >
>> > > Walt
>> > >

Another option that I've used to good effect is to attach to the
primary db block device via ISCSI as a read only disk from each VM.

Then, in each vm, mount the db block device, and union another
directory over that mount so all modifications are copied up to the VM.

-C
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug