gabriel rosenkoetter on 6 Jan 2004 03:39:02 -0000


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] distributing work


On Mon, Jan 05, 2004 at 07:03:22PM -0500, Kevin Brosius wrote:
> The nice feature is that all you need to do is kick off work
> that loads a single machine heavily (with multiple processes.)  As the
> load goes up, an OpenMosix system will move processes across the network
> away from the heavily loaded machine onto the other machines.

While this sounds good, process migration is an expensive task;
unless your compute process is really orders of magnitude bigger
than the costs associated with packaging a process for migration,
the benefit of only having to start processes on one machine is
pretty minor compared to the cost of losing the migration time.

Mosix *is* nice in that you can use it like Knoppix (walk around a
lab and drop CDs in all of the unused systems at closing time and
they'll boot, join your cluster, chug all night, and then drop out,
ejecting the CD and rebooting, in the morning), but it's got more
fluff than it sounded like Jeff was after.

> How well this performs depends heavily on the job type and the mix of
> machines involved.  It's easier to setup than a Beowulf though, because
> it's a kernel patch plus some utils and system setup.  Beowulf was a lot
> more painful, as I recall.

clusterit's a few shell scripts. :^>

I mentioned clusterit because I know its author (who looked around
at options like MPI, PVM, and GLUnix) wrote it out of disatisfaction
with the available options for a simple process distribution
mechanism.

Jeff, if you're interested in setting up a more permanent arrangment
to distribute compute processes around your lab, you may want to
have a look at Sun's Grid Engine (http://gridengine.sunsource.net/).
There's a non-trivial amount of setup involved there.

-- 
gabriel rosenkoetter
gr@eclipsed.net

Attachment: pgpklRJbRPIOW.pgp
Description: PGP signature