Re: [PLUG] recompiling a kernel for performance

[Austin Murphy <austin.murphy@gmail.com>]

Hi Mag,

On Mon, Dec 6, 2010 at 8:26 AM, Mag Gam <magawake@gmail.com> wrote:
...
> simulation may consist of a Octave, Python, R, and MATLAB process
> which reads data and generates data. Each process can take 60 mins to
> 70 hours. I am sure there are other tuning we can do such as -- tune
> I/O subsystem, tune network, etc...
...
> Assuming all the 'low bearing fruit' have been picked would
> recompiling with the latest 2.3.36 kernel help in computing speed?
...
> Also, are there any settings in the kernel I can set to enhance
> performance -- According to redhat you should stick with their build

I don't think you are going to find a hidden "turbo button" in the
kernel tunable options.

For the most part, the kernel is already configured for maximum speed
across a wide range of possible workloads without unreasonable
side-effects.   The tunable options give you a chance to make some
workloads faster at the expense of making other workloads slower.

If you have a 50 node environment, I'd guess that the biggest gains
will be seen in improving the performance of your shared storage.
Ethernet jumbo frames or TCP offload might help if you have the
hardware support.  Mounting with "noatime" can cut down on a lot of
unnecessary writes.

You might also want to oversubscribe your CPUs.  For example, if your
processes go like this: READ--COMPUTE--WRITE, there is probably a lot
of free CPU time available while reading and writing to run more
threads or jobs.  An 8 core server with sufficient RAM might be able
to run 12 or 16 jobs in about the same amount of time as 8 jobs.

Austin
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug