Re: [PLUG] recompiling a kernel for performance

[Mag Gam <magawake@gmail.com>]

Thanks everyone for your responses.

Yes, I did look for a turbo button and apparently one does exist!  HP
has a secret BIOS which lets you disable powersaving modes and
apparently it makes things faster. :-)

Yes, I meant 2.6.36


The programs are mostly CPU intensive. However , I/O is always a
factor so I am planning to implement a distributed files system  such
as Hadoop.  Regarding jumboframes, do I have to make the switch on the
client side or switch side ?

Austin,
READ-COMPUTE-WRITE is absolute correct for my case however the CPU is
always at 100%. I am assuming I am still CPU bound because, I read
about 90% of data and compute and then generate a result.

Eric,
Regarding, 'Do your cluster tools allow you to measure the performance
of specific nodes? ', this is a very good question. Yes, I am using
lapack provided by Intel but the problem is I am not sure what to look
for. Ideally, I want something like this... if I have a  processor, P
which can do X floating point operations in T,time...I run the program
and it should be very close to the manufacture specs. Then I know the
processor is at optimal state. I am not sure if anything like that
exists....please guide me if one does.


p.s 50 node cluster is our test cluster. We have a much larger one for
production :-)




On Mon, Dec 6, 2010 at 11:22 AM, Austin Murphy <austin.murphy@gmail.com> wrote:
> Hi Mag,
>
> On Mon, Dec 6, 2010 at 8:26 AM, Mag Gam <magawake@gmail.com> wrote:
> ...
>> simulation may consist of a Octave, Python, R, and MATLAB process
>> which reads data and generates data. Each process can take 60 mins to
>> 70 hours. I am sure there are other tuning we can do such as -- tune
>> I/O subsystem, tune network, etc...
> ...
>> Assuming all the 'low bearing fruit' have been picked would
>> recompiling with the latest 2.3.36 kernel help in computing speed?
> ...
>> Also, are there any settings in the kernel I can set to enhance
>> performance -- According to redhat you should stick with their build
>
> I don't think you are going to find a hidden "turbo button" in the
> kernel tunable options.
>
> For the most part, the kernel is already configured for maximum speed
> across a wide range of possible workloads without unreasonable
> side-effects. Â The tunable options give you a chance to make some
> workloads faster at the expense of making other workloads slower.
>
> If you have a 50 node environment, I'd guess that the biggest gains
> will be seen in improving the performance of your shared storage.
> Ethernet jumbo frames or TCP offload might help if you have the
> hardware support. ÂMounting with "noatime" can cut down on a lot of
> unnecessary writes.
>
> You might also want to oversubscribe your CPUs. ÂFor example, if your
> processes go like this: READ--COMPUTE--WRITE, there is probably a lot
> of free CPU time available while reading and writing to run more
> threads or jobs. ÂAn 8 core server with sufficient RAM might be able
> to run 12 or 16 jobs in about the same amount of time as 8 jobs.
>
> Austin
> ___________________________________________________________________________
> Philadelphia Linux Users Group     --    Âhttp://www.phillylinux.org
> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
> General Discussion Â-- Â http://lists.phillylinux.org/mailman/listinfo/plug
>
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug