Mag Gam on 6 Dec 2010 15:56:56 -0800

[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: [PLUG] recompiling a kernel for performance

Thanks everyone for your responses.

Yes, I did look for a turbo button and apparently one does exist!  HP
has a secret BIOS which lets you disable powersaving modes and
apparently it makes things faster. :-)

Yes, I meant 2.6.36

The programs are mostly CPU intensive. However , I/O is always a
factor so I am planning to implement a distributed files system  such
as Hadoop.  Regarding jumboframes, do I have to make the switch on the
client side or switch side ?

READ-COMPUTE-WRITE is absolute correct for my case however the CPU is
always at 100%. I am assuming I am still CPU bound because, I read
about 90% of data and compute and then generate a result.

Regarding, 'Do your cluster tools allow you to measure the performance
of specific nodes? ', this is a very good question. Yes, I am using
lapack provided by Intel but the problem is I am not sure what to look
for. Ideally, I want something like this... if I have a  processor, P
which can do X floating point operations in T,time...I run the program
and it should be very close to the manufacture specs. Then I know the
processor is at optimal state. I am not sure if anything like that
exists....please guide me if one does.

p.s 50 node cluster is our test cluster. We have a much larger one for
production :-)

On Mon, Dec 6, 2010 at 11:22 AM, Austin Murphy <> wrote:
> Hi Mag,
> On Mon, Dec 6, 2010 at 8:26 AM, Mag Gam <> wrote:
> ...
>> simulation may consist of a Octave, Python, R, and MATLAB process
>> which reads data and generates data. Each process can take 60 mins to
>> 70 hours. I am sure there are other tuning we can do such as -- tune
>> I/O subsystem, tune network, etc...
> ...
>> Assuming all the 'low bearing fruit' have been picked would
>> recompiling with the latest 2.3.36 kernel help in computing speed?
> ...
>> Also, are there any settings in the kernel I can set to enhance
>> performance -- According to redhat you should stick with their build
> I don't think you are going to find a hidden "turbo button" in the
> kernel tunable options.
> For the most part, the kernel is already configured for maximum speed
> across a wide range of possible workloads without unreasonable
> side-effects. Â The tunable options give you a chance to make some
> workloads faster at the expense of making other workloads slower.
> If you have a 50 node environment, I'd guess that the biggest gains
> will be seen in improving the performance of your shared storage.
> Ethernet jumbo frames or TCP offload might help if you have the
> hardware support. ÂMounting with "noatime" can cut down on a lot of
> unnecessary writes.
> You might also want to oversubscribe your CPUs. ÂFor example, if your
> processes go like this: READ--COMPUTE--WRITE, there is probably a lot
> of free CPU time available while reading and writing to run more
> threads or jobs. ÂAn 8 core server with sufficient RAM might be able
> to run 12 or 16 jobs in about the same amount of time as 8 jobs.
> Austin
> ___________________________________________________________________________
> Philadelphia Linux Users Group     --    Â
> Announcements -
> General Discussion Â-- Â
Philadelphia Linux Users Group         --
Announcements -
General Discussion  --