Re: [PLUG] To Drop or Not Drop Caches

[bergman@merctech.com]

In the message dated: Mon, 16 Nov 2009 13:06:39 EST,
The pithy ruminations from "Brian Vagnoni" on 
<[PLUG] To Drop or Not Drop Caches> were:
=> So I had a problem that returned to me. My system caches were eating up all my free ram, and slo
=> wing down my system. I found this and it works:

Can you give more background about how caching filesystem data in unused memory 
is substantially slowing down your system (hardware, application requirements, 
how caching was determined to be the problem, etc.)? As I understand it, the
algorithm is roughly:

	[1] if there's unused memory, cache recently requested filesystem data
	[2] for filesystem access, check the cache before doing real I/O
	[3] if there's a request for memory in excess of what's available, 
		flush [some] data from the cache

Yes, step [2] has some overhead in the case of a cache miss (ie., scanning memory 
and then doing disk I/O is marginally slower than just doing the I/O), and 
similarly step [3] has some overhead...however, in most situations, those 
delays are usually orders-of-magnitude smaller than getting filesystem data 
from cache vrs. I/O. I guess that faster I/O (ie., ssd) would tip some of the 
balance away from having the system use RAM as an I/O cache.

The only situations that I've seen or heard of when the cache is detrimental 
are when:

	the cache will not be re-used (for example, when doing a full backup, there
	is filesystem data being stored into and then emptied from the cache,
	without being referenced again...there are no cache "hits" as the cached
	files aren't being used more than once before being flushed)

	there's a very large amount of RAM, such that cache lookups and flushes
	take a substantial amount of time. As RAM and CPU speeds have increased,
	the value of "very large" has increased. IME, it used to be in the low 
	GB...now I'd start to be concerned about cache overhead in the 30+ GB 
	RAM range....maybe...

	when the CPU is very slow (relative to the amount of cached data)...perhaps
	this would be the case on an embedded system...

If you've got a system that has a lot of physical memory, that has a lot of free
memory, that has a lot of filesystem data cached that will never be accessed
again, and then you run something that requests a lot of memory, I can imagine
that the presence of the system cache would slow things down. On the other hand,
I still have trouble imagining that the next step in the program that's
requested a lot of RAM (ie., a computational step or an I/O step) wouldn't be
orders of magnitude slower than the act of flushing the cache before statisfying
a malloc() request.
	
=> 
=> http://linux-mm.org/Drop_Caches
=> 
=> I understand that free ram is waisted, or under-utilized ram. But does it have to be all or noth
=> ing? So to cache, or not to cache, is the question. I would prefer some middle ground, like a tr
=> ip wire value that would be some percentage of available ram, rather the just leave like 80mb of
=>  free ram, on my now, slowed system.


80MB free? Are you running on an embedded platform?

If you run w/o cache, does performance really improve?

Mark


=> 
=> 
=> --------------------------------------------------
=> Brian Vagnoni
=> PGP Digital Fingerprint
=> F076 6EEE 06E5 BEEF EBBD  BD36 F29E 850D FC32 3955
=> --------------------------------------------------
=> ___________________________________________________________________________
=> Philadelphia Linux Users Group         --        http://www.phillylinux.org
=> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
=> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug
=> 



___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug