|
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
Many of you will recall when Walt gave a talk, maybe two years ago, about when
NOT to use Memoize.pm. Walt tried to memoize a trivial function, and the
program's performance decreased instead of increased.
The docs for Memoize.pm also include some other scenarios when Memoize should
not be used.
I've found another one. :-) I have a long-running process, which does a
ridiculous number of database queries (on the order of 200 million). The
program is very poorly designed, and I'm in the process of rewriting it. But
as a stopgap, I figured I'd memoize some of the more-frequently called database
queries. This worked well... up to a point. Here's a snapshot of top(1) on a
machine where I had two of the processes running:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
25756 eroode 20 5 1590M 648M 11820 S N 4.5 32.1 36:59 mgq_tofile.pl
25856 eroode 20 5 1624M 605M 3552 S N 4.1 30.0 37:23 mgq_tofile.pl
For those not familiar with top's output, what this tells you is that each of
the processes was using over 1.5GB of virtual memory -- nearly all of it in one
gigantic hash, the array-context cache from Memoize.pm.
Some time later, these processes exhausted the computer's virtual memory, and
were killed by the OS.
Ah well. It was a good idea. Now I need to install Memoize::ExpireLRU, which
limits the size of Memoize's cache.
On the positive side, I proved that Perl can indeed create and use amazingly
large data structures with no difficulty....
--
Eric
___________________________________________________________________
Privacy Notice: This message has been sent via www.zoemail.net using
patented e-mail protection technology developed by AT&T Labs. Reply to
the "keyed address" above to ensure delivery.
-
**Majordomo list services provided by PANIX <URL:http://www.panix.com>**
**To Unsubscribe, send "unsubscribe phl" to majordomo@lists.pm.org**
|
|