|
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
|
Re: December 15th - RSVP - Tornado with Gavin Roy
|
On Dec 7, 2010, at 1:50 PM, paul santa clara wrote:
> Thanks, Toby && Phillip. The GIL problem similarly plagues ruby/eventmachine.
Hey Paul,
Just to be clear, and I know Toby alluded to this, while Python threads via the `threading' module are in fact hampered by the GIL[1], if you have a truly CPU-bound problem that lends itself to parallelism, Python's `multiprocessing' module is very capable and will keep all cores busy (i.e. you are literally firing off subprocesses albeit shielded behind a thread-like or other[2] high-level api provided by multiprocessing). If you are I/O-bound typically using threads (so long as the thread count won't get too high) or using some existing / rolling your own async framework is the way to go -- as others in this discussion have already alluded to.
-Tom.
[1] - It should be noted that Python threads are _real_ OS-level threads they are not "green threads." However, CPython only allows a single thread to be executing within the interpreter at once ... this is enforced by the GIL. For example, using the threading module in Jython will take advantage of the underlying JVM threading ... which may or may not be truly concurrent. Likewise using threading in IronPython will use the underlying CLR's threading capabilities ... which may or may not be truly concurrent.
[2] - Of interest to this group in particular, one of the many high-level interfaces provided by multiprocessing is Process Pools which essentially give you a "map-reduce-like" interface suitable for all kinds of concurrent data processing tasks.
|
|