Apr
11
2007
Python Threads and the GIL
Posted by: Gustavo Felisberto in Ciência/Tecnologia, Gentoo, tags: gil, python, threadsThe Global Interpreter Lock is python’s way of solving the issues of multiple threads running on multiple cpu’s thrashing common data.
What happens is that only one thread runs at any time.
In the last week I’ve been implementing some search algorithms that would greatly benefit from multiple threads on multiple cpu’s. Some of you might say:
If you are using Unix or Cygwin you can use os.fork()
Yes I can, but I’m not even going to delve into the issues around Threads VS Fork in terms of the time it will take to spawn a thread or spawn another process.
In the work I was doing I would use threads to do very small (as in time) tasks, but spawning many threads in a small amount of time. Seems I’ll have to re-think the approach…..
Entries (RSS)
Have you ever heard about Parallel Python? It’s a Python module to do what it seems to me that you need, for Python 2.3 or later.
No, but as far as I can tell it uses some “trick” to use multiple processes (making fork portable). This is not what i need
Not really, unless your definition of “trick” is different than mine. PP overcomes GIL by using processes and IPC to schedule its threads, but that’s really no different of what would be done in GIL internally if hacking GIL to properly support multi-processors was being considered. As a matter of fact this was already discussed previously, regarding Python 3000… Let me see… Here it is, follow this thread.
OK, PP maybe isn’t the best sollution ever, but it works, and it will perform better than Python’s regular non-parallelable threads (or forking() as hell, for that matter).
I was a bit fast in my reply. I did not dig the PP internals to see how it works. If it creates a poll of processes that are just “there” waiting to receive tasks via IPC calls then it will probably work very well and I can probably use it
Ola!
Vou fazer um comment que de longe se aproxima do post que fizeste :p Mas já sabes que sou assim… LoL.
Como não consegui falar contigo no messenger, queria-te dizer que tenho um livro para te entregar já algum tempo (que emprestaste à Filipa) e queria que soubesses que faço intenções de to entregar!!!
Temos de combinar, num dia que vás ao DEI ou algo do género…
Mas pronto, ficas a saber que o livro ainda está inteiro e na minha posse…
Beijinhos.