Multiprocessing with Python | Linux Today

Multiprocessing with Python

Written By
Web Webster
Web Webster
Mar 26, 2009

[ Thanks to An Anonymous Reader for
this link. ]

“In a previous article for IBM® developerWorks, I
demonstrated a simple and effective pattern for implementing
threaded programming in Python. One downside of this approach,
though, is that it won’t always speed up your application, because
the GIL (global interpreter lock) effectively limits threads to one
core. If you need to use all of the cores on your machine, then
typically you will need to fork processes, to increase speed.
Dealing with a flock of processes can be a challenge, because if
communication between processes is needed, it can often get
complicated to coordinate all of the calls.

“Fortunately, as of version 2.6, Python includes a module called
“multiprocessing” to help you deal with processes. The API of the
processing module has some similarities to the way the threading
API works, but there are also few differences to keep in mind. One
of the main differences is that processes have subtle underlying
behavior that a high-level API will never be able to completely
abstract away. You can read more about this in the official
documentation for the multiprocessing module (see the Resources
section).”


Complete Story

Web Webster

Web Webster

Web Webster has more than 20 years of writing and editorial experience in the tech sector. He’s written and edited news, demand generation, user-focused, and thought leadership content for business software solutions, consumer tech, and Linux Today, he edits and writes for a portfolio of tech industry news and analysis websites including webopedia.com, and DatabaseJournal.com.

Linux Today Logo

LinuxToday is a trusted, contributor-driven news resource supporting all types of Linux users. Our thriving international community engages with us through social media and frequent content contributions aimed at solving problems ranging from personal computing to enterprise-level IT operations. LinuxToday serves as a home for a community that struggles to find comparable information elsewhere on the web.

Property of TechnologyAdvice. © 2026 TechnologyAdvice. All Rights Reserved

Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.