Parallelism with processes for CPU-bound work

CPU-bound work, such as number crunching, is limited by processor speed rather than waiting. To run it in parallel you need multiple cores doing useful work at once. In the default CPython build the global interpreter lock lets only one thread execute Python bytecode at a time, so threads cannot speed this work up. The multiprocessing module sidesteps the GIL by starting separate worker processes, each with its own interpreter and memory.

from multiprocessing import Pool


def count_primes(limit):
    return sum(1 for n in range(2, limit) if all(n % d for d in range(2, int(n ** 0.5) + 1)))


if __name__ == "__main__":
    limits = [20000, 20000, 20000, 20000]
    with Pool(2) as pool:
        results = pool.map(count_primes, limits)
    print(results)
    print(results == [count_primes(limit) for limit in limits])