Synchronization: locks and the bounded buffer
Protect shared data with a lock and coordinate a producer and consumer with a blocking queue.
import queue
import threading
counter = 0
lock = threading.Lock()
def work():
global counter
for _ in range(10000):
with lock:
counter += 1
workers = [threading.Thread(target=work) for _ in range(4)]
for worker in workers:
worker.start()
for worker in workers:
worker.join()
buffer = queue.Queue(maxsize=2)
results = []
def producer():
for item in range(1, 6):
buffer.put(item)
buffer.put(None)
def consumer():
while True:
item = buffer.get()
if item is None:
break
results.append(item * item)
threads = [threading.Thread(target=producer), threading.Thread(target=consumer)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
print(results)
print(counter)[1, 4, 9, 16, 25] 40000
Explanation
When threads share data, the result of an operation can depend on how their steps interleave. That is a race condition. The code that touches shared data is a critical section, and mutual exclusion means only one thread at a time may be inside it. A lock (mutex) enforces this: a thread acquires the lock before the critical section and releases it afterwards.
The statement counter += 1 looks like one step but is really read, add, and write. Two threads can both read the same old value and one update is lost. Python's global interpreter lock does not make such compound operations safe, so a Lock is still required. The with statement releases the lock even if an exception occurs. The first part of the program increments a shared counter from four threads under a lock, and the total is always exactly 40000.
The second part is the bounded-buffer producer–consumer problem. queue.Queue is thread-safe and its put blocks when the queue is full, while get blocks when it is empty. The producer therefore cannot overrun the consumer, and the consumer waits for data. A None value signals that no more items are coming. Acquiring locks in inconsistent orders can cause deadlock, so keep critical sections small and use a consistent lock order.
Explore the dry run
import queueimport threading counter = 0lock = threading.Lock() def work(): global counter for _ in range(10000): with lock: counter += 1 workers = [threading.Thread(target=work) for _ in range(4)]for worker in workers: worker.start()for worker in workers: worker.join() buffer = queue.Queue(maxsize=2)results = [] def producer(): for item in range(1, 6): buffer.put(item) buffer.put(None) def consumer(): while True: item = buffer.get() if item is None: break results.append(item * item) threads = [threading.Thread(target=producer), threading.Thread(target=consumer)]for thread in threads: thread.start()for thread in threads: thread.join()print(results)print(counter)Follow the example
Start to inspect the source, values, and output. These prepared steps explain the published sample; grouped loop steps describe multiple iterations.
View the complete dry-run table
| Step / line | What happens | State after step | Output so far |
|---|---|---|---|
| Only one thread at a time may hold the lock, so counter += 1 never interleaves. | 4 threads × 10000 increments | — | |
| Wait for all four workers to finish before reading the counter. | 4 threads × 10000 increments | — | |
| put blocks while the 2-slot buffer is full, so the producer cannot run ahead of the consumer. | 4 threads × 10000 increments | — | |
| The single consumer takes items in FIFO order and squares them. | 4 threads × 10000 increments | — | |
| The lock guarantees no lost updates: exactly 40000. | counter = 40000 | [1, 4, 9, 16, 25] 40000 |
Cost and common mistakes
The counter does 40000 locked updates, O(n) total, with contention making each one slower than an unlocked update.
- Without the lock, counter += 1 from several threads can lose updates and give an unpredictable total. The GIL does not prevent this.
- Holding a lock while doing slow work, such as I/O, makes other threads wait. Lock only the critical section.
- Two threads that acquire two locks in opposite orders can deadlock. Always acquire multiple locks in one agreed order.
Try it yourself
What would happen if the consumer never received the None value?
Reveal answer and reasoning
After the five items were consumed, get() would block forever on the empty queue and the program would hang. The sentinel tells the consumer to stop.