concurrent.futures: The Unified Interface for Threads and Processes
You've learned three ways to do concurrent work. Your manager asked which one to use. Here's the answer: a single interface, two implementations, and a decision tree.
I just realized something. We've spent this whole week looking at threading, multiprocessing, and async. Each one works. Each one has a different reason for existing. But when I'm writing actual code, when I sit down to process orders or fetch data, how do I decide which one to use? My manager asked for a status update on the pipeline refactor by end of day, and I need to answer this question right now.
That question is exactly why concurrent.futures exists. It is not a new concurrency model. It is a unified interface that abstracts both threading and multiprocessing behind a single API. You write the code once. You choose the executor at import time. Everything else flows from that one decision.
So it is like... the API that lets you swap the implementation?
Exactly. concurrent.futures has two main executors: ThreadPoolExecutor and ProcessPoolExecutor. They share identical APIs. Your code does not change. Only the executor changes. ThreadPoolExecutor spawns threads that share memory. ProcessPoolExecutor spawns child processes that do not. Everything else is the same from your perspective.
Okay, so I pick ThreadPoolExecutor when I have I/O-bound work, and ProcessPoolExecutor when I have CPU-bound work. That is the decision tree I learned on Days 10 and 11.
That is the decision tree. But concurrent.futures adds one more layer on top of it. Instead of manually creating threads with threading.Thread, or managing a multiprocessing.Pool, you create an executor and submit work to it. The executor manages the pool for you. You do not write thread or process management code. You write submission code.
That sounds simpler than everything we've done so far.
It is. It is intentionally simple. The tradeoff is: you lose some control. With threading.Thread, you can do anything. With ThreadPoolExecutor, you submit a callable and get back a Future. You wait for the Future to complete. That is the contract.
A Future. That is a new pattern this week.
A Future is a promise. You submit work to the executor. The executor returns immediately with a Future — a handle to work that may or may not be complete yet. You can ask the Future if the work is done, wait for it to finish, or ask it for the result. The Future bridges the gap between "I submitted work" and "the work is done and I have the answer."
So if I submit a task that takes 10 seconds, and I immediately ask the Future for its result, what happens?
If the work is not done yet, future.result() blocks. It waits until the work finishes, then returns the result. If an exception was raised inside the task, future.result() re-raises it to you. The Future is your window into what happened in the executor.
That sounds like join() on a thread — you wait for it to finish.
Similar idea, cleaner API. Here is the basic pattern:
from concurrent.futures import ThreadPoolExecutor
import time
def fetch_order(order_id):
"""Simulate an external API call that takes time."""
time.sleep(2) # I/O-bound: network latency
return f'Order {order_id} fetched'
with ThreadPoolExecutor(max_workers=3) as executor:
# Submit three tasks in parallel
future1 = executor.submit(fetch_order, 'ORD-001')
future2 = executor.submit(fetch_order, 'ORD-002')
future3 = executor.submit(fetch_order, 'ORD-003')
# Wait for results (each waited in parallel — total ~2 seconds, not 6)
print(future1.result())
print(future2.result())
print(future3.result())
Three tasks, each taking 2 seconds. Because they run in parallel on three threads, the total time is about 2 seconds, not 6. The ThreadPoolExecutor manages the pool. The with statement cleans it up automatically.
Okay. And if I have a lot of tasks — not just three — I would not do submit() one at a time and then result() one at a time. That defeats the parallelism.
Right. If you have 100 tasks, you would submit all 100, then iterate through the results. But there is a more elegant pattern: as_completed().
from concurrent.futures import ThreadPoolExecutor, as_completed
import time
def fetch_order(order_id):
time.sleep(2)
return f'Order {order_id} fetched'
order_ids = ['ORD-001', 'ORD-002', 'ORD-003']
with ThreadPoolExecutor(max_workers=3) as executor:
# Submit all tasks at once — returns a dict of {order_id: future}
futures = {executor.submit(fetch_order, oid): oid for oid in order_ids}
# as_completed yields futures as they finish — not in submission order
for future in as_completed(futures):
order_id = futures[future]
result = future.result()
print(f'{order_id}: {result}')
as_completed() is the key. You submit all 100 tasks immediately. as_completed() yields them back to you as they finish — whichever finishes first comes out first. You process results as they arrive, not in submission order. That is the pattern for bulk I/O operations: submit everything, then consume results as they become available.
So it is like... you dump all your orders into the pool, and as soon as each one finishes fetching, you grab the result. You do not sit and wait for them in order.
That is exactly right. If task 2 finishes before task 1, you get task 2's result first. That is perfect for API calls where some endpoints are faster than others, or some orders are simpler to fetch than others. Your code does not block waiting for the slowest task in the queue.
And if one of the tasks raises an exception? If fetch_order raises an error?
future.result() re-raises the exception. If you want to handle it:
for future in as_completed(futures):
order_id = futures[future]
try:
result = future.result()
print(f'{order_id}: {result}')
except Exception as e:
print(f'{order_id}: failed with {e}')
The exception does not crash the whole loop. Each task's exception is isolated. You handle it or skip it, then move to the next task.
What about .map() — I feel like there was a .map() method on pools?
Both multiprocessing.Pool and concurrent.futures have .map(). It submits a bunch of tasks and returns results in submission order — like Python's built-in map(). The difference between .map() and as_completed():
.map()— returns results in the order you submitted them. You get a list or iterator that comes out in order, even if task 5 finishes before task 1.as_completed()— returns results as they finish. You process them out of order. More efficient if you want to consume results immediately.
.map() is simpler if you care about order. as_completed() is better if you want to act on results as soon as they are ready. For order processing, as_completed() is usually better — you want to store results to the database as soon as each one finishes, not wait for them to come back in order.
And the choice between ThreadPoolExecutor and ProcessPoolExecutor?
You already know this. ThreadPoolExecutor for I/O-bound work — network calls, file reads, database queries. ProcessPoolExecutor for CPU-bound work — calculations, transformations, anything that uses significant CPU time. The GIL is still there. If you use ThreadPoolExecutor for CPU-bound work, you will not get parallelism. You will just have thread overhead.
So if I have 100 orders to fetch from an external API, I use ThreadPoolExecutor with max_workers=10. Each worker fetches one order. Fetching involves network I/O, which releases the GIL. Parallelism works.
Exactly. If you have 100 orders to process with some heavy calculation — say, training a small ML model on each order's data — you use ProcessPoolExecutor. Each worker gets a process. CPU time is actual parallelism, not threads fighting over the GIL.
And max_workers? How do I choose that?
For ThreadPoolExecutor with I/O: usually the number of concurrent I/O operations you want. If your network bandwidth supports 10 concurrent requests, use 10 workers. If the external API rate-limits you to 5, use 5. Do not use 100 workers on a single API endpoint — you will just pile up requests.
For ProcessPoolExecutor with CPU: usually the number of cores on your machine. If you have 4 cores, 4 workers gives you full parallelism. Adding more workers does not help — they just contend for CPU time.
And what if I have a mix? Some tasks are I/O, some are CPU?
That is when you need to think harder. The general rule: pick the bottleneck. If your code spends 80% of its time waiting for network and 20% computing, use ThreadPoolExecutor. The I/O is your constraint. If it is the opposite — 80% CPU, 20% waiting — use ProcessPoolExecutor.
Or: use async. async and ThreadPoolExecutor solve similar problems differently. async avoids thread overhead entirely by cooperatively yielding control. ThreadPoolExecutor uses real threads and lets the OS scheduler manage them. For very high concurrency (1000+ concurrent tasks), async is often better. For simpler cases — 10 concurrent API calls — ThreadPoolExecutor is less mental overhead.
So the decision tree is: Is it I/O-bound or CPU-bound? If I/O-bound and low concurrency, ThreadPoolExecutor. If I/O-bound and very high concurrency, consider async. If CPU-bound, ProcessPoolExecutor.
That is the exact decision tree. And because both executors share the same API, you can write the code with ThreadPoolExecutor, then switch to ProcessPoolExecutor if you measure and find that I/O is not actually your bottleneck. That is why the interface exists.
Let me walk through the exercise. I have a batch of orders. Each order needs to be fetched from an external API. That is I/O-bound. I use ThreadPoolExecutor. I submit all orders to the executor, then iterate through as_completed() to process results as they arrive. Each result is stored to the database immediately.
That is the exact pattern. One more thing: what if you want to cancel a task that is still queued or running?
Can you do that?
future.cancel() tries to cancel a task. If the task has not started yet, it cancels successfully. If the task is already running, it does not interrupt — Python does not have preemptive cancellation. But you can check future.cancelled() and skip processing results from tasks that were cancelled.
So cancellation only works on tasks that have not started yet.
Right. And if you need to interrupt running tasks, that is when you look at async with asyncio.CancelledError, or you structure your task function to check a shutdown flag.
Okay. ThreadPoolExecutor for I/O, ProcessPoolExecutor for CPU. Submit with .submit() or .map(). Consume results with as_completed() or in order with .map(). Handle exceptions when you call .result(). That is the whole interface.
That is the whole interface. concurrent.futures is a thin wrapper around threading and multiprocessing that gives you a unified submission-and-result model. You lose some low-level control — you cannot tune individual thread stacks or process startup behavior — but you gain simplicity. For 95% of use cases, that is the right tradeoff.
And next week we start with type safety. My concurrent code is going to be fast — now it needs to also be type-safe.
Your concurrent code is going to be fast and type-safe. And then we will discover that the type checker has opinions about Futures, Executors, and callbacks that you did not expect. But that is next week. For now — you have chosen your tool. You know when to use threads, processes, async, and now, the unified interface that makes choosing between threads and processes simple.
Day 14. Week 2 is almost done. I can now look at any concurrent problem and ask the right question: Is this I/O or CPU? How much concurrency do I need? And then pick the tool.
Practice your skills
Sign up to write and run code in this lesson.
concurrent.futures: The Unified Interface for Threads and Processes
You've learned three ways to do concurrent work. Your manager asked which one to use. Here's the answer: a single interface, two implementations, and a decision tree.
I just realized something. We've spent this whole week looking at threading, multiprocessing, and async. Each one works. Each one has a different reason for existing. But when I'm writing actual code, when I sit down to process orders or fetch data, how do I decide which one to use? My manager asked for a status update on the pipeline refactor by end of day, and I need to answer this question right now.
That question is exactly why concurrent.futures exists. It is not a new concurrency model. It is a unified interface that abstracts both threading and multiprocessing behind a single API. You write the code once. You choose the executor at import time. Everything else flows from that one decision.
So it is like... the API that lets you swap the implementation?
Exactly. concurrent.futures has two main executors: ThreadPoolExecutor and ProcessPoolExecutor. They share identical APIs. Your code does not change. Only the executor changes. ThreadPoolExecutor spawns threads that share memory. ProcessPoolExecutor spawns child processes that do not. Everything else is the same from your perspective.
Okay, so I pick ThreadPoolExecutor when I have I/O-bound work, and ProcessPoolExecutor when I have CPU-bound work. That is the decision tree I learned on Days 10 and 11.
That is the decision tree. But concurrent.futures adds one more layer on top of it. Instead of manually creating threads with threading.Thread, or managing a multiprocessing.Pool, you create an executor and submit work to it. The executor manages the pool for you. You do not write thread or process management code. You write submission code.
That sounds simpler than everything we've done so far.
It is. It is intentionally simple. The tradeoff is: you lose some control. With threading.Thread, you can do anything. With ThreadPoolExecutor, you submit a callable and get back a Future. You wait for the Future to complete. That is the contract.
A Future. That is a new pattern this week.
A Future is a promise. You submit work to the executor. The executor returns immediately with a Future — a handle to work that may or may not be complete yet. You can ask the Future if the work is done, wait for it to finish, or ask it for the result. The Future bridges the gap between "I submitted work" and "the work is done and I have the answer."
So if I submit a task that takes 10 seconds, and I immediately ask the Future for its result, what happens?
If the work is not done yet, future.result() blocks. It waits until the work finishes, then returns the result. If an exception was raised inside the task, future.result() re-raises it to you. The Future is your window into what happened in the executor.
That sounds like join() on a thread — you wait for it to finish.
Similar idea, cleaner API. Here is the basic pattern:
from concurrent.futures import ThreadPoolExecutor
import time
def fetch_order(order_id):
"""Simulate an external API call that takes time."""
time.sleep(2) # I/O-bound: network latency
return f'Order {order_id} fetched'
with ThreadPoolExecutor(max_workers=3) as executor:
# Submit three tasks in parallel
future1 = executor.submit(fetch_order, 'ORD-001')
future2 = executor.submit(fetch_order, 'ORD-002')
future3 = executor.submit(fetch_order, 'ORD-003')
# Wait for results (each waited in parallel — total ~2 seconds, not 6)
print(future1.result())
print(future2.result())
print(future3.result())
Three tasks, each taking 2 seconds. Because they run in parallel on three threads, the total time is about 2 seconds, not 6. The ThreadPoolExecutor manages the pool. The with statement cleans it up automatically.
Okay. And if I have a lot of tasks — not just three — I would not do submit() one at a time and then result() one at a time. That defeats the parallelism.
Right. If you have 100 tasks, you would submit all 100, then iterate through the results. But there is a more elegant pattern: as_completed().
from concurrent.futures import ThreadPoolExecutor, as_completed
import time
def fetch_order(order_id):
time.sleep(2)
return f'Order {order_id} fetched'
order_ids = ['ORD-001', 'ORD-002', 'ORD-003']
with ThreadPoolExecutor(max_workers=3) as executor:
# Submit all tasks at once — returns a dict of {order_id: future}
futures = {executor.submit(fetch_order, oid): oid for oid in order_ids}
# as_completed yields futures as they finish — not in submission order
for future in as_completed(futures):
order_id = futures[future]
result = future.result()
print(f'{order_id}: {result}')
as_completed() is the key. You submit all 100 tasks immediately. as_completed() yields them back to you as they finish — whichever finishes first comes out first. You process results as they arrive, not in submission order. That is the pattern for bulk I/O operations: submit everything, then consume results as they become available.
So it is like... you dump all your orders into the pool, and as soon as each one finishes fetching, you grab the result. You do not sit and wait for them in order.
That is exactly right. If task 2 finishes before task 1, you get task 2's result first. That is perfect for API calls where some endpoints are faster than others, or some orders are simpler to fetch than others. Your code does not block waiting for the slowest task in the queue.
And if one of the tasks raises an exception? If fetch_order raises an error?
future.result() re-raises the exception. If you want to handle it:
for future in as_completed(futures):
order_id = futures[future]
try:
result = future.result()
print(f'{order_id}: {result}')
except Exception as e:
print(f'{order_id}: failed with {e}')
The exception does not crash the whole loop. Each task's exception is isolated. You handle it or skip it, then move to the next task.
What about .map() — I feel like there was a .map() method on pools?
Both multiprocessing.Pool and concurrent.futures have .map(). It submits a bunch of tasks and returns results in submission order — like Python's built-in map(). The difference between .map() and as_completed():
.map()— returns results in the order you submitted them. You get a list or iterator that comes out in order, even if task 5 finishes before task 1.as_completed()— returns results as they finish. You process them out of order. More efficient if you want to consume results immediately.
.map() is simpler if you care about order. as_completed() is better if you want to act on results as soon as they are ready. For order processing, as_completed() is usually better — you want to store results to the database as soon as each one finishes, not wait for them to come back in order.
And the choice between ThreadPoolExecutor and ProcessPoolExecutor?
You already know this. ThreadPoolExecutor for I/O-bound work — network calls, file reads, database queries. ProcessPoolExecutor for CPU-bound work — calculations, transformations, anything that uses significant CPU time. The GIL is still there. If you use ThreadPoolExecutor for CPU-bound work, you will not get parallelism. You will just have thread overhead.
So if I have 100 orders to fetch from an external API, I use ThreadPoolExecutor with max_workers=10. Each worker fetches one order. Fetching involves network I/O, which releases the GIL. Parallelism works.
Exactly. If you have 100 orders to process with some heavy calculation — say, training a small ML model on each order's data — you use ProcessPoolExecutor. Each worker gets a process. CPU time is actual parallelism, not threads fighting over the GIL.
And max_workers? How do I choose that?
For ThreadPoolExecutor with I/O: usually the number of concurrent I/O operations you want. If your network bandwidth supports 10 concurrent requests, use 10 workers. If the external API rate-limits you to 5, use 5. Do not use 100 workers on a single API endpoint — you will just pile up requests.
For ProcessPoolExecutor with CPU: usually the number of cores on your machine. If you have 4 cores, 4 workers gives you full parallelism. Adding more workers does not help — they just contend for CPU time.
And what if I have a mix? Some tasks are I/O, some are CPU?
That is when you need to think harder. The general rule: pick the bottleneck. If your code spends 80% of its time waiting for network and 20% computing, use ThreadPoolExecutor. The I/O is your constraint. If it is the opposite — 80% CPU, 20% waiting — use ProcessPoolExecutor.
Or: use async. async and ThreadPoolExecutor solve similar problems differently. async avoids thread overhead entirely by cooperatively yielding control. ThreadPoolExecutor uses real threads and lets the OS scheduler manage them. For very high concurrency (1000+ concurrent tasks), async is often better. For simpler cases — 10 concurrent API calls — ThreadPoolExecutor is less mental overhead.
So the decision tree is: Is it I/O-bound or CPU-bound? If I/O-bound and low concurrency, ThreadPoolExecutor. If I/O-bound and very high concurrency, consider async. If CPU-bound, ProcessPoolExecutor.
That is the exact decision tree. And because both executors share the same API, you can write the code with ThreadPoolExecutor, then switch to ProcessPoolExecutor if you measure and find that I/O is not actually your bottleneck. That is why the interface exists.
Let me walk through the exercise. I have a batch of orders. Each order needs to be fetched from an external API. That is I/O-bound. I use ThreadPoolExecutor. I submit all orders to the executor, then iterate through as_completed() to process results as they arrive. Each result is stored to the database immediately.
That is the exact pattern. One more thing: what if you want to cancel a task that is still queued or running?
Can you do that?
future.cancel() tries to cancel a task. If the task has not started yet, it cancels successfully. If the task is already running, it does not interrupt — Python does not have preemptive cancellation. But you can check future.cancelled() and skip processing results from tasks that were cancelled.
So cancellation only works on tasks that have not started yet.
Right. And if you need to interrupt running tasks, that is when you look at async with asyncio.CancelledError, or you structure your task function to check a shutdown flag.
Okay. ThreadPoolExecutor for I/O, ProcessPoolExecutor for CPU. Submit with .submit() or .map(). Consume results with as_completed() or in order with .map(). Handle exceptions when you call .result(). That is the whole interface.
That is the whole interface. concurrent.futures is a thin wrapper around threading and multiprocessing that gives you a unified submission-and-result model. You lose some low-level control — you cannot tune individual thread stacks or process startup behavior — but you gain simplicity. For 95% of use cases, that is the right tradeoff.
And next week we start with type safety. My concurrent code is going to be fast — now it needs to also be type-safe.
Your concurrent code is going to be fast and type-safe. And then we will discover that the type checker has opinions about Futures, Executors, and callbacks that you did not expect. But that is next week. For now — you have chosen your tool. You know when to use threads, processes, async, and now, the unified interface that makes choosing between threads and processes simple.
Day 14. Week 2 is almost done. I can now look at any concurrent problem and ask the right question: Is this I/O or CPU? How much concurrency do I need? And then pick the tool.