Python Context Managers and the with Statement: Guaranteed Cleanup
Priya's connection closes at the end of her script—unless an exception fires. With the with statement, cleanup is absolutely guaranteed, no matter what happens.
Okay so I have this database connection at the top of my script, and I close it at the bottom. Works fine.
connection = get_db_connection()
try:
result = connection.execute(query)
process(result)
finally:
connection.close()
That finally block is you fighting the language. You know connection.close() needs to run, so you wrapped it in finally to make sure. But you had to know to do that. And you had to write it every single time.
So there's a better way?
The with statement. Watch:
with get_db_connection() as connection:
result = connection.execute(query)
process(result)
No try, no finally, no explicit close(). When the with block ends — even if an exception fires — connection.close() runs automatically.
How does it know to call close()? It's not in my code.
Because get_db_connection() returns something called a context manager. It's an object with two special methods: __enter__ and __exit__. When you use with, Python calls __enter__ when you step into the block, and __exit__ when you leave it. No matter how you leave.
So the object itself is responsible for cleanup?
Exactly. The object says: "I know how to set myself up and how to tear myself down. You don't have to remember." It's like a fire exit. You don't have to carry the fire extinguisher out of the building every time you leave. The exit is always there, always available.
Show me what it looks like inside.
Here's a simple context manager:
class Database:
def __init__(self, connection_string):
self.connection_string = connection_string
self.conn = None
def __enter__(self):
print(f"Opening connection to {self.connection_string}")
self.conn = really_open_connection(self.connection_string)
return self.conn # This is what 'as connection' gets
def __exit__(self, exc_type, exc_value, traceback):
print("Closing connection")
if self.conn:
self.conn.close()
Now you can write:
with Database("postgres://...") as connection:
result = connection.execute(query)
What are exc_type, exc_value, traceback in exit? Those are weird names.
They're the exception information. If an exception fires inside the with block, exit receives the details. If no exception fires, all three are None.
So exit always runs, but it gets information about whether something went wrong?
Yes. And if exit returns True, it suppresses the exception. If it returns False (or returns nothing, which is the default), the exception keeps propagating.
So the cleanup code can decide whether to hide an error?
It can. But usually it doesn't. Usually exit cleans up and lets the exception bubble up so the caller knows something went wrong.
def __exit__(self, exc_type, exc_value, traceback):
print("Closing connection")
if self.conn:
self.conn.close()
return False # Don't suppress exceptions
So the pattern is: enter sets things up and returns the resource. exit tears things down, no matter what.
That's the whole thing.
But writing a whole class just to close a connection seems like overkill.
That's why Python has contextlib.contextmanager. It's a decorator that turns a function with yield into a context manager.
from contextlib import contextmanager
@contextmanager
def database_connection(connection_string):
print(f"Opening {connection_string}")
conn = really_open_connection(connection_string)
try:
yield conn # This is what 'as connection' gets
finally:
print("Closing")
conn.close()
# Now you use it exactly like before:
with database_connection("postgres://...") as connection:
result = connection.execute(query)
Wait, that's cleaner. You just yield the resource, then put cleanup code after the yield?
And if an exception fires during the with block, the finally still runs. Yield is the boundary. Code before yield is enter. Code after yield is exit.
So @contextmanager is just syntactic sugar for writing enter and exit manually?
Yes. Pick whichever is clearer. A class when you have multiple methods and state to track. The decorator when it's simple: setup, yield, cleanup.
Can I see something more realistic? Like, something from your codebase?
Sure. Imagine you're processing orders and you want to track whether the whole operation succeeded before you commit to the database.
@contextmanager
def transaction(database):
print("Starting transaction")
database.begin()
try:
yield database
except Exception as e:
print(f"Rolling back due to {e}")
database.rollback()
raise # Re-raise the exception
else:
print("Committing")
database.commit()
Now when you process orders, the context manager handles rollback and commit for you:
def process_orders(orders, database):
with transaction(database) as db:
for order in orders:
db.insert_order(order) # If this fails, rollback happens automatically
So if one order fails, the entire transaction rolls back? And you don't have to write the rollback code every time?
Exactly. The context manager is responsible for transaction lifecycle. Your code just says what to do, and the context manager handles success and failure paths.
But what if I need to do something after the context manager closes? Like, log a message after the connection closes?
You can't easily do that inside exit. exit runs as the context manager closes, so at that point your resource is already cleaned up. If you need post-cleanup code, you'd handle it outside the with block.
with database_connection("postgres://...") as connection:
result = connection.execute(query)
print("Connection is closed now") # This runs after the context manager exits
Okay so with statement: setup, use, cleanup. All automatic. And contextmanager decorator is the easiest way to write simple ones.
For most cases, yes. Use the class when you need enter and exit to be separate methods — like when you have other methods on the context manager object itself. Use @contextmanager for the simple yield-based pattern.
When would I actually need to write context managers? Like, in what situations?
Anywhere you have setup and teardown. Database connections — set up connection, execute queries, close connection. File handles — Python's open() is a context manager, which is why you write with open() as f:. Locks in multithreaded code — acquire lock, use resource, release lock. Database transactions — begin, execute, commit or rollback.
So with open() is using the context manager protocol?
Yes. File objects implement enter and exit. enter makes sure the file is open. exit closes it. That's why people say "always use with when you open files" — the context manager guarantees closure.
And if you don't use with, you have to remember to close manually?
Or the file stays open until Python garbage collects the object. Which might be soon, might be much later. With is the guarantee. No ambiguity.
Can you nest context managers? Like, open two files at once?
Yes:
with open("input.txt") as input_file:
with open("output.txt", "w") as output_file:
output_file.write(input_file.read())
Or more concisely:
with open("input.txt") as input_file, open("output.txt", "w") as output_file:
output_file.write(input_file.read())
Both input_file and output_file go out of scope at the end, and both close automatically.
That's clean. Both resources get cleaned up in the right order.
And if an exception fires, both still close. Try doing that with manual close() calls without a try/finally for each one.
Alright so here's my question: when I see with in Amir's code, what's actually happening?
Amir is using an object that implements enter and exit. When execution reaches the with statement, Python calls enter. When execution leaves the with block for any reason — normal exit, exception, break, return — exit runs.
So with is not language-level magic. It's a protocol. Objects opt into the protocol by implementing enter and exit.
Correct. And you can write context managers for anything that needs setup and teardown. Want a context manager that times a block of code? Done.
import time
from contextlib import contextmanager
@contextmanager
def timer(label):
start = time.time()
try:
yield
finally:
elapsed = time.time() - start
print(f"{label}: {elapsed:.2f}s")
with timer("database query"):
perform_expensive_query()
So you're saying I could write a context manager that logs when I enter and exit a function? For debugging?
Absolutely. Context managers are tools for any kind of setup and cleanup. Files, connections, transactions, timing, logging, state management. Anywhere you have a "before this block" and "after this block" responsibility.
Okay I get it. But here's where I'm confused. Yesterday we learned about exception handlers — try/except/finally. Today we're learning with statements. Are they solving the same problem?
Different problems. Try/except handles what to do when an error occurs. With handles setup and teardown. They work together. With usually has a try/finally under the hood, but you don't write it.
So the context manager is the try/finally, and you're just not seeing it.
Yes. The context manager author writes the try/finally (or the enter/exit protocol, which Python translates to try/finally). You just write with and trust that cleanup happens.
What's the order? Does exit run before or after my except block?
exit is part of the with protocol. If an exception fires inside the with block, exit runs first. Then the exception propagates outside the with. If your code has an except outside the with, that catches the propagated exception.
try:
with database_connection() as conn:
conn.execute(bad_query) # Raises ValueError
except ValueError:
print("Query was invalid")
Order: execute bad_query → ValueError raised → exit runs (cleanup) → ValueError caught by except.
So exit runs before my except?
exit is called automatically. Your except is outside the with block, so it catches whatever exception exit doesn't suppress.
So if I'm using with statements for setup and cleanup, I don't need to write finally blocks myself?
Correct. The context manager handles it. That's the whole point. You describe what you want to do (use a database, open a file, lock a resource), and the context manager handles the guaranteed cleanup.
Tomorrow we're learning what?
Exception hierarchies and chains. What happens when one exception triggers another, and how to keep track of the whole story.
So tomorrow it gets even more complex?
Yes. Today: "This thing needs cleanup, guaranteed." Tomorrow: "What if cleanup fails? What if five things fail in sequence and you need to know which one caused which?" Today you're guaranteeing exit. Tomorrow we make sure that exit knows what went wrong.
Practice your skills
Sign up to write and run code in this lesson.
Python Context Managers and the with Statement: Guaranteed Cleanup
Priya's connection closes at the end of her script—unless an exception fires. With the with statement, cleanup is absolutely guaranteed, no matter what happens.
Okay so I have this database connection at the top of my script, and I close it at the bottom. Works fine.
connection = get_db_connection()
try:
result = connection.execute(query)
process(result)
finally:
connection.close()
That finally block is you fighting the language. You know connection.close() needs to run, so you wrapped it in finally to make sure. But you had to know to do that. And you had to write it every single time.
So there's a better way?
The with statement. Watch:
with get_db_connection() as connection:
result = connection.execute(query)
process(result)
No try, no finally, no explicit close(). When the with block ends — even if an exception fires — connection.close() runs automatically.
How does it know to call close()? It's not in my code.
Because get_db_connection() returns something called a context manager. It's an object with two special methods: __enter__ and __exit__. When you use with, Python calls __enter__ when you step into the block, and __exit__ when you leave it. No matter how you leave.
So the object itself is responsible for cleanup?
Exactly. The object says: "I know how to set myself up and how to tear myself down. You don't have to remember." It's like a fire exit. You don't have to carry the fire extinguisher out of the building every time you leave. The exit is always there, always available.
Show me what it looks like inside.
Here's a simple context manager:
class Database:
def __init__(self, connection_string):
self.connection_string = connection_string
self.conn = None
def __enter__(self):
print(f"Opening connection to {self.connection_string}")
self.conn = really_open_connection(self.connection_string)
return self.conn # This is what 'as connection' gets
def __exit__(self, exc_type, exc_value, traceback):
print("Closing connection")
if self.conn:
self.conn.close()
Now you can write:
with Database("postgres://...") as connection:
result = connection.execute(query)
What are exc_type, exc_value, traceback in exit? Those are weird names.
They're the exception information. If an exception fires inside the with block, exit receives the details. If no exception fires, all three are None.
So exit always runs, but it gets information about whether something went wrong?
Yes. And if exit returns True, it suppresses the exception. If it returns False (or returns nothing, which is the default), the exception keeps propagating.
So the cleanup code can decide whether to hide an error?
It can. But usually it doesn't. Usually exit cleans up and lets the exception bubble up so the caller knows something went wrong.
def __exit__(self, exc_type, exc_value, traceback):
print("Closing connection")
if self.conn:
self.conn.close()
return False # Don't suppress exceptions
So the pattern is: enter sets things up and returns the resource. exit tears things down, no matter what.
That's the whole thing.
But writing a whole class just to close a connection seems like overkill.
That's why Python has contextlib.contextmanager. It's a decorator that turns a function with yield into a context manager.
from contextlib import contextmanager
@contextmanager
def database_connection(connection_string):
print(f"Opening {connection_string}")
conn = really_open_connection(connection_string)
try:
yield conn # This is what 'as connection' gets
finally:
print("Closing")
conn.close()
# Now you use it exactly like before:
with database_connection("postgres://...") as connection:
result = connection.execute(query)
Wait, that's cleaner. You just yield the resource, then put cleanup code after the yield?
And if an exception fires during the with block, the finally still runs. Yield is the boundary. Code before yield is enter. Code after yield is exit.
So @contextmanager is just syntactic sugar for writing enter and exit manually?
Yes. Pick whichever is clearer. A class when you have multiple methods and state to track. The decorator when it's simple: setup, yield, cleanup.
Can I see something more realistic? Like, something from your codebase?
Sure. Imagine you're processing orders and you want to track whether the whole operation succeeded before you commit to the database.
@contextmanager
def transaction(database):
print("Starting transaction")
database.begin()
try:
yield database
except Exception as e:
print(f"Rolling back due to {e}")
database.rollback()
raise # Re-raise the exception
else:
print("Committing")
database.commit()
Now when you process orders, the context manager handles rollback and commit for you:
def process_orders(orders, database):
with transaction(database) as db:
for order in orders:
db.insert_order(order) # If this fails, rollback happens automatically
So if one order fails, the entire transaction rolls back? And you don't have to write the rollback code every time?
Exactly. The context manager is responsible for transaction lifecycle. Your code just says what to do, and the context manager handles success and failure paths.
But what if I need to do something after the context manager closes? Like, log a message after the connection closes?
You can't easily do that inside exit. exit runs as the context manager closes, so at that point your resource is already cleaned up. If you need post-cleanup code, you'd handle it outside the with block.
with database_connection("postgres://...") as connection:
result = connection.execute(query)
print("Connection is closed now") # This runs after the context manager exits
Okay so with statement: setup, use, cleanup. All automatic. And contextmanager decorator is the easiest way to write simple ones.
For most cases, yes. Use the class when you need enter and exit to be separate methods — like when you have other methods on the context manager object itself. Use @contextmanager for the simple yield-based pattern.
When would I actually need to write context managers? Like, in what situations?
Anywhere you have setup and teardown. Database connections — set up connection, execute queries, close connection. File handles — Python's open() is a context manager, which is why you write with open() as f:. Locks in multithreaded code — acquire lock, use resource, release lock. Database transactions — begin, execute, commit or rollback.
So with open() is using the context manager protocol?
Yes. File objects implement enter and exit. enter makes sure the file is open. exit closes it. That's why people say "always use with when you open files" — the context manager guarantees closure.
And if you don't use with, you have to remember to close manually?
Or the file stays open until Python garbage collects the object. Which might be soon, might be much later. With is the guarantee. No ambiguity.
Can you nest context managers? Like, open two files at once?
Yes:
with open("input.txt") as input_file:
with open("output.txt", "w") as output_file:
output_file.write(input_file.read())
Or more concisely:
with open("input.txt") as input_file, open("output.txt", "w") as output_file:
output_file.write(input_file.read())
Both input_file and output_file go out of scope at the end, and both close automatically.
That's clean. Both resources get cleaned up in the right order.
And if an exception fires, both still close. Try doing that with manual close() calls without a try/finally for each one.
Alright so here's my question: when I see with in Amir's code, what's actually happening?
Amir is using an object that implements enter and exit. When execution reaches the with statement, Python calls enter. When execution leaves the with block for any reason — normal exit, exception, break, return — exit runs.
So with is not language-level magic. It's a protocol. Objects opt into the protocol by implementing enter and exit.
Correct. And you can write context managers for anything that needs setup and teardown. Want a context manager that times a block of code? Done.
import time
from contextlib import contextmanager
@contextmanager
def timer(label):
start = time.time()
try:
yield
finally:
elapsed = time.time() - start
print(f"{label}: {elapsed:.2f}s")
with timer("database query"):
perform_expensive_query()
So you're saying I could write a context manager that logs when I enter and exit a function? For debugging?
Absolutely. Context managers are tools for any kind of setup and cleanup. Files, connections, transactions, timing, logging, state management. Anywhere you have a "before this block" and "after this block" responsibility.
Okay I get it. But here's where I'm confused. Yesterday we learned about exception handlers — try/except/finally. Today we're learning with statements. Are they solving the same problem?
Different problems. Try/except handles what to do when an error occurs. With handles setup and teardown. They work together. With usually has a try/finally under the hood, but you don't write it.
So the context manager is the try/finally, and you're just not seeing it.
Yes. The context manager author writes the try/finally (or the enter/exit protocol, which Python translates to try/finally). You just write with and trust that cleanup happens.
What's the order? Does exit run before or after my except block?
exit is part of the with protocol. If an exception fires inside the with block, exit runs first. Then the exception propagates outside the with. If your code has an except outside the with, that catches the propagated exception.
try:
with database_connection() as conn:
conn.execute(bad_query) # Raises ValueError
except ValueError:
print("Query was invalid")
Order: execute bad_query → ValueError raised → exit runs (cleanup) → ValueError caught by except.
So exit runs before my except?
exit is called automatically. Your except is outside the with block, so it catches whatever exception exit doesn't suppress.
So if I'm using with statements for setup and cleanup, I don't need to write finally blocks myself?
Correct. The context manager handles it. That's the whole point. You describe what you want to do (use a database, open a file, lock a resource), and the context manager handles the guaranteed cleanup.
Tomorrow we're learning what?
Exception hierarchies and chains. What happens when one exception triggers another, and how to keep track of the whole story.
So tomorrow it gets even more complex?
Yes. Today: "This thing needs cleanup, guaranteed." Tomorrow: "What if cleanup fails? What if five things fail in sequence and you need to know which one caused which?" Today you're guaranteeing exit. Tomorrow we make sure that exit knows what went wrong.