Context Managers Deep Dive: __enter__, __exit__, and Reusable Cleanup
Master __enter__ and __exit__ protocol to write reusable database transaction managers with guaranteed cleanup, even on exceptions.
I was reviewing my own code this morning, and I noticed I've written try/finally three times this week. Set up a database connection, try to run a query, finally close it. Every time, the same pattern.
You've been writing defensively. Try/finally is safe. But Python has a pattern for exactly that.
What pattern?
Context managers. The with statement. You know it from file handling — with open('file.txt') as f:. Same idea here. Setup, action, cleanup. Guaranteed cleanup even if the action fails.
So instead of try/finally, I'd write with something as x: and it automatically handles cleanup?
Exactly. The magic is two methods: __enter__ and __exit__. When you enter a with block, Python calls __enter__. When you exit the block — success or failure — Python calls __exit__.
So I can write a database context manager, and every time I use it with with, cleanup is automatic?
That's the whole point. Let me show you the simplest possible context manager:
class SimpleContextManager:
def __enter__(self):
print("Setting up")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print("Tearing down")
return False
with SimpleContextManager() as cm:
print("Inside the block")
Output:
Setting up
Inside the block
Tearing down
__enter__ runs first. It can do setup — acquire a resource, start a transaction. It returns something (often self, often something else). That thing is what the as variable gets. __exit__ runs last, even if an exception happens in the block.
But what are exc_type, exc_val, exc_tb? Those look ominous.
They're the exception information. If the with block raises an exception, those tell you what it was. exc_type is the exception class (ValueError, KeyError). exc_val is the actual exception object. exc_tb is the traceback. If no exception happened, all three are None.
And what does return False do?
That's critical. If __exit__ returns False (or nothing, which is the same as False), the exception propagates — passes through to the caller. If you return True, you're saying "I handled this, suppress the exception." Usually you return False — let the exception bubble up. Only suppress if you truly handled it.
So if a database query fails inside the with block, __exit__ gets called with the exception information, and then the exception keeps going?
Yes. __exit__ can log it, cleanup resources, whatever. Then if it returns False, the exception propagates. The caller has to deal with it. That's the right design.
Okay, so for a database context manager, __enter__ starts a transaction, __exit__ commits or rolls back depending on whether there was an exception?
Exactly! And you write the logic once, in __exit__. Then every place you use the context manager, transactions are handled automatically.
What's the actual implementation look like?
Here's a transaction context manager:
class ManagedTransaction:
def __init__(self, db_connection):
self.connection = db_connection
self.transaction = None
def __enter__(self):
print(f"[Transaction] BEGIN")
self.transaction = self.connection.begin()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is None:
# No exception — commit
print(f"[Transaction] COMMIT")
self.transaction.commit()
else:
# Exception occurred — rollback
print(f"[Transaction] ROLLBACK (due to {exc_type.__name__})")
self.transaction.rollback()
return False # Let exception propagate
# Usage:
with ManagedTransaction(db) as txn:
db.execute("INSERT INTO orders VALUES (...)")
# If this line raises an exception, rollback happens automatically
So the logic is: if there's an exception (exc_type is not None), rollback. Otherwise commit. And return False means the exception still gets raised if there was one.
That's it. Clean separation. Setup in __enter__, teardown logic in __exit__, exception handling in the condition inside __exit__.
But there's another pattern I've seen — @contextlib.contextmanager. That's a decorator that does this with generators?
Yes. It's a shorthand for the class-based approach. Instead of writing a class with __enter__ and __exit__, you write a generator function:
from contextlib import contextmanager
@contextmanager
def managed_transaction(db_connection):
print("[Transaction] BEGIN")
transaction = db_connection.begin()
try:
yield transaction # This is what 'as' gets
except Exception as e:
print(f"[Transaction] ROLLBACK (due to {type(e).__name__})")
transaction.rollback()
raise # Re-raise the exception
else:
# This block runs if no exception
print("[Transaction] COMMIT")
transaction.commit()
# Usage is identical:
with managed_transaction(db) as txn:
db.execute("INSERT INTO orders VALUES (...)")
The magic: code before yield is __enter__. Code after yield is __exit__. The function is a generator — it pauses at yield, runs the block, then resumes to finish cleanup.
So @contextmanager is the syntax sugar. Decorator turns a generator into a context manager. Less typing, same result.
Exactly. Choose based on what you're reading. If you see a class with __enter__ and __exit__, you're looking at structured cleanup. If you see @contextmanager, you're looking at shorthand.
When would I use one versus the other?
Class-based if the context manager is complex — multiple methods, state, multiple resources. Decorator-based if it's simple — setup, action, teardown. For a database transaction, a decorator works great. For a connection pool manager, a class is cleaner.
Okay, but what if __exit__ throws its own exception during cleanup? Like, the rollback fails?
That's a real problem. If __exit__ raises an exception while trying to handle the original exception, Python chains them. You see the original exception, then the cleanup exception. Bad.
Best practice: catch exceptions in __exit__ and log them, don't let them propagate.
def __exit__(self, exc_type, exc_val, exc_tb):
try:
if exc_type is None:
self.transaction.commit()
else:
self.transaction.rollback()
except Exception as cleanup_error:
# Log the cleanup error but don't raise
print(f"Error during cleanup: {cleanup_error}")
return False
You're guaranteed to tell the original exception. Cleanup failures get logged but don't mask the real problem.
So the pattern is: setup in __enter__, teardown in __exit__, exception-safe cleanup with try/except inside __exit__?
That's the professional approach. And now you never write try/finally again for resource cleanup. You write a context manager once, use it everywhere.
What about using context managers with things that aren't database transactions? Like acquiring a lock?
Same pattern:
class LockManager:
def __init__(self, lock):
self.lock = lock
def __enter__(self):
self.lock.acquire()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.lock.release()
return False
with LockManager(my_lock):
# Critical section — lock is held
do_something_dangerous()
# Lock is released automatically, even if do_something_dangerous() fails
Setup, action, teardown. Guaranteed. That's why with is everywhere in Python.
So I can write context managers for file handles, database connections, locks, temporary directories, any resource that needs cleanup?
Any resource. And here's the beautiful part: the cleanup happens automatically. You can't forget. If you write with, cleanup is guaranteed.
One more thing — can I nest context managers? Like, multiple transactions at once?
Yes, multiple ways:
# Nested with statements
with managed_transaction(db) as txn1:
with managed_transaction(db) as txn2:
# Both active
pass
# Or comma-separated (Python 3.10+)
with managed_transaction(db) as txn1, managed_transaction(db) as txn2:
# Both active
pass
Each context manager's __exit__ is called in reverse order when exiting. txn2's __exit__ first, then txn1's. That's the right order — cleanup in reverse of setup.
Why is reverse order important?
If txn1 acquired a lock and txn2 acquired a file handle, you want to release the file handle (most recent) before the lock. Cleanup in reverse prevents deadlocks and resource contention.
Alright, so for today's code, I'm writing a ManagedOrderTransaction context manager that handles order processing with automatic commit or rollback?
Exactly. Start with the class-based version. __enter__ starts a transaction, __exit__ commits or rolls back. Make sure you handle exceptions safely. Then write the same thing with @contextmanager as a bonus.
This is going to replace a lot of my try/finally code.
That's the point. Once you understand __enter__ and __exit__, you'll see them everywhere — the standard library, third-party libraries. They're foundational.
And tomorrow is iterators, right?
Tomorrow is Iterator protocol — __iter__ and __next__. The pattern that powers for loops. You'll see how generators are iterators. Same philosophy: a protocol, a contract, and Python taking care of the plumbing.
Week 4 is clicking. Protocols, generics, type hints, context managers — it's all fitting together.
That's the goal. You've spent three weeks reading internals. This week you're composing. By Friday you'll be reading Amir's codebase and understanding every pattern.
Practice your skills
Sign up to write and run code in this lesson.
Context Managers Deep Dive: __enter__, __exit__, and Reusable Cleanup
Master __enter__ and __exit__ protocol to write reusable database transaction managers with guaranteed cleanup, even on exceptions.
I was reviewing my own code this morning, and I noticed I've written try/finally three times this week. Set up a database connection, try to run a query, finally close it. Every time, the same pattern.
You've been writing defensively. Try/finally is safe. But Python has a pattern for exactly that.
What pattern?
Context managers. The with statement. You know it from file handling — with open('file.txt') as f:. Same idea here. Setup, action, cleanup. Guaranteed cleanup even if the action fails.
So instead of try/finally, I'd write with something as x: and it automatically handles cleanup?
Exactly. The magic is two methods: __enter__ and __exit__. When you enter a with block, Python calls __enter__. When you exit the block — success or failure — Python calls __exit__.
So I can write a database context manager, and every time I use it with with, cleanup is automatic?
That's the whole point. Let me show you the simplest possible context manager:
class SimpleContextManager:
def __enter__(self):
print("Setting up")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print("Tearing down")
return False
with SimpleContextManager() as cm:
print("Inside the block")
Output:
Setting up
Inside the block
Tearing down
__enter__ runs first. It can do setup — acquire a resource, start a transaction. It returns something (often self, often something else). That thing is what the as variable gets. __exit__ runs last, even if an exception happens in the block.
But what are exc_type, exc_val, exc_tb? Those look ominous.
They're the exception information. If the with block raises an exception, those tell you what it was. exc_type is the exception class (ValueError, KeyError). exc_val is the actual exception object. exc_tb is the traceback. If no exception happened, all three are None.
And what does return False do?
That's critical. If __exit__ returns False (or nothing, which is the same as False), the exception propagates — passes through to the caller. If you return True, you're saying "I handled this, suppress the exception." Usually you return False — let the exception bubble up. Only suppress if you truly handled it.
So if a database query fails inside the with block, __exit__ gets called with the exception information, and then the exception keeps going?
Yes. __exit__ can log it, cleanup resources, whatever. Then if it returns False, the exception propagates. The caller has to deal with it. That's the right design.
Okay, so for a database context manager, __enter__ starts a transaction, __exit__ commits or rolls back depending on whether there was an exception?
Exactly! And you write the logic once, in __exit__. Then every place you use the context manager, transactions are handled automatically.
What's the actual implementation look like?
Here's a transaction context manager:
class ManagedTransaction:
def __init__(self, db_connection):
self.connection = db_connection
self.transaction = None
def __enter__(self):
print(f"[Transaction] BEGIN")
self.transaction = self.connection.begin()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is None:
# No exception — commit
print(f"[Transaction] COMMIT")
self.transaction.commit()
else:
# Exception occurred — rollback
print(f"[Transaction] ROLLBACK (due to {exc_type.__name__})")
self.transaction.rollback()
return False # Let exception propagate
# Usage:
with ManagedTransaction(db) as txn:
db.execute("INSERT INTO orders VALUES (...)")
# If this line raises an exception, rollback happens automatically
So the logic is: if there's an exception (exc_type is not None), rollback. Otherwise commit. And return False means the exception still gets raised if there was one.
That's it. Clean separation. Setup in __enter__, teardown logic in __exit__, exception handling in the condition inside __exit__.
But there's another pattern I've seen — @contextlib.contextmanager. That's a decorator that does this with generators?
Yes. It's a shorthand for the class-based approach. Instead of writing a class with __enter__ and __exit__, you write a generator function:
from contextlib import contextmanager
@contextmanager
def managed_transaction(db_connection):
print("[Transaction] BEGIN")
transaction = db_connection.begin()
try:
yield transaction # This is what 'as' gets
except Exception as e:
print(f"[Transaction] ROLLBACK (due to {type(e).__name__})")
transaction.rollback()
raise # Re-raise the exception
else:
# This block runs if no exception
print("[Transaction] COMMIT")
transaction.commit()
# Usage is identical:
with managed_transaction(db) as txn:
db.execute("INSERT INTO orders VALUES (...)")
The magic: code before yield is __enter__. Code after yield is __exit__. The function is a generator — it pauses at yield, runs the block, then resumes to finish cleanup.
So @contextmanager is the syntax sugar. Decorator turns a generator into a context manager. Less typing, same result.
Exactly. Choose based on what you're reading. If you see a class with __enter__ and __exit__, you're looking at structured cleanup. If you see @contextmanager, you're looking at shorthand.
When would I use one versus the other?
Class-based if the context manager is complex — multiple methods, state, multiple resources. Decorator-based if it's simple — setup, action, teardown. For a database transaction, a decorator works great. For a connection pool manager, a class is cleaner.
Okay, but what if __exit__ throws its own exception during cleanup? Like, the rollback fails?
That's a real problem. If __exit__ raises an exception while trying to handle the original exception, Python chains them. You see the original exception, then the cleanup exception. Bad.
Best practice: catch exceptions in __exit__ and log them, don't let them propagate.
def __exit__(self, exc_type, exc_val, exc_tb):
try:
if exc_type is None:
self.transaction.commit()
else:
self.transaction.rollback()
except Exception as cleanup_error:
# Log the cleanup error but don't raise
print(f"Error during cleanup: {cleanup_error}")
return False
You're guaranteed to tell the original exception. Cleanup failures get logged but don't mask the real problem.
So the pattern is: setup in __enter__, teardown in __exit__, exception-safe cleanup with try/except inside __exit__?
That's the professional approach. And now you never write try/finally again for resource cleanup. You write a context manager once, use it everywhere.
What about using context managers with things that aren't database transactions? Like acquiring a lock?
Same pattern:
class LockManager:
def __init__(self, lock):
self.lock = lock
def __enter__(self):
self.lock.acquire()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.lock.release()
return False
with LockManager(my_lock):
# Critical section — lock is held
do_something_dangerous()
# Lock is released automatically, even if do_something_dangerous() fails
Setup, action, teardown. Guaranteed. That's why with is everywhere in Python.
So I can write context managers for file handles, database connections, locks, temporary directories, any resource that needs cleanup?
Any resource. And here's the beautiful part: the cleanup happens automatically. You can't forget. If you write with, cleanup is guaranteed.
One more thing — can I nest context managers? Like, multiple transactions at once?
Yes, multiple ways:
# Nested with statements
with managed_transaction(db) as txn1:
with managed_transaction(db) as txn2:
# Both active
pass
# Or comma-separated (Python 3.10+)
with managed_transaction(db) as txn1, managed_transaction(db) as txn2:
# Both active
pass
Each context manager's __exit__ is called in reverse order when exiting. txn2's __exit__ first, then txn1's. That's the right order — cleanup in reverse of setup.
Why is reverse order important?
If txn1 acquired a lock and txn2 acquired a file handle, you want to release the file handle (most recent) before the lock. Cleanup in reverse prevents deadlocks and resource contention.
Alright, so for today's code, I'm writing a ManagedOrderTransaction context manager that handles order processing with automatic commit or rollback?
Exactly. Start with the class-based version. __enter__ starts a transaction, __exit__ commits or rolls back. Make sure you handle exceptions safely. Then write the same thing with @contextmanager as a bonus.
This is going to replace a lot of my try/finally code.
That's the point. Once you understand __enter__ and __exit__, you'll see them everywhere — the standard library, third-party libraries. They're foundational.
And tomorrow is iterators, right?
Tomorrow is Iterator protocol — __iter__ and __next__. The pattern that powers for loops. You'll see how generators are iterators. Same philosophy: a protocol, a contract, and Python taking care of the plumbing.
Week 4 is clicking. Protocols, generics, type hints, context managers — it's all fitting together.
That's the goal. You've spent three weeks reading internals. This week you're composing. By Friday you'll be reading Amir's codebase and understanding every pattern.