Introduction

Python asyncio Task Cancelled errors occur when a coroutine is cancelled during execution, raising asyncio.CancelledError which, if not handled properly, causes unhandled exceptions, incomplete cleanup, and application instability. Task cancellation is a fundamental asyncio mechanism for stopping coroutines gracefully, but mishandling leads to resources not being released, connections left open, and data corruption. Common causes include explicit task.cancel() calls without proper exception handling, context manager __aexit__ not awaiting cleanup, timeout (asyncio.wait_for, asyncio.timeout) cancelling inner coroutine, application shutdown cancelling all pending tasks, parent task cancellation propagating to children, task group exiting cancelling remaining tasks, event loop closure cancelling pending tasks, and CancelledError being caught and suppressed incorrectly (preventing proper cancellation propagation). The fix requires understanding asyncio cancellation semantics, implementing graceful cancellation handlers, ensuring cleanup in finally blocks, and properly re-raising CancelledError after cleanup. This guide provides production-proven patterns for asyncio task cancellation across Python 3.7+ with async/await syntax, context managers, task groups, and application shutdown handlers.

Symptoms

  • asyncio.exceptions.CancelledError unhandled exception
  • Task logs show "Task was destroyed but it is pending!"
  • Application shutdown hangs with pending tasks
  • Resources (connections, files) not released after cancellation
  • CancelledError caught but not re-raised, task appears stuck
  • Timeout causes entire operation to fail instead of partial result
  • Child tasks not cancelled when parent cancelled
  • Task group exits prematurely cancelling all tasks
  • Signal handler cancels tasks but cleanup incomplete
  • Background tasks disappear silently after cancellation

Common Causes

  • Task.cancel() called without awaiting task completion
  • CancelledError caught and swallowed (not re-raised)
  • Cleanup code in except block instead of finally
  • Async context manager __aexit__ not handling cancellation
  • asyncio.wait_for timeout cancelling long-running operation
  • Application shutdown (Ctrl+C) cancelling all tasks
  • Task group (asyncio.TaskGroup) exiting on exception
  • Parent task cancelled, children not shielded
  • Event loop closed with pending tasks
  • Cancellation during critical section (non-atomic operation)

Step-by-Step Fix

### 1. Understand asyncio cancellation semantics

CancelledError behavior:

```python import asyncio

# CancelledError is a BaseException, not Exception # It should propagate to signal task completion

async def simple_task(): try: await asyncio.sleep(10) except asyncio.CancelledError: print("Task was cancelled") # WRONG: Just logging and returning # This suppresses cancellation - task appears to complete normally return

# CORRECT: Always re-raise after cleanup # raise

async def correct_cancellation_handling(): try: await asyncio.sleep(10) except asyncio.CancelledError: print("Task was cancelled, cleaning up...") # Do cleanup (but keep it fast!) await cleanup_resources() # CRITICAL: Re-raise to indicate cancellation complete raise

# Cancellation is a request, not a command # Task must cooperate by: # 1. Catching CancelledError # 2. Doing necessary cleanup # 3. Re-raising to confirm cancellation

async def main(): task = asyncio.create_task(correct_cancellation_handling())

await asyncio.sleep(0.1) # Let task start

# Request cancellation task.cancel()

# Await task completion (cancellation) try: await task except asyncio.CancelledError: print("Task cancellation confirmed")

asyncio.run(main()) ```

Task cancellation states:

```python async def check_task_state(): task = asyncio.create_task(asyncio.sleep(10))

# Check cancellation status print(f"Task cancelled: {task.cancelled()}") # False initially print(f"Task done: {task.done()}") # False initially print(f"Task result: {task.result()}") # Raises InvalidStateError if not done

task.cancel()

# After cancel() but before await print(f"Task cancelled: {task.cancelled()}") # Still False (not processed yet) print(f"Task done: {task.done()}") # False

try: await task except asyncio.CancelledError: pass

# After awaiting cancellation print(f"Task cancelled: {task.cancelled()}") # True print(f"Task done: {task.done()}") # True

asyncio.run(check_task_state()) ```

### 2. Fix cancellation in async functions

Basic cancellation handling:

```python async def worker(name: str, delay: float): """Worker that handles cancellation gracefully.""" print(f"Worker {name} starting")

try: iteration = 0 while True: print(f"Worker {name} iteration {iteration}") await asyncio.sleep(delay) iteration += 1

except asyncio.CancelledError: # Cancellation requested - do cleanup print(f"Worker {name} cancelled, cleaning up...")

# Do async cleanup (with timeout to prevent hanging) try: await asyncio.wait_for(cleanup_worker(name), timeout=5.0) except asyncio.TimeoutError: print(f"Worker {name} cleanup timed out")

# Re-raise to confirm cancellation raise

finally: # Finally always runs (even after CancelledError re-raised) print(f"Worker {name} stopped")

async def cleanup_worker(name: str): """Clean up worker resources.""" await asyncio.sleep(0.1) # Simulate cleanup print(f"Worker {name} cleanup complete")

async def main(): # Start worker task = asyncio.create_task(worker("main-worker", 1.0))

# Let it run for a bit await asyncio.sleep(2.5)

# Cancel and wait task.cancel()

try: await task except asyncio.CancelledError: print("Worker cancellation confirmed")

asyncio.run(main())

# Output: # Worker main-worker starting # Worker main-worker iteration 0 # Worker main-worker iteration 1 # Worker main-worker iteration 2 # Worker main-worker cancelled, cleaning up... # Worker main-worker cleanup complete # Worker main-worker stopped # Worker cancellation confirmed ```

Cancellation with state management:

```python class CancellableWorker: def __init__(self, name: str): self.name = name self.state = "idle" self._cancelled = False self._task = None

async def run(self): """Main worker loop with cancellation handling.""" self._cancelled = False self.state = "running"

try: while not self._cancelled: self.state = "working" await self.do_work()

self.state = "waiting" await asyncio.sleep(1.0)

except asyncio.CancelledError: self.state = "cancelling" print(f"Worker {self.name} received cancellation")

# Clean up with timeout await asyncio.wait_for(self.cleanup(), timeout=5.0)

self.state = "cancelled" raise # Re-raise after cleanup

finally: self.state = "stopped" print(f"Worker {self.name} stopped (state: {self.state})")

async def do_work(self): """Do actual work - also handles cancellation.""" try: await asyncio.sleep(0.5) except asyncio.CancelledError: # Work was cancelled, let it propagate raise

async def cleanup(self): """Clean up resources.""" print(f"Worker {self.name} cleaning up") await asyncio.sleep(0.1)

def start(self): """Start the worker task.""" self._task = asyncio.create_task(self.run()) return self._task

async def stop(self): """Request graceful stop.""" self._cancelled = True if self._task: try: await self._task except asyncio.CancelledError: pass

async def main(): worker = CancellableWorker("worker-1") task = worker.start()

await asyncio.sleep(2.5)

# Graceful stop await worker.stop()

print(f"Final state: {worker.state}")

asyncio.run(main()) ```

### 3. Fix timeout-induced cancellation

asyncio.wait_for handling:

```python async def slow_operation(): """Operation that might timeout.""" await asyncio.sleep(10) return "result"

async def operation_with_timeout(): """Handle timeout cancellation properly.""" try: result = await asyncio.wait_for(slow_operation(), timeout=5.0) return result

except asyncio.TimeoutError: # Timeout wraps CancelledError as TimeoutError print("Operation timed out")

# The inner coroutine was cancelled # Do any necessary cleanup here await cleanup_after_timeout()

# Decide: return default, raise custom error, or re-raise raise CustomTimeoutError("Operation took too long")

except asyncio.CancelledError: # This is different - the outer task was cancelled print("Outer task cancelled (not timeout)") raise

async def cleanup_after_timeout(): await asyncio.sleep(0.1)

class CustomTimeoutError(Exception): pass

# Nested timeouts - be careful! async def nested_timeouts(): try: # Outer timeout: 10 seconds async with asyncio.timeout(10): try: # Inner timeout: 5 seconds result = await asyncio.wait_for(slow_operation(), timeout=5.0) except asyncio.TimeoutError: print("Inner timeout") raise except asyncio.TimeoutError: print("Outer timeout") raise

asyncio.run(operation_with_timeout()) ```

Python 3.11+ asyncio.timeout:

```python # Python 3.11+ has new timeout context manager async def operation_with_context_timeout(): try: async with asyncio.timeout(5.0) as timeout: await slow_operation() # timeout.remaining() shows time left print(f"Time remaining: {timeout.remaining()}")

except TimeoutError: print("Operation timed out") raise

# Custom timeout with cleanup async def timeout_with_cleanup(coro, timeout_seconds: float): """Run coroutine with timeout and guaranteed cleanup.""" try: async with asyncio.timeout(timeout_seconds): return await coro except TimeoutError: # Do cleanup on timeout await cleanup_after_timeout() raise except asyncio.CancelledError: # Do cleanup on cancellation (different from timeout) await cleanup_after_cancel() raise ```

### 4. Fix task group cancellation

asyncio.TaskGroup handling (Python 3.11+):

```python # Python 3.11+ TaskGroup async def task_group_example(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(worker("worker-1", 1.0)) task2 = tg.create_task(worker("worker-2", 1.0))

# If any task raises exception (not CancelledError), # all other tasks are cancelled

# TaskGroup waits for all tasks on exit

async def task_group_with_cancellation(): try: async with asyncio.TaskGroup() as tg: tg.create_task(worker("worker-1", 1.0)) tg.create_task(worker("worker-2", 1.0))

# Simulate error in main code await asyncio.sleep(0.5) raise ValueError("Something went wrong")

except ExceptionGroup as eg: # TaskGroup raises ExceptionGroup containing all exceptions print(f"Task group failed: {eg}")

# All tasks were cancelled due to the exception # But their CancelledError is wrapped in ExceptionGroup

# Shielding tasks from group cancellation async def shielded_in_group(): async with asyncio.TaskGroup() as outer: # Normal task - will be cancelled on group exit normal_task = outer.create_task(worker("normal", 1.0))

# Shielded task - won't be cancelled by group async with asyncio.TaskGroup() as inner: shielded = inner.create_task( asyncio.shield(worker("shielded", 1.0)) )

asyncio.run(task_group_example()) ```

Pre-3.11 task group pattern:

```python async def create_task_group(): """TaskGroup-like pattern for Python 3.7-3.10.""" tasks = []

try: # Create tasks tasks.append(asyncio.create_task(worker("worker-1", 1.0))) tasks.append(asyncio.create_task(worker("worker-2", 1.0)))

# Wait for all await asyncio.gather(*tasks)

except Exception: # If any task fails, cancel all others for task in tasks: if not task.done(): task.cancel()

# Wait for cancellation to complete await asyncio.gather(*tasks, return_exceptions=True) raise

async def main(): await create_task_group()

asyncio.run(main()) ```

### 5. Fix application shutdown handling

Graceful shutdown with signal handlers:

```python import signal

class Application: def __init__(self): self.tasks = [] self._shutdown = False

async def run(self): """Main application loop.""" # Setup signal handlers loop = asyncio.get_running_loop()

for sig in (signal.SIGINT, signal.SIGTERM): loop.add_signal_handler( sig, lambda: asyncio.create_task(self.shutdown()) )

# Start background tasks self.tasks.append(asyncio.create_task(self.background_worker())) self.tasks.append(asyncio.create_task(self.periodic_task()))

# Main application logic try: while not self._shutdown: await asyncio.sleep(1.0) # Do main work except asyncio.CancelledError: pass finally: await self.cleanup()

async def background_worker(self): """Background task with cancellation handling.""" try: while not self._shutdown: await asyncio.sleep(0.5) # Do background work except asyncio.CancelledError: # Cleanup on cancellation await self.worker_cleanup() raise

async def periodic_task(self): """Periodic task with cancellation handling.""" try: while not self._shutdown: await asyncio.sleep(5.0) # Do periodic work except asyncio.CancelledError: await self.periodic_cleanup() raise

async def shutdown(self): """Handle shutdown signal.""" print("Shutdown requested") self._shutdown = True

# Cancel all tasks for task in self.tasks: if not task.done(): task.cancel()

# Wait for tasks to complete (with timeout) if self.tasks: results = await asyncio.gather(*self.tasks, return_exceptions=True)

for i, result in enumerate(results): if isinstance(result, Exception) and not isinstance(result, asyncio.CancelledError): print(f"Task {i} failed: {result}")

self.tasks.clear()

async def cleanup(self): """Final cleanup.""" print("Application cleanup") await asyncio.sleep(0.1)

async def worker_cleanup(self): print("Worker cleanup")

async def periodic_cleanup(self): print("Periodic task cleanup")

async def main(): app = Application() await app.run()

asyncio.run(main()) ```

### 6. Use asyncio.shield correctly

Shielding tasks from cancellation:

```python async def critical_operation(): """Operation that must complete even if cancelled.""" print("Starting critical operation") await asyncio.sleep(2.0) print("Critical operation complete") return "result"

async def caller(): """Caller that might be cancelled.""" try: # Shield the operation from cancellation result = await asyncio.shield(critical_operation()) return result

except asyncio.CancelledError: # Caller was cancelled, but critical_operation continues print("Caller cancelled, but critical operation still running") # Note: shield doesn't prevent the operation from completing # It just prevents the cancellation from propagating raise

# Shield with timeout async def shield_with_timeout(): try: result = await asyncio.wait_for( asyncio.shield(critical_operation()), timeout=5.0 ) return result except asyncio.TimeoutError: # Timeout still works with shield print("Shielded operation timed out") raise

# Shield pattern for cleanup async def operation_with_guaranteed_cleanup(): """Ensure cleanup runs even if operation is cancelled.""" result = None cleanup_needed = False

try: result = await do_operation() cleanup_needed = True return result

except asyncio.CancelledError: if cleanup_needed: # Shield cleanup from cancellation await asyncio.shield(do_cleanup(result)) raise

finally: # Finally always runs print("Operation finished (or was cancelled)")

async def do_operation(): await asyncio.sleep(1.0) return "data"

async def do_cleanup(data): print(f"Cleaning up: {data}") await asyncio.sleep(0.1)

asyncio.run(caller()) ```

### 7. Debug cancellation issues

Enable cancellation debugging:

```python import asyncio import logging

# Enable asyncio debug logging logging.basicConfig(level=logging.DEBUG) asyncio.get_event_loop().set_debug(True)

async def debug_cancellation(): """Run with PYTHONASYNCIODEBUG=1 for detailed logs."""

task = asyncio.create_task(worker("debug", 1.0))

await asyncio.sleep(0.5) task.cancel()

try: await task except asyncio.CancelledError: pass

# Track task lifecycle class TrackedTask: def __init__(self, name: str): self.name = name self.task = None self._cancelled = False

async def run(self): print(f"[{self.name}] Task started")

try: while True: print(f"[{self.name}] Working...") await asyncio.sleep(1.0)

except asyncio.CancelledError: print(f"[{self.name}] CancelledError caught") await asyncio.sleep(0.1) # Cleanup print(f"[{self.name}] Cleanup complete, re-raising") raise

finally: print(f"[{self.name}] Finally block executed")

def start(self): self.task = asyncio.create_task(self.run()) return self.task

async def cancel_and_wait(self): if self.task: self.task.cancel() try: await self.task except asyncio.CancelledError: print(f"[{self.name}] Cancellation confirmed")

async def main(): tracked = TrackedTask("test") tracked.start()

await asyncio.sleep(2.5) await tracked.cancel_and_wait()

asyncio.run(main())

# Check for pending tasks at exit async def check_pending_tasks(): tasks = asyncio.all_tasks() if tasks: print(f"Pending tasks: {len(tasks)}") for task in tasks: print(f" - {task}")

asyncio.run(check_pending_tasks()) ```

Prevention

  • Always re-raise CancelledError after cleanup
  • Put cleanup code in finally blocks, not just except
  • Use asyncio.shield for operations that must complete
  • Set timeouts on cleanup operations to prevent hanging
  • Implement graceful shutdown with signal handlers
  • Use TaskGroup (Python 3.11+) for structured concurrency
  • Test cancellation paths in unit tests
  • Log cancellation events for debugging
  • Document cancellation behavior in function docstrings
  • Avoid catching CancelledError without re-raising
  • **Asyncio timeout error**: Operation exceeded time limit
  • **Asyncio incomplete read**: Stream closed prematurely
  • **Asyncio connection reset**: Network connection lost
  • **Asyncio gather exception**: One of gathered tasks failed
  • **RuntimeError event loop closed**: Operations after loop closed