# Async Optimization for Python

This guide covers the non-obvious parts of async Python performance — the things that are easy to get wrong or miss entirely. For asyncio debug mode details, see `asyncio-debug-mode.md`. For concurrency patterns, see `concurrency-patterns.md`. For blocking function catalogs and framework-specific guidance (FastAPI, Django, aiohttp), see `blocking-detection.md`. For code quality and correctness patterns (cancellation safety, resource lifecycle, structured concurrency), see `code-quality.md`.

## The Core Problem: Event Loop Blocking

The asyncio event loop runs in a single thread. When any coroutine blocks (sync I/O, CPU work, `time.sleep`), *every other coroutine* stops making progress. This is the root cause of most async performance issues.

**Symptoms**:
- Endpoints that should take 100ms take 2-5s under load
- Websocket connections dropping during heavy processing
- asyncio debug mode shows "Executing ... took X.XXX seconds"
- Throughput doesn't scale with concurrency — stays flat or drops

**Diagnosis**: Enable asyncio debug mode (see `asyncio-debug-mode.md`) and look for slow callback warnings. Then use the static analyzer to find the blocking calls.

## Sequential Awaits: The Most Common Performance Bug

When independent I/O operations are awaited one after another, total time is the sum of all waits:

```python
# BEFORE: 3 sequential awaits — total time = t1 + t2 + t3
async def get_dashboard(user_id):
    profile = await fetch_profile(user_id)      # 200ms
    orders = await fetch_orders(user_id)         # 300ms
    recommendations = await fetch_recs(user_id)  # 250ms
    return build_dashboard(profile, orders, recommendations)
    # Total: ~750ms

# AFTER: concurrent awaits — total time = max(t1, t2, t3)
async def get_dashboard(user_id):
    profile, orders, recommendations = await asyncio.gather(
        fetch_profile(user_id),
        fetch_orders(user_id),
        fetch_recs(user_id),
    )
    return build_dashboard(profile, orders, recommendations)
    # Total: ~300ms (limited by slowest call)
```

### When NOT to gather

- **Data dependency**: If `fetch_orders` needs the result of `fetch_profile`, they must be sequential.
- **Shared mutable state**: If coroutines modify the same object, concurrent execution causes races.
- **Rate-limited APIs**: Gathering 100 calls at once may trigger rate limits. Use semaphore-bounded concurrency.
- **Connection pool exhaustion**: Gathering more tasks than the pool size causes queueing. Bound with semaphore.

## Await in Loop: The Second Most Common Bug

```python
# BEFORE: sequential — N round trips, total = N * latency
async def process_items(items):
    results = []
    for item in items:
        result = await process_one(item)  # 100ms each
        results.append(result)
    return results  # 100 items = 10 seconds

# AFTER: concurrent — 1 round of N operations
async def process_items(items):
    return await asyncio.gather(*(process_one(item) for item in items))
    # 100 items = ~100ms (limited by slowest)

# BETTER: bounded concurrency for large N
async def process_items(items, max_concurrent=20):
    semaphore = asyncio.Semaphore(max_concurrent)
    async def bounded(item):
        async with semaphore:
            return await process_one(item)
    return await asyncio.gather(*(bounded(item) for item in items))
```

### Python 3.11+ — use TaskGroup instead of gather

```python
async def process_items(items, max_concurrent=20):
    semaphore = asyncio.Semaphore(max_concurrent)
    results = {}

    async with asyncio.TaskGroup() as tg:
        for i, item in enumerate(items):
            async def bounded(idx=i, it=item):
                async with semaphore:
                    results[idx] = await process_one(it)
            tg.create_task(bounded())

    return [results[i] for i in range(len(items))]
```

TaskGroup advantages over gather:
- **Structured concurrency**: if one task fails, all others are cancelled
- **Better error reporting**: exceptions include the full traceback of the failing task
- **No silent failures**: unlike `gather(return_exceptions=True)` which swallows errors into the result list

## Blocking Calls in Async Functions

These functions block the event loop and must be replaced or wrapped:

| Blocking call | Async replacement |
|---------------|-------------------|
| `time.sleep(n)` | `await asyncio.sleep(n)` |
| `requests.get/post/...` | `httpx.AsyncClient` or `aiohttp.ClientSession` |
| `open()` / file read/write | `aiofiles.open()` or `await loop.run_in_executor(None, ...)` |
| `subprocess.run()` | `await asyncio.create_subprocess_exec()` |
| `os.system()` | `await asyncio.create_subprocess_shell()` |
| `socket.*` (sync) | `await asyncio.open_connection()` / `asyncio.start_server()` |
| `sqlite3.connect()` | `aiosqlite.connect()` |
| CPU-heavy computation | `await asyncio.to_thread(func)` (3.9+) or `loop.run_in_executor(executor, func)` |

### run_in_executor vs asyncio.to_thread

```python
# run_in_executor (all Python versions)
loop = asyncio.get_running_loop()
result = await loop.run_in_executor(None, blocking_function, arg1, arg2)

# asyncio.to_thread (3.9+) — simpler syntax, supports kwargs
result = await asyncio.to_thread(blocking_function, arg1, arg2, key=value)
```

Both run the function in a ThreadPoolExecutor, releasing the event loop. For CPU-bound work, use `ProcessPoolExecutor`:

```python
from concurrent.futures import ProcessPoolExecutor

executor = ProcessPoolExecutor(max_workers=4)
result = await loop.run_in_executor(executor, cpu_heavy_function, data)
```

## @cache / @lru_cache on Async Functions

This is a correctness bug, not just a performance issue:

```python
@functools.cache
async def fetch_user(user_id):
    return await db.get_user(user_id)

# First call: works — creates and awaits coroutine, caches the result
# Second call: returns the cached coroutine object (already consumed)
# await on it raises RuntimeError or returns None
```

Solutions:
```python
# Manual async memoization
_user_cache = {}

async def fetch_user(user_id):
    if user_id not in _user_cache:
        _user_cache[user_id] = await db.get_user(user_id)
    return _user_cache[user_id]

# Or with TTL and bounded size — use aiocache or cachetools with async wrapper
```

## Unbounded Concurrency

Gathering thousands of coroutines at once causes:
- Connection pool exhaustion (max connections reached, new requests queue)
- Memory spikes (all coroutine frames alive simultaneously)
- Rate limit violations on external APIs
- OS file descriptor limits (each connection = 1 fd)

```python
# BAD: 10K concurrent HTTP requests
results = await asyncio.gather(*(fetch(url) for url in urls))  # 10K urls

# GOOD: bounded to 50 concurrent
async def bounded_fetch(url, semaphore):
    async with semaphore:
        return await fetch(url)

semaphore = asyncio.Semaphore(50)
results = await asyncio.gather(*(bounded_fetch(url, semaphore) for url in urls))
```

### Choosing the right concurrency limit

| Resource | Typical limit | How to find |
|----------|--------------|-------------|
| HTTP connection pool | 100 (httpx default) | Client constructor docs |
| Database pool | 5-20 (depends on DB) | DB driver config |
| File descriptors | 1024 (default ulimit) | `ulimit -n` |
| External API rate limit | Varies | API docs |

Set semaphore to `min(pool_size, rate_limit)` to avoid both exhaustion and throttling.

## Event Loop Starvation

When a coroutine runs CPU-bound code for extended periods without yielding, the event loop can't process other tasks:

```python
# BAD: 500ms of CPU work blocks the event loop
async def process_data(data):
    result = heavy_computation(data)  # 500ms, no await
    return result

# GOOD: offload to thread pool
async def process_data(data):
    result = await asyncio.to_thread(heavy_computation, data)
    return result
```

**Detection**: asyncio debug mode logs "Executing <Task ...> took X.XXX seconds" for any callback exceeding `slow_callback_duration` (default 100ms).

## Connection Management

### HTTP client sessions

```python
# BAD: new connection per request
async def fetch(url):
    async with httpx.AsyncClient() as client:  # TCP+TLS handshake each time
        return await client.get(url)

# GOOD: reuse session across requests
class Service:
    def __init__(self):
        self.client = httpx.AsyncClient(
            timeout=30,
            limits=httpx.Limits(max_connections=100, max_keepalive_connections=20),
        )

    async def fetch(self, url):
        return await self.client.get(url)

    async def close(self):
        await self.client.aclose()
```

### Database connections

Same principle — create pool once, reuse across requests. Most async DB drivers (asyncpg, aiomysql, aiosqlite) support connection pooling natively.

## Framework-Specific Patterns

For detailed framework-specific async guidance (FastAPI sync-vs-async endpoints, Django `sync_to_async` wrapping, aiohttp session lifecycle, connection pooling), see `blocking-detection.md`.

## Pitfalls

- **`asyncio.run()` inside async function**: Raises "cannot be called from a running event loop". Use `await` directly.
- **Blocking in `__init__`**: `__init__` can't be async. Do async setup in a factory method or `async def start()`.
- **CancelledError handling**: On Python 3.9+, `CancelledError` inherits from `BaseException`, not `Exception`. `except Exception:` won't catch it (good). On 3.8, it inherits from `Exception` — be careful with bare `except Exception:` in async code.
- **gather with return_exceptions=True**: Errors become result values instead of raising. Easy to miss failures. Prefer TaskGroup (3.11+) or explicit error checking.
- **Forgetting to close sessions/pools**: Use `async with` or explicit cleanup in shutdown hooks.

## uvloop

Drop-in replacement event loop for asyncio, typically 2-4x faster than the default loop. Written in Cython, wraps libuv.

```bash
pip install uvloop
```

```python
# Option 1: install globally before asyncio.run()
import uvloop
uvloop.install()
asyncio.run(main())

# Option 2: pass as loop_factory (3.12+)
asyncio.run(main(), loop_factory=uvloop.new_event_loop)
```

**Caveats**: CPython only (no PyPy), not compatible with Windows, doesn't support all asyncio features (e.g., subprocess on some platforms). Always benchmark first — gains vary by workload. Biggest wins for I/O-heavy servers with many concurrent connections.

## Streaming & Server-Sent Events

Pattern: async generator yielding chunks for streaming responses. The generator suspends at each `yield` until the consumer reads, providing implicit backpressure.

```python
async def stream_results(query):
    async for batch in run_query_batches(query):
        yield json.dumps(batch) + "\n"
        # Implicit backpressure: yield suspends until consumer reads

# FastAPI
@app.get("/stream")
async def stream_endpoint():
    return StreamingResponse(stream_results("SELECT ..."), media_type="text/event-stream")
```

**Backpressure concerns**:
- If the consumer is slow and the producer doesn't respect backpressure, data accumulates in memory. For raw `StreamWriter`, always pair `write()` with `await drain()`.
- WebSocket pattern: use a bounded send queue. Detect slow consumers by checking queue depth and disconnect if they fall too far behind.

```python
# WebSocket with bounded queue — disconnect slow consumers
async def ws_handler(ws):
    queue = asyncio.Queue(maxsize=64)
    async def sender():
        while True:
            msg = await queue.get()
            await ws.send(msg)
    sender_task = asyncio.create_task(sender())
    try:
        async for event in event_stream():
            try:
                queue.put_nowait(event)
            except asyncio.QueueFull:
                break  # Consumer too slow — disconnect
    finally:
        sender_task.cancel()
```

## Connection Pool Exhaustion

**Symptom**: requests hang or timeout after running fine initially. The pool is full and new requests wait for a connection that never becomes available.

**Diagnosis**: check pool stats — `httpx`: inspect `client._transport._pool`, `asyncpg`: `pool.get_size()` / `pool.get_idle_size()`.

**Common causes**:
- Not closing responses or sessions (`async with` not used)
- Semaphore limit much higher than pool size — allows more concurrent tasks than available connections
- Long-running queries holding connections while other tasks wait

**Fix**: match semaphore limit to pool capacity, use `async with` for automatic cleanup, set pool `max_size` explicitly.

```python
# WRONG: semaphore allows 100 concurrent, but pool only has 10 connections
sem = asyncio.Semaphore(100)
pool = asyncpg.create_pool(max_size=10)

# RIGHT: match concurrency to pool capacity
sem = asyncio.Semaphore(10)
pool = asyncpg.create_pool(max_size=10)
```

## Reference Files

- `asyncio-debug-mode.md` — Enabling debug mode, interpreting warnings, slow callback detection
- `concurrency-patterns.md` — gather, TaskGroup, Semaphore, Queue, wait patterns
- `blocking-detection.md` — Complete blocking function catalog, wrapping strategies, framework-specific guidance
- `code-quality.md` — Async code quality patterns: cancellation safety, resource lifecycle, structured concurrency, separating I/O from logic
