
# Memory Profiling

## Tool Selection

Choose the right profiler for the situation:

| Tool | Install | Best for | Limitations |
|------|---------|----------|-------------|
| **tracemalloc** | stdlib (always available) | Quick allocation tracing, snapshot diffs, per-line attribution | Python-only allocations; no C extension visibility; overhead on long runs |
| **memray** | Add as dev dep (`uv add --dev memray`, `poetry add -G dev memray`, or `pip install memray`) | Full-process profiling, C extensions, flamegraphs, leak detection | Requires install; `PYTHONMALLOC=malloc` + `--native` needed for accuracy |

**Default choice**: Use tracemalloc first — it's zero-install and gives per-line allocation stats with snapshot comparison. If you need C extension visibility, leak detection, or flamegraph visualization, install and use memray.

Both tools are effective for identifying dominant allocators. The key is to **actually profile** rather than guess from source code — transient spikes and indirect allocations are invisible without measurement.

### Per-stage snapshot profiling (primary method for pipelines)

For any code with sequential stages (load → transform → output, parse → validate → enrich, etc.), profile each stage individually with snapshot diffs. This is the most reliable way to identify dominant allocators.

**Why not just before/after with `compare_to("lineno")`?** Line-level grouping fragments a stage's allocations across its individual source lines. If a stage allocates through 5 different lines, each shows as a separate entry — and a noisier stage with fewer but larger per-line allocations can push the real culprit out of the top results. Per-stage snapshots give you one number per stage, making the dominant allocator immediately obvious.

```python
import tracemalloc

tracemalloc.start()
snap0 = tracemalloc.take_snapshot()

result_a = stage_a(input_data)
snap1 = tracemalloc.take_snapshot()

result_b = stage_b(result_a)
snap2 = tracemalloc.take_snapshot()

result_c = stage_c(result_b)
snap3 = tracemalloc.take_snapshot()

# Per-stage deltas — one number per stage, ranked by size
stages = [
    ("stage_a", snap0, snap1),
    ("stage_b", snap1, snap2),
    ("stage_c", snap2, snap3),
]
for name, before, after in stages:
    delta = sum(s.size_diff for s in after.compare_to(before, "filename"))
    print(f"{name}: {delta / 1024 / 1024:+.1f} MB")

print(f"\nPeak: {tracemalloc.get_traced_memory()[1] / 1024 / 1024:.1f} MB")
```

**Investigation order: strict delta ranking.** After profiling, sort stages by measured delta and investigate them in that order. Do NOT let source appearance re-order this list. A stage with high measured overhead but clean-looking source is the most important finding — it means there's a non-obvious allocator that source reading alone would miss. Stages that look expensive in source but measure low (e.g., triple hashing that produces only temporary objects) are red herrings.

Common "invisible" patterns that measure high but look clean in source:
- Setting many attributes per object in a loop (each attr is small, N objects makes it huge)
- Building small dicts per item (CPython dict overhead is 200+ bytes even for 2-3 keys)
- Copying references that could be shared (list/set copies per item instead of per group)

Drill into the top stages with `compare_to("lineno")` to find the specific lines:

```python
# Drill into the dominant stage
diff = snap2.compare_to(snap1, "lineno")
for stat in diff[:10]:
    print(stat)
```

### Simple before/after diff (for non-pipeline code)

For isolated functions or single code paths where there's no pipeline structure:

```python
import tracemalloc
tracemalloc.start()
snap_before = tracemalloc.take_snapshot()

# ... run the code under investigation ...

snap_after = tracemalloc.take_snapshot()
print(f"Peak: {tracemalloc.get_traced_memory()[1] / 1024 / 1024:.1f} MB")

diff = snap_after.compare_to(snap_before, "lineno")
for stat in diff[:10]:
    print(stat)
```

## Memray: Capture Flags

This section covers the non-obvious parts of memray — the things that are easy to get wrong or miss entirely. For basic CLI commands and flags, see `cli-reference.md`.

### Always Disable pymalloc and Enable Native

By default, Python uses pymalloc — a fast internal allocator that batches small objects (<512 bytes) into arenas. This makes many allocations invisible to memray, because memray only sees the large arena-level mmap calls, not the individual objects inside them.

Always disable pymalloc by setting `PYTHONMALLOC=malloc` so every allocation goes through the system allocator and is visible to memray:

```bash
PYTHONMALLOC=malloc $RUNNER -m memray run --native -o /tmp/profile.bin my_script.py
```

Why this matters:
- **`PYTHONMALLOC=malloc`**: Disables pymalloc entirely. Every allocation — no matter how small — goes through the system malloc, so memray sees it. This is simpler and more accurate than `--trace-python-allocators` (which hooks into pymalloc's internals but adds overhead and larger capture files).
- **`--native`**: Without this, allocations from C extensions (NumPy, ONNX, PIL, etc.) show up as opaque Python-level calls. You'll see "numpy.array()" but not which internal C function actually allocated.

For quick sanity checks, plain `memray run` is fine. For actual investigation, always use both `PYTHONMALLOC=malloc` and `--native`.

When comparing captures (before/after optimization), always compare like-for-like — never compare a `--native` capture against a non-native one, or a `PYTHONMALLOC=malloc` capture against one using pymalloc.

## Leak Detection Workflow

This is the area where getting the approach right matters most. There are two complementary methods — use both for thorough analysis.

### Method 1: CLI reporters with `--leaks`

The `--leaks` flag changes any reporter from showing peak-memory allocations to showing only unfreed allocations:

```bash
PYTHONMALLOC=malloc $RUNNER -m memray run --native -o /tmp/capture.bin my_service.py

# Flamegraph of leaked memory only
$RUNNER -m memray flamegraph --leaks /tmp/capture.bin -o /tmp/leaks.html

# Stats for leaked memory
$RUNNER -m memray stats --leaks /tmp/capture.bin
```

Without `--leaks`, reporters show the high watermark — what was alive at peak. With `--leaks`, they show what was still alive at exit. For a well-behaved program these are different; for a leaking one they converge.

### Method 2: Programmatic with FileReader

For deeper analysis, write a script using the leak-specific API:

```python
from memray import FileReader

reader = FileReader("capture.bin")

# get_leaked_allocation_records() — NOT get_high_watermark_allocation_records()
leaked = list(reader.get_leaked_allocation_records(merge_threads=True))
peak = list(reader.get_high_watermark_allocation_records(merge_threads=True))

leaked_total = sum(r.size for r in leaked)
peak_total = sum(r.size for r in peak)
leak_ratio = leaked_total / peak_total if peak_total else 0

print(f"Peak: {peak_total / 1024**2:.1f} MB")
print(f"Leaked: {leaked_total / 1024**2:.1f} MB")
print(f"Leak ratio: {leak_ratio:.0%}")  # >80% = almost certainly leaking

# Rank leaked allocations by size
leaked.sort(key=lambda r: r.size, reverse=True)
for record in leaked[:10]:
    size_mb = record.size / 1024**2
    stack = record.stack_trace()
    if stack:
        func, file, line = stack[0]
        print(f"  {size_mb:.1f} MB  {func} @ {file}:{line}")
```

The leak ratio (leaked / peak) is a powerful diagnostic: values near 100% confirm a true leak, while low values suggest the program freed most of its memory and the "leaks" are just caches.

### False Positive Leaks

Python's internal caches produce false leak reports: regex compilation, logging level lookups, bytecode caching, object reuse pools. Strategies:
- Always capture with `PYTHONMALLOC=malloc` — this bypasses pymalloc's arenas so leak detection sees real allocations, not arena-level noise
- Use `filter_fn` with `@pytest.mark.limit_leaks` to suppress known cache patterns
- Allow a small per-location budget rather than zero tolerance
- Run the code in a loop so one-time caches are amortized

## Per-Test vs Process-Wide Peak

This is a common source of confusion when using pytest-memray:

| Tool | What it measures | Import-time included? |
|------|-----------------|----------------------|
| `pytest --memray` | Peak within the **test function body only** | **NO** — module imports happen before Tracker starts |
| `memray run` | Peak across the **entire process lifetime** | **YES** — includes all imports and setup |

If you load a 200 MB model at import time, `pytest --memray` won't count it. Only allocations inside the test function body contribute to the budget. This means:
- Import-time optimizations are invisible to `pytest --memray`
- Fixture allocations that run before the test function are excluded
- `memray run -m pytest` gives the process-wide view including imports

When setting `@pytest.mark.limit_memory` budgets, profile first with `pytest --memray --most-allocations=0` to see per-test peaks, then add ~20% headroom to account for pymalloc arena overhead and run-to-run variance.

## Programmatic Analysis with FileReader

Beyond leak detection, FileReader is the right tool whenever you need to answer specific questions about allocations — not just look at a flamegraph. See `python-api.md` for the full API.

**Rank allocations at peak:**
```python
from memray import FileReader

reader = FileReader("profile.bin")
records = list(reader.get_high_watermark_allocation_records())
records.sort(key=lambda r: r.size, reverse=True)

total = sum(r.size for r in records)
for record in records[:10]:
    size_mb = record.size / 1024**2
    pct = record.size / total * 100
    stack = record.stack_trace()
    loc = f"{stack[0][0]} @ {stack[0][1]}:{stack[0][2]}" if stack else "unknown"
    print(f"{size_mb:8.1f} MB ({pct:4.1f}%)  {loc}")
```

**Find allocations from a specific module:**
```python
def find_by_module(reader, pattern):
    matches = []
    for record in reader.get_high_watermark_allocation_records():
        for func, file, line in record.stack_trace():
            if pattern in (file or "") or pattern in func:
                matches.append(record)
                break
    return sorted(matches, key=lambda r: r.size, reverse=True)
```

## Common Memory Traps

These are patterns that cause unexpected memory growth — they show up constantly in real profiling sessions.

### `functools.cache` on instance methods

`@functools.cache` (or `@lru_cache`) on an instance method captures `self` as a cache key. This means the instance — and everything it references — can never be garbage collected as long as the cache exists:

```python
class ModelManager:
    @functools.cache  # BUG: self is captured as cache key → instance lives forever
    def load_model(self, name):
        return load_heavy_model(name)
```

Solutions: use a module-level cache keyed on the relevant value (not self), use `@lru_cache(maxsize=N)` with explicit `cache_clear()` in `__del__`, or cache on the instance itself (`self._cache`).

### Variables retain memory until function returns

Python locals stay alive until the function returns, not when you stop using them. In a function that loads data, transforms it, and saves results, all intermediate objects coexist at peak:

```python
def process():
    raw = load_big_file()       # 500 MB — alive until return
    parsed = parse(raw)         # 500 MB — alive until return
    result = transform(parsed)  # 500 MB — all three alive simultaneously = 1.5 GB peak
    save(result)
```

Fix: break into smaller functions so each intermediate is freed when its function returns, or explicitly `del raw` / `del parsed` after use.

## Circular References & Weak References

Circular references between objects normally get collected by Python's cycle detector — unless any object in the cycle defines `__del__`. In that case, the collector can't determine a safe destruction order and moves them to `gc.garbage` instead.

**Detection:**
```python
import gc
gc.set_debug(gc.DEBUG_SAVEALL)
gc.collect()
print(f"Uncollectable: {len(gc.garbage)}")
for obj in gc.garbage[:5]:
    print(type(obj))
```

**Classic pattern — parent/child cycle:**
```python
# BUG: mutual strong references → leak if either has __del__
class Parent:
    def __init__(self):
        self.children = []
    def add_child(self, child):
        self.children.append(child)
        child.parent = self  # strong back-reference creates cycle

# FIX: break the cycle with weakref
import weakref

class Child:
    def set_parent(self, parent):
        self._parent_ref = weakref.ref(parent)  # weak — won't prevent GC

    @property
    def parent(self):
        return self._parent_ref()  # returns None if parent was collected
```

For caches where values should not prevent key collection, use `weakref.WeakValueDictionary` or `weakref.WeakKeyDictionary`.

## Common Framework Memory Leaks

- **Django `QuerySet._result_cache`** — iterating a queryset caches all rows in memory. For large tables, use `.iterator()` to stream rows without caching
- **SQLAlchemy Session accumulation** — the identity map tracks every loaded object for the session's lifetime. Call `session.close()` or `session.expire_all()` after bulk operations. Use scoped sessions with request-level teardown
- **FastAPI/Starlette background tasks** — background tasks that capture `request` or large response objects keep them alive after the response is sent. Copy only the needed values before spawning the task
- **General ORM pattern** — any ORM that caches query results by default (Django, SQLAlchemy, Peewee) will accumulate memory during batch processing. Always check for an iterator/streaming mode when processing large result sets

## Pitfalls

- **`memray tree` / `memray live`** open interactive TUIs — never use in automated contexts. Use `stats`, `summary`, or `flamegraph` instead.
- **`--aggregate` mode** cannot detect temporary allocations and doesn't work with the `stats` reporter.
- **Only one Tracker per process** — you cannot nest `memray.Tracker` context managers.
- **Native tracking portability** — reports captured with `--native` must be analyzed on the same machine (needs shared library symbols).
- **Heap ≠ RSS** — memray tracks heap allocations (malloc/free). Resident Set Size (RSS) includes memory-mapped files, shared libraries, and lazy pages. If RSS is high but memray shows low heap, the culprit is likely mmap'd files or shared memory, not Python allocations.

## Reference Files

- `cli-reference.md` — All CLI commands, flags, and reporter options
- `pytest-memray.md` — pytest markers, configuration, CI setup, gotchas
- `python-api.md` — Tracker, FileReader, programmatic analysis patterns
- `reference.md` — Per-stage profiling template, micro-benchmark template, allocation patterns
