# Data Structure Selection for Python

This guide covers the non-obvious parts of choosing the right data structure in Python — the tradeoffs that matter in practice, not just in theory. For complete stdlib container docs, see `stdlib-containers.md`.

## Version Awareness

CPython's performance characteristics have changed significantly across versions. When giving advice, always consider the project's Python version:

| Version | Key changes affecting optimization advice |
|---------|------------------------------------------|
| **3.10 and below** | All classic optimizations apply at full strength |
| **3.11** | PEP 659: Specializing adaptive interpreter. Caches attribute lookups, method calls, binary ops. Reduces gains from local variable hoisting and `operator.itemgetter`. |
| **3.12** | PEP 709: Comprehension inlining. List/set/dict comprehensions no longer create a separate frame. `map(c_func)` vs comprehension gap is now negligible. |
| **3.13** | Experimental JIT (PEP 744). Tier-2 micro-ops optimizer may further reduce micro-optimization value. Not yet default. |

**Rule of thumb**: Algorithmic optimizations (O(n^2) -> O(n), set vs list, deque vs list) and data-layout optimizations (`__slots__`, `weakref`) are **always relevant** regardless of version. Bytecode-level micro-optimizations (hoisting, operator.itemgetter, map vs comp) have **diminishing returns on 3.11+**.

## The Decision Framework

Before changing any data structure, answer three questions:

1. **What operations dominate?** (lookup, insert, delete, iterate, sort)
2. **What's the data size?** (10 items = doesn't matter; 10K+ = big-O matters)
3. **Is this on the hot path?** (confirm with cProfile before optimizing)

If you can't answer #3 with profiler data, profile first. Never optimize cold code.

## Container Selection Guide

### When to use what

| Need | Use | Not | Why |
|------|-----|-----|-----|
| FIFO queue (append + pop front) | `collections.deque` | `list` | list.pop(0) is O(n), deque.popleft() is O(1). Algorithmic — always relevant. |
| Membership testing | `set` / `frozenset` | `list` | list `in` is O(n), set `in` is O(1) average. Crossover at ~4-8 elements. |
| Key-value lookup | `dict` | list of tuples | dict lookup is O(1), linear scan is O(n) |
| Grouped collection | `defaultdict(list)` | manual setdefault | Cleaner, marginally faster |
| Counting elements | `Counter` | manual dict increment | Purpose-built, supports arithmetic, most_common() |
| Sorted insertion | `bisect.insort` | append + sort | insort is O(n) for insertion, sort is O(n log n) |
| Top-k elements | `heapq.nsmallest/nlargest` | sorted()[:k] | O(n log k) vs O(n log n) |
| Priority queue | `heapq` | sorted list | heappush/heappop are O(log n) |
| Fixed-type numeric array | `array.array` | `list` | 3-4x less memory, faster for numeric ops |
| Immutable record | `tuple` / `NamedTuple` | `dict` | Less memory, faster access, hashable |
| Mutable record with fields | `@dataclass` | `dict` | Faster attribute access, type safety |
| Many instances of same class | `__slots__` / `dataclass(slots=True)` | plain class | ~50% memory savings per instance. See below. |
| Cache that shouldn't prevent GC | `weakref.WeakValueDictionary` | plain dict | Entries auto-removed when values are GC'd |
| Stack (LIFO) | `list` | `deque` | list.append/pop are O(1), deque adds overhead for this use case |
| Ordered unique elements | `dict` (3.7+) | `OrderedDict` | Plain dict preserves insertion order since Python 3.7 |

### Memory comparison (approximate, 64-bit Python 3.11+)

| Container | Overhead per element | Notes |
|-----------|---------------------|-------|
| `list` | 8 bytes (pointer) + 56 bytes base | Over-allocates by ~12.5% |
| `tuple` | 8 bytes (pointer) + 40 bytes base | No over-allocation, immutable |
| `set` | ~50 bytes | Hash table with open addressing |
| `dict` | ~70 bytes per key-value pair | Compact dict since 3.6 |
| `deque` | ~8 bytes (pointer) + block overhead | 64-element blocks, linked |
| `array.array('d')` | 8 bytes per double | No boxing — raw C doubles |
| `NamedTuple` | same as tuple | Zero extra overhead vs tuple |
| Instance with `__dict__` | ~100-170 bytes overhead | Full dict per instance |
| Instance with `__slots__` | ~8 bytes per slot + 40 bytes base | No `__dict__` |

## `__slots__` — Eliminating Per-Instance `__dict__`

One of the highest-impact single optimizations for data-heavy Python code. Always relevant regardless of Python version (data layout, not bytecode).

By default, every Python instance carries a `__dict__` — a full dictionary storing its attributes. For a class with 3 attributes, an instance costs ~360 bytes. With `__slots__`, it drops to ~80 bytes.

```python
# BEFORE: each instance has a __dict__ (~360 bytes for 3 attributes)
class Point:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

# AFTER: ~80 bytes per instance — 4.5x savings
class Point:
    __slots__ = ('x', 'y', 'z')
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

# Python 3.10+: dataclass shorthand
from dataclasses import dataclass

@dataclass(slots=True)
class Point:
    x: float
    y: float
    z: float
```

### When to use `__slots__`

- **Many instances** (thousands+) of the same class — memory savings compound
- **Data classes / records** — simple attribute containers
- **Inner loop objects** — objects created and discarded in hot paths

### When NOT to use `__slots__`

- **Few instances** — overhead isn't meaningful
- **Need dynamic attributes** — `__slots__` prevents adding arbitrary attributes
- **Inheritance complexity** — all classes in the hierarchy must define `__slots__`, or you get `__dict__` back
- **Pickle compatibility** — slotted objects need `__getstate__`/`__setstate__` for older pickle protocols

### Impact at scale

With 1 million instances of a 3-attribute class:
- Without `__slots__`: ~360 MB
- With `__slots__`: ~80 MB
- Savings: **~280 MB**

## `weakref` — Caches That Don't Leak

When building caches or registries, a plain dict holds strong references — cached objects can never be garbage collected. `weakref.WeakValueDictionary` solves this.

```python
import weakref

# BEFORE: objects cached forever — memory leak in long-running process
_cache = {}
def get_resource(key):
    if key not in _cache:
        _cache[key] = load_expensive_resource(key)
    return _cache[key]

# AFTER: cache entries auto-removed when resource is GC'd elsewhere
_cache = weakref.WeakValueDictionary()
def get_resource(key):
    resource = _cache.get(key)
    if resource is None:
        resource = load_expensive_resource(key)
        _cache[key] = resource
    return resource
```

Also available: `WeakSet` (for tracking live objects without preventing GC) and `weakref.finalize()` (modern replacement for `__del__` for cleanup).

## `copy.deepcopy()` — A Hidden Performance Trap

`copy.deepcopy()` recursively traverses the entire object graph and maintains a memo dict to handle circular references. In hot paths, it's an order of magnitude slower than alternatives.

```python
# BEFORE: deepcopy in loop — very expensive
import copy
for item in items:
    snapshot = copy.deepcopy(item)
    process(snapshot)

# AFTER: construct directly if possible
for item in items:
    snapshot = Item(x=item.x, y=item.y, z=item.z)
    process(snapshot)

# Or use shallow copy if nested objects aren't mutated
for item in items:
    snapshot = copy.copy(item)
    process(snapshot)
```

**When deepcopy is actually needed**: Only when you have nested mutable objects that will be mutated independently. In most cases, shallow copy or direct construction suffices.

## Algorithmic Patterns That Matter

### The Index-First Pattern (O(n*m) -> O(n+m))

The single most impactful optimization. Whenever you see nested loops where the inner loop searches for a match:

```python
# BEFORE: O(n * m)
for order in orders:
    for customer in customers:
        if customer.id == order.customer_id:
            process(order, customer)
            break

# AFTER: O(n + m)
customer_by_id = {c.id: c for c in customers}
for order in orders:
    customer = customer_by_id.get(order.customer_id)
    if customer:
        process(order, customer)
```

This pattern applies everywhere: matching records, deduplication, grouping, joining datasets.

### The Precompute Pattern

When the same derived value is computed repeatedly:

```python
# BEFORE: recomputes set(tags) for every comparison
def has_overlap(item, target_tags):
    return bool(set(item.tags) & target_tags)

for item in items:  # called 10K times
    if has_overlap(item, target_tags):
        ...

# AFTER: precompute once
items_with_tag_sets = [(item, frozenset(item.tags)) for item in items]
target_set = frozenset(target_tags)
for item, tag_set in items_with_tag_sets:
    if tag_set & target_set:
        ...
```

### The Accumulate Pattern

Building a result incrementally:

```python
# BEFORE: string concatenation in loop
result = ""
for line in lines:
    result += process(line) + "\n"

# AFTER: O(n) guaranteed
parts = []
for line in lines:
    parts.append(process(line))
result = "\n".join(parts)

# Or with generator:
result = "\n".join(process(line) for line in lines)
```

**Version note**: CPython has optimized `str +=` to resize in-place when refcount is 1 (since ~2004), making it amortized O(n) in many cases. However, this is a CPython implementation detail — `.join()` is O(n) by design and works reliably across all Python implementations.

### The Batch Pattern

Avoid repeated single-element operations when batch is available:

```python
# BEFORE: n individual set.add calls
seen = set()
for item in items:
    seen.add(item.id)

# AFTER: single set construction
seen = {item.id for item in items}

# BEFORE: repeated dict.update
for k, v in pairs:
    result[k] = v

# AFTER: dict comprehension or direct construction
result = dict(pairs)
```

## Common Traps

### `sorted()` vs `.sort()`

`sorted()` creates a new list. `list.sort()` sorts in-place. In a hot loop, the extra allocation from `sorted()` can matter:

```python
# If you don't need the original order:
data.sort(key=lambda x: x.score)  # in-place, no allocation

# If you need to preserve original:
ranked = sorted(data, key=lambda x: x.score)  # new list
```

### `in` operator complexity varies by type

| Type | `x in container` | Notes |
|------|-------------------|-------|
| `list` | O(n) | Linear scan |
| `tuple` | O(n) | Linear scan |
| `set` | O(1) average | Hash lookup |
| `frozenset` | O(1) average | Hash lookup |
| `dict` | O(1) average | Key lookup |
| `str` | O(n*m) worst case | Substring search |
| `range` | O(1) | Arithmetic check |

### `collections.deque` maxlen for bounded buffers

```python
from collections import deque

# Auto-evicts oldest when full — no manual size management
recent_events = deque(maxlen=1000)
recent_events.append(event)  # if len > 1000, leftmost is dropped
```

### `heapq` for streaming top-k

Don't sort the entire dataset to get the top 10:

```python
import heapq

# O(n log k) instead of O(n log n)
top_10 = heapq.nlargest(10, items, key=lambda x: x.score)

# For streaming data, maintain a min-heap of size k:
heap = []
for item in stream:
    if len(heap) < k:
        heapq.heappush(heap, (item.score, item))
    elif item.score > heap[0][0]:
        heapq.heapreplace(heap, (item.score, item))
```

### dict/set iteration after heavy deletion

Iteration cost is O(n) where n is the **maximum historical size**, not current size. After deleting many keys, the internal hash table retains its oversized allocation until a resize is triggered.

```python
# After heavy deletion, iteration is slower than expected
d = {i: i for i in range(1_000_000)}
for i in range(999_000):
    del d[i]
# d has 1000 items but iterating walks 1M slots

# Fix: rebuild the dict to compact it
d = dict(d)  # or {k: v for k, v in d.items()}
```

### Caching traps

**`@functools.cache` / `@lru_cache` on instance methods**: Captures `self` as cache key — instances can never be garbage collected:

```python
class Manager:
    @functools.cache  # BUG: self captured as key, instance lives forever
    def load(self, name):
        return expensive(name)
```

Fix: use a module-level cache keyed on the relevant value, use `lru_cache` with explicit `cache_clear()` in `__del__`, or cache on the instance itself.

**Unbounded `@cache` on long-running processes**: `@cache` (alias for `lru_cache(maxsize=None)`) never evicts. On servers and daemons, this silently grows memory forever. Use `@lru_cache(maxsize=N)` with an appropriate bound.

## Version-Specific Guidance

### Optimizations worth recommending at any version

These are algorithmic or data-layout changes — CPython interpreter improvements cannot obsolete them:

- Container swaps (list -> set, list -> deque, list -> dict)
- `__slots__` / `dataclass(slots=True)` for memory
- `weakref.WeakValueDictionary` for caches
- Index-first pattern (nested loop -> dict index)
- `heapq` for top-k
- `bisect` for sorted data operations
- `Counter` / `defaultdict` for counting/grouping
- Avoiding `copy.deepcopy()` in hot paths

### Optimizations to check against Python version

| Optimization | < 3.11 | 3.11+ | 3.12+ |
|---|---|---|---|
| Local variable hoisting (`append = result.append`) | ~15-30% gain | ~5-10% (PEP 659 caches LOAD_ATTR). Only recommend for chained access like `self.data.append`. | Same as 3.11 |
| `operator.itemgetter` vs lambda | ~30-50% faster | ~15-30% faster. Still wins (C callable avoids Python frame). | Same as 3.11 |
| `map(c_func)` vs list comprehension | `map` is faster | `map` marginally faster | Negligible gap (PEP 709 inlined comprehensions). Prefer comprehensions for readability. |
| `dict.get()` vs `try/except KeyError` | Within ~10% | Within ~10% | Within ~10%. Choose by semantics, not speed. |
| String `+=` in loop vs `.join()` | `.join()` recommended | `.join()` recommended (CPython refcount-1 optimization exists since 2004 but is implementation-specific) | Same |

## Profiling for Data Structure Issues

### Identifying the problem

Data structure issues manifest as:
- **High `tottime` in built-in operations**: `list.index`, `list.remove`, `list.__contains__`
- **Unexpected quadratic scaling**: doubling input size -> 4x runtime
- **Hot functions doing simple lookups**: a function called 100K times that does `if x in some_list`

### cProfile workflow

```bash
# Profile and sort by cumulative time
$RUNNER -m cProfile -s cumtime script.py 2>&1 | head -40

# Sort by total time (time in function itself, excluding callees)
$RUNNER -m cProfile -s tottime script.py 2>&1 | head -40

# Save for detailed analysis
$RUNNER -m cProfile -o /tmp/profile.prof script.py
```

### timeit for A/B comparison

```python
import timeit

# Compare list membership vs set membership
setup = "data = list(range(10000)); s = set(data); target = 9999"
print(timeit.timeit("target in data", setup=setup, number=10000))   # ~1.5s
print(timeit.timeit("target in s", setup=setup, number=10000))       # ~0.001s
```

### tracemalloc for memory profiling (stdlib, no install needed)

```python
import tracemalloc

tracemalloc.start()

# ... code under test ...

snapshot = tracemalloc.take_snapshot()
top = snapshot.statistics('lineno')
for stat in top[:10]:
    print(stat)
```

Useful for quick memory auditing without installing memray. Shows where memory was allocated, line by line.

### Scaling test

The most reliable way to confirm a complexity issue — run with 1x, 2x, 4x input size:

```python
import time

for scale in [1, 2, 4, 8]:
    data = generate_test_data(n=1000 * scale)
    start = time.perf_counter()
    target_function(data)
    elapsed = time.perf_counter() - start
    print(f"n={1000*scale:>8d}  time={elapsed:.3f}s  ratio={elapsed/base:.1f}x" if scale > 1 else f"n={1000*scale:>8d}  time={elapsed:.3f}s")
    if scale == 1:
        base = elapsed
```

If ratio doubles when n doubles = O(n). If ratio quadruples = O(n^2).

## Pitfalls

- **Don't optimize cold code**: Profile first. The slow function might only be called once at startup.
- **Don't assume data size**: A O(n^2) algorithm on 10 items is faster than O(n) with a dict due to constant factors. Measure.
- **set is not always faster**: For tiny collections (<5 items), list iteration can beat set hashing. The crossover is around 4-8 elements depending on element type.
- **dict key requirements**: Keys must be hashable. If you need to index by mutable objects, consider using an immutable projection (e.g., `id()`, a frozen attribute).
- **Memory vs speed tradeoff**: Building an index (dict/set) trades memory for speed. On memory-constrained systems, the O(n) scan might be preferable.
- **Thread safety**: `deque.append` and `deque.popleft` are atomic in CPython (GIL), but this is an implementation detail, not a guarantee. Use `queue.Queue` for explicit thread safety.
- **Don't chase micro-optimizations on 3.11+**: The specializing interpreter has absorbed many classic tricks. Focus on algorithmic improvements and data layout, not bytecode-level tweaks.
- **Don't hoist attributes into locals**: Caching `self.x` into a local variable (e.g., `local_x = self.x` before a loop) is a common micro-optimization pattern that maintainers frequently reject. On Python 3.11+, PEP 659 caches attribute lookups anyway, making the benefit negligible (~1-2%). Worse, it changes code style in a way that obscures intent. Only recommend attribute hoisting for deeply chained access (`self.a.b.c`) where the intermediate objects might change, and even then, flag it as a style tradeoff.

## Bytecode Specialization Analysis (Python 3.11+)

On Python 3.11+, CPython's specializing adaptive interpreter (PEP 659) rewrites bytecodes after ~8 executions at each call site. Inspecting bytecode *after warmup* reveals optimization opportunities invisible to source-level analysis:

- **Unspecialized (ADAPTIVE) opcodes** on hot paths indicate type instability — the interpreter couldn't optimize because types vary across invocations
- **Specialized opcodes** confirm type stability — e.g., `LOAD_ATTR_SLOT` means `__slots__` is working, `BINARY_OP_ADD_INT` means pure integer arithmetic
- **Upgrade paths** are visible: `LOAD_ATTR_INSTANCE_VALUE` (dict-based) can be upgraded to `LOAD_ATTR_SLOT` by adding `__slots__`

### Quick reference: what ADAPTIVE opcodes mean

| ADAPTIVE opcode | Problem | Fix |
|----------------|---------|-----|
| `LOAD_ATTR_ADAPTIVE` | Attribute target type varies | Add `__slots__`, ensure consistent types |
| `STORE_ATTR_ADAPTIVE` | Store target type varies | Same as LOAD_ATTR |
| `BINARY_OP_ADAPTIVE` | Mixed numeric types (int + float) | Normalize types early |
| `BINARY_SUBSCR_ADAPTIVE` | Container type varies (list vs dict) | Pin container type |
| `CALL_ADAPTIVE` | Callable varies (callback patterns) | Direct calls or if/elif dispatch |
| `COMPARE_OP_ADAPTIVE` | Operand types vary | Consistent types in comparisons |
| `FOR_ITER_ADAPTIVE` | Iterator type varies | Accept specific iterable type |

### When to use bytecode analysis

- **After profiling identifies hot functions**: Bytecode analysis tells you *why* those functions are slow at the interpreter level
- **When `__slots__` is a candidate**: Confirm `LOAD_ATTR_INSTANCE_VALUE` → `LOAD_ATTR_SLOT` upgrade
- **When type instability is suspected**: ADAPTIVE opcodes are direct evidence
- **On 3.11+ only**: Older versions don't have the adaptive interpreter

### Running the analyzer

```bash
$RUNNER -c "import dis; import module; dis.dis(module)"
# Or for a specific function:
$RUNNER -c "import dis; import module; dis.dis(module.target_function)"
```

See `bytecode-guide.md` for the full opcode vocabulary and interpretation guide.

## NumPy & Pandas Antipatterns

These are extremely common in data-heavy Python code. Recognize these during source reading.

### Row-by-row iteration instead of vectorized operations

```python
# SLOW: iterrows is Python-speed
for idx, row in df.iterrows():
    df.at[idx, "result"] = row["a"] + row["b"]

# FAST: vectorized — runs in C
df["result"] = df["a"] + df["b"]
```

### apply() with Python function instead of vectorized

```python
# SLOW: apply calls Python function per row
df["upper"] = df["name"].apply(lambda x: x.upper())

# FAST: vectorized string method
df["upper"] = df["name"].str.upper()
```

### Growing a DataFrame in a loop

```python
# SLOW: O(n²) — each concat copies everything
df = pd.DataFrame()
for item in items:
    df = pd.concat([df, pd.DataFrame([item])])

# FAST: build list, create DataFrame once
rows = [item for item in items]
df = pd.DataFrame(rows)
```

### NumPy: creating temporary arrays

```python
# SLOW: creates 3 temporary arrays
result = np.sqrt(x**2 + y**2)

# FASTER: in-place operations
result = x**2
np.add(result, y**2, out=result)
np.sqrt(result, out=result)
```

### Wrong dtype

```python
# Uses 8 bytes per int by default
arr = np.array(small_ints)  # int64

# Use smallest sufficient dtype
arr = np.array(small_ints, dtype=np.int16)  # 2 bytes per int
```

### Detection

Look for `iterrows`, `itertuples` in hot paths, `.apply(` with lambda, `pd.concat` inside loops, `np.append` in loops.

## Reference Files

- `stdlib-containers.md` — Detailed stdlib container docs with complexity tables
- `algorithmic-patterns.md` — Common algorithmic transformations with before/after examples
- `profiling-guide.md` — cProfile, timeit, tracemalloc, line_profiler usage and interpretation
- `bytecode-guide.md` — CPython 3.11+ bytecode specialization opcode reference and interpretation
