# Algorithmic Patterns for Python Optimization

## Pattern 1: Index-First (Nested Loop Elimination)

**Problem**: Nested loops where inner loop searches for a match.

```python
# BEFORE: O(n * m)
results = []
for order in orders:
    for product in products:
        if product.id == order.product_id:
            results.append((order, product))
            break
```

```python
# AFTER: O(n + m)
product_by_id = {p.id: p for p in products}
results = []
for order in orders:
    product = product_by_id.get(order.product_id)
    if product:
        results.append((order, product))
```

**Variants:**
- Multi-key index: `defaultdict(list)` when multiple items share a key
- Composite key: `(field1, field2)` tuple as dict key
- Bidirectional: build index for both sides when doing cross-matching

---

## Pattern 2: Set Operations (Membership + Set Algebra)

**Problem**: Checking membership against a collection, or computing intersections/differences.

```python
# BEFORE: O(n * m) — list membership in loop
valid_ids = [1, 2, 3, ...]  # large list
filtered = [item for item in items if item.id in valid_ids]
```

```python
# AFTER: O(n + m)
valid_ids = {1, 2, 3, ...}  # set
filtered = [item for item in items if item.id in valid_ids]
```

**Set algebra shortcuts:**

```python
# Instead of manual loop intersection
common = set(a) & set(b)

# Instead of manual loop difference
only_in_a = set(a) - set(b)

# Instead of checking all pairs for duplicates
has_dupes = len(items) != len(set(items))
```

---

## Pattern 3: Sliding Window

**Problem**: Computing aggregates over a moving window of fixed size.

```python
# BEFORE: O(n * k) — recompute sum for each position
def moving_average(data, k):
    result = []
    for i in range(len(data) - k + 1):
        result.append(sum(data[i:i+k]) / k)
    return result
```

```python
# AFTER: O(n) — maintain running sum
from collections import deque

def moving_average(data, k):
    window = deque(maxlen=k)
    window_sum = 0
    result = []
    for x in data:
        if len(window) == k:
            window_sum -= window[0]
        window.append(x)
        window_sum += x
        if len(window) == k:
            result.append(window_sum / k)
    return result
```

**Variants:**
- Sliding min/max: use `deque` storing indices of monotonic sequence
- Sliding distinct count: use `Counter` as the window

---

## Pattern 4: Two Pointers (Sorted Data)

**Problem**: Finding pairs or ranges in sorted data.

```python
# BEFORE: O(n^2) — check all pairs
def two_sum_sorted(nums, target):
    for i in range(len(nums)):
        for j in range(i+1, len(nums)):
            if nums[i] + nums[j] == target:
                return (i, j)
    return None
```

```python
# AFTER: O(n) — two pointers converging
def two_sum_sorted(nums, target):
    left, right = 0, len(nums) - 1
    while left < right:
        s = nums[left] + nums[right]
        if s == target:
            return (left, right)
        elif s < target:
            left += 1
        else:
            right -= 1
    return None
```

---

## Pattern 5: Frequency Map (Counting + Grouping)

**Problem**: Counting occurrences, grouping by key, or histogram construction.

```python
# BEFORE: manual counting
counts = {}
for item in items:
    key = item.category
    if key not in counts:
        counts[key] = 0
    counts[key] += 1
```

```python
# AFTER: Counter
from collections import Counter
counts = Counter(item.category for item in items)
top_5 = counts.most_common(5)
```

**Grouping variant:**

```python
# BEFORE: manual grouping
groups = {}
for item in items:
    key = item.category
    if key not in groups:
        groups[key] = []
    groups[key].append(item)

# AFTER: defaultdict
from collections import defaultdict
groups = defaultdict(list)
for item in items:
    groups[item.category].append(item)
```

---

## Pattern 6: Lazy Evaluation (Generator Pipelines)

**Problem**: Building large intermediate lists that are only iterated once.

```python
# BEFORE: O(n) memory for each intermediate list
data = read_large_file()                    # list of lines
parsed = [parse_line(l) for l in data]      # list of records
filtered = [r for r in parsed if r.valid]   # list of valid records
result = [transform(r) for r in filtered]   # list of results
```

```python
# AFTER: O(1) memory pipeline
def pipeline(filepath):
    data = read_large_file_iter(filepath)       # generator
    parsed = (parse_line(l) for l in data)      # generator
    filtered = (r for r in parsed if r.valid)   # generator
    return (transform(r) for r in filtered)     # generator

for item in pipeline("data.txt"):
    process(item)
```

**When NOT to use generators:**
- When you need `len()`, indexing, or multiple passes
- When the full list is small anyway
- When you need to backtrack or look ahead

---

## Pattern 7: Top-K Selection

**Problem**: Finding the k largest/smallest elements from n items.

```python
# BEFORE: O(n log n) — sort everything
top_10 = sorted(items, key=lambda x: x.score, reverse=True)[:10]
```

```python
# AFTER: O(n log k)
import heapq
top_10 = heapq.nlargest(10, items, key=lambda x: x.score)
```

**When sorted() is actually better:**
- k is close to n (e.g., top 50% of items)
- k = 1 — just use `max()` or `min()` which are O(n)
- You need the full sorted order anyway

---

## Pattern 8: Prefix Sum (Range Queries)

**Problem**: Repeated range sum queries on static data.

```python
# BEFORE: O(q * n) for q queries on n elements
def range_sum(data, left, right):
    return sum(data[left:right+1])
```

```python
# AFTER: O(n) precomputation, O(1) per query
from itertools import accumulate

prefix = [0] + list(accumulate(data))

def range_sum(left, right):
    return prefix[right+1] - prefix[left]
```

---

## Pattern 9: Deduplication Preserving Order

**Problem**: Remove duplicates while preserving first-occurrence order.

```python
# BEFORE: O(n^2) — check if seen in list
seen = []
unique = []
for item in items:
    if item not in seen:
        seen.append(item)
        unique.append(item)
```

```python
# AFTER: O(n) — set for O(1) membership
seen = set()
unique = []
for item in items:
    if item not in seen:
        seen.add(item)
        unique.append(item)

# Or using dict.fromkeys (Python 3.7+ preserves order):
unique = list(dict.fromkeys(items))
```

---

## Pattern 10: Early Termination

**Problem**: Processing continues after the answer is already found.

```python
# BEFORE: processes everything
def has_valid(items):
    return len([i for i in items if is_valid(i)]) > 0
```

```python
# AFTER: stops at first match
def has_valid(items):
    return any(is_valid(i) for i in items)

# Similarly:
all(condition(i) for i in items)   # stops at first False
next((i for i in items if pred(i)), None)  # first match or None
```

---

## Pattern 11: Avoid Unnecessary Deep Copies

**Problem**: `copy.deepcopy()` used where shallow copy or direct construction suffices.

```python
# BEFORE: deepcopy in loop — traverses entire object graph each time
import copy
for template in templates:
    instance = copy.deepcopy(template)
    instance.name = generate_name()
    results.append(instance)
```

```python
# AFTER: construct directly
for template in templates:
    instance = Template(
        name=generate_name(),
        config=template.config,  # shared reference is fine if not mutated
        data=template.data,
    )
    results.append(instance)

# Or shallow copy if structure is flat
import copy
for template in templates:
    instance = copy.copy(template)
    instance.name = generate_name()
    results.append(instance)
```

**When deepcopy IS needed**: Only when you have nested mutable objects (e.g., list of lists, dict of dicts) that will be mutated independently after copying.

---

## Pattern 12: `__slots__` for Memory-Heavy Classes

**Problem**: Millions of instances of a simple class, each carrying a full `__dict__`.

```python
# BEFORE: ~360 bytes per instance
class Point:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z
```

```python
# AFTER: ~80 bytes per instance
class Point:
    __slots__ = ('x', 'y', 'z')
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

# Or with dataclass (Python 3.10+):
from dataclasses import dataclass

@dataclass(slots=True)
class Point:
    x: float
    y: float
    z: float
```

This is a data-layout optimization — always relevant regardless of Python version.

---

## Complexity Quick Reference

| Operation | list | dict | set | deque | heapq |
|-----------|------|------|-----|-------|-------|
| Access by index | O(1) | — | — | O(n) | — |
| Access by key | — | O(1) | — | — | — |
| Search | O(n) | O(1) | O(1) | O(n) | O(n) |
| Insert at end | O(1)* | O(1) | O(1) | O(1) | O(log n) |
| Insert at start | O(n) | — | — | O(1) | — |
| Delete by value | O(n) | O(1) | O(1) | O(n) | O(n) |
| Min/Max | O(n) | O(n) | O(n) | O(n) | O(1)/O(n) |
| Sort | O(n log n) | — | — | — | O(n log n) |

\* amortized — occasional O(n) for reallocation
