# Experiment Loop — Data Structures Domain

> Base framework: `../shared/experiment-loop-base.md`

## Reasoning Checklist

Before writing any code, answer these 9 questions. If you can't answer 3-6 concretely, research more before coding.

1. **Pattern**: What antipattern or suboptimal choice? (see Antipattern Categories in the agent prompt)
2. **Hot path?** Is this code actually on the critical path? Confirm with profiler — don't optimize cold code.
3. **Complexity change?** What's the big-O before and after? (e.g., O(n^2) -> O(n), O(n) -> O(1))
4. **Data size?** How large is n in practice? A O(n^2) loop on 10 items doesn't matter.
5. **Exercised?** Does the target test actually exercise this code path with representative data?
6. **Mechanism**: HOW does your change improve performance? Be specific.
7. **Correctness**: Does this change behavior? Trace ALL code paths — especially polymorphic dispatch and virtual method calls. See Correctness Verification below.
8. **Conventions**: Does this match the project's existing style? Don't introduce micro-optimization patterns (attribute hoisting, wrapper removal) that conflict with project conventions.
9. **Verify cheaply**: Can you validate with a micro-benchmark or timeit before the full benchmark run?

## Correctness Verification

**Every optimization that changes control flow or adds a short-circuit MUST trace all code paths.** This is the #1 source of incorrect optimizations.

### Polymorphic dispatch traps

When you see a loop like `for x in items: x.do_thing(self)` and want to add a fast-path skip, you MUST:

1. **Find ALL implementations** of `do_thing` across the class hierarchy (use Grep for `def do_thing`).
2. **Verify the skip condition is valid for every implementation**, not just the most common one.
3. **Check if any implementation has internal guards** that already handle the skip — if so, your external check is redundant AND may be wrong for other implementations.

### Internal guard duplication

If a method you're calling already has an internal fast-path guard, adding the same check externally is:
- **Redundant** for that specific method
- **Potentially wrong** if other methods called from the same dispatch point don't have equivalent guards

Rule: Don't hoist guards out of polymorphic call targets.

### Control flow refactors

When refactoring loops (e.g., replacing a boolean flag with `for/else`, converting `if/else` to early `continue`), verify the new control flow by tracing each path:
- Does every case still reach the code it should?
- Does `continue` or `break` skip code that was previously reachable?
- Are both the fast path AND slow path exercised?

Run the compilation or test suite after every control flow refactor, not just at the end.

## Diff Hygiene

Before pushing, review the full diff against upstream (`git diff <base>..HEAD`) and check for:

1. **Unintended formatting changes** — rewrapped comments, changed whitespace, reformatted lines you didn't need to touch. These are noise that makes review harder.
2. **Deleted code you didn't mean to remove** — commented-out blocks, debug scaffolding, or TODOs that belong to the maintainer.
3. **Inconsistent style** — if the surrounding code uses a particular pattern, your changes should match.

The diff is what the maintainer reviews. Every unintended change is a distraction that erodes trust.

## Maintainer Acceptance

Optimizations that are correct and fast can still be rejected if they violate project conventions. Before submitting:

1. **Match existing style**. If the project has its own wrappers, conventions, or idioms, don't replace them with "standard" equivalents for marginal gain.
2. **Separate bug fixes from optimizations**. Bug fixes should be clearly identifiable in the commit/PR, not buried among micro-optimizations.
3. **Avoid micro-optimizations with marginal benefit**. Attribute hoisting (`local_var = self.attr`) typically saves <2% on Python 3.11+ and changes code style. Maintainers often reject these. Focus on algorithmic improvements and data structure changes — these are universally accepted.
4. **Provide reasoning per change**. In the PR body, explain *why* each change helps — not just *what* it does. Maintainers are more likely to accept changes they understand.
5. **Record rejections as conventions**. When a maintainer rejects a change for style or convention reasons, immediately add it to `.codeflash/conventions.md`. This prevents repeating the same mistake in future sessions. Format: what was rejected, why, and what to do instead.

## Domain-Specific Loop Steps

**Step 1 — Choose target** sources:
- **Profiler output**: Highest cumulative-time function from cProfile that has a known antipattern.
- **Static analysis**: Findings from `ds_analyze.py` that overlap with hot paths.
- **Deep source reading**: Trace the hot path for algorithmic inefficiencies (redundant computation, missing caching, early termination). Use Explore subagents for this.

Print: `[experiment N] Target: <description> (<pattern>, <est. impact>%)`

**Step 6 — Read results**: Print `[experiment N] baseline <X>s, optimized <Y>s — <Z>% faster` (or slower/same).

**Step 8 — Noise threshold**: If speedup is <5%, re-run 3 times to confirm not noise.

**Step 10 — Record**: Record immediately in both `.codeflash/results.tsv` and `.codeflash/HANDOFF.md`. Do not batch — write as soon as you keep or discard.

**Step 11 — Bytecode validation (Python 3.11+, optional)**: After keeping a data-layout optimization (e.g., adding `__slots__`, fixing type instability), re-run `bytecode_inspect.py` on the modified module to confirm the optimization took effect at the interpreter level.

**Step 12 — E2E benchmark (after KEEP)**: If `codeflash compare` is available (check `.codeflash/setup.md`), run `$RUNNER -m codeflash compare <pre-opt-sha> HEAD` for authoritative isolated measurement. Record e2e speedup in `results.tsv`. If e2e contradicts micro-bench, trust the e2e measurement. See `../shared/e2e-benchmarks.md`.

**Step 13 — Milestones**: Create `codeflash/ds-<tag>-v<N>` branch. At milestones, run `$RUNNER -m codeflash compare <baseline-sha> HEAD` for cumulative e2e measurement. Print `[milestone] vN — <total kept>/<total experiments>, cumulative speedup <X>%`.

## Keep/Discard Thresholds

```
Test passed?
+-- NO -> Fix or discard immediately
+-- YES -> Speedup measured?
    +-- YES (>=5%) -> KEEP
    +-- YES (<5%) -> Re-run 3x to confirm not noise
    |   +-- Confirmed -> KEEP
    |   +-- Noise -> DISCARD
    +-- YES (micro-bench only, >=20%) -> KEEP if hot path confirmed by profiler
    +-- NO -> DISCARD
```

## Strategy Rotation

If 3+ consecutive discards on the same type of optimization (e.g., container swaps), switch strategy:
- Container swaps -> Algorithmic restructuring -> Caching/precomputation -> Stdlib replacements

## Plateau Detection — Domain-Specific

3+ consecutive discards across all strategies = check if remaining hotspots are:
- I/O bound (network, disk) — not fixable with data structure changes
- Already using optimal structures
- Dominated by third-party/framework code

If top 3 hotspots are all non-optimizable, **stop and report to user** with what's left and why.

## Logging Format

Tab-separated `.codeflash/results.tsv`, 10 columns:

```
commit	target_test	baseline_s	optimized_s	speedup	tests_passed	tests_failed	status	pattern	description
```

- `target_test`: test name, `all`, or `micro:<name>`
- `baseline_s`: baseline timing in seconds
- `optimized_s`: optimized timing in seconds
- `speedup`: percentage improvement (e.g., `23%`)
- `status`: `keep`, `discard`, or `crash`
- `pattern`: antipattern category (e.g., `list-as-queue`, `quadratic-loop`)
