codeflash-internal/experiments/universe-optimize/optimization.md

# Codeflash Optimization Run

You are running an automated optimization session on this open-source project.
Your goal is to find provably-faster code implementations and create individual
PRs with benchmark results for each optimization.

## Your Mission

### Phase 1: Setup

1. Understand the project: read the README, project structure, and key source files.
2. Set up the project: install dependencies, verify tests pass.
3. Install codeflash: `pip install codeflash` (or `uv add --dev codeflash` if using uv).

### Phase 2: Optimize and Create PRs

Launch the optimization agent directly using the Agent tool (do NOT use `/codeflash-optimize` or AskUserQuestion — you are running non-interactively):

```
Agent(subagent_type: "codeflash-deep", name: "optimizer", run_in_background: true, prompt: "AUTONOMOUS MODE: The user has already been asked for context (included below). Do NOT ask the user any questions — work fully autonomously. Make all decisions yourself: generate a run tag from today's date, identify benchmark tiers from available tests, choose optimization targets from profiler output. If something is ambiguous, pick the reasonable default and document your choice in HANDOFF.md.\n\nOptimize this project for performance.\n\nUser context: go — no special constraints, optimize everything you can find.\n\nIMPORTANT — After each KEEP, immediately do the following before moving to the next target:\n1. Write a pytest-benchmark test for the optimized function with realistic inputs\n2. Run `codeflash compare <base-commit> <head-commit> --inject <benchmark-test-path>` to generate benchmark tables. If codeflash compare is not available, run the benchmark manually on base and head commits and format the results as a markdown table.\n3. Create a PR branch: `git checkout main && git checkout -b codeflash/opt-<function-name> && git cherry-pick <commit-sha> && git add <benchmark-test> && git commit --amend --no-edit`\n4. Push the branch: `git push origin codeflash/opt-<function-name>`\n5. Open a PR with `gh pr create` using the benchmark results in the body (see PR body format below)\n6. Switch back to the optimize branch: `git checkout codeflash/optimize`\n7. Record the PR URL in results.tsv and HANDOFF.md\n\nDo NOT batch PR creation for later. Each optimization gets its PR immediately after the experiment proves it works.\n\nPR Body Format:\n## Summary\n- <One-line description of what was optimized and how>\n\n## Benchmark\n### <Machine specs — e.g. Azure Standard_D4s_v5, 4 vCPU, 16 GiB RAM, Python 3.12>\n\n| Function | base | head | Delta | Speedup |\n|:---|---:|---:|:---|---:|\n| `<function_name>` | ... | ... | ... | X.Xx |\n\n---\n*Generated by codeflash agent*\n\n<details>\n<summary><b>Benchmark test source</b></summary>\n\n```python\n<benchmark test code>\n```\n</details>\n\n## Test plan\n- [x] All existing tests pass\n- [x] Benchmark shows measurable improvement")
```

Wait for the optimizer agent to complete.

### Phase 3: Collect Results

Create the results directory and write summary:

```bash
mkdir -p ~/results
```

Write `~/results/summary.json` with this structure:

```json
{
  "repo": "<org/repo>",
  "language": "<python|javascript|java>",
  "branch": "codeflash/optimize",
  "total_experiments": 0,
  "total_keeps": 0,
  "total_discards": 0,
  "optimizations": [
    {
      "commit": "<sha>",
      "function": "<function_name>",
      "file": "<file_path>",
      "description": "<what was optimized>",
      "cpu_speedup": "<e.g. 2.3x faster>",
      "memory_reduction": "<e.g. -50 MiB>",
      "technique": "<e.g. replaced list with set, eliminated deepcopy>",
      "pr_number": "<PR number if created>",
      "pr_url": "<PR URL if created>",
      "benchmark_test": "<path to benchmark test file>"
    }
  ],
  "headline_stats": {
    "best_single_speedup": "<e.g. 5x faster>",
    "best_function": "<function_name>",
    "total_cpu_improvement_pct": 0,
    "total_memory_saved_mb": 0
  },
  "pr_ready_commits": 0,
  "prs_created": [],
  "status": "completed|plateaued|failed",
  "error": "<if failed, why>"
}
```

Also copy `.codeflash/results.tsv` and `.codeflash/HANDOFF.md` to `~/results/` (if they exist).

Then push all branches to the remote fork:

```bash
git push origin --all
```

## Important

- Work fully autonomously. Do not ask questions -- make reasonable decisions.
- If tests fail during setup, note the pre-existing failures and work around them.
- If the project cannot be set up (missing deps, private packages), write summary.json with status "failed" and an error message, then stop.
- Time limit: aim to complete within 8 hours. If still running after 8 hours, wrap up, write summary.json with whatever results you have, and stop.
- Always push all branches before finishing, even if results are partial.
- Each optimization MUST have its own PR with benchmark results created immediately after the optimization is verified. Do NOT batch PR creation for later.

CRITICAL: Do NOT use AskUserQuestion at any point. You are running in fully autonomous non-interactive mode. Make all decisions yourself.