- Added a new JMH runner script (`jmh-runner.sh`) for automated benchmarking in Java, including options for baseline capture, GC profiling, and result comparison. - Updated experiment loop documentation to include mandatory baseline performance capture before code changes, emphasizing the importance of capturing performance metrics for accurate comparisons. - Revised micro-benchmarking steps to incorporate the new JMH runner and ensure consistent usage of GC profiling across benchmarks. - Improved end-to-end benchmarking instructions to utilize the JMH runner for authoritative measurements, ensuring results are recorded alongside micro-benchmark results. - Enhanced JSON result parsing in documentation to utilize `jq` for more efficient extraction of benchmark metrics. - Streamlined the experiment loop base documentation to clarify the steps for capturing original outputs and performance baselines, reinforcing the need for correctness verification before proceeding with optimizations. |
||
|---|---|---|
| .. | ||
| .claude-plugin | ||
| agents | ||
| commands | ||
| hooks | ||
| languages | ||
| references/shared | ||
| vendor/codex | ||
| ARCHITECTURE.md | ||
| README.md | ||
| ROADMAP.md | ||
Plugin Architecture
The codeflash plugin is a multi-language Claude Code plugin for autonomous performance optimization.
Session Flow
/codeflash-optimize start
└─ Skill asks user for context (AskUserQuestion)
└─ Skill launches language router (FOREGROUND, run_in_background: false)
└─ Router runs setup agent (waits for completion)
└─ Router reads CLAUDE.md, library research, etc.
└─ Router creates team: TeamCreate("codeflash-session")
└─ Router creates tasks: TaskCreate("Setup", "Baseline", "Experiment loop")
└─ Router launches optimizer (BACKGROUND, run_in_background: true)
└─ Router STAYS ALIVE coordinating:
├─ Receives [baseline], [progress], [experiment] via SendMessage
├─ Relays progress directly to user (foreground output)
├─ Handles [complete] → cleanup, changelog, TeamDelete
└─ Returns result to skill when session ends
The skill launches the language router in the foreground so its progress output streams directly to the user. The router launches the optimizer in the background as a named teammate and stays alive to coordinate via SendMessage. This is the Team Lead + Specialists pattern — the router is the team lead, the optimizer and any domain agents are specialists.
Agent Hierarchy
Tier 1: Top Router (plugin/agents/codeflash.md)
└─ Detects language, delegates immediately
Tier 2: Language Router / Team Lead
├─ codeflash-python (plugin/languages/python/agents/)
└─ codeflash-javascript (plugin/languages/javascript/agents/)
Tools: TeamCreate, TeamDelete, Agent, SendMessage, TaskCreate/Update
Tier 3: Deep Agent / Sub-Team Lead
├─ codeflash-deep (Python)
└─ codeflash-js-deep (JavaScript)
Tools: TeamCreate, Agent, SendMessage (can dispatch domain specialists)
Tier 4: Domain Specialists
├─ cpu, memory, async, structure (Python)
└─ cpu, memory, async, structure, bundle (JavaScript)
Tools: SendMessage only (report to parent, no team authority)
Shared (language-agnostic):
├─ codeflash-researcher (read-only, parallel investigation)
└─ codeflash-review (standalone or post-session gate)
File Layout
Language-agnostic (base)
agents/codeflash.md— language router that detects the project language and delegatesagents/codeflash-review.md— review agent (works on any language)agents/codeflash-researcher.md— research agent (works on any language)commands/— codex CLI commandsvendor/codex/— codex companion scripts and schemas (vendored)references/shared/— shared methodology (experiment loop, templates, benchmarks)hooks/— session lifecycle and review gate hooks
Python (languages/python/)
agents/codeflash-python.md— Python domain router (Team Lead)agents/codeflash-deep.md— primary optimizer (profiles all dimensions jointly)agents/codeflash-cpu.md,-memory.md,-async.md,-structure.md— one agent per domainagents/codeflash-setup.md— detects project env, installs depsagents/codeflash-scan.md,-ci.md,-pr-prep.md— scan, CI, and PR preparationskills/—/codeflash-optimizeentry point, memray profilingreferences/— Python-specific protocol implementations + domain deep-dive docs
JavaScript (languages/javascript/)
agents/codeflash-javascript.md— JS/TS domain router (Team Lead)agents/codeflash-js-deep.md— primary optimizer (profiles all dimensions jointly)agents/codeflash-js-cpu.md,-memory.md,-async.md,-structure.md,-bundle.md— one agent per domain (5 domains)agents/codeflash-js-setup.md— detects runtime/package manager, installs depsagents/codeflash-js-scan.md,-ci.md,-pr-prep.md— scan, CI, and PR preparationskills/—/codeflash-optimizeentry point, V8 profiling referencereferences/— JS-specific references (Prisma performance, domain deep-dives)
Adding a New Language
Follow this template — Team Lead + Deep Agent are required, domain specialists are added as needed:
| Component | Required? | Role |
|---|---|---|
codeflash-<lang> |
Yes | Language Router / Team Lead (TeamCreate, Agent, SendMessage) |
codeflash-<lang>-deep |
Yes | Primary optimizer / Sub-Team Lead (TeamCreate, Agent, SendMessage) |
codeflash-<lang>-setup |
Yes | Environment setup utility (one-off, no team membership) |
codeflash-<lang>-cpu |
As needed | CPU/algorithm specialist (SendMessage only) |
codeflash-<lang>-memory |
As needed | Memory specialist (SendMessage only) |
codeflash-<lang>-async |
As needed | Concurrency specialist (SendMessage only) |
codeflash-<lang>-structure |
As needed | Module/import specialist (SendMessage only) |
| Language-specific domains | As needed | e.g., bundle for JS, compile for Go/Rust |
codeflash-<lang>-scan |
Optional | Quick diagnostic (one-off) |
codeflash-<lang>-ci |
Optional | CI webhook handler |
codeflash-<lang>-pr-prep |
Optional | PR preparation |
Shared agents (codeflash-researcher, codeflash-review) work across all languages.
Build
make build-plugin # default: LANG=python → dist/ is a Python-only plugin
make build-plugin LANG=javascript # dist/ is a JavaScript-only plugin
make clean # remove dist/
Each build produces a single-language plugin in dist/. The Makefile copies language-agnostic files from plugin/, overlays plugin/languages/<LANG>/ (agents, references, skills), and rewrites internal paths so everything is flat. You pick the language at build time — there is no multi-language dist yet, because loading all languages at once is extremely heavy on context and compute.