codeflash-agent/plugin/languages/python/references/micro-benchmark.md

Micro-Benchmark — Python

Python-specific micro-benchmark tooling. Read ${CLAUDE_PLUGIN_ROOT}/references/shared/micro-benchmark.md first for the language-agnostic framework.

A/B Template

# /tmp/micro_bench_<name>.py
import sys

def bench_a():
    """Current approach."""
    # ... original code with real input

def bench_b():
    """Optimized approach."""
    # ... optimized code with same input

if __name__ == "__main__":
    {"a": bench_a, "b": bench_b}[sys.argv[1]]()

Running — by Domain

• Memory: memray run --native --trace-python-allocators -o /tmp/micro_{a,b}.bin /tmp/micro_bench_<name>.py {a,b} then memray stats
• CPU / Data Structures: wrap with timeit.timeit(fn, number=1000) inside the script
• Async: wrap with asyncio.run(fn()) and time.perf_counter() for wall-clock
• Structure: timeit.timeit(bench_import, number=10) with sys.modules cache clearing

$RUNNER /tmp/micro_bench_<name>.py a
$RUNNER /tmp/micro_bench_<name>.py b

Adapt commands for the project's specific setup (virtualenv, PYTHONPATH, working directory, etc.).

Domain Thresholds

Keep on micro-benchmark alone if above these thresholds:

• Memory: >10 MiB or >10%
• CPU / Data Structures: >20% or >2x, hot path confirmed by cProfile
• Async: >20% or >2x at representative concurrency