**Problem:**
When a project has @babel/core installed but no TypeScript transformer
(ts-jest, @swc/jest, etc.), Codeflash generated a Jest config with no
transform directive. Jest then defaulted to babel-jest (since @babel/core
was present), but without @babel/preset-typescript, Babel failed to
transform TypeScript files with error:
SyntaxError: Support for the experimental syntax 'flow' isn't currently enabled
This affected all TypeScript projects using Babel without explicit TypeScript
transformer configuration.
**Solution:**
- Added _ensure_babel_preset_typescript() function that checks if the preset
is available and installs it if needed
- Modified _detect_typescript_transformer() to use this fallback when @babel/core
exists but no TypeScript transformer is configured
- The fix only applies when @babel/core is present, avoiding unnecessary installs
**Impact:**
- Fixes 4/7 (57%) of current optimization failures in budibase monorepo
- Systematic fix for all TypeScript projects with Babel but no TS transformer
**Testing:**
- Added 4 new unit tests in test_typescript_babel_fallback.py
- All 34 existing JavaScript test runner tests pass
- No linting/type errors
**Related Traces:**
- 26117bae-39bb-4f2f-9047-f2eb6594b7eb
- 5562089f-85e9-4a6d-b790-260bcd9316cb
- c2f741b0-7eaa-4c93-b839-3832c46a3a34
- ec5e20f3-31cc-4bb4-bef2-990ee509c2b1
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Replace `import shutil as _shutil` with plain `import shutil` to
match the existing style in the same function
- Warn when --inject is used with --script mode (unsupported combo)
instead of silently dropping the flag
When benchmarking already-merged optimizations, the benchmark file
often doesn't exist at either the base or head ref. The --inject flag
copies specified files/directories from the working tree into both
worktrees before benchmark discovery and execution, eliminating the
need to cherry-pick benchmark commits onto temporary branches.
Usage:
codeflash compare <base> <head> --inject tests/benchmarks/test_bench.py
The index-only approach missed tests for class methods (imported by class name),
aliased imports (only alias was tracked), and namespace imports (e.g. math.calculate).
Adds class_name→methods index, tracks both original+alias names, and extracts
namespace member access via regex.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix Jest validation error that prevented all TypeScript/JavaScript optimizations
in projects using bundler moduleResolution.
## Problem
When optimizing JS/TS projects with bundler moduleResolution, Jest would fail with:
This blocked ALL optimizations for these projects since no tests could run.
## Root Cause
Two-stage Jest config generation caused conflicting options:
1. \_create_codeflash_jest_config() creates base config with `testRegex` (line 321)
2. \_create_runtime_jest_config() extends base config:
```javascript
module.exports = {
...baseConfig, // Spreads testRegex from base
testMatch: [...], // Adds testMatch - CONFLICT!
}
```
3. Jest sees both options and rejects the configuration
## Solution
Explicitly clear `testRegex` when setting `testMatch` in runtime config:
```javascript
testMatch: ['**/*.test.ts', ...],
testRegex: undefined, // Clear inherited testRegex
```
Jest config precedence allows explicit `undefined` to override inherited values.
## Testing
Before:
```
❌ Jest failed with returncode=1.
Validation Error: Configuration options testMatch and testRegex cannot be used together.
```
After:
```
✅ Jest runs without configuration errors
✅ Config validation passes
```
Tested on n8n packages/workflow which uses bundler moduleResolution.
## Impact
- Fixes all JS/TS projects that use:
- TypeScript with bundler moduleResolution (common in modern repos)
- Projects where Codeflash detects the need for ESM compatibility config
- Allows optimization to proceed past configuration stage
## Files Changed
- `codeflash/languages/javascript/test_runner.py` - _create_runtime_jest_config()
Fixes issue where TypeScript projects with bundler moduleResolution
would immediately fail Jest validation, preventing any optimizations.
Fix test discovery performance bottleneck that caused indefinite hangs on large codebases.
## Problem
The discover_tests() method had O(N×M) complexity where N is the number of test files
and M is the number of source functions. For large repos (e.g., n8n with 12,138 functions
and 5,502 test files), this created ~66 million iterations and caused the process to hang
indefinitely at the test discovery stage.
## Root Cause
Lines 258-265 iterated over ALL source functions for EVERY test file:
```python
for test_file in test_files: # N iterations
for func in source_functions: # M iterations per test file
if func.function_name in imported_names or func.function_name in source:
# map test to function
```
Additionally, the `func.function_name in source` check performed expensive string
containment searches on entire test files for every function, making it even slower.
## Solution
Rewrote algorithm to build a reverse index first, reducing complexity to O(N+M):
1. Build function_name → qualified_name dict once (O(M))
2. For each test file, only check imported names against the index (O(N))
This reduces iterations from ~66 million to ~17,640 for large repos.
## Performance Impact
Tested on n8n repository (12,138 functions, 5,502 test files):
- **Before**: Hung indefinitely (killed after 90+ seconds, never completed)
- **After**: 45.2 seconds total
- **Improvement**: 3,700x complexity reduction
Also removed the fallback `func.function_name in source` check as it was:
- Extremely expensive (substring search in entire file)
- Prone to false positives (matches in comments/strings)
- Unnecessary (functions must be imported to be used)
## Testing
- Verified on n8n repo: discovers 149,378 tests in 45s (previously hung)
- Verified on smaller repos: still works correctly with negligible overhead
Fixes performance issue where Codeflash would appear to hang after function discovery
when run with --all on large JavaScript/TypeScript monorepos.
PR #1968 changed JS to skip nested functions like Python, but the parity
test still expected JS to discover both outer and inner.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug #5 fix: The coverage exclusion error messages used
self.function_to_optimize.source_file_path but FunctionToOptimize
only has file_path attribute, not source_file_path. This caused
AttributeError when files were excluded from coverage.
Trace ID: 5c4a75fb-d8eb-4f75-9e57-893f0c44b9c7
Changes:
- Fixed lines 2797, 2803: source_file_path -> file_path
- Added regression test to verify correct attribute used
Testing:
- New test passes
- Linting passes
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The hot path shows `logger.debug` consuming 18.3% of original runtime despite appearing infrequently (141 hits), because formatting the f-string occurs unconditionally even when debug logging is disabled. Wrapping it with `logger.isEnabledFor(logging.DEBUG)` defers string construction until confirmed necessary, eliminating wasteful formatting. Replacing `lambda x: x[3]` with `operator.itemgetter(3)` in the sort key reduces per-comparison overhead from a Python function call to a C-level attribute access, and hoisting the division constant `1_000_000.0` outside the loop avoids repeated float literal construction. Line profiler confirms the sort line dropped from 568 µs to 197 µs (65% faster) and the debug call from 1102 µs to 124 µs (89% faster), yielding a 45% overall speedup with no correctness or metric trade-offs.
Issue: Test discovery incorrectly matched test files with source functions
when the function name appeared anywhere in the test file, including in
mocks, comments, or unrelated code. This caused 'Failed to instrument test
file' errors.
Root cause: In javascript/support.py line 259, naive substring matching
(func.function_name in source) matched function names even when they were
only mentioned in mocks like:
vi.mock('./file.js', () => ({ funcName: ... }))
Example: Function parseRestartRequestParams from restart-request.ts was
wrongly matched with update.test.ts because the test file mocked it.
Fix: Removed substring matching, now only matches explicitly imported
functions. This is more reliable and avoids false positives.
Trace ID: 0b575a96-62a8-4910-b163-1ad10e60ba79
Changes:
- Removed naive substring check in discover_tests()
- Only match functions that are explicitly imported
- Added regression tests (2 test cases)
Testing:
- All 70 JavaScript tests pass
- New tests verify fix works correctly
- Linting/type checks pass (uv run prek)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
## Problem
When a file is excluded from coverage by vitest.config.ts (e.g., via
`coverage.exclude: ["src/agents/**"]`), Codeflash reports misleading
"Test coverage is 0.0%" messages even though tests run successfully.
This happens because:
- Vitest doesn't include excluded files in coverage-final.json
- Codeflash detects this (status = NOT_FOUND) but shows generic 0% message
- Users don't know the file is excluded from coverage collection
## Solution
Detect when coverage status is NOT_FOUND and provide a clear, actionable
error message explaining:
1. No coverage data was found for the file
2. It may be excluded by test framework configuration
3. Where to check (coverage.exclude in vitest.config.ts, etc.)
## Changes
- function_optimizer.py: Check CoverageStatus.NOT_FOUND before reporting 0%
- Added clear warning log and user-facing error message
- New test file: test_vitest_coverage_exclusions.py
## Testing
- All existing JavaScript tests pass
- New tests verify NOT_FOUND status is returned correctly
- Manual verification with openclaw logs (trace: 2a84fe6b-9871-4916-96da-bdd79bca508a)
Fixes #BUG-1 (from autoresearch:debug workflow)
Trace IDs affected: All 10 log files showing 0% coverage in /workspace/logs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Allows running arbitrary benchmark scripts on both git refs and
rendering a styled comparison table. Supports optional --memory
via memray wrapping. No codeflash config required for script mode.
When --memory is used and no changed top-level functions are detected,
skip trace benchmarking but still run memray profiling. This fixes the
class method limitation where codeflash compare couldn't profile memory
for changes in class methods (which are excluded from @codeflash_trace
instrumentation due to pickle overhead).
**Problem:**
When running Codeflash-generated tests with coverage enabled, Vitest would
fail with returncode=1 due to project-level coverage thresholds not being met.
Generated tests typically cover only a single function (~1-2% of codebase),
which fails projects with thresholds like 70% lines/functions configured in
their vitest.config.ts.
**Root Cause:**
In vitest_runner.py line 450, Codeflash was adding --coverage flag without
disabling the project's global coverage thresholds. This caused false failures
even when all tests passed successfully.
**Solution:**
Added coverage threshold override flags when coverage is enabled:
- --coverage.thresholds.lines=0
- --coverage.thresholds.functions=0
- --coverage.thresholds.statements=0
- --coverage.thresholds.branches=0
These flags disable project-level thresholds, allowing coverage collection
without failing the test run. Coverage data is still collected for analysis,
but thresholds no longer cause false failures.
**Testing:**
- Added comprehensive unit tests in test_vitest_coverage_thresholds.py
- All 40 existing vitest-related tests pass
- Verified with uv run prek (linter + type checker)
**Related Issues:**
Trace IDs affected: 05a626f3, 932e7799, a145328d, aa9bb63f, d669202e, e6de097a
Fixes 6 out of 10 optimization failures in openclaw project.
The benchmark plugin now runs multiple rounds with calibrated
iterations. Tests need SELECT DISTINCT for row counts and must
extract median_ns from BenchmarkStats before validation.
Bug: Nested functions were being discovered and attempted to be optimized,
but the extraction logic only captured the nested function body without
parent scope variables, causing validation errors like:
'Undefined variable(s): base, streamFn, record, writer'
Root cause: The discover_functions method was allowing nested functions
(functions defined inside other functions) to be marked for optimization.
These nested functions depend on closure variables from their parent scope
and cannot be optimized in isolation.
Fix: Added explicit check to skip functions with parent_function set.
Nested functions are now filtered out during discovery phase.
Impact: Resolves 140+ trace failures with undefined variable errors.
Functions like 'wrapStreamFn.wrapped' will no longer be attempted.
Test: Added test_discover_functions.py with 4 test cases:
- test_discovers_top_level_function
- test_skips_nested_functions_in_closures (main bug fix test)
- test_discovers_class_methods (ensure methods still work)
- test_skips_nested_functions_with_multiple_levels
Affects trace IDs including: 02a59310-bb18-47e4-87cb-1e5144ce2d8c
and 140+ others with nested function extraction issues.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add -w flag for pnpm workspace roots to avoid ERR_PNPM_ADDING_TO_ROOT
- Use local package path (/opt/codeflash/packages/codeflash) in dev mode
- Improve error logging to show actual stderr at ERROR level instead of WARNING
- Add unit tests for workspace detection and local package usage
Fixes 9/13 optimization failures caused by 'Cannot find package codeflash'
Trace IDs affected: 08d594a2, 1722cff7, 23480bf7, 3074f19b, 6043236e,
b883f1bd, d01b03ce, e56507a4, f8f54e06
## Problem
The inject_test_globals() function was adding duplicate framework imports,
causing parse errors like "Identifier 'describe' has already been declared".
This occurred when:
1. AI service generated tests WITH vitest imports
2. CLI called inject_test_globals() which added its own import
3. String-based duplicate check failed because identifiers had different order
Result: TWO import statements declaring the same identifiers → parse error.
## Solution
Replace string-based duplicate detection with regex-based detection that
catches ANY import from the framework, regardless of identifier order.
## Changes
- Added regex patterns for Vitest, Jest, and Mocha imports
- Modified inject_test_globals() to use regex search
- Added comprehensive tests in test_inject_test_globals_duplicate.py
## Impact
Fixes HIGH severity bug blocking test generation for all Vitest projects
## Trace IDs
- 03a5a9d9-8e56-47e8-9c5e-0160fb9a529a
- 0be70f8d-884e-45e4-8fa2-28ed40cdf068
- 29c6d314-8561-4bb4-9b77-00b3b83943f0
