**Problem**: When a project uses `jest.config.ts` (TypeScript config), the
generated runtime config tries to `require('./jest.config.ts')`, which fails
because Node.js CommonJS cannot parse TypeScript syntax without compilation.
**Error**: `SyntaxError: Missing initializer in const declaration` at the
TypeScript type annotation (e.g., `const config: Config = ...`).
**Impact**: Affected 18 out of 38 optimization runs (~47%) in initial testing.
All TypeScript projects using `jest.config.ts` were unable to run tests.
**Root Cause**: Line 386 in test_runner.py used `base_config_path.name`
directly without checking the extension. The generated runtime config is
always a `.js` file, so it cannot use `require()` on `.ts` files.
**Solution**: Check if `base_config_path` is a TypeScript file (.ts). If so,
create a standalone runtime config without trying to extend it via require().
Jest will still discover and use the original TypeScript config naturally.
**Testing**:
- Added comprehensive test in test_jest_typescript_config_bug.py
- Test creates a realistic TypeScript Jest config and verifies the generated
runtime config loads without syntax errors
- Existing 34 JavaScript test runner tests still pass
- No linting/type errors from `uv run prek`
**Trace IDs affected**: 0fd176bf-5c7f-4f41-8396-77c46be86412 and 17 others
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
When JavaScript/TypeScript support generates test files in __tests__
subdirectories adjacent to source files (e.g., src/foo/__tests__/codeflash-generated/),
these test files are not within the configured tests_project_rootdir.
Previously, verifier.py:37 called module_name_from_file_path() without
handling the ValueError that occurs when the test path is outside tests_root,
causing optimization runs to crash.
This fix adds try-except handling with a fallback to using just the filename,
matching the pattern already used in javascript/parse.py:330-333.
Fixes trace ID: 84f5467f-8acf-427f-b468-02cb3342097e
Changes:
- codeflash/verification/verifier.py:37-48: Added try-except for path computation
- tests/verification/test_verifier_path_handling.py: Added unit tests
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Error messages in coverage_utils.py hardcoded "Jest" even when the
test framework was Vitest. This caused confusion in logs when Vitest
tests failed (e.g., "Jest coverage file not found" when using Vitest).
The JestCoverageUtils class is used for both Jest and Vitest since
they share the same Istanbul/v8 coverage format. Error messages
should be framework-agnostic.
Changes:
- "Jest coverage file not found" → "JavaScript coverage file not found"
- "Failed to parse Jest coverage file" → "Failed to parse JavaScript coverage file"
- "No coverage data found for X in Jest coverage" → "No coverage data found for X in JavaScript coverage"
- "Function X not found in Jest fnMap" → "Function X not found in JavaScript fnMap"
Affected trace IDs: 37e5a406, 735555fa, 940dfe80, c1e1de0e, dbec6c33, de96b1ab, fcf08c6b (7 logs from Apr 4 00:50 batch)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
## Problem
When Codeflash runs Vitest tests, coverage files are not collected even
though tests run successfully. This affects 14 out of ~20 optimization
runs, resulting in 0% coverage reported.
Root cause: The generated `codeflash.vitest.config.mjs` only overrides
`include`, `pool`, and `setupFiles`, but does NOT override coverage
settings. When the project's vitest.config.ts has custom coverage
settings (e.g., `reporter: ["text", "lcov"]`), Vitest's `mergeConfig()`
doesn't properly handle the nested coverage object merge with command-line
flags like `--coverage.reporter=json`, resulting in coverage files not
being written to the expected location.
## Solution
Explicitly override `coverage.reporter` to `['json']` in the generated
codeflash.vitest.config.mjs file. This ensures consistent behavior
regardless of Vitest version or project configuration.
## Testing
Added comprehensive unit tests in test_vitest_coverage_config.py that:
1. Verify coverage overrides are present in generated config
2. Test with and without existing project coverage settings
3. Confirm generated config includes expected coverage.reporter
All tests pass ✓
## References
- Trace IDs affected: 11f707d6, 69b271f5, and 12 others
- Related to iteration 2 investigation in fix_history.log
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
**Problem:**
1. Vitest tests were failing with 'Cannot find module .../test/setup.ts'
when testing functions in nested directories (e.g., extensions/discord/).
2. Root cause had two parts:
- _is_vitest_workspace() was doing substring search for 'workspace',
matching it even in comments, causing false positives
- Custom vitest config wasn't overriding setupFiles, leaving relative
paths from original config that resolved incorrectly
**Solution:**
1. Improved workspace detection (vitest_runner.py:172-191):
- Use regex to match actual workspace config patterns
- Match defineWorkspace( function calls
- Match workspace: [ property assignments
- Ignore 'workspace' in comments
2. Override setupFiles in custom config (vitest_runner.py:235-242):
- Set setupFiles: [] to disable project setup files
- Prevents relative path resolution issues
- Safe since Codeflash tests are self-contained
**Testing:**
Added test_vitest_setupfiles_fix.py with 2 test cases:
- Verifies setupFiles is overridden in generated config
- Verifies configs without setupFiles still work
**Trace ID:** 161e21be-9306-4a4d-a9dc-978f65a1af7a
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Replace `import shutil as _shutil` with plain `import shutil` to
match the existing style in the same function
- Warn when --inject is used with --script mode (unsupported combo)
instead of silently dropping the flag
When benchmarking already-merged optimizations, the benchmark file
often doesn't exist at either the base or head ref. The --inject flag
copies specified files/directories from the working tree into both
worktrees before benchmark discovery and execution, eliminating the
need to cherry-pick benchmark commits onto temporary branches.
Usage:
codeflash compare <base> <head> --inject tests/benchmarks/test_bench.py
- Rename `_get_js_project_root` → `get_js_project_root` (no leading underscores per convention)
- Remove redundant `from pathlib import Path` import inside method (already imported at top)
- Remove unnecessary docstring from new method
- Rewrite tests to use `tmp_path` fixture instead of `tempfile.TemporaryDirectory()`
- Add `.resolve()` calls and `encoding="utf-8"` per project conventions
- Simplify second test file to focus on the actual caching behavior
Co-authored-by: mohammed ahmed <undefined@users.noreply.github.com>
**Issue:**
When optimizing multiple functions in a monorepo with nested package.json
files (e.g., extensions/discord/package.json), the js_project_root was set
once for the first function and reused for all subsequent functions. This
caused vitest to look for setupFiles in the wrong directory.
**Root Cause:**
test_cfg.js_project_root was set during initial setup and never recalculated.
When function #1 was in extensions/discord/, all subsequent functions in
src/ inherited this wrong project root.
**Fix:**
- Added _get_js_project_root() method to FunctionOptimizer
- Calculate js_project_root fresh for each function using find_node_project_root()
- Updated all test execution paths (behavior, performance, line_profile)
**Impact:**
- Vitest now runs from the correct working directory for each function
- setupFiles can be resolved correctly
- Functions in different monorepo packages can be optimized correctly
Fixes trace IDs: 12d26b00-cbae-49a8-a3cd-c36024ee06ec, 1cde1c65-ef42-4072-afbc-165b0c235688, and 18 others
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Extended fix_jest_mock_paths() to handle vitest mock calls (vi.mock()) in
addition to jest.mock(). Previously, only jest.mock() paths were corrected,
causing vitest tests to fail with "Cannot find module" errors.
Problem:
- Source at src/agents/workspace.ts imports ../routing/session-key
- Generated test at test/test_workspace.test.ts used vi.mock('../routing/session-key')
- This resolves to /routing/session-key (wrong - goes up from test/, not found)
- Should be vi.mock('../src/routing/session-key') (correct path from test/)
Solution:
- Updated regex pattern to match both jest.mock() and vi.mock()
- Function now fixes relative paths for both test frameworks
- Added unit tests to verify both jest and vitest paths are corrected
Trace ID: 265059d4-f518-44da-8367-d90ca424092c
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The index-only approach missed tests for class methods (imported by class name),
aliased imports (only alias was tracked), and namespace imports (e.g. math.calculate).
Adds class_name→methods index, tracks both original+alias names, and extracts
namespace member access via regex.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This fixes a ValueError that occurs when generated tests are placed in
co-located __tests__ directories outside the configured tests_root.
Root cause:
The CLI's _find_codeflash_test_dir() method generates tests in co-located
__tests__ directories (e.g., src/gateway/server/__tests__/) when they exist,
but verifier.py tried to compute a module path relative to the configured
tests_root (e.g., /workspace/target/test), causing:
ValueError: '/workspace/target/src/gateway/server/__tests__/codeflash-generated/test_xxx.test.ts'
is not in the subpath of '/workspace/target/test'
Fix:
- Added traverse_up=True to module_name_from_file_path() call in verifier.py
- This allows the function to find a common ancestor directory and compute
the relative path from there, handling tests outside tests_root
Testing:
- Added comprehensive unit tests in test_module_name_from_file_path.py
- All existing tests pass ✅
- Linting passes ✅
Impact:
- Resolves crashes when optimizing functions with co-located test directories
- Enables proper handling of monorepo and __tests__ directory structures
Trace IDs affected: 7b97ddba-6ecd-42fd-b572-d40658746836
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fix Jest validation error that prevented all TypeScript/JavaScript optimizations
in projects using bundler moduleResolution.
## Problem
When optimizing JS/TS projects with bundler moduleResolution, Jest would fail with:
This blocked ALL optimizations for these projects since no tests could run.
## Root Cause
Two-stage Jest config generation caused conflicting options:
1. \_create_codeflash_jest_config() creates base config with `testRegex` (line 321)
2. \_create_runtime_jest_config() extends base config:
```javascript
module.exports = {
...baseConfig, // Spreads testRegex from base
testMatch: [...], // Adds testMatch - CONFLICT!
}
```
3. Jest sees both options and rejects the configuration
## Solution
Explicitly clear `testRegex` when setting `testMatch` in runtime config:
```javascript
testMatch: ['**/*.test.ts', ...],
testRegex: undefined, // Clear inherited testRegex
```
Jest config precedence allows explicit `undefined` to override inherited values.
## Testing
Before:
```
❌ Jest failed with returncode=1.
Validation Error: Configuration options testMatch and testRegex cannot be used together.
```
After:
```
✅ Jest runs without configuration errors
✅ Config validation passes
```
Tested on n8n packages/workflow which uses bundler moduleResolution.
## Impact
- Fixes all JS/TS projects that use:
- TypeScript with bundler moduleResolution (common in modern repos)
- Projects where Codeflash detects the need for ESM compatibility config
- Allows optimization to proceed past configuration stage
## Files Changed
- `codeflash/languages/javascript/test_runner.py` - _create_runtime_jest_config()
Fixes issue where TypeScript projects with bundler moduleResolution
would immediately fail Jest validation, preventing any optimizations.
Fix test discovery performance bottleneck that caused indefinite hangs on large codebases.
## Problem
The discover_tests() method had O(N×M) complexity where N is the number of test files
and M is the number of source functions. For large repos (e.g., n8n with 12,138 functions
and 5,502 test files), this created ~66 million iterations and caused the process to hang
indefinitely at the test discovery stage.
## Root Cause
Lines 258-265 iterated over ALL source functions for EVERY test file:
```python
for test_file in test_files: # N iterations
for func in source_functions: # M iterations per test file
if func.function_name in imported_names or func.function_name in source:
# map test to function
```
Additionally, the `func.function_name in source` check performed expensive string
containment searches on entire test files for every function, making it even slower.
## Solution
Rewrote algorithm to build a reverse index first, reducing complexity to O(N+M):
1. Build function_name → qualified_name dict once (O(M))
2. For each test file, only check imported names against the index (O(N))
This reduces iterations from ~66 million to ~17,640 for large repos.
## Performance Impact
Tested on n8n repository (12,138 functions, 5,502 test files):
- **Before**: Hung indefinitely (killed after 90+ seconds, never completed)
- **After**: 45.2 seconds total
- **Improvement**: 3,700x complexity reduction
Also removed the fallback `func.function_name in source` check as it was:
- Extremely expensive (substring search in entire file)
- Prone to false positives (matches in comments/strings)
- Unnecessary (functions must be imported to be used)
## Testing
- Verified on n8n repo: discovers 149,378 tests in 45s (previously hung)
- Verified on smaller repos: still works correctly with negligible overhead
Fixes performance issue where Codeflash would appear to hang after function discovery
when run with --all on large JavaScript/TypeScript monorepos.
PR #1968 changed JS to skip nested functions like Python, but the parity
test still expected JS to discover both outer and inner.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug #5 fix: The coverage exclusion error messages used
self.function_to_optimize.source_file_path but FunctionToOptimize
only has file_path attribute, not source_file_path. This caused
AttributeError when files were excluded from coverage.
Trace ID: 5c4a75fb-d8eb-4f75-9e57-893f0c44b9c7
Changes:
- Fixed lines 2797, 2803: source_file_path -> file_path
- Added regression test to verify correct attribute used
Testing:
- New test passes
- Linting passes
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The hot path shows `logger.debug` consuming 18.3% of original runtime despite appearing infrequently (141 hits), because formatting the f-string occurs unconditionally even when debug logging is disabled. Wrapping it with `logger.isEnabledFor(logging.DEBUG)` defers string construction until confirmed necessary, eliminating wasteful formatting. Replacing `lambda x: x[3]` with `operator.itemgetter(3)` in the sort key reduces per-comparison overhead from a Python function call to a C-level attribute access, and hoisting the division constant `1_000_000.0` outside the loop avoids repeated float literal construction. Line profiler confirms the sort line dropped from 568 µs to 197 µs (65% faster) and the debug call from 1102 µs to 124 µs (89% faster), yielding a 45% overall speedup with no correctness or metric trade-offs.
Issue: Test discovery incorrectly matched test files with source functions
when the function name appeared anywhere in the test file, including in
mocks, comments, or unrelated code. This caused 'Failed to instrument test
file' errors.
Root cause: In javascript/support.py line 259, naive substring matching
(func.function_name in source) matched function names even when they were
only mentioned in mocks like:
vi.mock('./file.js', () => ({ funcName: ... }))
Example: Function parseRestartRequestParams from restart-request.ts was
wrongly matched with update.test.ts because the test file mocked it.
Fix: Removed substring matching, now only matches explicitly imported
functions. This is more reliable and avoids false positives.
Trace ID: 0b575a96-62a8-4910-b163-1ad10e60ba79
Changes:
- Removed naive substring check in discover_tests()
- Only match functions that are explicitly imported
- Added regression tests (2 test cases)
Testing:
- All 70 JavaScript tests pass
- New tests verify fix works correctly
- Linting/type checks pass (uv run prek)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
## Problem
When a file is excluded from coverage by vitest.config.ts (e.g., via
`coverage.exclude: ["src/agents/**"]`), Codeflash reports misleading
"Test coverage is 0.0%" messages even though tests run successfully.
This happens because:
- Vitest doesn't include excluded files in coverage-final.json
- Codeflash detects this (status = NOT_FOUND) but shows generic 0% message
- Users don't know the file is excluded from coverage collection
## Solution
Detect when coverage status is NOT_FOUND and provide a clear, actionable
error message explaining:
1. No coverage data was found for the file
2. It may be excluded by test framework configuration
3. Where to check (coverage.exclude in vitest.config.ts, etc.)
## Changes
- function_optimizer.py: Check CoverageStatus.NOT_FOUND before reporting 0%
- Added clear warning log and user-facing error message
- New test file: test_vitest_coverage_exclusions.py
## Testing
- All existing JavaScript tests pass
- New tests verify NOT_FOUND status is returned correctly
- Manual verification with openclaw logs (trace: 2a84fe6b-9871-4916-96da-bdd79bca508a)
Fixes #BUG-1 (from autoresearch:debug workflow)
Trace IDs affected: All 10 log files showing 0% coverage in /workspace/logs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Allows running arbitrary benchmark scripts on both git refs and
rendering a styled comparison table. Supports optional --memory
via memray wrapping. No codeflash config required for script mode.
When --memory is used and no changed top-level functions are detected,
skip trace benchmarking but still run memray profiling. This fixes the
class method limitation where codeflash compare couldn't profile memory
for changes in class methods (which are excluded from @codeflash_trace
instrumentation due to pickle overhead).
