Merge remote-tracking branch 'origin/main' into fix/gradle-classpath-missing-jars

This commit is contained in:
HeshamHM28 2026-04-09 08:16:49 +00:00
commit ee70689573
51 changed files with 1387 additions and 173 deletions

View file

@ -21,7 +21,7 @@ codeflash/
├── api/ # AI service communication ├── api/ # AI service communication
├── code_utils/ # Code parsing, git utilities ├── code_utils/ # Code parsing, git utilities
├── models/ # Pydantic models and types ├── models/ # Pydantic models and types
├── languages/ # Multi-language support (Python, JavaScript/TypeScript, Java planned) ├── languages/ # Multi-language support (Python, JavaScript/TypeScript, Java)
│ ├── base.py # LanguageSupport protocol and shared data types │ ├── base.py # LanguageSupport protocol and shared data types
│ ├── registry.py # Language registration and lookup by extension/enum │ ├── registry.py # Language registration and lookup by extension/enum
│ ├── current.py # Current language singleton (set_current_language / current_language_support) │ ├── current.py # Current language singleton (set_current_language / current_language_support)
@ -35,11 +35,29 @@ codeflash/
│ │ ├── test_runner.py # Test subprocess execution for Python │ │ ├── test_runner.py # Test subprocess execution for Python
│ │ ├── instrument_codeflash_capture.py # Instrument __init__ with capture decorators │ │ ├── instrument_codeflash_capture.py # Instrument __init__ with capture decorators
│ │ └── parse_line_profile_test_output.py # Parse line profiler output │ │ └── parse_line_profile_test_output.py # Parse line profiler output
│ └── javascript/ │ ├── javascript/
│ ├── support.py # JavaScriptSupport (LanguageSupport implementation) │ │ ├── support.py # JavaScriptSupport (LanguageSupport implementation)
│ ├── function_optimizer.py # JavaScriptFunctionOptimizer subclass │ │ ├── function_optimizer.py # JavaScriptFunctionOptimizer subclass
│ ├── optimizer.py # JS project root finding & module preparation │ │ ├── optimizer.py # JS project root finding & module preparation
│ └── normalizer.py # JS/TS code normalization for deduplication │ │ └── normalizer.py # JS/TS code normalization for deduplication
│ └── java/
│ ├── support.py # JavaSupport (LanguageSupport implementation)
│ ├── function_optimizer.py # JavaFunctionOptimizer subclass
│ ├── build_tool_strategy.py # Abstract BuildToolStrategy for Maven/Gradle
│ ├── maven_strategy.py # Maven build tool strategy
│ ├── gradle_strategy.py # Gradle build tool strategy
│ ├── build_tools.py # Build tool detection and project info
│ ├── build_config_strategy.py # Config read/write for pom.xml / gradle.properties
│ ├── test_runner.py # Test execution via Maven/Gradle
│ ├── instrumentation.py # Behavior capture and benchmarking instrumentation
│ ├── discovery.py # Function discovery using tree-sitter
│ ├── test_discovery.py # Test discovery for JUnit/TestNG
│ ├── context.py # Code context extraction
│ ├── comparator.py # Test result comparison
│ ├── config.py # Java project detection and config
│ ├── formatter.py # Code formatting and normalization
│ ├── line_profiler.py # JVM bytecode agent-based line profiling
│ └── tracer.py # Two-stage JFR + argument capture tracer
├── setup/ # Config schema, auto-detection, first-run experience ├── setup/ # Config schema, auto-detection, first-run experience
├── picklepatch/ # Serialization/deserialization utilities ├── picklepatch/ # Serialization/deserialization utilities
├── tracing/ # Function call tracing ├── tracing/ # Function call tracing
@ -57,7 +75,7 @@ codeflash/
|------|------------| |------|------------|
| CLI arguments & commands | `cli_cmds/cli.py` (parsing), `main.py` (subcommand dispatch) | | CLI arguments & commands | `cli_cmds/cli.py` (parsing), `main.py` (subcommand dispatch) |
| Optimization orchestration | `optimization/optimizer.py``run()` | | Optimization orchestration | `optimization/optimizer.py``run()` |
| Per-function optimization | `languages/function_optimizer.py` (base), `languages/python/function_optimizer.py`, `languages/javascript/function_optimizer.py` | | Per-function optimization | `languages/function_optimizer.py` (base), `languages/python/function_optimizer.py`, `languages/javascript/function_optimizer.py`, `languages/java/function_optimizer.py` |
| Function discovery | `discovery/functions_to_optimize.py` | | Function discovery | `discovery/functions_to_optimize.py` |
| Context extraction | `languages/<lang>/context/code_context_extractor.py` | | Context extraction | `languages/<lang>/context/code_context_extractor.py` |
| Test execution | `languages/<lang>/support.py` (`run_behavioral_tests`, etc.), `verification/pytest_plugin.py` | | Test execution | `languages/<lang>/support.py` (`run_behavioral_tests`, etc.), `verification/pytest_plugin.py` |
@ -67,7 +85,7 @@ codeflash/
## LanguageSupport Protocol Methods ## LanguageSupport Protocol Methods
Core protocol in `languages/base.py`. Each language (`PythonSupport`, `JavaScriptSupport`) implements these. Core protocol in `languages/base.py`. Each language (`PythonSupport`, `JavaScriptSupport`, `JavaSupport`) implements these.
| Category | Method/Property | Purpose | | Category | Method/Property | Purpose |
|----------|----------------|---------| |----------|----------------|---------|

22
.github/dependabot.yml vendored Normal file
View file

@ -0,0 +1,22 @@
version: 2
updates:
# Python (root pyproject.toml)
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
# JavaScript (codeflash npm package)
- package-ecosystem: "npm"
directory: "/packages/codeflash"
schedule:
interval: "weekly"
# GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
# code_to_optimize/ directories are test fixtures — do NOT update them.
# Dependabot PRs for these always fail (missing secrets) and waste CI.

View file

@ -68,7 +68,7 @@ jobs:
- name: Run Claude Code - name: Run Claude Code
id: claude id: claude
uses: anthropics/claude-code-action@v1 uses: anthropics/claude-code-action@v1.0.89
with: with:
use_bedrock: "true" use_bedrock: "true"
use_sticky_comment: true use_sticky_comment: true
@ -328,7 +328,7 @@ jobs:
- name: Run Claude Code - name: Run Claude Code
id: claude id: claude
uses: anthropics/claude-code-action@v1 uses: anthropics/claude-code-action@v1.0.89
with: with:
use_bedrock: "true" use_bedrock: "true"
claude_args: '--model us.anthropic.claude-sonnet-4-6 --allowedTools "Read,Edit,Write,Glob,Grep,Bash(git status*),Bash(git diff*),Bash(git add *),Bash(git commit *),Bash(git push*),Bash(git log*),Bash(git merge*),Bash(git fetch*),Bash(git checkout*),Bash(git branch*),Bash(uv run prek *),Bash(prek *),Bash(uv run ruff *),Bash(uv run pytest *),Bash(uv run mypy *),Bash(uv run coverage *),Bash(gh pr comment*),Bash(gh pr view*),Bash(gh pr diff*),Bash(gh pr merge*),Bash(gh pr close*)"' claude_args: '--model us.anthropic.claude-sonnet-4-6 --allowedTools "Read,Edit,Write,Glob,Grep,Bash(git status*),Bash(git diff*),Bash(git add *),Bash(git commit *),Bash(git push*),Bash(git log*),Bash(git merge*),Bash(git fetch*),Bash(git checkout*),Bash(git branch*),Bash(uv run prek *),Bash(prek *),Bash(uv run ruff *),Bash(uv run pytest *),Bash(uv run mypy *),Bash(uv run coverage *),Bash(gh pr comment*),Bash(gh pr view*),Bash(gh pr diff*),Bash(gh pr merge*),Bash(gh pr close*)"'

View file

@ -3,7 +3,11 @@ name: E2E - Async
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Bubble Sort Benchmark
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Bubble Sort Pytest (No Git)
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Bubble Sort Unittest
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: Coverage E2E
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Futurehouse Structure
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Init Optimization
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:
concurrency: concurrency:

View file

@ -3,7 +3,12 @@ name: E2E - JS CommonJS Function
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'packages/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,12 @@ name: E2E - JS ESM Async
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'packages/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,12 @@ name: E2E - JS TypeScript Class
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'packages/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Topological Sort (Worktree)
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:

View file

@ -3,7 +3,11 @@ name: E2E - Tracer Replay
on: on:
pull_request: pull_request:
paths: paths:
- '**' # Trigger for all paths - 'codeflash/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/e2e-*.yaml'
workflow_dispatch: workflow_dispatch:
concurrency: concurrency:

View file

@ -211,6 +211,7 @@ def compare_branches(
functions: Optional[dict[Path, list[FunctionToOptimize]]] = None, functions: Optional[dict[Path, list[FunctionToOptimize]]] = None,
timeout: int = 600, timeout: int = 600,
memory: bool = False, memory: bool = False,
inject_paths: Optional[list[str]] = None,
) -> CompareResult: ) -> CompareResult:
"""Compare benchmark performance between two git refs. """Compare benchmark performance between two git refs.
@ -343,6 +344,24 @@ def compare_branches(
head_sha = repo.commit(head_ref).hexsha head_sha = repo.commit(head_ref).hexsha
repo.git.worktree("add", str(base_worktree), base_sha) repo.git.worktree("add", str(base_worktree), base_sha)
repo.git.worktree("add", str(head_worktree), head_sha) repo.git.worktree("add", str(head_worktree), head_sha)
# Inject files from working tree into both worktrees
if inject_paths:
import shutil
for path_str in inject_paths:
src = repo_root / path_str
if not src.exists():
logger.warning("Inject path does not exist: %s", src)
continue
for wt in [base_worktree, head_worktree]:
dst = wt / path_str
dst.parent.mkdir(parents=True, exist_ok=True)
if src.is_dir():
shutil.copytree(src, dst, dirs_exist_ok=True)
elif src.is_file():
shutil.copy2(src, dst)
step += 1 step += 1
live.update(build_panel(step)) live.update(build_panel(step))

View file

@ -404,6 +404,12 @@ def _build_parser() -> ArgumentParser:
help="Relative path to JSON results file produced by --script (required with --script)", help="Relative path to JSON results file produced by --script (required with --script)",
) )
compare_parser.add_argument("--config-file", type=str, dest="config_file", help="Path to pyproject.toml") compare_parser.add_argument("--config-file", type=str, dest="config_file", help="Path to pyproject.toml")
compare_parser.add_argument(
"--inject",
nargs="+",
default=None,
help="Files or directories to copy into both worktrees before benchmarking. Paths are relative to repo root.",
)
trace_optimize.add_argument( trace_optimize.add_argument(
"--max-function-count", "--max-function-count",

View file

@ -40,6 +40,9 @@ def run_compare(args: Namespace) -> None:
# Script mode: run an arbitrary benchmark command on each worktree (no codeflash config needed) # Script mode: run an arbitrary benchmark command on each worktree (no codeflash config needed)
script_cmd = getattr(args, "script", None) script_cmd = getattr(args, "script", None)
if script_cmd: if script_cmd:
if getattr(args, "inject", None):
logger.warning("--inject is not supported in --script mode and will be ignored")
script_output = getattr(args, "script_output", None) script_output = getattr(args, "script_output", None)
if not script_output: if not script_output:
logger.error("--script-output is required when using --script") logger.error("--script-output is required when using --script")
@ -108,6 +111,7 @@ def run_compare(args: Namespace) -> None:
functions=functions, functions=functions,
timeout=args.timeout, timeout=args.timeout,
memory=getattr(args, "memory", False), memory=getattr(args, "memory", False),
inject_paths=getattr(args, "inject", None),
) )
if not result.base_stats and not result.head_stats: if not result.base_stats and not result.head_stats:

View file

@ -17,7 +17,7 @@ import tomlkit
from codeflash.cli_cmds.console import logger, paneled_text from codeflash.cli_cmds.console import logger, paneled_text
from codeflash.code_utils.config_parser import find_pyproject_toml, get_all_closest_config_files from codeflash.code_utils.config_parser import find_pyproject_toml, get_all_closest_config_files
from codeflash.lsp.helpers import is_LSP_enabled from codeflash.lsp.helpers import is_LSP_enabled, is_subagent_mode
_INVALID_CHARS_NT = {"<", ">", ":", '"', "|", "?", "*"} _INVALID_CHARS_NT = {"<", ">", ":", '"', "|", "?", "*"}
@ -471,6 +471,11 @@ def exit_with_message(message: str, *, error_on_exit: bool = False) -> None:
if is_LSP_enabled(): if is_LSP_enabled():
logger.error(message) logger.error(message)
return return
if is_subagent_mode():
from xml.sax.saxutils import escape
sys.stdout.write(f"<codeflash-error>{escape(message)}</codeflash-error>\n")
sys.exit(1 if error_on_exit else 0)
paneled_text(message, panel_args={"style": "red"}) paneled_text(message, panel_args={"style": "red"})
sys.exit(1 if error_on_exit else 0) sys.exit(1 if error_on_exit else 0)

View file

@ -3085,6 +3085,16 @@ class FunctionOptimizer:
) )
) )
def get_js_project_root(self) -> Path | None:
# Only calculate for JavaScript/TypeScript projects
if self.function_to_optimize.language not in ("javascript", "typescript"):
return self.test_cfg.js_project_root # Fall back to cached value for non-JS
# For JS/TS, calculate fresh for each function to support monorepos
from codeflash.languages.javascript.test_runner import find_node_project_root
return find_node_project_root(Path(self.function_to_optimize.file_path))
def run_and_parse_tests( def run_and_parse_tests(
self, self,
testing_type: TestingMode, testing_type: TestingMode,
@ -3103,33 +3113,39 @@ class FunctionOptimizer:
coverage_config_file = None coverage_config_file = None
try: try:
if testing_type == TestingMode.BEHAVIOR: if testing_type == TestingMode.BEHAVIOR:
# Calculate js_project_root for the current function being optimized
# instead of using cached value from test_cfg, which may be from a different function
js_project_root = self.get_js_project_root()
result_file_path, run_result, coverage_database_file, coverage_config_file = ( result_file_path, run_result, coverage_database_file, coverage_config_file = (
self.language_support.run_behavioral_tests( self.language_support.run_behavioral_tests(
test_paths=test_files, test_paths=test_files,
test_env=test_env, test_env=test_env,
cwd=self.project_root, cwd=self.project_root,
timeout=INDIVIDUAL_TESTCASE_TIMEOUT, timeout=INDIVIDUAL_TESTCASE_TIMEOUT,
project_root=self.test_cfg.js_project_root, project_root=js_project_root,
enable_coverage=enable_coverage, enable_coverage=enable_coverage,
candidate_index=optimization_iteration, candidate_index=optimization_iteration,
) )
) )
elif testing_type == TestingMode.LINE_PROFILE: elif testing_type == TestingMode.LINE_PROFILE:
js_project_root = self.get_js_project_root()
result_file_path, run_result = self.language_support.run_line_profile_tests( result_file_path, run_result = self.language_support.run_line_profile_tests(
test_paths=test_files, test_paths=test_files,
test_env=test_env, test_env=test_env,
cwd=self.project_root, cwd=self.project_root,
timeout=INDIVIDUAL_TESTCASE_TIMEOUT, timeout=INDIVIDUAL_TESTCASE_TIMEOUT,
project_root=self.test_cfg.js_project_root, project_root=js_project_root,
line_profile_output_file=line_profiler_output_file, line_profile_output_file=line_profiler_output_file,
) )
elif testing_type == TestingMode.PERFORMANCE: elif testing_type == TestingMode.PERFORMANCE:
js_project_root = self.get_js_project_root()
result_file_path, run_result = self.language_support.run_benchmarking_tests( result_file_path, run_result = self.language_support.run_benchmarking_tests(
test_paths=test_files, test_paths=test_files,
test_env=test_env, test_env=test_env,
cwd=self.project_root, cwd=self.project_root,
timeout=INDIVIDUAL_TESTCASE_TIMEOUT, timeout=INDIVIDUAL_TESTCASE_TIMEOUT,
project_root=self.test_cfg.js_project_root, project_root=js_project_root,
min_loops=pytest_min_loops, min_loops=pytest_min_loops,
max_loops=pytest_max_loops, max_loops=pytest_max_loops,
target_duration_seconds=testing_time, target_duration_seconds=testing_time,

View file

@ -45,7 +45,8 @@ gradle.projectsEvaluated {
'spotbugsMain', 'spotbugsTest', 'spotbugsMain', 'spotbugsTest',
'pmdMain', 'pmdTest', 'pmdMain', 'pmdTest',
'rat', 'japicmp', 'rat', 'japicmp',
'jarHell', 'thirdPartyAudit' 'jarHell', 'thirdPartyAudit',
'spotlessCheck', 'spotlessApply', 'spotlessJava', 'spotlessKotlin', 'spotlessScala'
] ]
}.configureEach { }.configureEach {
enabled = false enabled = false

View file

@ -43,6 +43,8 @@ _MAVEN_VALIDATION_SKIP_FLAGS = [
"-Denforcer.skip=true", "-Denforcer.skip=true",
"-Djapicmp.skip=true", "-Djapicmp.skip=true",
"-Derrorprone.skip=true", "-Derrorprone.skip=true",
"-Dspotless.check.skip=true",
"-Dspotless.apply.skip=true",
"-Dmaven.compiler.failOnWarning=false", "-Dmaven.compiler.failOnWarning=false",
"-Dmaven.compiler.showWarnings=false", "-Dmaven.compiler.showWarnings=false",
] ]

View file

@ -1287,13 +1287,13 @@ def fix_imports_inside_test_blocks(test_code: str) -> str:
def fix_jest_mock_paths(test_code: str, test_file_path: Path, source_file_path: Path, tests_root: Path) -> str: def fix_jest_mock_paths(test_code: str, test_file_path: Path, source_file_path: Path, tests_root: Path) -> str:
"""Fix relative paths in jest.mock() calls to be correct from the test file's location. """Fix relative paths in jest.mock() and vi.mock() calls to be correct from the test file's location.
The AI sometimes generates jest.mock() calls with paths relative to the source file The AI sometimes generates mock calls with paths relative to the source file
instead of the test file. For example: instead of the test file. For example:
- Source at `src/queue/queue.ts` imports `../environment` (-> src/environment) - Source at `src/queue/queue.ts` imports `../environment` (-> src/environment)
- Test at `tests/test.test.ts` generates `jest.mock('../environment')` (-> ./environment, wrong!) - Test at `tests/test.test.ts` generates `jest.mock('../environment')` or `vi.mock('../environment')` (-> ./environment, wrong!)
- Should generate `jest.mock('../src/environment')` - Should generate `jest.mock('../src/environment')` or `vi.mock('../src/environment')`
This function detects relative mock paths and adjusts them based on the test file's This function detects relative mock paths and adjusts them based on the test file's
location relative to the source file's directory. location relative to the source file's directory.
@ -1318,8 +1318,8 @@ def fix_jest_mock_paths(test_code: str, test_file_path: Path, source_file_path:
test_dir = test_file_path.resolve().parent test_dir = test_file_path.resolve().parent
project_root = tests_root.resolve().parent if tests_root.name == "tests" else tests_root.resolve() project_root = tests_root.resolve().parent if tests_root.name == "tests" else tests_root.resolve()
# Pattern to match jest.mock() or jest.doMock() with relative paths # Pattern to match jest.mock(), jest.doMock(), or vi.mock() with relative paths
mock_pattern = re.compile(r"(jest\.(?:mock|doMock)\s*\(\s*['\"])(\.\./[^'\"]+|\.\/[^'\"]+)(['\"])") mock_pattern = re.compile(r"((?:jest|vi)\.(?:mock|doMock)\s*\(\s*['\"])(\.\./[^'\"]+|\.\/[^'\"]+)(['\"])")
def fix_mock_path(match: re.Match[str]) -> str: def fix_mock_path(match: re.Match[str]) -> str:
original = match.group(0) original = match.group(0)
@ -1359,7 +1359,7 @@ def fix_jest_mock_paths(test_code: str, test_file_path: Path, source_file_path:
if not new_rel_path.startswith("../") and not new_rel_path.startswith("./"): if not new_rel_path.startswith("../") and not new_rel_path.startswith("./"):
new_rel_path = f"./{new_rel_path}" new_rel_path = f"./{new_rel_path}"
logger.debug(f"Fixed jest.mock path: {rel_path} -> {new_rel_path}") logger.debug(f"Fixed mock path: {rel_path} -> {new_rel_path}")
return f"{prefix}{new_rel_path}{suffix}" return f"{prefix}{new_rel_path}{suffix}"
except (ValueError, OSError): except (ValueError, OSError):

View file

@ -7,6 +7,7 @@ using tree-sitter for code analysis and Jest for test execution.
from __future__ import annotations from __future__ import annotations
import logging import logging
import re
import subprocess import subprocess
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
from pathlib import Path from pathlib import Path
@ -230,11 +231,14 @@ class JavaScriptSupport:
""" """
result: dict[str, list[TestInfo]] = {} result: dict[str, list[TestInfo]] = {}
# Build index: function_name → qualified_name for O(1) lookup # Build indices for O(1) lookup per imported name (avoids O(NxM) loop)
# This avoids iterating all functions for every test file (was O(NxM), now O(N+M))
function_name_to_qualified: dict[str, str] = {} function_name_to_qualified: dict[str, str] = {}
class_name_to_qualified_names: dict[str, list[str]] = {}
for func in source_functions: for func in source_functions:
function_name_to_qualified[func.function_name] = func.qualified_name function_name_to_qualified[func.function_name] = func.qualified_name
for parent in func.parents:
if parent.type == "ClassDef":
class_name_to_qualified_names.setdefault(parent.name, []).append(func.qualified_name)
# Find all test files using language-specific patterns # Find all test files using language-specific patterns
test_patterns = self._get_test_patterns() test_patterns = self._get_test_patterns()
@ -249,28 +253,41 @@ class JavaScriptSupport:
analyzer = get_analyzer_for_file(test_file) analyzer = get_analyzer_for_file(test_file)
imports = analyzer.find_imports(source) imports = analyzer.find_imports(source)
# Build a set of imported function names # Build a set of imported names, resolving aliases and namespace member access
imported_names: set[str] = set() imported_names: set[str] = set()
for imp in imports: for imp in imports:
if imp.default_import: if imp.default_import:
imported_names.add(imp.default_import) imported_names.add(imp.default_import)
# Extract member access patterns: e.g. `math.calculate(...)` → "calculate"
for m in re.finditer(rf"\b{re.escape(imp.default_import)}\.(\w+)", source):
imported_names.add(m.group(1))
if imp.namespace_import:
imported_names.add(imp.namespace_import)
for m in re.finditer(rf"\b{re.escape(imp.namespace_import)}\.(\w+)", source):
imported_names.add(m.group(1))
for name, alias in imp.named_imports: for name, alias in imp.named_imports:
imported_names.add(alias or name) imported_names.add(name)
if alias:
imported_names.add(alias)
# Find test functions (describe/it/test blocks) # Find test functions (describe/it/test blocks)
test_functions = self._find_jest_tests(source, analyzer) test_functions = self._find_jest_tests(source, analyzer)
# Match source functions to tests using the index # Match via indices: function names and class names → qualified names
# Only check functions that are actually imported in this test file matched_qualified_names: set[str] = set()
for imported_name in imported_names: for imported_name in imported_names:
if imported_name in function_name_to_qualified: if imported_name in function_name_to_qualified:
qualified_name = function_name_to_qualified[imported_name] matched_qualified_names.add(function_name_to_qualified[imported_name])
if qualified_name not in result: if imported_name in class_name_to_qualified_names:
result[qualified_name] = [] matched_qualified_names.update(class_name_to_qualified_names[imported_name])
for test_name in test_functions:
result[qualified_name].append( for qualified_name in matched_qualified_names:
TestInfo(test_name=test_name, test_file=test_file, test_class=None) if qualified_name not in result:
) result[qualified_name] = []
for test_name in test_functions:
result[qualified_name].append(
TestInfo(test_name=test_name, test_file=test_file, test_class=None)
)
except Exception as e: except Exception as e:
logger.debug("Failed to analyze test file %s: %s", test_file, e) logger.debug("Failed to analyze test file %s: %s", test_file, e)
@ -2251,7 +2268,10 @@ class JavaScriptSupport:
source_without_ext = source_file_abs.with_suffix("") source_without_ext = source_file_abs.with_suffix("")
# Use os.path.relpath to compute relative path from tests_root to source file # Use os.path.relpath to compute relative path from tests_root to source file
rel_path = os.path.relpath(str(source_without_ext), str(tests_root_abs)) # Replace backslashes with forward slashes — JavaScript import/require paths
# must use forward slashes. Backslashes are escape chars in JS strings
# (e.g. \t → tab, \n → newline) and would break imports on Windows.
rel_path = os.path.relpath(str(source_without_ext), str(tests_root_abs)).replace("\\", "/")
# For ESM, add .js extension (TypeScript convention) # For ESM, add .js extension (TypeScript convention)
# TypeScript requires imports to reference the OUTPUT file extension (.js), # TypeScript requires imports to reference the OUTPUT file extension (.js),

View file

@ -369,7 +369,9 @@ def _create_runtime_jest_config(base_config_path: Path | None, project_root: Pat
runtime_config_path = config_dir / f"jest.codeflash.runtime.config{config_ext}" runtime_config_path = config_dir / f"jest.codeflash.runtime.config{config_ext}"
test_dirs_js = ", ".join(f"'{d}'" for d in sorted(test_dirs)) # Normalize to forward slashes — backslashes in JS strings are escape chars
# (e.g. \t → tab, \n → newline) and would corrupt paths on Windows.
test_dirs_js = ", ".join(f"'{d.replace(chr(92), '/')}'" for d in sorted(test_dirs))
# In monorepos, add the root node_modules to moduleDirectories so Jest # In monorepos, add the root node_modules to moduleDirectories so Jest
# can resolve workspace packages that are hoisted to the monorepo root. # can resolve workspace packages that are hoisted to the monorepo root.
@ -382,7 +384,13 @@ def _create_runtime_jest_config(base_config_path: Path | None, project_root: Pat
else: else:
module_dirs_line_no_base = "" module_dirs_line_no_base = ""
if base_config_path: project_root_posix = project_root.as_posix()
# TypeScript configs (.ts) cannot be required from CommonJS modules
# because Node.js cannot parse TypeScript syntax in require().
# When the base config is TypeScript, we create a standalone config
# instead of trying to extend it via require().
if base_config_path and base_config_path.suffix != ".ts":
require_path = f"./{base_config_path.name}" require_path = f"./{base_config_path.name}"
config_content = f"""// Auto-generated by codeflash - runtime config with test roots config_content = f"""// Auto-generated by codeflash - runtime config with test roots
const baseConfig = require('{require_path}'); const baseConfig = require('{require_path}');
@ -399,7 +407,7 @@ module.exports = {{
else: else:
config_content = f"""// Auto-generated by codeflash - runtime config with test roots config_content = f"""// Auto-generated by codeflash - runtime config with test roots
module.exports = {{ module.exports = {{
roots: ['{project_root}', {test_dirs_js}], roots: ['{project_root_posix}', {test_dirs_js}],
testMatch: ['**/*.test.ts', '**/*.test.js', '**/*.test.tsx', '**/*.test.jsx'], testMatch: ['**/*.test.ts', '**/*.test.js', '**/*.test.tsx', '**/*.test.jsx'],
{module_dirs_line_no_base}}}; {module_dirs_line_no_base}}};
""" """

View file

@ -7,6 +7,7 @@ verification and performance benchmarking.
from __future__ import annotations from __future__ import annotations
import os import os
import re
import subprocess import subprocess
import time import time
from pathlib import Path from pathlib import Path
@ -169,9 +170,24 @@ def _is_vitest_workspace(project_root: Path) -> bool:
return False return False
try: try:
content = vitest_config.read_text() content = vitest_config.read_text(encoding="utf-8")
# Check for workspace indicators # Check for actual workspace configuration patterns (not just the word "workspace" in comments)
return "workspace" in content.lower() or "defineWorkspace" in content # Valid indicators:
# - defineWorkspace() function call
# - workspace: [ array config
# - separate vitest.workspace.ts/js file
# Match defineWorkspace calls or workspace: property assignments
workspace_pattern = re.compile(
r"(?:^|[^a-zA-Z_])defineWorkspace\s*\(|" # defineWorkspace( function call
r"(?:^|[^a-zA-Z_])workspace\s*:\s*\[", # workspace: [ array
re.MULTILINE,
)
if workspace_pattern.search(content):
return True
# Also check for separate workspace config file
if (project_root / "vitest.workspace.ts").exists() or (project_root / "vitest.workspace.js").exists():
return True
return False
except Exception: except Exception:
return False return False
@ -238,6 +254,18 @@ export default mergeConfig(originalConfig, {{
include: ['**/*.test.ts', '**/*.test.js', '**/*.test.tsx', '**/*.test.jsx'], include: ['**/*.test.ts', '**/*.test.js', '**/*.test.tsx', '**/*.test.jsx'],
// Use forks pool so timing markers from process.stdout.write flow to parent stdout // Use forks pool so timing markers from process.stdout.write flow to parent stdout
pool: 'forks', pool: 'forks',
// Disable setupFiles to prevent relative path resolution issues in nested directories.
// Project setupFiles often use relative paths (e.g., "test/setup.ts") which resolve
// incorrectly when tests are in subdirectories (e.g., extensions/discord/test/).
// Codeflash-generated tests are self-contained and don't require project setup files.
setupFiles: [],
// Override coverage settings to ensure JSON reporter is used.
// Vitest's mergeConfig doesn't properly handle nested coverage object merge with
// command-line flags, so we explicitly set reporter here to guarantee coverage
// files are written to the expected location (coverage-final.json).
coverage: {{
reporter: ['json'],
}},
}}, }},
}}); }});
""" """
@ -254,6 +282,10 @@ export default defineConfig({
exclude: ['**/node_modules/**', '**/dist/**'], exclude: ['**/node_modules/**', '**/dist/**'],
// Use forks pool so timing markers from process.stdout.write flow to parent stdout // Use forks pool so timing markers from process.stdout.write flow to parent stdout
pool: 'forks', pool: 'forks',
// Override coverage settings to ensure JSON reporter is used
coverage: {
reporter: ['json'],
},
}, },
}); });
""" """

View file

@ -7,6 +7,7 @@ from posthog import Posthog
from codeflash.api.cfapi import get_user_id from codeflash.api.cfapi import get_user_id
from codeflash.cli_cmds.console import logger from codeflash.cli_cmds.console import logger
from codeflash.lsp.helpers import is_subagent_mode
from codeflash.version import __version__ from codeflash.version import __version__
_posthog = None _posthog = None
@ -36,7 +37,7 @@ def ph(event: str, properties: dict[str, Any] | None = None) -> None:
return return
properties = properties or {} properties = properties or {}
properties.update({"cli_version": __version__}) properties.update({"cli_version": __version__, "subagent": is_subagent_mode()})
user_id = get_user_id() user_id = get_user_id()

View file

@ -43,30 +43,33 @@ class JestCoverageUtils:
""" """
if not coverage_json_path or not coverage_json_path.exists(): if not coverage_json_path or not coverage_json_path.exists():
logger.debug(f"Jest coverage file not found: {coverage_json_path}") logger.debug(f"JavaScript coverage file not found: {coverage_json_path}")
return CoverageData.create_empty(source_code_path, function_name, code_context) return CoverageData.create_empty(source_code_path, function_name, code_context)
try: try:
with coverage_json_path.open(encoding="utf-8") as f: with coverage_json_path.open(encoding="utf-8") as f:
coverage_data = json.load(f) coverage_data = json.load(f)
except (json.JSONDecodeError, OSError) as e: except (json.JSONDecodeError, OSError) as e:
logger.warning(f"Failed to parse Jest coverage file: {e}") logger.warning(f"Failed to parse JavaScript coverage file: {e}")
return CoverageData.create_empty(source_code_path, function_name, code_context) return CoverageData.create_empty(source_code_path, function_name, code_context)
# Find the file entry in coverage data # Find the file entry in coverage data
# Jest uses absolute paths as keys # Jest/Vitest always writes coverage keys with forward slashes (POSIX paths),
# so we normalize our paths to POSIX for comparison — critical on Windows
# where Path.resolve() and str(Path) produce backslash paths.
file_coverage = None file_coverage = None
source_path_str = str(source_code_path.resolve()) source_path_posix = source_code_path.resolve().as_posix()
source_relative_posix = source_code_path.as_posix()
for file_path, file_data in coverage_data.items(): for file_path, file_data in coverage_data.items():
# Match exact path or path ending with full relative path from src/ # Match exact path or path ending with full relative path from src/
# Avoid matching files with same name in different directories (e.g., db/utils.ts vs utils/utils.ts) # Avoid matching files with same name in different directories (e.g., db/utils.ts vs utils/utils.ts)
if file_path == source_path_str or file_path.endswith(str(source_code_path)): if file_path == source_path_posix or file_path.endswith(source_relative_posix):
file_coverage = file_data file_coverage = file_data
break break
if not file_coverage: if not file_coverage:
logger.debug(f"No coverage data found for {source_code_path} in Jest coverage") logger.debug(f"No coverage data found for {source_code_path} in JavaScript coverage")
return CoverageData.create_empty(source_code_path, function_name, code_context) return CoverageData.create_empty(source_code_path, function_name, code_context)
# Extract line coverage from statement map and execution counts # Extract line coverage from statement map and execution counts
@ -94,7 +97,7 @@ class JestCoverageUtils:
# If function not found in fnMap, use entire file # If function not found in fnMap, use entire file
fn_start_line = 1 fn_start_line = 1
fn_end_line = 999999 fn_end_line = 999999
logger.debug(f"Function {function_name} not found in Jest fnMap, using file coverage") logger.debug(f"Function {function_name} not found in JavaScript fnMap, using file coverage")
# Calculate executed and unexecuted lines within the function # Calculate executed and unexecuted lines within the function
executed_lines = [] executed_lines = []

View file

@ -34,7 +34,20 @@ def generate_tests(
# TODO: Sometimes this recreates the original Class definition. This overrides and messes up the original # TODO: Sometimes this recreates the original Class definition. This overrides and messes up the original
# class import. Remove the recreation of the class definition # class import. Remove the recreation of the class definition
start_time = time.perf_counter() start_time = time.perf_counter()
test_module_path = Path(module_name_from_file_path(test_path, test_cfg.tests_project_rootdir))
# Compute test module path - handle case where test file is outside tests_project_rootdir
# (e.g., JavaScript/TypeScript tests generated in __tests__ subdirectories adjacent to source files)
# Similar to javascript/parse.py:330-333 fallback pattern
try:
# Use traverse_up=True to handle co-located __tests__ directories that may be outside
# the configured tests_root (e.g., src/gateway/__tests__/ when tests_root is test/)
test_module_path = Path(module_name_from_file_path(test_path, test_cfg.tests_project_rootdir, traverse_up=True))
except ValueError:
# Test file is not within tests_project_rootdir - use just the filename
# This can happen for JavaScript/TypeScript when get_test_dir_for_source()
# places tests adjacent to source files (e.g., in src/foo/__tests__/)
# instead of within the configured tests_root
test_module_path = Path(test_path.name)
# Detect module system via language support (non-None for JS/TS, None for Python) # Detect module system via language support (non-None for JS/TS, None for Python)
lang_support = current_language_support() lang_support = current_language_support()

View file

@ -29,7 +29,7 @@ Flags can be combined: `/optimize src/utils.py my_function`
### What happens behind the scenes ### What happens behind the scenes
1. The skill (defined in `skills/optimize/SKILL.md`) forks context and spawns the **optimizer agent** 1. The skill (defined in `skills/optimize/SKILL.md`) forks context and spawns the **optimizer agent**
2. The agent locates your project config (`pyproject.toml` or `package.json` or `codeflash.toml`) 2. The agent locates your project config (`pyproject.toml`, `package.json`, or `pom.xml`/`gradle.properties`)
3. It verifies the codeflash CLI is installed and the project is configured 3. It verifies the codeflash CLI is installed and the project is configured
4. It runs `codeflash --subagent` as a **background task** with a 10-minute timeout 4. It runs `codeflash --subagent` as a **background task** with a 10-minute timeout
5. You're notified when optimization completes with results 5. You're notified when optimization completes with results

View file

@ -3,20 +3,20 @@ title: "How Codeflash Works"
description: "Understand Codeflash's generate-and-verify approach to code optimization and correctness verification" description: "Understand Codeflash's generate-and-verify approach to code optimization and correctness verification"
icon: "gear" icon: "gear"
sidebarTitle: "How It Works" sidebarTitle: "How It Works"
keywords: ["architecture", "verification", "correctness", "testing", "optimization", "LLM", "benchmarking", "javascript", "typescript", "python"] keywords: ["architecture", "verification", "correctness", "testing", "optimization", "LLM", "benchmarking", "javascript", "typescript", "python", "java"]
--- ---
# How Codeflash Works # How Codeflash Works
Codeflash follows a "generate and verify" approach to optimize code. It uses LLMs to generate optimizations, then it rigorously verifies if those optimizations are indeed Codeflash follows a "generate and verify" approach to optimize code. It uses LLMs to generate optimizations, then it rigorously verifies if those optimizations are indeed
faster and if they have the same behavior. The basic unit of optimization is a function—Codeflash tries to speed up the function, and tries to ensure that it still behaves the same way. This way if you merge the optimized code, it simply runs faster without breaking any functionality. faster and if they have the same behavior. The basic unit of optimization is a function—Codeflash tries to speed up the function, and tries to ensure that it still behaves the same way. This way if you merge the optimized code, it simply runs faster without breaking any functionality.
Codeflash supports **Python**, **JavaScript**, and **TypeScript** projects. Codeflash supports **Python**, **JavaScript**, **TypeScript**, and **Java** projects.
## Analysis of your code ## Analysis of your code
Codeflash scans your codebase to identify all available functions. It locates existing unit tests in your projects and maps which functions they test. When optimizing a function, Codeflash runs these discovered tests to verify nothing has broken. Codeflash scans your codebase to identify all available functions. It locates existing unit tests in your projects and maps which functions they test. When optimizing a function, Codeflash runs these discovered tests to verify nothing has broken.
For Python, code analysis uses `libcst` and `jedi`. For JavaScript/TypeScript, it uses `tree-sitter` for AST parsing. For Python, code analysis uses `libcst` and `jedi`. For JavaScript/TypeScript and Java, it uses `tree-sitter` for AST parsing.
#### What kind of functions can Codeflash optimize? #### What kind of functions can Codeflash optimize?
@ -25,7 +25,7 @@ Codeflash supports optimizing async functions in all supported languages.
#### Test Discovery #### Test Discovery
Codeflash discovers tests that directly call the target function in their test body. For Python, it finds pytest and unittest tests. For JavaScript/TypeScript, it finds Jest and Vitest test files. Codeflash discovers tests that directly call the target function in their test body. For Python, it finds pytest and unittest tests. For JavaScript/TypeScript, it finds Jest and Vitest test files. For Java, it finds JUnit 5, JUnit 4, and TestNG test classes.
To discover tests that indirectly call the function, you can use the Codeflash Tracer. The Tracer analyzes your test suite and identifies all tests that eventually call a function. To discover tests that indirectly call the function, you can use the Codeflash Tracer. The Tracer analyzes your test suite and identifies all tests that eventually call a function.
@ -54,12 +54,12 @@ We recommend manually reviewing the optimized code since there might be importan
Codeflash generates two types of tests: Codeflash generates two types of tests:
- **LLM Generated tests** - Codeflash uses LLMs to create several regression test cases that cover typical function usage, edge cases, and large-scale inputs to verify both correctness and performance. This works for Python, JavaScript, and TypeScript. - **LLM Generated tests** - Codeflash uses LLMs to create several regression test cases that cover typical function usage, edge cases, and large-scale inputs to verify both correctness and performance. This works for Python, JavaScript, TypeScript, and Java.
- **Concolic coverage tests** - Codeflash uses state-of-the-art concolic testing with an SMT Solver (a theorem prover) to explore execution paths and generate function arguments. This aims to maximize code coverage for the function being optimized. Currently, this feature only supports Python (pytest). - **Concolic coverage tests** - Codeflash uses state-of-the-art concolic testing with an SMT Solver (a theorem prover) to explore execution paths and generate function arguments. This aims to maximize code coverage for the function being optimized. Currently, this feature only supports Python (pytest).
## Code Execution ## Code Execution
Codeflash runs tests for the target function on your machine. For Python, it uses pytest or unittest. For JavaScript/TypeScript, it uses Jest or Vitest. Running on your machine ensures access to your environment and dependencies, and provides accurate performance measurements since runtime varies by system. Codeflash runs tests for the target function on your machine. For Python, it uses pytest or unittest. For JavaScript/TypeScript, it uses Jest or Vitest. For Java, it uses Maven Surefire or Gradle's test task. Running on your machine ensures access to your environment and dependencies, and provides accurate performance measurements since runtime varies by system.
#### Performance benchmarking #### Performance benchmarking

View file

@ -1,43 +1,52 @@
--- ---
title: "Java Configuration" title: "Java Configuration"
description: "Configure Codeflash for Java projects using codeflash.toml" description: "Configure Codeflash for Java projects"
icon: "java" icon: "java"
sidebarTitle: "Java (codeflash.toml)" sidebarTitle: "Java"
keywords: keywords:
[ [
"configuration", "configuration",
"codeflash.toml",
"java", "java",
"maven", "maven",
"gradle", "gradle",
"junit", "junit",
"pom.xml",
"gradle.properties",
] ]
--- ---
# Java Configuration # Java Configuration
Codeflash stores its configuration in `codeflash.toml` under the `[tool.codeflash]` section. Codeflash stores its configuration inside your existing build file — `pom.xml` properties for Maven projects, or `gradle.properties` for Gradle projects. No separate config file is needed.
## Full Reference ## Maven Configuration
```toml For Maven projects, Codeflash writes properties under the `<properties>` section of your `pom.xml` with the `codeflash.` prefix:
[tool.codeflash]
# Required
module-root = "src/main/java"
tests-root = "src/test/java"
language = "java"
# Optional ```xml
test-framework = "junit5" # "junit5", "junit4", or "testng" <properties>
disable-telemetry = false <!-- Only non-default overrides are written -->
git-remote = "origin" <codeflash.moduleRoot>src/main/java</codeflash.moduleRoot>
ignore-paths = ["src/main/java/generated/"] <codeflash.testsRoot>src/test/java</codeflash.testsRoot>
<codeflash.gitRemote>origin</codeflash.gitRemote>
<codeflash.formatterCmds>mvn spotless:apply -DspotlessFiles=$file</codeflash.formatterCmds>
<codeflash.disableTelemetry>false</codeflash.disableTelemetry>
<codeflash.ignorePaths>src/main/java/generated/</codeflash.ignorePaths>
</properties>
``` ```
All file paths are relative to the directory containing `codeflash.toml`. ## Gradle Configuration
For Gradle projects, Codeflash writes settings to `gradle.properties` with the `codeflash.` prefix:
```properties
codeflash.moduleRoot=src/main/java
codeflash.testsRoot=src/test/java
codeflash.gitRemote=origin
```
<Info> <Info>
Codeflash auto-detects most settings from your project structure. Running `codeflash init` will set up the correct config — manual configuration is usually not needed. Codeflash auto-detects most settings from your project structure. Running `codeflash init` will set up the correct config — manual configuration is usually not needed. For standard Maven/Gradle layouts, Codeflash may write no config at all if all defaults are correct.
</Info> </Info>
## Auto-Detection ## Auto-Detection
@ -46,54 +55,42 @@ When you run `codeflash init`, Codeflash inspects your project and auto-detects:
| Setting | Detection logic | | Setting | Detection logic |
|---------|----------------| |---------|----------------|
| `module-root` | Looks for `src/main/java` (Maven/Gradle standard layout) | | **Source root** | Looks for `src/main/java` (Maven/Gradle standard layout), falls back to pom.xml `sourceDirectory` |
| `tests-root` | Looks for `src/test/java`, `test/`, `tests/` | | **Test root** | Looks for `src/test/java`, `test/`, `tests/` |
| `language` | Detected from build files (`pom.xml`, `build.gradle`) and `.java` files | | **Build tool** | Detects Maven (`pom.xml`) or Gradle (`build.gradle` / `build.gradle.kts`) |
| `test-framework` | Checks build file dependencies for JUnit 5, JUnit 4, or TestNG | | **Test framework** | Checks build file dependencies for JUnit 5, JUnit 4, or TestNG |
## Required Options ## Configuration Options
- **`module-root`**: The source directory to optimize. Only code under this directory is discovered for optimization. For standard Maven/Gradle projects, this is `src/main/java`. | Property | Description | Default |
- **`tests-root`**: The directory where your tests are located. Codeflash discovers existing tests and places generated replay tests here. |----------|-------------|---------|
- **`language`**: Must be set to `"java"` for Java projects. | `moduleRoot` | Source directory to optimize | `src/main/java` |
| `testsRoot` | Test directory | `src/test/java` |
| `gitRemote` | Git remote for pull requests | `origin` |
| `formatterCmds` | Code formatter command (`$file` placeholder for file path) | (none) |
| `disableTelemetry` | Disable anonymized telemetry | `false` |
| `ignorePaths` | Paths within source root to skip during optimization | (none) |
## Optional Options <Info>
Only non-default values are written to the config. If your project uses the standard `src/main/java` and `src/test/java` layout with the default `origin` remote, Codeflash may not need to write any config properties at all.
- **`test-framework`**: Test framework. Auto-detected from build dependencies. Supported values: `"junit5"` (default), `"junit4"`, `"testng"`. </Info>
- **`disable-telemetry`**: Disable anonymized telemetry. Defaults to `false`.
- **`git-remote`**: Git remote for pull requests. Defaults to `"origin"`.
- **`ignore-paths`**: Paths within `module-root` to skip during optimization.
## Multi-Module Projects ## Multi-Module Projects
For multi-module Maven/Gradle projects, place `codeflash.toml` at the project root and set `module-root` to the module you want to optimize: For multi-module Maven/Gradle projects, run `codeflash init` from the module you want to optimize. The config is written to that module's `pom.xml` or `gradle.properties`:
```text ```text
my-project/ my-project/
|- client/ |- client/
| |- src/main/java/com/example/client/ | |- src/main/java/com/example/client/
| |- src/test/java/com/example/client/ | |- src/test/java/com/example/client/
| |- pom.xml <-- run codeflash init here
|- server/ |- server/
| |- src/main/java/com/example/server/ | |- src/main/java/com/example/server/
|- pom.xml |- pom.xml
|- codeflash.toml
``` ```
```toml For non-standard layouts (like the Aerospike client where source is under `client/src/`), `codeflash init` will prompt you to override the detected paths.
[tool.codeflash]
module-root = "client/src/main/java"
tests-root = "client/src/test/java"
language = "java"
```
For non-standard layouts (like the Aerospike client where source is under `client/src/`), adjust paths accordingly:
```toml
[tool.codeflash]
module-root = "client/src"
tests-root = "test/src"
language = "java"
```
## Tracer Options ## Tracer Options
@ -124,15 +121,9 @@ my-app/
| |- test/java/com/example/ | |- test/java/com/example/
| |- AppTest.java | |- AppTest.java
|- pom.xml |- pom.xml
|- codeflash.toml
``` ```
```toml Standard layout — no extra config needed. `codeflash init` detects everything automatically.
[tool.codeflash]
module-root = "src/main/java"
tests-root = "src/test/java"
language = "java"
```
### Gradle project ### Gradle project
@ -142,12 +133,7 @@ my-lib/
| |- main/java/com/example/ | |- main/java/com/example/
| |- test/java/com/example/ | |- test/java/com/example/
|- build.gradle |- build.gradle
|- codeflash.toml |- gradle.properties <-- codeflash config written here if overrides needed
``` ```
```toml Standard layout — no extra config needed. `codeflash init` detects everything automatically.
[tool.codeflash]
module-root = "src/main/java"
tests-root = "src/test/java"
language = "java"
```

View file

@ -47,7 +47,38 @@ uv tool install codeflash
``` ```
</Step> </Step>
<Step title="Initialize your project"> <Step title="Authenticate with Codeflash">
Codeflash uses cloud-hosted AI models. You need to authenticate before running any commands.
**Option A: Browser login (recommended)**
```bash
codeflash auth login
```
This opens your browser to sign in with your GitHub account. Your API key is saved automatically to your shell profile.
If you're on a remote server without a browser, a URL will be displayed that you can open on any device.
**Option B: API key**
1. Visit the [Codeflash Web App](https://app.codeflash.ai/) and sign up with your GitHub account (free tier available)
2. Navigate to the [API Key](https://app.codeflash.ai/app/apikeys) page to generate your key
3. Set it as an environment variable:
```bash
export CODEFLASH_API_KEY="your-api-key-here"
```
Add this to your shell profile (`~/.bashrc`, `~/.zshrc`) so it persists across sessions.
<Info>
If you skip this step, `codeflash init` will prompt you to authenticate interactively.
</Info>
</Step>
<Step title="Initialize your project (recommended)">
Navigate to your Java project root (where `pom.xml` or `build.gradle` is) and run: Navigate to your Java project root (where `pom.xml` or `build.gradle` is) and run:
@ -55,32 +86,63 @@ Navigate to your Java project root (where `pom.xml` or `build.gradle` is) and ru
codeflash init codeflash init
``` ```
This will: The init command will:
- Detect your build tool (Maven/Gradle) 1. **Auto-detect your project** — find your build tool, source root (e.g., `src/main/java`), test root (e.g., `src/test/java`), and test framework
- Find your source and test directories 2. **Confirm settings** — show the detected values and ask if you want to change anything
- Update your pom.xml or gradle settings with codeflash java library. The java library instruments your code and verifies correctness. 3. **Configure formatter** — let you set up a code formatter (e.g., Spotless, google-java-format)
4. **Install GitHub App** — offer to set up the [Codeflash GitHub App](https://github.com/apps/codeflash-ai/installations/select_target) for automatic PR creation (see next step)
5. **Install GitHub Actions** — offer to add a CI workflow for automated optimization on PRs
Only non-default settings are written to your `pom.xml` properties (Maven) or `gradle.properties` (Gradle). For standard layouts, no config changes are needed.
<Info>
**Can I skip init?** Yes. For standard Maven/Gradle projects, Codeflash auto-detects your project structure from `pom.xml` or `build.gradle` at runtime. If you're already authenticated and your project uses a standard layout (`src/main/java`, `src/test/java`), you can skip straight to optimizing.
Init is recommended because it also sets up the GitHub App and Actions workflow, and lets you override paths for non-standard project layouts (e.g., multi-module projects where source is under `client/src/`).
</Info>
</Step>
<Step title="Install the Codeflash GitHub App (recommended)">
To have Codeflash create pull requests with optimizations automatically, install the GitHub App:
[Install Codeflash GitHub App](https://github.com/apps/codeflash-ai/installations/select_target)
Select the repositories you want Codeflash to optimize. This allows the codeflash-ai bot to open PRs with optimization suggestions in your repository.
<Info>
If you prefer to try Codeflash locally first, you can skip this step and use the `--no-pr` flag to apply optimizations directly to your local files (see next step).
</Info>
</Step> </Step>
<Step title="Run your first optimization"> <Step title="Run your first optimization">
Trace and optimize a Java program: Optimize a specific function:
```bash ```bash
codeflash optimize java -jar target/my-app.jar codeflash --file src/main/java/com/example/Utils.java --function myMethod
``` ```
Or with Maven: If you installed the GitHub App, Codeflash will create a pull request with the optimization. If you haven't installed the app yet, or prefer to review changes locally first, add `--no-pr`:
```bash ```bash
codeflash optimize mvn exec:java -Dexec.mainClass="com.example.Main" codeflash --file src/main/java/com/example/Utils.java --function myMethod --no-pr
```
Or optimize all functions in your project:
```bash
codeflash --all
``` ```
Codeflash will: Codeflash will:
1. Profile your program using JFR (Java Flight Recorder) 1. Discover optimizable functions in your source code
2. Capture method arguments using a bytecode instrumentation agent 2. Generate tests and optimization candidates using AI
3. Generate JUnit replay tests from the captured data to create a micro-benchmark. 3. Verify correctness by running tests (JUnit 5, JUnit 4, or TestNG)
4. Rank functions by performance impact 4. Benchmark performance improvements
5. Optimize the most impactful functions 5. Create a pull request with the optimization (or apply locally with `--no-pr`)
For advanced workflow tracing (profiling a running Java program), see [Trace & Optimize](/optimizing-with-codeflash/trace-and-optimize).
</Step> </Step>
</Steps> </Steps>

View file

@ -2,11 +2,11 @@
title: "Codeflash is an AI performance optimizer for your code" title: "Codeflash is an AI performance optimizer for your code"
icon: "rocket" icon: "rocket"
sidebarTitle: "Overview" sidebarTitle: "Overview"
keywords: ["python", "javascript", "typescript", "performance", "optimization", "AI", "code analysis", "benchmarking"] keywords: ["python", "javascript", "typescript", "java", "performance", "optimization", "AI", "code analysis", "benchmarking"]
--- ---
Codeflash speeds up your code by figuring out the best way to rewrite it while verifying that the behavior is unchanged, and verifying real speed Codeflash speeds up your code by figuring out the best way to rewrite it while verifying that the behavior is unchanged, and verifying real speed
gains through performance benchmarking. It supports **Python**, **JavaScript**, and **TypeScript**. gains through performance benchmarking. It supports **Python**, **JavaScript**, **TypeScript**, and **Java**.
The optimizations Codeflash finds are generally better algorithms, opportunities to remove wasteful compute, better logic, utilizing caching and utilization of more efficient library methods. Codeflash The optimizations Codeflash finds are generally better algorithms, opportunities to remove wasteful compute, better logic, utilizing caching and utilization of more efficient library methods. Codeflash
does not modify the system architecture of your code, but it tries to find the most efficient implementation of your current architecture. does not modify the system architecture of your code, but it tries to find the most efficient implementation of your current architecture.
@ -15,18 +15,21 @@ does not modify the system architecture of your code, but it tries to find the m
Pick your language to install and configure Codeflash: Pick your language to install and configure Codeflash:
<CardGroup cols={2}> <CardGroup cols={3}>
<Card title="Python" icon="python" href="/getting-started/local-installation"> <Card title="Python" icon="python" href="/getting-started/local-installation">
Install via pip, uv, or poetry. Configure in `pyproject.toml`. Install via pip, uv, or poetry. Configure in `pyproject.toml`.
</Card> </Card>
<Card title="JavaScript / TypeScript" icon="js" href="/getting-started/javascript-installation"> <Card title="JavaScript / TypeScript" icon="js" href="/getting-started/javascript-installation">
Install via npm, yarn, pnpm, or bun. Configure in `package.json`. Supports Jest, Vitest, and Mocha. Install via npm, yarn, pnpm, or bun. Configure in `package.json`. Supports Jest, Vitest, and Mocha.
</Card> </Card>
<Card title="Java" icon="java" href="/getting-started/java-installation">
Install via uv. Supports Maven and Gradle. JUnit 5, JUnit 4, and TestNG.
</Card>
</CardGroup> </CardGroup>
### How to use Codeflash ### How to use Codeflash
These commands work for both Python and JS/TS projects: These commands work for Python, JS/TS, and Java projects:
<CardGroup cols={2}> <CardGroup cols={2}>
<Card title="Optimize a Function" icon="bullseye" href="/optimizing-with-codeflash/one-function"> <Card title="Optimize a Function" icon="bullseye" href="/optimizing-with-codeflash/one-function">
@ -56,13 +59,16 @@ These commands work for both Python and JS/TS projects:
### Configuration Reference ### Configuration Reference
<CardGroup cols={2}> <CardGroup cols={3}>
<Card title="Python Config" icon="python" href="/configuration/python"> <Card title="Python Config" icon="python" href="/configuration/python">
`pyproject.toml` reference `pyproject.toml` reference
</Card> </Card>
<Card title="JS / TS Config" icon="js" href="/configuration/javascript"> <Card title="JS / TS Config" icon="js" href="/configuration/javascript">
`package.json` reference — includes monorepo, scattered tests, manual setup `package.json` reference — includes monorepo, scattered tests, manual setup
</Card> </Card>
<Card title="Java Config" icon="java" href="/configuration/java">
`pom.xml` / `gradle.properties` reference
</Card>
</CardGroup> </CardGroup>
### How does Codeflash verify correctness? ### How does Codeflash verify correctness?

View file

@ -3,13 +3,13 @@ title: "Optimize Your Entire Codebase"
description: "Automatically optimize all codepaths in your project with Codeflash's comprehensive analysis" description: "Automatically optimize all codepaths in your project with Codeflash's comprehensive analysis"
icon: "database" icon: "database"
sidebarTitle: "Optimize Entire Codebase" sidebarTitle: "Optimize Entire Codebase"
keywords: ["codebase optimization", "all functions", "batch optimization", "github app", "checkpoint", "recovery", "javascript", "typescript", "python"] keywords: ["codebase optimization", "all functions", "batch optimization", "github app", "checkpoint", "recovery", "javascript", "typescript", "python", "java"]
--- ---
# Optimize your entire codebase # Optimize your entire codebase
Codeflash can optimize your entire codebase by analyzing all the functions in your project and generating optimized versions of them. Codeflash can optimize your entire codebase by analyzing all the functions in your project and generating optimized versions of them.
It iterates through all the functions in your codebase and optimizes them one by one. This works for Python, JavaScript, and TypeScript projects. It iterates through all the functions in your codebase and optimizes them one by one. This works for Python, JavaScript, TypeScript, and Java projects.
To optimize your entire codebase, run the following command in your project directory: To optimize your entire codebase, run the following command in your project directory:
@ -45,6 +45,11 @@ codeflash --all path/to/dir
codeflash optimize --trace-only --vitest ; codeflash --all codeflash optimize --trace-only --vitest ; codeflash --all
``` ```
</Tab> </Tab>
<Tab title="Java">
```bash
codeflash optimize --timeout 60 java -cp target/classes com.example.Main ; codeflash --all
```
</Tab>
</Tabs> </Tabs>
This runs your test suite, traces all the code covered by your tests, ensuring higher correctness guarantees This runs your test suite, traces all the code covered by your tests, ensuring higher correctness guarantees

View file

@ -13,6 +13,7 @@ keywords:
"javascript", "javascript",
"typescript", "typescript",
"python", "python",
"java",
] ]
--- ---
@ -45,6 +46,11 @@ codeflash --file path/to/your/file.js --function functionName
codeflash --file path/to/your/file.ts --function functionName codeflash --file path/to/your/file.ts --function functionName
``` ```
</Tab> </Tab>
<Tab title="Java">
```bash
codeflash --file src/main/java/com/example/Utils.java --function methodName
```
</Tab>
</Tabs> </Tabs>
If you have installed the GitHub App to your repository, the above command will open a pull request with the optimized function. If you have installed the GitHub App to your repository, the above command will open a pull request with the optimized function.
@ -61,6 +67,11 @@ codeflash --file path/to/your/file.py --function function_name --no-pr
codeflash --file path/to/your/file.ts --function functionName --no-pr codeflash --file path/to/your/file.ts --function functionName --no-pr
``` ```
</Tab> </Tab>
<Tab title="Java">
```bash
codeflash --file src/main/java/com/example/Utils.java --function methodName --no-pr
```
</Tab>
</Tabs> </Tabs>
### Optimizing class methods ### Optimizing class methods
@ -78,4 +89,11 @@ codeflash --file path/to/your/file.py --function ClassName.method_name
codeflash --file path/to/your/file.ts --function ClassName.methodName codeflash --file path/to/your/file.ts --function ClassName.methodName
``` ```
</Tab> </Tab>
<Tab title="Java">
```bash
codeflash --file src/main/java/com/example/Utils.java --function methodName
```
In Java, use just the method name — no `ClassName.` prefix is needed. Codeflash discovers the method by name within the specified file.
</Tab>
</Tabs> </Tabs>

View file

@ -60,12 +60,12 @@ codeflash optimize --language javascript script.js
To trace and optimize a running Java program, replace your `java` command with `codeflash optimize java`: To trace and optimize a running Java program, replace your `java` command with `codeflash optimize java`:
```bash ```bash
# JAR application # Class with classpath (recommended — works with any compiled project)
codeflash optimize java -jar target/my-app.jar --app-args
# Class with classpath
codeflash optimize java -cp target/classes com.example.Main codeflash optimize java -cp target/classes com.example.Main
# Executable JAR (requires maven-jar-plugin or equivalent with Main-Class manifest)
codeflash optimize java -jar target/my-app.jar --app-args
# Maven exec # Maven exec
codeflash optimize mvn exec:java -Dexec.mainClass="com.example.Main" codeflash optimize mvn exec:java -Dexec.mainClass="com.example.Main"
``` ```
@ -73,7 +73,7 @@ codeflash optimize mvn exec:java -Dexec.mainClass="com.example.Main"
For long-running programs (servers, benchmarks), use `--timeout` to limit each tracing stage: For long-running programs (servers, benchmarks), use `--timeout` to limit each tracing stage:
```bash ```bash
codeflash optimize --timeout 30 java -jar target/my-app.jar codeflash optimize --timeout 30 java -cp target/classes com.example.Main
``` ```
</Tab> </Tab>
</Tabs> </Tabs>
@ -228,13 +228,15 @@ The Java tracer uses a **two-stage approach**: JFR (Java Flight Recorder) for ac
Replace your `java` command with `codeflash optimize java`: Replace your `java` command with `codeflash optimize java`:
```bash ```bash
# JAR application # Class with classpath (recommended — works with any compiled project)
codeflash optimize java -jar target/my-app.jar --app-args
# Class with classpath
codeflash optimize java -cp target/classes com.example.Main codeflash optimize java -cp target/classes com.example.Main
# Executable JAR (requires maven-jar-plugin or equivalent with Main-Class manifest)
codeflash optimize java -jar target/my-app.jar --app-args
``` ```
The `-cp` approach works with any project after `mvn compile` or `gradle build`. The `-jar` approach requires your project to produce an executable JAR with a `Main-Class` entry in the manifest — this is not the default Maven behavior.
Codeflash will run your program twice (once for profiling, once for argument capture), generate JUnit replay tests, then optimize the most impactful functions. Codeflash will run your program twice (once for profiling, once for argument capture), generate JUnit replay tests, then optimize the most impactful functions.
2. **Long-running programs** 2. **Long-running programs**
@ -242,7 +244,7 @@ The Java tracer uses a **two-stage approach**: JFR (Java Flight Recorder) for ac
For servers, benchmarks, or programs that don't terminate on their own, use `--timeout` to limit each tracing stage: For servers, benchmarks, or programs that don't terminate on their own, use `--timeout` to limit each tracing stage:
```bash ```bash
codeflash optimize --timeout 30 java -jar target/my-benchmark.jar codeflash optimize --timeout 30 java -cp target/classes com.example.Main
``` ```
Each stage runs for at most 30 seconds, then the program is terminated and captured data is processed. Each stage runs for at most 30 seconds, then the program is terminated and captured data is processed.

View file

@ -0,0 +1,109 @@
"""Test for false positive test discovery bug (Bug #4)."""
from pathlib import Path
from tempfile import TemporaryDirectory
import pytest
from codeflash.discovery.functions_to_optimize import FunctionToOptimize
from codeflash.languages.javascript.support import TypeScriptSupport
from codeflash.models.models import CodePosition
def test_discover_tests_should_not_match_mocked_functions():
"""Test that functions mentioned only in mocks are not matched as test targets.
Regression test for Bug #4: False positive test discovery due to substring matching.
When a test file mocks a function (e.g., vi.mock("./restart-request.js", () => ({...}))),
that function should NOT be considered as tested by that file, since it's only mocked,
not actually called or tested.
"""
support = TypeScriptSupport()
with TemporaryDirectory() as tmpdir:
test_root = Path(tmpdir)
# Create a test file that MOCKS parseRestartRequestParams but doesn't test it
test_file = test_root / "update.test.ts"
test_file.write_text(
'''
import { updateSomething } from "./update.js";
vi.mock("./restart-request.js", () => ({
parseRestartRequestParams: (params: any) => ({ sessionKey: undefined }),
}));
describe("updateSomething", () => {
it("should update successfully", () => {
const result = updateSomething();
expect(result).toBe(true);
});
});
'''
)
# Source function that is only mocked, not tested
source_function = FunctionToOptimize(
qualified_name="parseRestartRequestParams",
function_name="parseRestartRequestParams",
file_path=test_root / "restart-request.ts",
starting_line=1,
ending_line=10,
function_signature="",
code_position=CodePosition(line_no=1, col_no=0),
file_path_relative_to_project_root="restart-request.ts",
)
# Discover tests
result = support.discover_tests(test_root, [source_function])
# The bug: discovers update.test.ts as a test for parseRestartRequestParams
# because "parseRestartRequestParams" appears as a substring in the mock
# Expected: should NOT match (empty result)
assert (
source_function.qualified_name not in result or len(result[source_function.qualified_name]) == 0
), f"Should not match mocked function, but found: {result.get(source_function.qualified_name, [])}"
def test_discover_tests_should_match_actually_imported_functions():
"""Test that functions actually imported and tested ARE correctly matched.
This is the positive case to ensure we don't break legitimate test discovery.
"""
support = TypeScriptSupport()
with TemporaryDirectory() as tmpdir:
test_root = Path(tmpdir)
# Create a test file that ACTUALLY imports and tests the function
test_file = test_root / "restart-request.test.ts"
test_file.write_text(
'''
import { parseRestartRequestParams } from "./restart-request.js";
describe("parseRestartRequestParams", () => {
it("should parse valid params", () => {
const result = parseRestartRequestParams({ sessionKey: "abc" });
expect(result.sessionKey).toBe("abc");
});
});
'''
)
source_function = FunctionToOptimize(
qualified_name="parseRestartRequestParams",
function_name="parseRestartRequestParams",
file_path=test_root / "restart-request.ts",
starting_line=1,
ending_line=10,
function_signature="",
code_position=CodePosition(line_no=1, col_no=0),
file_path_relative_to_project_root="restart-request.ts",
)
result = support.discover_tests(test_root, [source_function])
# Should match: function is imported and tested
assert source_function.qualified_name in result, f"Should match imported function, but got: {result}"
assert len(result[source_function.qualified_name]) > 0, "Should find at least one test"

View file

@ -0,0 +1,79 @@
"""Test that Codeflash Vitest config properly overrides coverage settings."""
from pathlib import Path
import pytest
from codeflash.languages.javascript.vitest_runner import _ensure_codeflash_vitest_config
def test_codeflash_vitest_config_overrides_coverage(tmp_path: Path) -> None:
project_root = tmp_path.resolve()
vitest_config = project_root / "vitest.config.ts"
vitest_config.write_text(
"""
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ['test/**/*.test.ts'],
coverage: {
provider: 'v8',
reporter: ['text', 'lcov'],
all: false,
thresholds: {
lines: 70,
functions: 70,
},
},
},
});
""",
encoding="utf-8",
)
config_path = _ensure_codeflash_vitest_config(project_root)
assert config_path is not None, "Config should be created"
assert config_path.exists(), "Config file should exist"
config_content = config_path.read_text(encoding="utf-8")
assert "mergeConfig" in config_content, "Should use mergeConfig"
assert "import originalConfig from './vitest.config.ts'" in config_content
assert "coverage:" in config_content, (
"Config must explicitly override coverage settings to ensure "
"json reporter is used regardless of project config"
)
assert "reporter:" in config_content, "Config must override coverage.reporter to ['json']"
assert "['json']" in config_content or '["json"]' in config_content, (
"Coverage reporter must be set to ['json'] to ensure coverage files are written in the expected format"
)
def test_codeflash_vitest_config_without_original_coverage(tmp_path: Path) -> None:
project_root = tmp_path.resolve()
vitest_config = project_root / "vitest.config.ts"
vitest_config.write_text(
"""
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ['test/**/*.test.ts'],
},
});
""",
encoding="utf-8",
)
config_path = _ensure_codeflash_vitest_config(project_root)
assert config_path is not None
assert config_path.exists()
config_content = config_path.read_text(encoding="utf-8")
assert "coverage:" in config_content, "Config must explicitly set coverage even when original doesn't have it"

View file

@ -0,0 +1,61 @@
from pathlib import Path
import pytest
from codeflash.languages.javascript.vitest_runner import _ensure_codeflash_vitest_config
def test_codeflash_vitest_config_overrides_setupfiles(tmp_path: Path) -> None:
project_root = tmp_path.resolve()
# Create a project with setup file
(project_root / "test").mkdir()
(project_root / "test" / "setup.ts").write_text("// Setup file\n", encoding="utf-8")
vitest_config = """import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
setupFiles: ["test/setup.ts"], // Relative path - will cause issues
include: ["src/**/*.test.ts"],
},
});
"""
(project_root / "vitest.config.ts").write_text(vitest_config, encoding="utf-8")
codeflash_config_path = _ensure_codeflash_vitest_config(project_root)
assert codeflash_config_path is not None
assert codeflash_config_path.exists()
config_content = codeflash_config_path.read_text(encoding="utf-8")
assert "setupFiles" in config_content, (
"Generated config must explicitly handle setupFiles to prevent "
"relative path resolution issues. Current config:\n" + config_content
)
assert "setupFiles: []" in config_content or "setupFiles:" in config_content, (
"setupFiles must be explicitly set in the merged config"
)
def test_codeflash_vitest_config_without_setupfiles(tmp_path: Path) -> None:
project_root = tmp_path.resolve()
vitest_config = """import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ["src/**/*.test.ts"],
},
});
"""
(project_root / "vitest.config.ts").write_text(vitest_config, encoding="utf-8")
codeflash_config_path = _ensure_codeflash_vitest_config(project_root)
assert codeflash_config_path is not None
assert codeflash_config_path.exists()
config_content = codeflash_config_path.read_text(encoding="utf-8")
assert "mergeConfig" in config_content or "defineConfig" in config_content

View file

@ -8,6 +8,7 @@ import pytest
from codeflash.code_utils.code_utils import ( from codeflash.code_utils.code_utils import (
cleanup_paths, cleanup_paths,
exit_with_message,
file_name_from_test_module_name, file_name_from_test_module_name,
file_path_from_module_name, file_path_from_module_name,
get_all_function_names, get_all_function_names,
@ -751,3 +752,33 @@ class MyClass:
""" """
result = validate_python_code(code) result = validate_python_code(code)
assert result == code assert result == code
class TestExitWithMessageSubagent:
@patch("codeflash.code_utils.code_utils.is_subagent_mode", return_value=True)
def test_outputs_structured_xml_in_subagent_mode(self, _mock_subagent: MagicMock, capsys: pytest.CaptureFixture[str]) -> None:
with pytest.raises(SystemExit) as exc_info:
exit_with_message("Something went wrong", error_on_exit=True)
assert exc_info.value.code == 1
captured = capsys.readouterr()
assert "<codeflash-error>" in captured.out
assert "Something went wrong" in captured.out
assert "</codeflash-error>" in captured.out
@patch("codeflash.code_utils.code_utils.is_subagent_mode", return_value=True)
def test_escapes_xml_special_chars(self, _mock_subagent: MagicMock, capsys: pytest.CaptureFixture[str]) -> None:
with pytest.raises(SystemExit):
exit_with_message('File <foo> & "bar" not found', error_on_exit=True)
captured = capsys.readouterr()
assert "&lt;foo&gt;" in captured.out
assert "&amp;" in captured.out
@patch("codeflash.code_utils.code_utils.is_subagent_mode", return_value=False)
@patch("codeflash.code_utils.code_utils.is_LSP_enabled", return_value=False)
def test_no_xml_when_not_subagent(
self, _mock_lsp: MagicMock, _mock_subagent: MagicMock, capsys: pytest.CaptureFixture[str]
) -> None:
with pytest.raises(SystemExit):
exit_with_message("Normal error", error_on_exit=True)
captured = capsys.readouterr()
assert "<codeflash-error>" not in captured.out

View file

@ -0,0 +1,94 @@
"""Test fix_jest_mock_paths function with vitest mocks."""
from pathlib import Path
from codeflash.languages.javascript.instrument import fix_jest_mock_paths
def test_fix_vitest_mock_paths():
"""Test that vi.mock() paths are fixed correctly."""
# Simulate source at src/agents/workspace.ts importing from ../routing/session-key
# Test at test/test_workspace.test.ts should mock ../src/routing/session-key, not ../routing/session-key
test_code = """
vi.mock('../routing/session-key', () => ({
isSubagentSessionKey: vi.fn(),
isCronSessionKey: vi.fn(),
}));
import { filterBootstrapFilesForSession } from '../src/agents/workspace.js';
"""
# Create temp directories and files for testing
import tempfile
with tempfile.TemporaryDirectory() as tmpdir:
project = Path(tmpdir)
# Create directory structure
src = project / "src"
src_agents = src / "agents"
src_routing = src / "routing"
test_dir = project / "test"
src_agents.mkdir(parents=True)
src_routing.mkdir(parents=True)
test_dir.mkdir(parents=True)
# Create files
source_file = src_agents / "workspace.ts"
source_file.write_text("export function filterBootstrapFilesForSession() {}")
routing_file = src_routing / "session-key.ts"
routing_file.write_text("export function isSubagentSessionKey() {}")
test_file = test_dir / "test_workspace.test.ts"
test_file.write_text(test_code)
# Fix the paths
fixed = fix_jest_mock_paths(test_code, test_file, source_file, test_dir)
# Should change ../routing/session-key to ../src/routing/session-key
assert "../src/routing/session-key" in fixed, f"Expected path to be fixed, got: {fixed}"
assert "../routing/session-key" not in fixed or "../src/routing/session-key" in fixed
def test_fix_jest_mock_paths_still_works():
"""Test that jest.mock() paths are still fixed correctly."""
test_code = """
jest.mock('../routing/session-key', () => ({
isSubagentSessionKey: jest.fn(),
}));
"""
import tempfile
with tempfile.TemporaryDirectory() as tmpdir:
project = Path(tmpdir)
src = project / "src"
src_agents = src / "agents"
src_routing = src / "routing"
test_dir = project / "test"
src_agents.mkdir(parents=True)
src_routing.mkdir(parents=True)
test_dir.mkdir(parents=True)
source_file = src_agents / "workspace.ts"
source_file.write_text("")
routing_file = src_routing / "session-key.ts"
routing_file.write_text("")
test_file = test_dir / "test_workspace.test.ts"
test_file.write_text(test_code)
fixed = fix_jest_mock_paths(test_code, test_file, source_file, test_dir)
assert "../src/routing/session-key" in fixed
if __name__ == "__main__":
test_fix_vitest_mock_paths()
test_fix_jest_mock_paths_still_works()
print("All tests passed!")

View file

@ -0,0 +1,66 @@
"""Test that js_project_root is recalculated per function, not cached."""
from pathlib import Path
from codeflash.languages.javascript.test_runner import find_node_project_root
def test_find_node_project_root_returns_different_roots_for_different_files(tmp_path: Path) -> None:
"""Test that find_node_project_root returns the correct root for each file."""
# Create main project structure
main_project = (tmp_path / "project").resolve()
main_project.mkdir()
(main_project / "package.json").write_text("{}", encoding="utf-8")
(main_project / "src").mkdir()
main_file = (main_project / "src" / "main.ts").resolve()
main_file.write_text("// main file", encoding="utf-8")
# Create extension subdirectory with its own package.json
extension_dir = (main_project / "extensions" / "discord").resolve()
extension_dir.mkdir(parents=True)
(extension_dir / "package.json").write_text("{}", encoding="utf-8")
(extension_dir / "src").mkdir()
extension_file = (extension_dir / "src" / "accounts.ts").resolve()
extension_file.write_text("// extension file", encoding="utf-8")
# Extension file should return extension directory
result1 = find_node_project_root(extension_file)
assert result1 == extension_dir, f"Expected {extension_dir}, got {result1}"
# Main file should return main project directory
result2 = find_node_project_root(main_file)
assert result2 == main_project, f"Expected {main_project}, got {result2}"
# Calling again with extension file should still return extension dir
result3 = find_node_project_root(extension_file)
assert result3 == extension_dir, f"Expected {extension_dir}, got {result3}"
def test_js_project_root_recalculated_per_function(tmp_path: Path) -> None:
"""Each function in a monorepo should resolve to its own nearest package.json root."""
# Create main project
main_project = (tmp_path / "project").resolve()
main_project.mkdir()
(main_project / "package.json").write_text('{"name": "main"}', encoding="utf-8")
(main_project / "src").mkdir()
# Create extension with its own package.json
extension_dir = (main_project / "extensions" / "discord").resolve()
extension_dir.mkdir(parents=True)
(extension_dir / "package.json").write_text('{"name": "discord-extension"}', encoding="utf-8")
(extension_dir / "src").mkdir()
extension_file = (extension_dir / "src" / "accounts.ts").resolve()
extension_file.write_text("export function foo() {}", encoding="utf-8")
main_file = (main_project / "src" / "commands.ts").resolve()
main_file.write_text("export function bar() {}", encoding="utf-8")
js_project_root_1 = find_node_project_root(extension_file)
assert js_project_root_1 == extension_dir
js_project_root_2 = find_node_project_root(main_file)
assert js_project_root_2 == main_project, (
f"Expected {main_project}, got {js_project_root_2}. "
f"Happens when js_project_root is not recalculated per function."
)

View file

@ -641,3 +641,28 @@ class TestGradleEnsureRuntimeMultiModule:
assert result is True assert result is True
nested_build = (nested / "build.gradle.kts").read_text(encoding="utf-8") nested_build = (nested / "build.gradle.kts").read_text(encoding="utf-8")
assert "codeflash-runtime" in nested_build assert "codeflash-runtime" in nested_build
class TestValidationSkipFlags:
"""Tests that validation skip flags include all known static analysis and formatting plugins."""
def test_maven_skip_flags_include_spotless(self):
from codeflash.languages.java.maven_strategy import _MAVEN_VALIDATION_SKIP_FLAGS
flags_str = " ".join(_MAVEN_VALIDATION_SKIP_FLAGS)
assert "-Dspotless.check.skip=true" in flags_str
assert "-Dspotless.apply.skip=true" in flags_str
def test_maven_skip_flags_include_all_known_plugins(self):
from codeflash.languages.java.maven_strategy import _MAVEN_VALIDATION_SKIP_FLAGS
flags_str = " ".join(_MAVEN_VALIDATION_SKIP_FLAGS)
for plugin in ["rat", "checkstyle", "spotbugs", "pmd", "enforcer", "japicmp", "errorprone", "spotless"]:
assert plugin in flags_str, f"Missing skip flag for {plugin}"
def test_gradle_skip_script_includes_spotless(self):
from codeflash.languages.java.gradle_strategy import _GRADLE_SKIP_VALIDATION_INIT_SCRIPT
assert "spotlessCheck" in _GRADLE_SKIP_VALIDATION_INIT_SCRIPT
assert "spotlessApply" in _GRADLE_SKIP_VALIDATION_INIT_SCRIPT
assert "spotlessJava" in _GRADLE_SKIP_VALIDATION_INIT_SCRIPT

View file

@ -512,13 +512,16 @@ public class PreciseWaiterTest {
stddev_runtime = statistics.stdev(runtimes) stddev_runtime = statistics.stdev(runtimes)
coefficient_of_variation = stddev_runtime / mean_runtime coefficient_of_variation = stddev_runtime / mean_runtime
# Target: 10ms (10,000,000 ns), allow <5% coefficient of variation # Target: 10ms (10,000,000 ns), allow <15% coefficient of variation.
# (accounts for JIT warmup - first iteration is cold, subsequent are optimized) # The first iteration per test method runs with cold JIT, and shared CI VMs
# (especially Windows) have ~15ms scheduler granularity that adds noise.
# 15% still catches instrumentation bugs (e.g., 0ms or 100ms outliers)
# while the ±5% mean check below validates timing accuracy.
expected_ns = 10_000_000 expected_ns = 10_000_000
runtimes_ms = [r / 1_000_000 for r in runtimes] runtimes_ms = [r / 1_000_000 for r in runtimes]
assert coefficient_of_variation < 0.05, ( assert coefficient_of_variation < 0.15, (
f"Timing variance too high: CV={coefficient_of_variation:.2%} (should be <5%). " f"Timing variance too high: CV={coefficient_of_variation:.2%} (should be <15%). "
f"Runtimes: {runtimes_ms} ms (mean={mean_runtime / 1_000_000:.3f}ms)" f"Runtimes: {runtimes_ms} ms (mean={mean_runtime / 1_000_000:.3f}ms)"
) )
@ -597,13 +600,16 @@ public class PreciseWaiterMultiTest {
stddev_runtime = statistics.stdev(runtimes) stddev_runtime = statistics.stdev(runtimes)
coefficient_of_variation = stddev_runtime / mean_runtime coefficient_of_variation = stddev_runtime / mean_runtime
# Target: 10ms (10,000,000 ns), allow <5% coefficient of variation # Target: 10ms (10,000,000 ns), allow <15% coefficient of variation.
# (accounts for JIT warmup - first iteration is cold, subsequent are optimized) # The first iteration per test method runs with cold JIT, and shared CI VMs
# (especially Windows) have ~15ms scheduler granularity that adds noise.
# 15% still catches instrumentation bugs (e.g., 0ms or 100ms outliers)
# while the ±5% mean check below validates timing accuracy.
expected_ns = 10_000_000 expected_ns = 10_000_000
runtimes_ms = [r / 1_000_000 for r in runtimes] runtimes_ms = [r / 1_000_000 for r in runtimes]
assert coefficient_of_variation < 0.05, ( assert coefficient_of_variation < 0.15, (
f"Timing variance too high: CV={coefficient_of_variation:.2%} (should be <5%). " f"Timing variance too high: CV={coefficient_of_variation:.2%} (should be <15%). "
f"Runtimes: {runtimes_ms} ms (mean={mean_runtime / 1_000_000:.3f}ms)" f"Runtimes: {runtimes_ms} ms (mean={mean_runtime / 1_000_000:.3f}ms)"
) )

View file

@ -122,7 +122,7 @@ class TestJestRootsConfiguration:
runtime_configs = [f for f in get_created_config_files() if "codeflash.runtime" in f.name] runtime_configs = [f for f in get_created_config_files() if "codeflash.runtime" in f.name]
assert len(runtime_configs) == 1, f"Expected 1 runtime config, got {len(runtime_configs)}" assert len(runtime_configs) == 1, f"Expected 1 runtime config, got {len(runtime_configs)}"
config_content = runtime_configs[0].read_text(encoding="utf-8") config_content = runtime_configs[0].read_text(encoding="utf-8")
assert str(external_path) in config_content, "Runtime config should contain external test directory" assert external_path.as_posix() in config_content, "Runtime config should contain external test directory"
clear_created_config_files() clear_created_config_files()

View file

@ -0,0 +1,155 @@
"""Test for TypeScript Jest config require bug.
Regression test for the issue where _create_runtime_jest_config generates
code that tries to require('./jest.config.ts'), which fails because Node.js
CommonJS cannot load TypeScript files directly.
Bug: https://github.com/codeflash-ai/codeflash/issues/XXX
Affects: 18 out of 38 optimization runs in initial testing
"""
import subprocess
import tempfile
from pathlib import Path
import pytest
class TestTypeScriptJestConfigRequire:
"""Test that runtime config correctly handles TypeScript base configs."""
def test_runtime_config_with_typescript_base_config_loads_without_error(self):
"""Runtime config should NOT try to require .ts files directly.
When base_config_path points to jest.config.ts, the generated runtime
config must not use require('./jest.config.ts') because Node.js cannot
parse TypeScript syntax in CommonJS require().
This test creates a jest.config.ts file and verifies that the generated
runtime config can be successfully loaded by Node.js without syntax errors.
"""
from codeflash.languages.javascript.test_runner import _create_runtime_jest_config
with tempfile.TemporaryDirectory() as tmpdir:
project_path = Path(tmpdir).resolve()
# Create a TypeScript Jest config (realistic content with TS syntax)
ts_config_path = project_path / "jest.config.ts"
ts_config_content = """import { Config } from "jest"
const config: Config = {
testEnvironment: 'node',
testMatch: ['**/*.test.ts'],
moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json', 'node'],
}
export default config
"""
ts_config_path.write_text(ts_config_content, encoding="utf-8")
# Create runtime config with the TS base config
test_dirs = {str(project_path / "test")}
runtime_config_path = _create_runtime_jest_config(
base_config_path=ts_config_path,
project_root=project_path,
test_dirs=test_dirs
)
assert runtime_config_path is not None, "Runtime config should be created"
assert runtime_config_path.exists(), "Runtime config file should exist"
# Read the generated content
runtime_content = runtime_config_path.read_text(encoding="utf-8")
# CRITICAL CHECK: Should NOT contain require('./jest.config.ts')
# This is the bug we're fixing
assert "require('./jest.config.ts')" not in runtime_content, (
"Runtime config should not try to require .ts files directly"
)
# The config should handle TypeScript configs appropriately:
# - Either omit the extension (let Node resolve to .js)
# - Or use a TypeScript loader (ts-node)
# - Or skip requiring TS configs entirely
# Verify the generated config can be loaded by Node.js without errors
test_script = project_path / "test_load_config.js"
test_script_content = f"""
try {{
const config = require('./{runtime_config_path.name}');
console.log('SUCCESS');
process.exit(0);
}} catch (err) {{
console.error('FAILED:', err.message);
process.exit(1);
}}
"""
test_script.write_text(test_script_content, encoding="utf-8")
result = subprocess.run(
["node", str(test_script)],
capture_output=True,
text=True,
cwd=project_path,
timeout=30,
)
assert result.returncode == 0, (
f"Generated runtime config should load without errors.\n"
f"Config path: {runtime_config_path}\n"
f"Config content:\n{runtime_content}\n"
f"Node output:\n{result.stdout}\n{result.stderr}"
)
assert "SUCCESS" in result.stdout
def test_runtime_config_with_js_base_config_works(self):
"""Verify that .js base configs still work correctly (control test)."""
from codeflash.languages.javascript.test_runner import _create_runtime_jest_config
with tempfile.TemporaryDirectory() as tmpdir:
project_path = Path(tmpdir).resolve()
# Create a JavaScript Jest config
js_config_path = project_path / "jest.config.js"
js_config_content = """module.exports = {
testEnvironment: 'node',
testMatch: ['**/*.test.js'],
}
"""
js_config_path.write_text(js_config_content, encoding="utf-8")
# Create runtime config with the JS base config
test_dirs = {str(project_path / "test")}
runtime_config_path = _create_runtime_jest_config(
base_config_path=js_config_path,
project_root=project_path,
test_dirs=test_dirs
)
assert runtime_config_path is not None
assert runtime_config_path.exists()
# Verify it loads without errors
test_script = project_path / "test_load_config.js"
test_script_content = f"""
try {{
const config = require('./{runtime_config_path.name}');
console.log('SUCCESS');
process.exit(0);
}} catch (err) {{
console.error('FAILED:', err.message);
process.exit(1);
}}
"""
test_script.write_text(test_script_content, encoding="utf-8")
result = subprocess.run(
["node", str(test_script)],
capture_output=True,
text=True,
cwd=project_path,
timeout=30,
)
assert result.returncode == 0, f"JS config should load: {result.stderr}"
assert "SUCCESS" in result.stdout

View file

@ -0,0 +1,85 @@
"""Tests for module_name_from_file_path with co-located test directories."""
import pytest
from pathlib import Path
from codeflash.code_utils.code_utils import module_name_from_file_path
class TestModuleNameFromFilePath:
"""Test module name resolution for various directory structures."""
def test_file_inside_project_root(self, tmp_path: Path) -> None:
"""Test normal case where file is inside project root."""
project_root = tmp_path / "project"
project_root.mkdir()
test_file = project_root / "test" / "test_foo.py"
test_file.parent.mkdir()
test_file.touch()
result = module_name_from_file_path(test_file, project_root)
assert result == "test.test_foo"
def test_file_outside_project_root_without_traverse_up(self, tmp_path: Path) -> None:
"""Test that file outside project root raises ValueError by default."""
project_root = tmp_path / "project" / "test"
project_root.mkdir(parents=True)
# File is in a sibling directory, not under project_root
test_file = tmp_path / "project" / "src" / "__tests__" / "test_foo.py"
test_file.parent.mkdir(parents=True)
test_file.touch()
with pytest.raises(ValueError, match="is not within the project root"):
module_name_from_file_path(test_file, project_root)
def test_file_outside_project_root_with_traverse_up(self, tmp_path: Path) -> None:
"""Test that traverse_up=True handles files outside project root."""
project_root = tmp_path / "project" / "test"
project_root.mkdir(parents=True)
# File is in a sibling directory, not under project_root
test_file = tmp_path / "project" / "src" / "__tests__" / "codeflash-generated" / "test_foo.py"
test_file.parent.mkdir(parents=True)
test_file.touch()
# With traverse_up=True, it should find a common ancestor
result = module_name_from_file_path(test_file, project_root, traverse_up=True)
# Should return a relative path from some ancestor directory
assert "test_foo" in result
assert not result.startswith(".")
def test_colocated_test_directory_structure(self, tmp_path: Path) -> None:
"""Test real-world scenario with co-located __tests__ directory.
This reproduces the bug from trace 7b97ddba-6ecd-42fd-b572-d40658746836:
- Source: /workspace/target/src/gateway/server/ws-connection/connect-policy.ts
- Tests root: /workspace/target/test
- Generated test: /workspace/target/src/gateway/server/__tests__/codeflash-generated/test_xxx.test.ts
Without traverse_up=True, this should fail.
"""
project_root = tmp_path / "target"
project_root.mkdir()
tests_root = project_root / "test"
tests_root.mkdir()
# Source file location
source_file = project_root / "src" / "gateway" / "server" / "ws-connection" / "connect-policy.ts"
source_file.parent.mkdir(parents=True)
source_file.touch()
# Generated test in co-located __tests__ directory
test_file = project_root / "src" / "gateway" / "server" / "__tests__" / "codeflash-generated" / "test_resolveControlUiAuthPolicy.test.ts"
test_file.parent.mkdir(parents=True)
test_file.touch()
# This should fail WITHOUT traverse_up
with pytest.raises(ValueError, match="is not within the project root"):
module_name_from_file_path(test_file, tests_root)
# This should succeed WITH traverse_up
result = module_name_from_file_path(test_file, tests_root, traverse_up=True)
assert "test_resolveControlUiAuthPolicy" in result

View file

@ -0,0 +1,57 @@
"""Test that test_cfg.js_project_root caching bug is demonstrated and bypassed by the fix."""
from pathlib import Path
from unittest.mock import patch
from codeflash.languages.javascript.support import JavaScriptSupport
from codeflash.verification.verification_utils import TestConfig
@patch("codeflash.languages.javascript.optimizer.verify_js_requirements")
def test_js_project_root_cached_in_test_cfg(mock_verify: object, tmp_path: Path) -> None:
"""Demonstrates that test_cfg.js_project_root is set once per setup_test_config call.
This test shows the root cause: test_cfg caches the project root from the first function.
The fix bypasses this cache in FunctionOptimizer.get_js_project_root() instead of
changing how test_cfg stores the value.
"""
mock_verify.return_value = [] # type: ignore[attr-defined]
# Create main project
main_project = (tmp_path / "project").resolve()
main_project.mkdir()
(main_project / "package.json").write_text('{"name": "main"}', encoding="utf-8")
(main_project / "src").mkdir()
(main_project / "test").mkdir()
(main_project / "node_modules").mkdir()
# Create extension with its own package.json
extension_dir = (main_project / "extensions" / "discord").resolve()
extension_dir.mkdir(parents=True)
(extension_dir / "package.json").write_text('{"name": "discord-extension"}', encoding="utf-8")
(extension_dir / "src").mkdir()
(extension_dir / "node_modules").mkdir()
test_cfg = TestConfig(
tests_root=main_project / "test",
project_root_path=main_project,
tests_project_rootdir=main_project / "test",
)
test_cfg.set_language("javascript")
js_support = JavaScriptSupport()
extension_file = (extension_dir / "src" / "accounts.ts").resolve()
extension_file.write_text("export function foo() {}", encoding="utf-8")
success = js_support.setup_test_config(test_cfg, extension_file, current_worktree=None)
assert success, "setup_test_config should succeed"
# After setup for extension file, js_project_root is the extension directory
assert test_cfg.js_project_root == extension_dir
# test_cfg is NOT re-initialized for subsequent functions — js_project_root stays cached
main_file = (main_project / "src" / "commands.ts").resolve()
main_file.write_text("export function bar() {}", encoding="utf-8")
# The cached value is still extension_dir, not main_project — this is the root cause
assert test_cfg.js_project_root == extension_dir

View file

@ -0,0 +1,91 @@
"""Test that coverage error messages are framework-agnostic."""
import tempfile
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from codeflash.languages.language_enum import Language
from codeflash.models.models import CodeOptimizationContext
from codeflash.verification.coverage_utils import JestCoverageUtils
class TestCoverageUtilsFrameworkAgnostic:
"""Test that error messages don't hardcode 'Jest' when used for Vitest."""
def test_missing_coverage_file_message_is_framework_agnostic(self, caplog):
"""When coverage file is missing, error message should not say 'Jest' specifically.
This class is used for both Jest and Vitest (they use the same Istanbul/v8 format).
Error messages should be generic, not hardcode 'Jest'.
"""
# Set log level to DEBUG to capture all messages
caplog.set_level("DEBUG")
# Create minimal context
context = MagicMock(spec=CodeOptimizationContext)
context.language = Language.JAVASCRIPT
context.target_code = "export function test() {}"
context.helper_functions = []
nonexistent_path = Path("/tmp/nonexistent_coverage_12345.json")
# Load coverage from non-existent file
result = JestCoverageUtils.load_from_jest_json(
coverage_json_path=nonexistent_path,
function_name="testFunc",
code_context=context,
source_code_path=Path("/tmp/test.ts")
)
# Should return empty coverage data
assert result.status.name in ("NOT_FOUND", "EMPTY")
# Error message should NOT hardcode "Jest" - it should be framework-agnostic
# since this util is used for both Jest and Vitest
log_messages = [record.message for record in caplog.records]
# Check that if there's a message about coverage file, it doesn't say "Jest"
coverage_messages = [msg for msg in log_messages if "coverage file not found" in msg.lower()]
if coverage_messages:
# The message should NOT contain "Jest" specifically
# It should say something like "Coverage file not found" or "JavaScript coverage file not found"
for msg in coverage_messages:
assert "Jest" not in msg, (
f"Error message should not hardcode 'Jest' since this util is used for Vitest too. "
f"Got: {msg}"
)
def test_parse_error_message_is_framework_agnostic(self, tmp_path, caplog):
"""When coverage file is malformed, error should not say 'Jest' specifically."""
# Set log level to capture all messages
caplog.set_level("DEBUG")
# Create invalid JSON file
coverage_file = tmp_path / "invalid_coverage.json"
coverage_file.write_text("{invalid json")
context = MagicMock(spec=CodeOptimizationContext)
context.language = Language.JAVASCRIPT
context.target_code = "export function test() {}"
context.helper_functions = []
result = JestCoverageUtils.load_from_jest_json(
coverage_json_path=coverage_file,
function_name="testFunc",
code_context=context,
source_code_path=Path("/tmp/test.ts")
)
# Should return empty coverage
assert result.status.name in ("NOT_FOUND", "EMPTY")
# Check log messages don't hardcode "Jest"
log_messages = [record.message for record in caplog.records]
parse_error_messages = [msg for msg in log_messages if "parse" in msg.lower() and "coverage" in msg.lower()]
for msg in parse_error_messages:
assert "Jest" not in msg, (
f"Parse error message should not hardcode 'Jest'. Got: {msg}"
)

View file

@ -0,0 +1,55 @@
"""Test that verifier.py handles test files outside tests_project_rootdir gracefully.
This tests the fix for the bug where JavaScript/TypeScript test files generated
in __tests__ subdirectories (adjacent to source files) caused ValueError when
verifier.py tried to compute their module path relative to tests_project_rootdir.
Trace ID: 84f5467f-8acf-427f-b468-02cb3342097e
"""
from pathlib import Path
import pytest
from codeflash.code_utils.code_utils import module_name_from_file_path
class TestVerifierPathHandling:
"""Test path handling in verifier.py for test files outside tests_root."""
def test_module_name_from_file_path_raises_valueerror_when_outside_root(self) -> None:
"""Verify that module_name_from_file_path raises ValueError when file is outside root.
This is the current behavior that causes the bug in verifier.py line 37.
Scenario:
- JavaScript support generates test at: /workspace/target/src/gateway/server/__tests__/codeflash-generated/test_foo.test.ts
- tests_project_rootdir is: /workspace/target/test
- Test file is NOT within tests_root, so relative_to() fails
"""
test_path = Path("/workspace/target/src/gateway/server/__tests__/codeflash-generated/test_foo.test.ts")
tests_root = Path("/workspace/target/test")
# This should raise ValueError before the fix
with pytest.raises(ValueError, match="is not within the project root"):
module_name_from_file_path(test_path, tests_root)
def test_module_name_from_file_path_with_fallback_succeeds(self) -> None:
"""Test that adding a fallback (try-except) allows graceful handling.
This is the pattern used in javascript/parse.py:330-333 that should
also be applied to verifier.py:37.
"""
test_path = Path("/workspace/target/src/gateway/server/__tests__/codeflash-generated/test_foo.test.ts")
tests_root = Path("/workspace/target/test")
# Simulate the fix: try-except with fallback to filename
try:
test_module_path = module_name_from_file_path(test_path, tests_root)
except ValueError:
# Fallback: use just the filename (or relative path from parent)
# This is what javascript/parse.py does
test_module_path = test_path.name
# After fallback, we should have a valid path
assert test_module_path == "test_foo.test.ts"