Commit graph

7285 commits

Author SHA1 Message Date
Sarthak Agarwal
2e1ef202d1 Merge branch 'main' into fix/js-vitest-benchmarking-and-mocha-cjs 2026-03-04 10:47:13 +05:30
Sarthak Agarwal
12294cafb6 fix looping with JS/TS 2026-03-04 10:46:44 +05:30
Kevin Turcios
92ab600edc fix: resolve remaining test failures from main sync
- Update test_inject_profiling_used_frameworks, test_async_run_and_parse,
  test_pickle_patcher to use new inject_profiling_into_existing_test API
  (test_string param removed)
- Add parse_line_profile_results function to parse_line_profile_test_output
  module (imported by main's PythonFunctionOptimizer and test_instrument_tests)
2026-03-04 00:13:27 -05:00
Kevin Turcios
af7ce7fce2 fix: cherry-pick main improvements into omni-java branch
- Take main's JS improvements: Mocha CJS support, ESM/CJS handling,
  sanitize_mocha_imports, vitest benchmarking fixes
- Update instrument_existing_test API: remove test_string param, read from
  file internally (aligned across Python, JS, Java support classes)
- Take main's equivalence.py with pass_fail_only parameter
- Take main's models.py, critic.py, env_utils.py, replay_test.py fixes
- Take main's PythonFunctionOptimizer parse_line_profile fix
- Skip files where our branch has Java-specific additions main doesn't
  have (create_pr, discover_unit_tests, parse_test_output, optimizer,
  verification_utils, config_parser, cmd_init, detector, support.py
  protocol methods)
2026-03-03 23:59:26 -05:00
Kevin Turcios
bccc02aade merge: incorporate omni-main-java sync work
Merges the omni-main-java branch which synced main into omni-java,
including JavaFunctionOptimizer, removal of is_java()/is_python() guards,
protocol dispatch for parse_test_xml, and deletion of concolic_testing.py.
2026-03-03 23:42:39 -05:00
codeflash-ai[bot]
5a3f8cdd86
Optimize add_runtime_comments
The optimization adds `@lru_cache(maxsize=4096)` to `format_runtime_comment`, which is called once per annotated line (2484 hits in the profiler) with frequently repeated input tuples—particularly when multiple test cases measure identical nanosecond pairs or when performance data clusters around similar values. Without caching, each call re-executes `format_perf`, `performance_gain`, and two `format_time` calls, accounting for 78% of `format_runtime_comment`'s total time (9.55ms) and 63.5% of `add_runtime_comments`'s wall time (17.7ms). With caching, cache hits eliminate these formatting steps entirely, reducing the hot-path line from 7120ns to 4292ns per hit—a 39.7% per-call improvement that compounds across thousands of invocations to yield the observed 111% speedup.
2026-03-04 04:15:28 +00:00
codeflash-ai[bot]
ef83035589
Optimize _generate_sqlite_write_code
The optimization replaced 34 repeated string interpolations of `{iter_id}_{call_counter}` with a single pre-computed `id_pair` variable, reducing per-call runtime from ~24ms to ~13ms (87% faster). Line profiler shows the list-building return statement consumed 38.9% of original time (~42ms) versus 41.2% optimized (~47ms absolute), but the absolute wall-clock improvement came from eliminating redundant f-string evaluations: each of the 34 format operations that previously recalculated `f"{iter_id}_{call_counter}"` now references a cached result, cutting allocation overhead. Tests across 1000+ iterations confirm identical output with consistent speedup ranging from 44–92% depending on input size.
2026-03-04 04:00:27 +00:00
Kevin Turcios
8a445621f9 revert: undo unused loop variable fix — tests_in_file_list is used on omni-java
The variable is unused on main (which extracted the logic) but is still
actively used in 6 places on omni-java's instrument_existing_tests.
2026-03-03 22:55:41 -05:00
Kevin Turcios
6da2319dcb style: fix unused loop variable in function_optimizer.py
(cherry-picked from 192cf0f8 — manually applied)
2026-03-03 22:50:57 -05:00
Kevin Turcios
62010ecdc1 fix: add missing languages/code_replacer.py for JS function optimizer
JavaScriptFunctionOptimizer.replace_function_and_helpers_with_optimized_code
imports from codeflash.languages.code_replacer, but the file was never
ported to omni-java. Copied from main.

Fixes PYTHON-CLI-44S.
2026-03-03 22:40:49 -05:00
Kevin Turcios
c550bf1561 fix: restore QName test to match omni-java's enrich_testgen_context behavior
The cherry-picked test fix from main assumed stdlib classes are extracted,
but omni-java's implementation still skips them.
2026-03-03 22:27:44 -05:00
Kevin Turcios
efa4a00400 fix: handle non-UTF-8 subprocess output with errors='replace'
(cherry-picked from b59882e1 — manually applied shell_utils.py change only)
2026-03-03 22:22:37 -05:00
Kevin Turcios
a0249afc7e fix: use PythonFunctionOptimizer in tests that depend on Python-specific hooks 2026-03-03 22:22:14 -05:00
Kevin Turcios
01bada9ae0 chore: add TODO for get_code_optimization_context extraction, update follow_up.md 2026-03-03 22:22:10 -05:00
Kevin Turcios
2112d620ae refactor: remove is_python() guards from code-restore sites in run_optimized_candidate 2026-03-03 22:22:05 -05:00
Kevin Turcios
c11f52321e fix: correct pre-existing test failures in test_code_context_extractor
Fix 10 failing tests: remove wrong assertions expecting import statements
inside extracted class code, use substring matching for UserDict class
signature, and rewrite click-dependent tests as project-local equivalents.
Add tests for resolve_instance_class_name, enhanced extract_init_stub_from_class,
and enrich_testgen_context instance resolution.
2026-03-03 22:22:00 -05:00
claude[bot]
412d131d06 fix: resolve ruff errors in code_context_extractor
Move Path import out of TYPE_CHECKING block (TC004) since it is used at runtime, and replace missing safe_relative_to call with inline try/except pattern matching the rest of the PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 22:22:00 -05:00
Kevin Turcios
c7854fb605 chore: remove simplify step from Claude PR review and add prek rule
Drop the /simplify step that caused unprompted refactors and scope
creep in PR reviews. Also add prek pre-commit rule to project config
so the PR bot and all contributors see it.
2026-03-03 22:21:55 -05:00
Kevin Turcios
6256fd7efe refactor: deduplicate format_code and validate_syntax between JS and TS support
Use self.default_file_extension in format_code and a treesitter_language
property in validate_syntax so TypeScriptSupport inherits both methods
from JavaScriptSupport instead of duplicating them.
2026-03-03 22:21:55 -05:00
Sarthak Agarwal
84ac714a89 fix: use pattern matching for collocated tests in monorepos
When testsRoot overlaps moduleRoot (common in JS/TS monorepos like Ghost
where both point to "src"), the directory-based filter incorrectly
excluded ALL source files. Switch to filename/directory pattern matching
(*.test.*, *.spec.*, __tests__/) when roots overlap, preserving the
existing directory-based filter for standard layouts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 22:21:55 -05:00
mashraf-222
82e4627b03
Merge pull request #1740 from codeflash-ai/fix/java-comparator-and-errorprone
fix: suppress Error Prone CheckReturnValue in instrumented tests and fix stale pom dependency
2026-03-04 05:21:24 +02:00
Kevin Turcios
d46908faa2
Merge pull request #1752 from codeflash-ai/chore/pre-push-prek-rule
chore: add pre-push prek rule and fix pr-review close permission
2026-03-04 03:10:30 +00:00
misrasaurabh1
26f2c502fb feat: add inline runtime comments and unique _cf_mod/_cf_cls markers for Java tests
- Use instrumented class name in _cf_mod/_cf_cls markers to disambiguate
  existing vs generated tests sharing the same original class name
- Encode line number in invocation IDs (L{line}_{counter}) for deterministic
  call-site identification in inline runtime comments
- Rewrite add_runtime_comments() to annotate each call line with inline
  performance data instead of a summary block at top
- Strip assertions before instrumenting so both modes share the same base source
- Update test expected strings for new marker format

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:58:20 -08:00
Kevin Turcios
7c4ae58b18
Merge pull request #1751 from codeflash-ai/merge/language-refactoring-into-omni-java
feat: port main improvements into omni-java
2026-03-04 02:51:23 +00:00
Kevin Turcios
ec73c2c692 revert: undo Vitest benchmarking and capture.js changes
Reverts ed2594bf which added --pool=forks to Vitest commands and
changed capture.js to use process.stdout.write and Vitest worker API.
These changes broke JS E2E tests (CJS, ESM, TS class) by altering
how all JS tests run, not just Vitest benchmarking.
2026-03-03 21:34:59 -05:00
Kevin Turcios
b2b8201541 fix: update test_get_helper_code expectation for blank line after imports 2026-03-03 21:10:35 -05:00
Kevin Turcios
8f50f31251 fix: add gh pr close permission to pr-review workflow
The merge_optimization_prs step needs to close stale PRs but
gh pr close was missing from the allowed tools list.
2026-03-03 21:04:14 -05:00
Kevin Turcios
c2fe5261a3 chore: add pre-push prek rule to match CI behavior 2026-03-03 21:02:44 -05:00
Kevin Turcios
fb6a4ab587 fix: update test expectations for extra blank line after imports
The CST comment position fix changed how blank lines are preserved
after import blocks, adding a PEP 8-compliant double blank line.
2026-03-03 20:58:32 -05:00
Kevin Turcios
c6a1b9a16d style: add missing TYPE_CHECKING imports for ValidCode and TestConfig 2026-03-03 20:51:41 -05:00
Sarthak Agarwal
f5d48841f0 fix mocha test runner 2026-03-03 20:50:50 -05:00
Sarthak Agarwal
6e34dcce6b fix: regenerate stale vitest config and add .js extension for ESM imports
Two fixes discovered during end-to-end testing:

1. _ensure_codeflash_vitest_config() reused existing configs that lacked
   pool: 'forks', causing benchmarking to fall back to wall-clock timing
   on re-runs. Now checks for the required setting and regenerates if
   missing.

2. ESM projects require explicit .js file extensions in import specifiers
   (e.g., `import foo from './lib/foo.js'`). The verifier stripped the
   extension but never added .js back for ESM, causing
   ERR_MODULE_NOT_FOUND in Mocha ESM projects like axios.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:49:09 -05:00
Sarthak Agarwal
4f2c65daec fix: strip CJS require('vitest') and require('@jest/globals') in Mocha tests
The AI backend generates vitest/jest-style imports for Mocha projects.
Our sanitize_mocha_imports() stripped ESM `import { ... } from 'vitest'`,
but process_generated_test_strings() runs BEFORE postprocessing and calls
ensure_module_system_compatibility() which converts these to CJS requires.
Result: `const { ... } = require('vitest')` survived sanitization.

Added regexes for the CJS variants of vitest and @jest/globals requires.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:49:04 -05:00
Sarthak Agarwal
74150c2a7f fix: resolve Vitest benchmarking showing wall-clock time instead of per-function timing
Root cause: Vitest performance tests reported "20.0 seconds over 1 loop"
(JUnit XML wall-clock fallback) instead of actual per-function nanosecond
timing. This was a chain of two issues:

1. **stdout interception**: Vitest's default `threads` pool intercepts
   process.stdout.write() and console.log(), preventing timing markers
   from flowing to the parent process. Fixed by adding `--pool=forks`
   to all Vitest commands and config files. The `forks` pool uses child
   processes where stdout flows directly to the parent.

2. **test name detection**: Even after markers flowed through (43,000+
   found in stdout), the parser couldn't match them to JUnit XML
   testcases because all markers had "unknown" as the test name. This
   happened because Vitest doesn't inject `beforeEach` as a global
   (unlike Jest), so capture.js's Jest-style hook to set
   `currentTestName` never fired.

   Fixed by adding Vitest-specific test name detection in capture.js:
   - Primary: `expect.getState().currentTestName` (full describe path)
   - Fallback: `__vitest_worker__.current.fullTestName`
   - Defense-in-depth: parser fallback matches "unknown" markers to
     the first testcase when no name match is found

Result: cheerio's `isHtml` went from "20.0s / 1 loop" to
"902μs / 20,853 loops" with proper speedup analysis.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:48:59 -05:00
Sarthak Agarwal
b39f2a2e9b fix: support Mocha CJS projects and sanitize incorrect framework imports
Three related fixes for Mocha test generation in CommonJS projects:

1. inject_test_globals() now accepts module_system param — emits
   `require('node:assert/strict')` for CJS instead of ESM import syntax
2. ensure_module_system_compatibility() now converts ESM→CJS even when
   the source has mixed imports (was skipping when both ESM and CJS were
   detected, leaving the ESM import from inject_test_globals unconverted)
3. New sanitize_mocha_imports() strips vitest/jest/@jest/globals imports
   that the AI sometimes generates for Mocha projects — Mocha provides
   describe/it/before*/after* as globals

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:48:53 -05:00
Sarthak Agarwal
15fcf4741d add mocha support 2026-03-03 20:48:39 -05:00
Sarthak Agarwal
e6b2f7975b feat: bundle JUnit XML reporter for Jest, replacing external jest-junit dependency
Ship a zero-dependency jest-reporter.js inside the codeflash runtime package
instead of requiring the external jest-junit npm package. This ensures the
reporter is always available when codeflash is installed, fixing Jest-based
projects (Strapi, Moleculer) that failed because jest-junit wasn't installed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:48:34 -05:00
Kevin Turcios
894803c8fd style: fix ruff formatting issues from cherry-picked commits 2026-03-03 20:45:29 -05:00
Kevin Turcios
8f0ff84ecf fix: update tests for consolidated CST discovery behavior
Nested functions are now skipped by FunctionVisitor, and
discover_functions no longer swallows parse/IO errors — callers
handle them. Update test expectations accordingly.
2026-03-03 20:42:33 -05:00
Kevin Turcios
92a6460ac3 refactor: consolidate Python function discovery to CST path only
Remove the AST-based discovery path for Python, routing all languages
through the unified CST-based `_find_all_functions_via_language_support`.
Delete dead code: `find_functions_with_return_statement`,
`_find_all_functions_in_python_file`, `function_has_return_statement`,
`function_is_a_property`, and associated constants. Fix FunctionVisitor
to skip nested functions and exclude @property/@cached_property, and let
parse errors propagate for correct empty-dict behavior on invalid files.
2026-03-03 20:42:27 -05:00
Sarthak Agarwal
cb1ea5adc3 fix: use pattern matching for collocated tests in monorepos
When testsRoot overlaps moduleRoot (common in JS/TS monorepos like Ghost
where both point to "src"), the directory-based filter incorrectly
excluded ALL source files. Switch to filename/directory pattern matching
(*.test.*, *.spec.*, __tests__/) when roots overlap, preserving the
existing directory-based filter for standard layouts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:41:45 -05:00
Sarthak Agarwal
a429f47d10 fix: show actionable error when JS/TS function exists but is not exported
When a user targets a function that exists in the file but is not exported,
show a clear message suggesting they add an export statement, instead of
the generic "function not found" error.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:41:40 -05:00
Sarthak Agarwal
095b8b76d0 feat: add skip_confirm and skip_api_key params to JS init for non-interactive mode
Allow init_js_project(), should_modify_package_json_config(), and
collect_js_setup_info() to run without interactive prompts when
skip_confirm=True. Uses auto-detected defaults instead of prompting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:40:54 -05:00
Sarthak Agarwal
a14ef40d15 fix: raise clear error for unsupported JS test frameworks instead of silent fallback
Add NotImplementedError guard in all 3 test dispatchers (behavioral,
benchmarking, line-profile) for frameworks other than jest and vitest.
Previously, mocha and other frameworks silently fell through to Jest,
causing confusing failures. Now users get a clear error message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:40:49 -05:00
HeshamHM28
ade8558742
Update codeflash/languages/java/support.py
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
2026-03-03 17:40:11 -08:00
Sarthak Agarwal
800ebed837 feat: discover object methods exported via CJS module.exports = variable
Resolve `module.exports = varName` where varName is an object literal
containing methods. For patterns like `const utils = { match() {} };
module.exports = utils;`, the individual methods are now recognized as
exported. This fixes function discovery for CJS libraries like Moleculer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:40:06 -05:00
Sarthak Agarwal
b3d4e225e5 feat: discover const arrow functions exported via named export clauses
Post-process find_functions() to mark functions as exported when they appear
in named export clauses like `export { joinBy }`. This fixes discovery for
TypeScript codebases (e.g., Strapi) that define const arrow functions and
export them via a separate export statement.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:40:01 -05:00
claude[bot]
2606f483de fix: re-export jest patterns for backward compatibility 2026-03-03 20:39:00 -05:00
Kevin Turcios
a89dc4b09b test: add regression tests for comment position preservation 2026-03-03 20:36:56 -05:00
Kevin Turcios
b5fab57499 fix: preserve comment position by passing CST module directly to import adder
parse_code_and_prune_cst now returns cst.Module instead of str.
add_needed_imports_from_module accepts cst.Module | str, skipping re-parse
when a Module is passed. This eliminates the string round-trip that caused
comments to migrate from statement leading_lines to Module.header,
resulting in comments appearing above imports instead of at their
original position.
2026-03-03 20:36:52 -05:00