## Summary
- Add `instrumented_perf_test` field to `OptimizationFeatures` model
- Update `log_features` function to accept and store performance
instrumented tests
---------
Co-authored-by: Sarthak Agarwal <sarthak.saga@gmail.com>
Use postprocessing_testgen_pipeline with source_code_being_tested parameter
instead of calling add_missing_imports separately. Add debug logging for
investigating 'i' import bug.
## Summary
Adds a line-by-line performance profiler visualization to the webapp,
allowing users to compare execution times between original and optimized
code.
## Changes
### New Line Profiler View
- **`LineProfilerView.tsx`**: Side-by-side comparison component showing:
- Line-by-line execution times with heat map visualization
- Syntax highlighting using `prism-react-renderer`
- Collapsible function blocks
- Light/dark mode support
- Heat legend (cold → hot based on % time)
- **`lineProfilerParser.ts`**: Parser utilities for line profiler data:
- `parseLineProfilerResults()` - parses markdown table output from
Python's line_profiler
- `formatTime()` - converts timer units to human-readable format (ns,
µs, ms, s)
- `getHeatLevel()` - determines heat coloring based on % time
- **`/review-optimizations/[traceId]/profiler/page.tsx`**: New route for
the profiler view
### API Changes
- **`create-pr.ts`**: Adds "📊 Performance Profile" link to PR
description when profiler data exists
- **`github-app.ts`**: Removes line profiler data from metadata when PR
is closed/merged
- **`create-staging.ts`**, **`suggest-pr-changes.ts`**: Handle line
profiler data in staging
- **`staging-storage-strategy.ts`**: Interface updates for line profiler
fields
### Webapp Integration
- **`page.tsx`**: Added "Performance Profile" button (only visible when
profiler data exists)
- **`action.ts`**: Sends line profiler data when creating PR from webapp
Fixes CF-1018
https://codeflash-ai.slack.com/files/U08MSR1UN6L/F0A9YVDJY75/screen_recording_2026-01-21_at_10.03.18___pm.movhttps://github.com/HeshamHM28/my-best-repo/pull/21
linked to https://github.com/codeflash-ai/codeflash/pull/1139
---------
Co-authored-by: Aseem Saxena <aseem.bits@gmail.com>
Remove prompt parameter from claude-mention job so Claude runs in
interactive mode and naturally receives @mention context. Move prek
formatting instructions to CLAUDE.md where Claude reads them for any
request.
- Add standalone fix-formatting.yml workflow for `/fix-formatting` command
- Uses prek's native --from-ref to only format changed files
- Properly handles prek exit codes and reports errors
- Enable show_full_output in claude-mention job for debugging
Update the prompt to explicitly mention "pre-k" and "prek" triggers
and provide step-by-step instructions for running the formatter,
committing, and pushing changes.
## ⚡️ This pull request contains optimizations for PR #2247
If you approve this dependent PR, these changes will be merged into the
original PR branch `multi-language`.
>This PR will be automatically closed if the original PR is merged.
----
#### 📄 18% (0.18x) speedup for ***`_has_test_functions` in
`django/aiservice/testgen/testgen_javascript.py`***
⏱️ Runtime : **`740 microseconds`** **→** **`627 microseconds`** (best
of `76` runs)
#### 📝 Explanation and details
The optimized code achieves an **18% runtime improvement** by
eliminating repeated regex compilation overhead.
**Key optimization:**
- **Precompiled regex pattern**: The pattern
`r"(?:test|it)\s*\(\s*['\"]"` is compiled once at module load time into
`_TEST_FUNC_RE`, rather than being recompiled on every function call. In
Python, `re.search()` with a raw string pattern incurs compilation cost
each time it's invoked.
**Performance breakdown from line profiler:**
- Original: 2.70ms spent in `re.search(test_pattern, code)` (96.5% of
total time)
- Optimized: 862μs for the direct pattern search (100% of total time,
but 3.1x faster overall)
- The pattern string assignment overhead (97μs in original) is
eliminated entirely
**Why this matters for the workload:**
Based on `function_references`, this function is called from
`parse_and_validate_js_output()` during LLM response validation. This is
a **hot path** operation that executes on every test generation request.
The validation flow checks multiple conditions including syntax
validation before checking for test functions, meaning this function
runs repeatedly during normal operations.
**Test case performance:**
- **Small inputs** (single test functions): 50-80% faster (e.g., 2.80μs
→ 1.83μs)
- **Empty/minimal strings**: 130-140% faster (e.g., 1.80μs → 750ns)
- **Large inputs** (500-1000 lines): 1-8% faster depending on match
location
- **Early matches** benefit most since regex short-circuits on first
match
The optimization is most effective when processing typical-sized
JavaScript test code (dozens to hundreds of lines), which aligns with
the common use case of validating LLM-generated test functions.
✅ **Correctness verification report:**
| Test | Status |
| --------------------------- | ----------------- |
| ⚙️ Existing Unit Tests | 🔘 **None Found** |
| 🌀 Generated Regression Tests | ✅ **102 Passed** |
| ⏪ Replay Tests | 🔘 **None Found** |
| 🔎 Concolic Coverage Tests | 🔘 **None Found** |
|📊 Tests Coverage | 100.0% |
<details>
<summary>🌀 Click to see Generated Regression Tests</summary>
```python
from __future__ import annotations
import re
# imports
import pytest # used for our unit tests
from testgen.testgen_javascript import _has_test_functions
def test_basic_test_call_double_quotes():
# Basic: a standard Jest test call using double quotes should be detected.
code = 'test("my test name", () => { expect(true).toBe(true); });'
codeflash_output = _has_test_functions(code) # 2.83μs -> 1.78μs (59.3% faster)
def test_basic_it_call_single_quotes():
# Basic: a standard it() call using single quotes should be detected.
code = "it('does something', function() { /* ... */ });"
codeflash_output = _has_test_functions(code) # 2.80μs -> 1.83μs (53.0% faster)
def test_whitespace_and_newlines_between_name_and_paren():
# Edge: whitespace/newlines between the function name and '(' and between '(' and the quote
# The regex allows arbitrary whitespace, so this should still match.
code = "it \n (\n 'handles newlines'\n )"
codeflash_output = _has_test_functions(code) # 2.90μs -> 1.89μs (53.2% faster)
def test_empty_string_returns_false():
# Edge: empty input must return False (no tests found).
code = ""
codeflash_output = _has_test_functions(code) # 1.80μs -> 750ns (139% faster)
def test_uppercase_function_name_not_matched():
# Edge: the regex is case-sensitive; 'Test' should NOT match.
code = "Test('capitalized should not match', () => {});"
codeflash_output = _has_test_functions(code) # 3.20μs -> 2.23μs (43.7% faster)
def test_backtick_template_not_matched():
# Edge: template literals use backticks; pattern looks only for single/double quotes.
code = "test(`template literal name`, () => {});"
codeflash_output = _has_test_functions(code) # 3.34μs -> 2.27μs (47.0% faster)
def test_numeric_first_arg_not_matched():
# Edge: if the first argument is not a quoted string (e.g., a number), pattern should not match.
code = "test(123, () => {});"
codeflash_output = _has_test_functions(code) # 3.01μs -> 1.86μs (61.7% faster)
def test_test_call_inside_comment_still_matches():
# Important behavioral note: the function does not ignore comments.
# A 'test(' occurrence inside a JS comment still matches because the function only does regex search.
code = "// test('in a single-line comment')\n/* test(\"in block comment\") */"
# Both comment forms contain test('...') / test("...") which the regex will find.
codeflash_output = _has_test_functions(code) # 2.96μs -> 1.84μs (61.1% faster)
def test_substring_in_identifier_matches():
# The regex is permissive and will match occurrences where 'test' or 'it' appear as suffixes
# of other identifiers (e.g., 'latesttest(' or 'split('). This test documents that behavior.
code_latest = "function latesttest(){}\nlatesttest('x')"
code_split = "const arr = ['a']; arr.split('a');"
# Both contain the substring "test('..." or "it('...", so they should be considered matches by the implementation.
codeflash_output = _has_test_functions(
code_latest
) # 3.77μs -> 2.59μs (45.5% faster)
codeflash_output = _has_test_functions(code_split) # 1.16μs -> 793ns (46.7% faster)
def test_comment_between_paren_blocks_prevents_match():
# If there is a non-whitespace token (like a block comment) between '(' and the starting quote,
# the current regex will not match because it expects only whitespace between '(' and the quote.
code = "test(/* important note */ 'name in comment')"
codeflash_output = _has_test_functions(code) # 3.31μs -> 2.30μs (43.5% faster)
def test_multiple_test_and_it_occurrences():
# A file with multiple matches should still return True (boolean).
code = """
describe('suite', () => {
it('first case', () => {});
// some other code
test("second case", () => {});
});
"""
codeflash_output = _has_test_functions(code) # 3.34μs -> 2.16μs (54.9% faster)
def test_large_scale_no_match_performance():
# Large-scale: many lines without any test/it(...) occurrences should return False.
# Keep size under 1000 to respect constraints. We use 900 repeated lines.
repeated = "const filler = 0;\n" * 900 # 900 lines of filler
codeflash_output = _has_test_functions(repeated) # 76.6μs -> 75.5μs (1.47% faster)
def test_large_scale_match_near_end():
# Large-scale: many lines of filler followed by a single test at the end should return True.
# This ensures the search scans through large input and finds a late occurrence.
repeated = "const filler = 0;\n" * 900 # 900 lines of filler
code = repeated + " // real test follows\n test('final case', () => {});"
codeflash_output = _has_test_functions(code) # 77.5μs -> 76.3μs (1.59% faster)
def test_it_with_newline_between_name_and_paren():
# Verify that a newline immediately after 'it' and before '(' is allowed by the regex (\s* covers newline).
code = "it\n('newline-allowed')"
codeflash_output = _has_test_functions(code) # 3.22μs -> 1.94μs (66.0% faster)
def test_quoted_string_with_escaped_quotes_still_matches():
# Even if the string contains escaped quotes, the regex only checks the opening quote, so it should match.
code = r'test("contains an escaped quote: \" here", () => {});'
codeflash_output = _has_test_functions(code) # 2.90μs -> 1.83μs (58.2% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
```
```python
import re
import pytest
from testgen.testgen_javascript import _has_test_functions
def test_simple_test_function_with_single_quotes():
"""Test detection of test() function with single quotes."""
code = "test('should work', () => {})"
codeflash_output = _has_test_functions(code) # 3.78μs -> 2.29μs (65.1% faster)
def test_simple_it_function_with_single_quotes():
"""Test detection of it() function with single quotes."""
code = "it('should work', () => {})"
codeflash_output = _has_test_functions(code) # 3.10μs -> 1.99μs (56.0% faster)
def test_simple_test_function_with_double_quotes():
"""Test detection of test() function with double quotes."""
code = 'test("should work", () => {})'
codeflash_output = _has_test_functions(code) # 3.07μs -> 1.89μs (62.8% faster)
def test_simple_it_function_with_double_quotes():
"""Test detection of it() function with double quotes."""
code = 'it("should work", () => {})'
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.86μs (68.2% faster)
def test_no_test_functions():
"""Test code without any test functions returns False."""
code = "function myFunction() { return 42; }"
codeflash_output = _has_test_functions(code) # 3.08μs -> 1.85μs (66.5% faster)
def test_test_function_with_single_whitespace():
"""Test detection with single space between function name and parenthesis."""
code = "test ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.20μs -> 1.96μs (63.0% faster)
def test_it_function_with_single_whitespace():
"""Test detection with single space between function name and parenthesis."""
code = "it ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.98μs -> 1.88μs (58.5% faster)
def test_test_function_with_multiple_whitespaces():
"""Test detection with multiple spaces between function name and parenthesis."""
code = "test ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.04μs -> 1.95μs (55.5% faster)
def test_it_function_with_multiple_whitespaces():
"""Test detection with multiple spaces between function name and parenthesis."""
code = "it ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.06μs -> 1.89μs (62.1% faster)
def test_test_function_with_tab_character():
"""Test detection with tab character between function name and parenthesis."""
code = "test\t('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.05μs -> 1.93μs (57.8% faster)
def test_it_function_with_tab_character():
"""Test detection with tab character between function name and parenthesis."""
code = "it\t('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.99μs -> 1.85μs (61.6% faster)
def test_test_function_with_newline():
"""Test detection with newline between function name and parenthesis."""
code = "test\n('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.02μs -> 1.93μs (56.5% faster)
def test_it_function_with_newline():
"""Test detection with newline between function name and parenthesis."""
code = "it\n('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.95μs -> 1.90μs (55.4% faster)
def test_multiple_test_functions():
"""Test detection with multiple test functions in the same code."""
code = """
test('first test', () => {});
it('second test', () => {});
"""
codeflash_output = _has_test_functions(code) # 3.12μs -> 1.92μs (62.3% faster)
def test_test_function_in_multiline_code():
"""Test detection of test function within multiline code."""
code = """
const helper = () => {};
test('actual test', () => {});
const another = () => {};
"""
codeflash_output = _has_test_functions(code) # 3.19μs -> 2.12μs (50.1% faster)
def test_it_function_in_multiline_code():
"""Test detection of it function within multiline code."""
code = """
const helper = () => {};
it('actual test', () => {});
const another = () => {};
"""
codeflash_output = _has_test_functions(code) # 3.46μs -> 2.23μs (55.1% faster)
def test_test_word_in_comment_not_matched():
"""Test that test() in comments is still detected by regex (no comment parsing)."""
code = "// test('in comment', () => {})"
# Note: The function uses regex without comment awareness, so it will match
codeflash_output = _has_test_functions(code) # 3.10μs -> 1.94μs (60.2% faster)
def test_test_word_in_string_variable():
"""Test that test word in string variable doesn't match pattern."""
code = 'const description = "this is a test of something";'
codeflash_output = _has_test_functions(code) # 3.56μs -> 2.29μs (55.3% faster)
def test_test_as_variable_name_not_matched():
"""Test that 'test' as variable name doesn't match without parenthesis."""
code = "const test = 5;"
codeflash_output = _has_test_functions(code) # 3.11μs -> 2.03μs (53.1% faster)
def test_testing_as_word_not_matched():
"""Test that 'testing' word doesn't match."""
code = "const testing = 'some value';"
codeflash_output = _has_test_functions(code) # 3.17μs -> 2.04μs (55.4% faster)
def test_it_as_pronoun_not_matched():
"""Test that 'it' as pronoun doesn't match without proper pattern."""
code = "// it is a good day"
codeflash_output = _has_test_functions(code) # 3.15μs -> 1.98μs (59.1% faster)
def test_it_as_variable_not_matched():
"""Test that 'it' as variable name doesn't match without parenthesis."""
code = "const it = 5;"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.92μs (63.7% faster)
def test_empty_string():
"""Test with empty string input."""
codeflash_output = _has_test_functions("") # 1.74μs -> 757ns (129% faster)
def test_only_whitespace():
"""Test with only whitespace."""
codeflash_output = _has_test_functions(" \n\t ") # 2.05μs -> 890ns (131% faster)
def test_test_function_with_special_test_name():
"""Test detection with special characters in test name."""
code = "test('test-name_123!@#', () => {})"
codeflash_output = _has_test_functions(code) # 2.95μs -> 1.90μs (55.1% faster)
def test_it_function_with_special_test_name():
"""Test detection with special characters in test name."""
code = "it('it-name_123!@#', () => {})"
codeflash_output = _has_test_functions(code) # 3.04μs -> 1.88μs (62.0% faster)
def test_test_function_with_empty_string_name():
"""Test detection with empty string as test name."""
code = "test('', () => {})"
codeflash_output = _has_test_functions(code) # 2.90μs -> 1.84μs (58.0% faster)
def test_it_function_with_empty_string_name():
"""Test detection with empty string as test name."""
code = "it('', () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.86μs (59.7% faster)
def test_test_with_carriage_return():
"""Test detection with carriage return character."""
code = "test\r('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.02μs -> 1.89μs (60.2% faster)
def test_it_with_carriage_return():
"""Test detection with carriage return character."""
code = "it\r('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.12μs -> 1.91μs (63.8% faster)
def test_test_with_form_feed():
"""Test detection with form feed character."""
code = "test\f('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.98μs -> 1.85μs (60.7% faster)
def test_it_with_form_feed():
"""Test detection with form feed character."""
code = "it\f('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.91μs (64.4% faster)
def test_test_with_vertical_tab():
"""Test detection with vertical tab character."""
code = "test\v('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.08μs -> 1.87μs (65.0% faster)
def test_it_with_vertical_tab():
"""Test detection with vertical tab character."""
code = "it\v('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.83μs (71.4% faster)
def test_test_with_non_breaking_space():
"""Test that non-breaking space might not work depending on whitespace regex."""
code = "test\u00a0('my test', () => {})"
# Non-breaking space might not be treated as \s in regex
codeflash_output = _has_test_functions(code)
result = codeflash_output # 3.25μs -> 2.07μs (56.6% faster)
def test_test_with_zero_width_space():
"""Test with zero-width space."""
code = "test\u200b('my test', () => {})"
codeflash_output = _has_test_functions(code)
result = codeflash_output # 3.82μs -> 2.74μs (39.4% faster)
def test_only_test_keyword():
"""Test with only the word 'test' without parenthesis."""
code = "test"
codeflash_output = _has_test_functions(code) # 2.85μs -> 1.83μs (55.3% faster)
def test_only_it_keyword():
"""Test with only the word 'it' without parenthesis."""
code = "it"
codeflash_output = _has_test_functions(code) # 1.89μs -> 793ns (138% faster)
def test_test_with_parenthesis_but_no_quote():
"""Test function call without string argument."""
code = "test(variable)"
codeflash_output = _has_test_functions(code) # 3.39μs -> 2.12μs (59.8% faster)
def test_it_with_parenthesis_but_no_quote():
"""Test it function call without string argument."""
code = "it(variable)"
codeflash_output = _has_test_functions(code) # 3.24μs -> 2.07μs (56.5% faster)
def test_test_followed_by_string_literal_without_parenthesis():
"""Test with string literal but missing parenthesis."""
code = "test 'string'"
codeflash_output = _has_test_functions(code) # 3.17μs -> 1.93μs (64.5% faster)
def test_it_followed_by_string_literal_without_parenthesis():
"""Test with string literal but missing parenthesis."""
code = "it 'string'"
codeflash_output = _has_test_functions(code) # 3.24μs -> 2.02μs (59.9% faster)
def test_test_with_backtick_quotes():
"""Test with backtick quotes (template literals)."""
code = "test(`my test`, () => {})"
codeflash_output = _has_test_functions(code) # 3.49μs -> 2.34μs (49.5% faster)
def test_it_with_backtick_quotes():
"""Test it with backtick quotes (template literals)."""
code = "it(`my test`, () => {})"
codeflash_output = _has_test_functions(code) # 3.38μs -> 2.23μs (51.4% faster)
def test_describe_function_not_matched():
"""Test that describe() function is not matched."""
code = "describe('suite', () => {})"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.94μs (61.2% faster)
def test_beforeEach_function_not_matched():
"""Test that beforeEach() function is not matched."""
code = "beforeEach(() => {})"
codeflash_output = _has_test_functions(code) # 1.98μs -> 901ns (119% faster)
def test_afterEach_function_not_matched():
"""Test that afterEach() function is not matched."""
code = "afterEach(() => {})"
codeflash_output = _has_test_functions(code) # 2.68μs -> 1.50μs (78.6% faster)
def test_test_method_on_object():
"""Test with test as method call on object."""
code = "obj.test('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.12μs -> 1.94μs (61.4% faster)
def test_it_method_on_object():
"""Test with it as method call on object."""
code = "obj.it('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.10μs -> 2.00μs (55.0% faster)
def test_test_substring_in_longer_identifier():
"""Test when test is part of longer identifier."""
code = "mytest('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.00μs -> 1.90μs (58.0% faster)
def test_it_substring_in_longer_identifier():
"""Test when it is part of longer identifier."""
code = "unit('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.07μs -> 1.94μs (58.2% faster)
def test_test_with_unicode_test_name():
"""Test detection with unicode characters in test name."""
code = "test('\u4e2d\u6587\u6d4b\u8bd5', () => {})"
codeflash_output = _has_test_functions(code) # 3.57μs -> 2.31μs (54.4% faster)
def test_it_with_unicode_test_name():
"""Test detection with unicode characters in test name."""
code = "it('\u4e2d\u6587\u6d4b\u8bd5', () => {})"
codeflash_output = _has_test_functions(code) # 3.09μs -> 2.08μs (48.8% faster)
def test_test_with_emoji():
"""Test detection with emoji in test name."""
code = "test('\u263a emoji test', () => {})"
codeflash_output = _has_test_functions(code) # 3.14μs -> 2.02μs (55.2% faster)
def test_it_with_emoji():
"""Test detection with emoji in test name."""
code = "it('\u263a emoji test', () => {})"
codeflash_output = _has_test_functions(code) # 3.09μs -> 2.00μs (54.0% faster)
def test_very_long_test_name():
"""Test detection with very long test name."""
long_name = "a" * 5000
code = f"test('{long_name}', () => {{}})"
codeflash_output = _has_test_functions(code) # 3.00μs -> 1.77μs (69.5% faster)
def test_very_long_code_without_tests():
"""Test with very long code but no test functions."""
code = "const x = 1;\n" * 500
codeflash_output = _has_test_functions(code) # 29.7μs -> 28.5μs (4.29% faster)
def test_test_with_escaped_quote():
"""Test with escaped quote in test name."""
code = "test('test\\'s name', () => {})"
codeflash_output = _has_test_functions(code) # 3.15μs -> 2.03μs (55.2% faster)
def test_it_with_escaped_quote():
"""Test with escaped quote in test name."""
code = "it('it\\'s name', () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.87μs (59.0% faster)
def test_test_with_double_quote_in_single_quote():
"""Test with double quote inside single quoted test name."""
code = "test('has \"double\" quotes', () => {})"
codeflash_output = _has_test_functions(code) # 2.93μs -> 1.82μs (60.7% faster)
def test_it_with_double_quote_in_single_quote():
"""Test with double quote inside single quoted test name."""
code = "it('has \"double\" quotes', () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.83μs (62.6% faster)
def test_test_with_single_quote_in_double_quote():
"""Test with single quote inside double quoted test name."""
code = "test(\"has 'single' quotes\", () => {})"
codeflash_output = _has_test_functions(code) # 3.00μs -> 1.90μs (57.8% faster)
def test_it_with_single_quote_in_double_quote():
"""Test with single quote inside double quoted test name."""
code = "it(\"has 'single' quotes\", () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.78μs (66.7% faster)
def test_test_case_sensitive():
"""Test that TEST (uppercase) is not matched."""
code = "TEST('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.03μs -> 1.81μs (67.1% faster)
def test_it_case_sensitive():
"""Test that IT (uppercase) is not matched."""
code = "IT('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.81μs -> 1.84μs (52.8% faster)
def test_test_with_mixed_case():
"""Test that TeSt (mixed case) is not matched."""
code = "TeSt('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.05μs -> 1.85μs (65.2% faster)
def test_it_with_mixed_case():
"""Test that It (mixed case) is not matched."""
code = "It('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.94μs -> 1.91μs (54.3% faster)
def test_code_with_many_non_test_functions():
"""Test performance with many non-test functions."""
# Build code with 500 non-test function definitions
code_lines = [f"function func{i}() {{ return {i}; }}" for i in range(500)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 77.0μs -> 75.9μs (1.52% faster)
def test_code_with_many_functions_and_one_test():
"""Test detection of single test among many non-test functions."""
# Build code with 500 non-test functions and 1 test function
code_lines = [f"function func{i}() {{ return {i}; }}" for i in range(250)]
code_lines.append("test('the actual test', () => {})")
code_lines.extend(
[f"function func{i}() {{ return {i}; }}" for i in range(250, 500)]
)
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 40.3μs -> 38.9μs (3.41% faster)
def test_code_with_many_test_functions():
"""Test detection with many test functions."""
# Build code with 100 test functions
code_lines = [f"test('test {i}', () => {{}})" for i in range(100)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.94μs -> 1.74μs (69.4% faster)
def test_code_with_many_it_functions():
"""Test detection with many it functions."""
# Build code with 100 it functions
code_lines = [f"it('test {i}', () => {{}})" for i in range(100)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.91μs -> 1.61μs (80.4% faster)
def test_code_with_alternating_test_and_it_functions():
"""Test detection with alternating test and it functions."""
# Build code with 100 alternating test and it functions
code_lines = []
for i in range(50):
code_lines.append(f"test('test {i}', () => {{}})")
code_lines.append(f"it('it {i}', () => {{}})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.93μs -> 1.64μs (78.7% faster)
def test_code_with_many_non_matching_similar_patterns():
"""Test performance with many similar but non-matching patterns."""
# Build code with 500 similar patterns that don't match
code_lines = [f"test{i}('name', () => {{}})" for i in range(500)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 64.3μs -> 62.9μs (2.27% faster)
def test_large_code_with_test_at_end():
"""Test detection when test function is at end of large code."""
# Build code with 500 lines and test at the end
code_lines = [f"const var{i} = {i};" for i in range(500)]
code_lines.append("test('test at end', () => {})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 41.4μs -> 40.3μs (2.65% faster)
def test_large_code_with_it_at_end():
"""Test detection when it function is at end of large code."""
# Build code with 500 lines and it at the end
code_lines = [f"const var{i} = {i};" for i in range(500)]
code_lines.append("it('it at end', () => {})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 41.5μs -> 40.3μs (3.00% faster)
def test_large_code_with_test_at_beginning():
"""Test detection when test function is at beginning of large code."""
# Build code with test at beginning and 500 lines after
code_lines = ["test('test at beginning', () => {})"]
code_lines.extend([f"const var{i} = {i};" for i in range(500)])
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 3.09μs -> 1.90μs (62.2% faster)
def test_large_code_with_it_at_beginning():
"""Test detection when it function is at beginning of large code."""
# Build code with it at beginning and 500 lines after
code_lines = ["it('it at beginning', () => {})"]
code_lines.extend([f"const var{i} = {i};" for i in range(500)])
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 3.02μs -> 1.81μs (67.1% faster)
def test_code_with_multiple_tests_scattered():
"""Test detection with multiple test functions scattered throughout large code."""
# Build code with 20 test functions scattered among 480 non-test lines
code_lines = []
for i in range(500):
if i % 25 == 0:
code_lines.append(f"test('scattered test {i}', () => {{}})")
else:
code_lines.append(f"const var{i} = {i};")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.86μs -> 1.79μs (59.8% faster)
def test_code_with_very_large_test_name():
"""Test performance with very long test name."""
# Create a test with name of 10000 characters
long_name = "x" * 10000
code = f"test('{long_name}', () => {{}})"
codeflash_output = _has_test_functions(code) # 2.99μs -> 1.84μs (62.1% faster)
def test_code_with_deeply_nested_structures():
"""Test detection in deeply nested code structures."""
# Build nested structure with test at bottom
code = "const nested = { level1: { level2: { level3: { level4: { " * 50
code += "test('nested test', () => {})"
code += " } } } } };" * 50
codeflash_output = _has_test_functions(code) # 14.6μs -> 13.5μs (8.56% faster)
def test_code_with_many_whitespace_variations():
"""Test detection with many different whitespace patterns."""
code_lines = []
for i in range(100):
if i % 4 == 0:
code_lines.append(f"test('test {i}', () => {{}})")
elif i % 4 == 1:
code_lines.append(f"test ('test {i}', () => {{}})")
elif i % 4 == 2:
code_lines.append(f"test ('test {i}', () => {{}})")
else:
code_lines.append(f"test\t('test {i}', () => {{}})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.92μs -> 1.71μs (70.7% faster)
def test_code_return_type_is_boolean():
"""Test that return value is always boolean regardless of input size."""
# Various test inputs
test_inputs = [
"",
"test",
"test('name', () => {})",
"const x = 1;" * 100,
"test('name', () => {})" + "const x = 1;" * 100,
]
for test_input in test_inputs:
codeflash_output = _has_test_functions(test_input)
result = codeflash_output # 11.9μs -> 9.34μs (27.0% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
```
</details>
To edit these changes `git checkout
codeflash/optimize-pr2247-2026-01-25T08.57.25` and push.
[](https://codeflash.ai)
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: Kevin Turcios <106575910+KRRT7@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Fix CST tree corruption issues that caused 'NoneType' object has no
attribute 'visit' errors
- Consolidate testgen postprocessing into a single pipeline with
tuple-based pattern
- Improve markdown code extraction to prefer filepath-annotated blocks
- Add diagnostic context to optimization failure logs
## Changes
- Handle empty `SimpleStatementLine` and `StatementHandler` body to
prevent malformed CST
- Add trace_id logging to optimization and import failure paths
- Refactor testgen postprocessing into consolidated pipeline
- Fix code extraction for LLM responses with multiple code blocks
## Test plan
- [x] Added integration tests for full testgen pipeline
- [x] Added tests for markdown extraction with filepath preference
- [x] Existing tests pass
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
## Summary
- Remove the hardcoded check that blocked PR suggestions for roboflow
repositories
- Remove the corresponding unit test that validated the roboflow
restriction
