Add markdown code block parsing, detailed syntax error locations with
line/col info, and structured logging to the JavaScript/TypeScript
validators.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update type hints for `add_months_safe` and `get_next_subscription_period`
to accept both datetime.datetime and datetime.date, and add ty:ignore
comment for Django ORM field type that ty cannot infer correctly.
Co-authored-by: Aseem Saxena <aseembits93@users.noreply.github.com>
Auth now attaches fetched organization/subscription to the request so
TrackUsageMiddleware reuses them instead of re-querying. RateLimitMiddleware
caches restricted_paths at init and uses async cache methods. LLM call
recording is fire-and-forget via asyncio.create_task to avoid blocking
responses on DB writes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move JIT instructions appending from the per-call level
(optimize_python_code_line_profiler_single) to the endpoint level
(optimize endpoint), matching the regular optimizer's pattern.
This removes the is_numerical_code parameter threading through
the call chain.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When is_numerical_code is true, the LLM sometimes outputs conditional
fallback paths (try/except, if/else) instead of applying the JIT
decorator directly. Add explicit output format instructions to prevent
this behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ty type checker correctly flags that list[str] is not a subtype of list[str | None] due to list invariance. Added explicit cast.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The JSON parsing path returned the LLM's explicit ranking array,
which sometimes contradicted its own per-dimension scores. Use
_scores_to_ranking() to compute the ranking from weighted scores
when available, falling back to the LLM ranking only when scores
are absent.
The ranker LLM was rewarding candidates that cache global variables
into locals as a performance win. Add an explicit rule: this is only
relevant on Python ≤3.10; on 3.11+ LOAD_GLOBAL uses adaptive
specialization and is nearly as fast as LOAD_FAST.
The non-greedy regex in FIRST_CODE_BLOCK_PATTERN stopped at the first
``` occurrence, even inside triple-quoted strings or nested code fence
blocks. This truncated the extracted code and lost test functions when
LLMs embedded function definitions using ```python:filepath syntax.
Switch to greedy matching and require the closing ``` to be alone on
its line so intermediate backticks are skipped.
The get_or_create defaults passed test lists without positional
indexing, so when a higher test_index created the row first its
content landed at index 0 and was overwritten by the lower index
update, losing a test.
Add explicit guidance to avoid generating tests that check for specific
exception types, since JIT compilers (numba, torch.compile) produce
different error types than uncompiled code. This ensures generated tests
work consistently for both compiled and uncompiled versions.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The LLM prompt preprocessing now highlights __init__ signatures for
regular classes, not just @dataclass ones, reducing brute-force
constructor guessing and pytest.skip() fallbacks in generated tests.
log_features() appended test results in call-completion order, causing
model attribution swaps when LLM responses arrived out of order. Pass
test_index through and use positional insertion instead of append.
## Summary
- Restructure CLAUDE.md hierarchy so Claude Code auto-discovers
project-specific instructions
- Delete dead `AGENTS.md` files (referenced non-existent
`.tessl/RULES.md`)
- Rename `django/aiservice/AGENTS.md` → `CLAUDE.md` for auto-discovery
- Create `js/CLAUDE.md` with package commands and gotchas
- Move PR review guidelines to `.claude/rules/pr-review.md` (auto-loaded
rule)
- Move prek workflow to `.claude/skills/fix-prek.md` (on-demand skill)
- Add path-scoped rules for Python and Next.js patterns
- Add domain glossary, service architecture diagram, and per-package
gotchas
## Test plan
- Verify `CLAUDE.md` files exist at root, `django/aiservice/`, and `js/`
- Verify no remaining references to `AGENTS.md` or `.tessl/`
- Verify `.claude/rules/` and `.claude/skills/` files are committed
## Summary
- Extract testgen and optimizer API routers from
`core/languages/python/` into `core/shared/` with lazy imports,
eliminating cross-module coupling between language modules
- Delete stale JavaScript prompt files left in the Python module after
migration to `js_ts/`
- Remove backward-compat fallback paths for prompt files that already
exist at their new locations
- Remove unused `is_multi_context_any()` and its cross-language imports
- Remove unused `BEGIN_PATCH`/`END_PATCH` constants and stale TODO
## Test plan
- [ ] Verify testgen endpoint dispatches correctly for Python, JS/TS,
and Java
- [ ] Verify optimizer endpoint dispatches correctly for all languages
- [ ] Run existing testgen and optimizer tests
## Summary
- Pass test_index through LLM call context so observability chat can
attribute responses to specific test generation calls
- Fix SSE streaming to send keepalive pings from the start
CF-504
# Pull Request Checklist
## Description
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: HeshamHM28 <HeshamMohamedFathy@outlook.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-39-200.ec2.internal>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Kevin Turcios <turcioskevinr@gmail.com>
Co-authored-by: Kevin Turcios <106575910+KRRT7@users.noreply.github.com>
## Problem
The JS/TS language handler (`core/languages/js_ts/`) was importing
models, schemas, config, prompts, and helpers directly from the Python
language handler. This created a confusing architectural dependency and
risked serving wrong language-specific prompt content.
## What Changed
- Created `core/shared/` for genuinely language-agnostic code (optimizer
schemas, models, config, testgen models, context helpers)
- Moved JS/TS-specific prompts and context helpers into
`core/languages/js_ts/`
- Updated all consumers (20+ files) to import from the correct locations
- Removed backwards-compat re-exports from the Python module
## Result
- **Before:** 11 imports from `core.languages.python` in
`core/languages/js_ts/`
- **After:** 0
## Summary
- Chat panel on the observability timeline that uses Claude to answer
questions about optimization traces
- Tool-based context retrieval (fetches candidates, tests, errors on
demand instead of stuffing everything upfront)
- Uses `@anthropic-ai/sdk` via Azure AI Foundry
- Strengthened testgen prompts to ban mocks/fakes for test inputs
Store qualified function name (e.g., HttpInterface.__init__) and
file_path in testgen metadata instead of bare function_name (__init__).
Update the frontend parser to handle qualified names by splitting into
class + method and searching within the correct class using both
tree-sitter and regex. Prioritize the file matching filePath before
searching all files.
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
…ndencies
The old rule ("NOT in libraries such as numpy, pandas etc.") forced LLMs
to reinvent helpers like np.allclose using slow / inaccurate Python
loops. The new rule allows assertions from packages already imported by
the function under test.
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Reorganizes `django/aiservice/` from feature-first layout (separate
`optimizer/`, `testgen/`, `code_repair/` dirs) to language-first layout
under `core/languages/{python,js_ts}/`
- Adds handler/registry/dispatcher pattern for routing requests to
language-specific implementations
- All existing module code preserved via `git mv` for history tracking;
no logic changes to existing modules
## What changed
- New `core/` app with registry, dispatcher, protocols, and error
hierarchy
- `PythonHandler` and `JSTypeScriptHandler` delegate to existing module
functions
- All imports updated across the codebase (views, tests,
adaptive_optimizer, etc.)
- Integration tests for handler registration and dispatch
- 155 files changed, ~880 additions / ~207 deletions (mostly import path
updates and moves)
## Test plan
- [ ] `python manage.py check` passes
- [ ] Integration tests in
`tests/integration/test_handler_integration.py` pass
- [ ] Existing test suite passes with updated import paths
- [ ] Ruff and ty clean on all new infrastructure files
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
## Summary
- Rewrite testgen system prompts from constraint-heavy to positive-first
structure with chain-of-thought instructions
- Simplify LLM message structure from `[system, user, user, user]` to
`[system, user]` by absorbing plan_content guidelines into system
prompts
- Observability UI: add search to LLM debug dialog, expand timeline view
- Fix data capture: raw LLM responses, all user messages in prompt
column, nested code fences, empty notes handling
## Test plan
- [ ] Verify testgen produces valid test suites with the new prompt
structure
- [ ] Verify observability timeline displays LLM prompts/responses
correctly
- [ ] Check that search works in the LLM debug dialog
## Summary
- Restructure the refinement system prompt into clear numbered sections
(Preserve Behavior, Minimize Diff, Revert Anti-Patterns, Maintain
Readability) with an explicit 6-step refinement process
- Extract inline prompt strings into separate markdown files
(`refinement_system_prompt.md`, `refinement_user_prompt.md`), matching
the convention used by other optimizer prompts
- Add `AuthenticatedRequest` type hint to `refine()` endpoint and fix
grammar in tool use section
## Test plan
- [ ] Verify refinement endpoint still works end-to-end with a test
optimization candidate
- [ ] Confirm prompt content is loaded correctly from markdown files at
startup
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
## Summary
- Re-adds the enriched observability context from CF-1041 that was
reverted
- Passes `module_path`, `test_module_path`, `helper_function_names`,
`is_async`, and `function_to_optimize` details to `call_llm` in testgen
## Test plan
- [ ] Verify testgen LLM calls include the enriched context
- [ ] Confirm no regressions in test generation flow
introducing this due to pain points in V1, not a complete rewrite, based
off v1
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Kevin Turcios <KRRT7@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
1. languages/js_ts/testgen.py:
- Updated parse_and_validate_js_output to accept a language parameter
- Uses validate_typescript_syntax when language="typescript", otherwise
uses validate_javascript_syntax
- Updated generate_and_validate_js_test_code to accept and pass the
language parameter
- Updated the call chain to pass language through to the validation
2. optimizer/context_utils/refiner_context.py:
- Added import for validate_typescript_syntax
- Fixed is_valid_refinement method to use correct validator based on
language
- Fixed validate_code_syntax in SingleRefinerContext class
- Fixed validate_code_syntax in MultiRefinerContext class
3. tests/optimizer/test_javascript_validator.py:
- Added test_typescript_type_assertion_valid_in_ts - verifies as unknown
as number is valid TypeScript
- Added test_typescript_type_assertion_invalid_in_js - verifies as
unknown as number is INVALID JavaScript (this would have caught the
original bug)
- Added test_typescript_generic_valid_in_ts - verifies generics are
valid TypeScript
- Added test_typescript_generic_invalid_in_js - verifies generics are
INVALID JavaScript
Files Already Correct (no changes needed):
- languages/js_ts/optimizer.py - already correctly checks language
- languages/js_ts/optimizer_lp.py - already correctly checks language
- optimizer/optimizer_line_profiler.py - already correctly checks
language
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
The JavaScript test generation prompt contained `{fn}` as part of
example code showing import syntax. However, Python's `.format()`
method interprets this as a placeholder and tries to substitute it,
causing a KeyError.
Fixed by escaping the curly braces as `{{fn}}` so they render as
literal `{fn}` in the final prompt.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
LLMs often add .js extensions to TypeScript import paths (e.g.,
`import { func } from '../module.js'`), but TypeScript/Jest module
resolution doesn't require explicit extensions. This causes
"Cannot find module" errors.
This change adds `strip_js_extensions()` function that removes
.js/.ts/.tsx/.jsx/.mjs/.mts extensions from relative import paths
in generated tests. The function handles:
- ES module imports: import { x } from '../path.js'
- CommonJS requires: require('../path.js')
- Jest mocks: jest.mock('../path.js'), jest.doMock(), etc.
External package imports (lodash, react, etc.) are preserved.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add more explicit instructions to prevent LLMs from adding .js/.ts
extensions to import paths. The previous guidance was being ignored
by some models.
- Add dedicated "CRITICAL: IMPORT PATH RULES" section with examples
- Show both WRONG and CORRECT patterns explicitly
- Remind to copy the provided import statement exactly
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Python's str.isidentifier() validates Python identifiers, not JavaScript
identifiers. This caused valid JS identifiers like '$handler' to be
rejected (since $ is not valid in Python identifiers).
Changed to use a regex pattern that matches JavaScript identifier rules:
- Can start with letter, underscore, or $
- Can contain letters, digits, underscores, or $
Added tests for $ identifiers to ensure they are correctly handled.
- Add _is_valid_js_identifier() to check for reserved words (module, exports, prototype, etc.)
- Only use class import pattern for single-dot names where class name is valid identifier
- Fall back to module import for:
- Multiple dots (e.g., Constructor.prototype.method)
- Reserved words (e.g., module.exports)
- Add comprehensive tests for edge cases
When generating test imports for class methods like `Validator.validateRequest`,
the previous code produced invalid JavaScript:
const { Validator.validateRequest } = require('../middlewares/Validator');
This is invalid because dots are not allowed in destructuring patterns.
The fix:
- Add _generate_import_statement() function to detect class methods (names with dots)
- For class methods: generate `const ClassName = require('...')`
- For simple functions: keep destructuring `const { funcName } = require('...')`
- Update prompt templates to use {import_statement} placeholder
Includes unit tests for the new import generation logic.
Add better error diagnostics for TypeScript/JavaScript syntax validation:
- Add line numbers and code snippets to error messages
- Log warnings when markdown parsing finds no code blocks
- Show the actual problematic code in error logs
- Help debug "Invalid syntax" errors by showing exact location
This helps diagnose issues where the API rejects code that tree-sitter
parses correctly on the client side by providing more context in the
error messages and logs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
## Summary
- Add `instrumented_perf_test` field to `OptimizationFeatures` model
- Update `log_features` function to accept and store performance
instrumented tests
---------
Co-authored-by: Sarthak Agarwal <sarthak.saga@gmail.com>
## ⚡️ This pull request contains optimizations for PR #2247
If you approve this dependent PR, these changes will be merged into the
original PR branch `multi-language`.
>This PR will be automatically closed if the original PR is merged.
----
#### 📄 18% (0.18x) speedup for ***`_has_test_functions` in
`django/aiservice/testgen/testgen_javascript.py`***
⏱️ Runtime : **`740 microseconds`** **→** **`627 microseconds`** (best
of `76` runs)
#### 📝 Explanation and details
The optimized code achieves an **18% runtime improvement** by
eliminating repeated regex compilation overhead.
**Key optimization:**
- **Precompiled regex pattern**: The pattern
`r"(?:test|it)\s*\(\s*['\"]"` is compiled once at module load time into
`_TEST_FUNC_RE`, rather than being recompiled on every function call. In
Python, `re.search()` with a raw string pattern incurs compilation cost
each time it's invoked.
**Performance breakdown from line profiler:**
- Original: 2.70ms spent in `re.search(test_pattern, code)` (96.5% of
total time)
- Optimized: 862μs for the direct pattern search (100% of total time,
but 3.1x faster overall)
- The pattern string assignment overhead (97μs in original) is
eliminated entirely
**Why this matters for the workload:**
Based on `function_references`, this function is called from
`parse_and_validate_js_output()` during LLM response validation. This is
a **hot path** operation that executes on every test generation request.
The validation flow checks multiple conditions including syntax
validation before checking for test functions, meaning this function
runs repeatedly during normal operations.
**Test case performance:**
- **Small inputs** (single test functions): 50-80% faster (e.g., 2.80μs
→ 1.83μs)
- **Empty/minimal strings**: 130-140% faster (e.g., 1.80μs → 750ns)
- **Large inputs** (500-1000 lines): 1-8% faster depending on match
location
- **Early matches** benefit most since regex short-circuits on first
match
The optimization is most effective when processing typical-sized
JavaScript test code (dozens to hundreds of lines), which aligns with
the common use case of validating LLM-generated test functions.
✅ **Correctness verification report:**
| Test | Status |
| --------------------------- | ----------------- |
| ⚙️ Existing Unit Tests | 🔘 **None Found** |
| 🌀 Generated Regression Tests | ✅ **102 Passed** |
| ⏪ Replay Tests | 🔘 **None Found** |
| 🔎 Concolic Coverage Tests | 🔘 **None Found** |
|📊 Tests Coverage | 100.0% |
<details>
<summary>🌀 Click to see Generated Regression Tests</summary>
```python
from __future__ import annotations
import re
# imports
import pytest # used for our unit tests
from testgen.testgen_javascript import _has_test_functions
def test_basic_test_call_double_quotes():
# Basic: a standard Jest test call using double quotes should be detected.
code = 'test("my test name", () => { expect(true).toBe(true); });'
codeflash_output = _has_test_functions(code) # 2.83μs -> 1.78μs (59.3% faster)
def test_basic_it_call_single_quotes():
# Basic: a standard it() call using single quotes should be detected.
code = "it('does something', function() { /* ... */ });"
codeflash_output = _has_test_functions(code) # 2.80μs -> 1.83μs (53.0% faster)
def test_whitespace_and_newlines_between_name_and_paren():
# Edge: whitespace/newlines between the function name and '(' and between '(' and the quote
# The regex allows arbitrary whitespace, so this should still match.
code = "it \n (\n 'handles newlines'\n )"
codeflash_output = _has_test_functions(code) # 2.90μs -> 1.89μs (53.2% faster)
def test_empty_string_returns_false():
# Edge: empty input must return False (no tests found).
code = ""
codeflash_output = _has_test_functions(code) # 1.80μs -> 750ns (139% faster)
def test_uppercase_function_name_not_matched():
# Edge: the regex is case-sensitive; 'Test' should NOT match.
code = "Test('capitalized should not match', () => {});"
codeflash_output = _has_test_functions(code) # 3.20μs -> 2.23μs (43.7% faster)
def test_backtick_template_not_matched():
# Edge: template literals use backticks; pattern looks only for single/double quotes.
code = "test(`template literal name`, () => {});"
codeflash_output = _has_test_functions(code) # 3.34μs -> 2.27μs (47.0% faster)
def test_numeric_first_arg_not_matched():
# Edge: if the first argument is not a quoted string (e.g., a number), pattern should not match.
code = "test(123, () => {});"
codeflash_output = _has_test_functions(code) # 3.01μs -> 1.86μs (61.7% faster)
def test_test_call_inside_comment_still_matches():
# Important behavioral note: the function does not ignore comments.
# A 'test(' occurrence inside a JS comment still matches because the function only does regex search.
code = "// test('in a single-line comment')\n/* test(\"in block comment\") */"
# Both comment forms contain test('...') / test("...") which the regex will find.
codeflash_output = _has_test_functions(code) # 2.96μs -> 1.84μs (61.1% faster)
def test_substring_in_identifier_matches():
# The regex is permissive and will match occurrences where 'test' or 'it' appear as suffixes
# of other identifiers (e.g., 'latesttest(' or 'split('). This test documents that behavior.
code_latest = "function latesttest(){}\nlatesttest('x')"
code_split = "const arr = ['a']; arr.split('a');"
# Both contain the substring "test('..." or "it('...", so they should be considered matches by the implementation.
codeflash_output = _has_test_functions(
code_latest
) # 3.77μs -> 2.59μs (45.5% faster)
codeflash_output = _has_test_functions(code_split) # 1.16μs -> 793ns (46.7% faster)
def test_comment_between_paren_blocks_prevents_match():
# If there is a non-whitespace token (like a block comment) between '(' and the starting quote,
# the current regex will not match because it expects only whitespace between '(' and the quote.
code = "test(/* important note */ 'name in comment')"
codeflash_output = _has_test_functions(code) # 3.31μs -> 2.30μs (43.5% faster)
def test_multiple_test_and_it_occurrences():
# A file with multiple matches should still return True (boolean).
code = """
describe('suite', () => {
it('first case', () => {});
// some other code
test("second case", () => {});
});
"""
codeflash_output = _has_test_functions(code) # 3.34μs -> 2.16μs (54.9% faster)
def test_large_scale_no_match_performance():
# Large-scale: many lines without any test/it(...) occurrences should return False.
# Keep size under 1000 to respect constraints. We use 900 repeated lines.
repeated = "const filler = 0;\n" * 900 # 900 lines of filler
codeflash_output = _has_test_functions(repeated) # 76.6μs -> 75.5μs (1.47% faster)
def test_large_scale_match_near_end():
# Large-scale: many lines of filler followed by a single test at the end should return True.
# This ensures the search scans through large input and finds a late occurrence.
repeated = "const filler = 0;\n" * 900 # 900 lines of filler
code = repeated + " // real test follows\n test('final case', () => {});"
codeflash_output = _has_test_functions(code) # 77.5μs -> 76.3μs (1.59% faster)
def test_it_with_newline_between_name_and_paren():
# Verify that a newline immediately after 'it' and before '(' is allowed by the regex (\s* covers newline).
code = "it\n('newline-allowed')"
codeflash_output = _has_test_functions(code) # 3.22μs -> 1.94μs (66.0% faster)
def test_quoted_string_with_escaped_quotes_still_matches():
# Even if the string contains escaped quotes, the regex only checks the opening quote, so it should match.
code = r'test("contains an escaped quote: \" here", () => {});'
codeflash_output = _has_test_functions(code) # 2.90μs -> 1.83μs (58.2% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
```
```python
import re
import pytest
from testgen.testgen_javascript import _has_test_functions
def test_simple_test_function_with_single_quotes():
"""Test detection of test() function with single quotes."""
code = "test('should work', () => {})"
codeflash_output = _has_test_functions(code) # 3.78μs -> 2.29μs (65.1% faster)
def test_simple_it_function_with_single_quotes():
"""Test detection of it() function with single quotes."""
code = "it('should work', () => {})"
codeflash_output = _has_test_functions(code) # 3.10μs -> 1.99μs (56.0% faster)
def test_simple_test_function_with_double_quotes():
"""Test detection of test() function with double quotes."""
code = 'test("should work", () => {})'
codeflash_output = _has_test_functions(code) # 3.07μs -> 1.89μs (62.8% faster)
def test_simple_it_function_with_double_quotes():
"""Test detection of it() function with double quotes."""
code = 'it("should work", () => {})'
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.86μs (68.2% faster)
def test_no_test_functions():
"""Test code without any test functions returns False."""
code = "function myFunction() { return 42; }"
codeflash_output = _has_test_functions(code) # 3.08μs -> 1.85μs (66.5% faster)
def test_test_function_with_single_whitespace():
"""Test detection with single space between function name and parenthesis."""
code = "test ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.20μs -> 1.96μs (63.0% faster)
def test_it_function_with_single_whitespace():
"""Test detection with single space between function name and parenthesis."""
code = "it ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.98μs -> 1.88μs (58.5% faster)
def test_test_function_with_multiple_whitespaces():
"""Test detection with multiple spaces between function name and parenthesis."""
code = "test ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.04μs -> 1.95μs (55.5% faster)
def test_it_function_with_multiple_whitespaces():
"""Test detection with multiple spaces between function name and parenthesis."""
code = "it ('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.06μs -> 1.89μs (62.1% faster)
def test_test_function_with_tab_character():
"""Test detection with tab character between function name and parenthesis."""
code = "test\t('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.05μs -> 1.93μs (57.8% faster)
def test_it_function_with_tab_character():
"""Test detection with tab character between function name and parenthesis."""
code = "it\t('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.99μs -> 1.85μs (61.6% faster)
def test_test_function_with_newline():
"""Test detection with newline between function name and parenthesis."""
code = "test\n('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.02μs -> 1.93μs (56.5% faster)
def test_it_function_with_newline():
"""Test detection with newline between function name and parenthesis."""
code = "it\n('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.95μs -> 1.90μs (55.4% faster)
def test_multiple_test_functions():
"""Test detection with multiple test functions in the same code."""
code = """
test('first test', () => {});
it('second test', () => {});
"""
codeflash_output = _has_test_functions(code) # 3.12μs -> 1.92μs (62.3% faster)
def test_test_function_in_multiline_code():
"""Test detection of test function within multiline code."""
code = """
const helper = () => {};
test('actual test', () => {});
const another = () => {};
"""
codeflash_output = _has_test_functions(code) # 3.19μs -> 2.12μs (50.1% faster)
def test_it_function_in_multiline_code():
"""Test detection of it function within multiline code."""
code = """
const helper = () => {};
it('actual test', () => {});
const another = () => {};
"""
codeflash_output = _has_test_functions(code) # 3.46μs -> 2.23μs (55.1% faster)
def test_test_word_in_comment_not_matched():
"""Test that test() in comments is still detected by regex (no comment parsing)."""
code = "// test('in comment', () => {})"
# Note: The function uses regex without comment awareness, so it will match
codeflash_output = _has_test_functions(code) # 3.10μs -> 1.94μs (60.2% faster)
def test_test_word_in_string_variable():
"""Test that test word in string variable doesn't match pattern."""
code = 'const description = "this is a test of something";'
codeflash_output = _has_test_functions(code) # 3.56μs -> 2.29μs (55.3% faster)
def test_test_as_variable_name_not_matched():
"""Test that 'test' as variable name doesn't match without parenthesis."""
code = "const test = 5;"
codeflash_output = _has_test_functions(code) # 3.11μs -> 2.03μs (53.1% faster)
def test_testing_as_word_not_matched():
"""Test that 'testing' word doesn't match."""
code = "const testing = 'some value';"
codeflash_output = _has_test_functions(code) # 3.17μs -> 2.04μs (55.4% faster)
def test_it_as_pronoun_not_matched():
"""Test that 'it' as pronoun doesn't match without proper pattern."""
code = "// it is a good day"
codeflash_output = _has_test_functions(code) # 3.15μs -> 1.98μs (59.1% faster)
def test_it_as_variable_not_matched():
"""Test that 'it' as variable name doesn't match without parenthesis."""
code = "const it = 5;"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.92μs (63.7% faster)
def test_empty_string():
"""Test with empty string input."""
codeflash_output = _has_test_functions("") # 1.74μs -> 757ns (129% faster)
def test_only_whitespace():
"""Test with only whitespace."""
codeflash_output = _has_test_functions(" \n\t ") # 2.05μs -> 890ns (131% faster)
def test_test_function_with_special_test_name():
"""Test detection with special characters in test name."""
code = "test('test-name_123!@#', () => {})"
codeflash_output = _has_test_functions(code) # 2.95μs -> 1.90μs (55.1% faster)
def test_it_function_with_special_test_name():
"""Test detection with special characters in test name."""
code = "it('it-name_123!@#', () => {})"
codeflash_output = _has_test_functions(code) # 3.04μs -> 1.88μs (62.0% faster)
def test_test_function_with_empty_string_name():
"""Test detection with empty string as test name."""
code = "test('', () => {})"
codeflash_output = _has_test_functions(code) # 2.90μs -> 1.84μs (58.0% faster)
def test_it_function_with_empty_string_name():
"""Test detection with empty string as test name."""
code = "it('', () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.86μs (59.7% faster)
def test_test_with_carriage_return():
"""Test detection with carriage return character."""
code = "test\r('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.02μs -> 1.89μs (60.2% faster)
def test_it_with_carriage_return():
"""Test detection with carriage return character."""
code = "it\r('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.12μs -> 1.91μs (63.8% faster)
def test_test_with_form_feed():
"""Test detection with form feed character."""
code = "test\f('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.98μs -> 1.85μs (60.7% faster)
def test_it_with_form_feed():
"""Test detection with form feed character."""
code = "it\f('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.91μs (64.4% faster)
def test_test_with_vertical_tab():
"""Test detection with vertical tab character."""
code = "test\v('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.08μs -> 1.87μs (65.0% faster)
def test_it_with_vertical_tab():
"""Test detection with vertical tab character."""
code = "it\v('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.83μs (71.4% faster)
def test_test_with_non_breaking_space():
"""Test that non-breaking space might not work depending on whitespace regex."""
code = "test\u00a0('my test', () => {})"
# Non-breaking space might not be treated as \s in regex
codeflash_output = _has_test_functions(code)
result = codeflash_output # 3.25μs -> 2.07μs (56.6% faster)
def test_test_with_zero_width_space():
"""Test with zero-width space."""
code = "test\u200b('my test', () => {})"
codeflash_output = _has_test_functions(code)
result = codeflash_output # 3.82μs -> 2.74μs (39.4% faster)
def test_only_test_keyword():
"""Test with only the word 'test' without parenthesis."""
code = "test"
codeflash_output = _has_test_functions(code) # 2.85μs -> 1.83μs (55.3% faster)
def test_only_it_keyword():
"""Test with only the word 'it' without parenthesis."""
code = "it"
codeflash_output = _has_test_functions(code) # 1.89μs -> 793ns (138% faster)
def test_test_with_parenthesis_but_no_quote():
"""Test function call without string argument."""
code = "test(variable)"
codeflash_output = _has_test_functions(code) # 3.39μs -> 2.12μs (59.8% faster)
def test_it_with_parenthesis_but_no_quote():
"""Test it function call without string argument."""
code = "it(variable)"
codeflash_output = _has_test_functions(code) # 3.24μs -> 2.07μs (56.5% faster)
def test_test_followed_by_string_literal_without_parenthesis():
"""Test with string literal but missing parenthesis."""
code = "test 'string'"
codeflash_output = _has_test_functions(code) # 3.17μs -> 1.93μs (64.5% faster)
def test_it_followed_by_string_literal_without_parenthesis():
"""Test with string literal but missing parenthesis."""
code = "it 'string'"
codeflash_output = _has_test_functions(code) # 3.24μs -> 2.02μs (59.9% faster)
def test_test_with_backtick_quotes():
"""Test with backtick quotes (template literals)."""
code = "test(`my test`, () => {})"
codeflash_output = _has_test_functions(code) # 3.49μs -> 2.34μs (49.5% faster)
def test_it_with_backtick_quotes():
"""Test it with backtick quotes (template literals)."""
code = "it(`my test`, () => {})"
codeflash_output = _has_test_functions(code) # 3.38μs -> 2.23μs (51.4% faster)
def test_describe_function_not_matched():
"""Test that describe() function is not matched."""
code = "describe('suite', () => {})"
codeflash_output = _has_test_functions(code) # 3.13μs -> 1.94μs (61.2% faster)
def test_beforeEach_function_not_matched():
"""Test that beforeEach() function is not matched."""
code = "beforeEach(() => {})"
codeflash_output = _has_test_functions(code) # 1.98μs -> 901ns (119% faster)
def test_afterEach_function_not_matched():
"""Test that afterEach() function is not matched."""
code = "afterEach(() => {})"
codeflash_output = _has_test_functions(code) # 2.68μs -> 1.50μs (78.6% faster)
def test_test_method_on_object():
"""Test with test as method call on object."""
code = "obj.test('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.12μs -> 1.94μs (61.4% faster)
def test_it_method_on_object():
"""Test with it as method call on object."""
code = "obj.it('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.10μs -> 2.00μs (55.0% faster)
def test_test_substring_in_longer_identifier():
"""Test when test is part of longer identifier."""
code = "mytest('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.00μs -> 1.90μs (58.0% faster)
def test_it_substring_in_longer_identifier():
"""Test when it is part of longer identifier."""
code = "unit('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.07μs -> 1.94μs (58.2% faster)
def test_test_with_unicode_test_name():
"""Test detection with unicode characters in test name."""
code = "test('\u4e2d\u6587\u6d4b\u8bd5', () => {})"
codeflash_output = _has_test_functions(code) # 3.57μs -> 2.31μs (54.4% faster)
def test_it_with_unicode_test_name():
"""Test detection with unicode characters in test name."""
code = "it('\u4e2d\u6587\u6d4b\u8bd5', () => {})"
codeflash_output = _has_test_functions(code) # 3.09μs -> 2.08μs (48.8% faster)
def test_test_with_emoji():
"""Test detection with emoji in test name."""
code = "test('\u263a emoji test', () => {})"
codeflash_output = _has_test_functions(code) # 3.14μs -> 2.02μs (55.2% faster)
def test_it_with_emoji():
"""Test detection with emoji in test name."""
code = "it('\u263a emoji test', () => {})"
codeflash_output = _has_test_functions(code) # 3.09μs -> 2.00μs (54.0% faster)
def test_very_long_test_name():
"""Test detection with very long test name."""
long_name = "a" * 5000
code = f"test('{long_name}', () => {{}})"
codeflash_output = _has_test_functions(code) # 3.00μs -> 1.77μs (69.5% faster)
def test_very_long_code_without_tests():
"""Test with very long code but no test functions."""
code = "const x = 1;\n" * 500
codeflash_output = _has_test_functions(code) # 29.7μs -> 28.5μs (4.29% faster)
def test_test_with_escaped_quote():
"""Test with escaped quote in test name."""
code = "test('test\\'s name', () => {})"
codeflash_output = _has_test_functions(code) # 3.15μs -> 2.03μs (55.2% faster)
def test_it_with_escaped_quote():
"""Test with escaped quote in test name."""
code = "it('it\\'s name', () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.87μs (59.0% faster)
def test_test_with_double_quote_in_single_quote():
"""Test with double quote inside single quoted test name."""
code = "test('has \"double\" quotes', () => {})"
codeflash_output = _has_test_functions(code) # 2.93μs -> 1.82μs (60.7% faster)
def test_it_with_double_quote_in_single_quote():
"""Test with double quote inside single quoted test name."""
code = "it('has \"double\" quotes', () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.83μs (62.6% faster)
def test_test_with_single_quote_in_double_quote():
"""Test with single quote inside double quoted test name."""
code = "test(\"has 'single' quotes\", () => {})"
codeflash_output = _has_test_functions(code) # 3.00μs -> 1.90μs (57.8% faster)
def test_it_with_single_quote_in_double_quote():
"""Test with single quote inside double quoted test name."""
code = "it(\"has 'single' quotes\", () => {})"
codeflash_output = _has_test_functions(code) # 2.97μs -> 1.78μs (66.7% faster)
def test_test_case_sensitive():
"""Test that TEST (uppercase) is not matched."""
code = "TEST('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.03μs -> 1.81μs (67.1% faster)
def test_it_case_sensitive():
"""Test that IT (uppercase) is not matched."""
code = "IT('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.81μs -> 1.84μs (52.8% faster)
def test_test_with_mixed_case():
"""Test that TeSt (mixed case) is not matched."""
code = "TeSt('my test', () => {})"
codeflash_output = _has_test_functions(code) # 3.05μs -> 1.85μs (65.2% faster)
def test_it_with_mixed_case():
"""Test that It (mixed case) is not matched."""
code = "It('my test', () => {})"
codeflash_output = _has_test_functions(code) # 2.94μs -> 1.91μs (54.3% faster)
def test_code_with_many_non_test_functions():
"""Test performance with many non-test functions."""
# Build code with 500 non-test function definitions
code_lines = [f"function func{i}() {{ return {i}; }}" for i in range(500)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 77.0μs -> 75.9μs (1.52% faster)
def test_code_with_many_functions_and_one_test():
"""Test detection of single test among many non-test functions."""
# Build code with 500 non-test functions and 1 test function
code_lines = [f"function func{i}() {{ return {i}; }}" for i in range(250)]
code_lines.append("test('the actual test', () => {})")
code_lines.extend(
[f"function func{i}() {{ return {i}; }}" for i in range(250, 500)]
)
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 40.3μs -> 38.9μs (3.41% faster)
def test_code_with_many_test_functions():
"""Test detection with many test functions."""
# Build code with 100 test functions
code_lines = [f"test('test {i}', () => {{}})" for i in range(100)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.94μs -> 1.74μs (69.4% faster)
def test_code_with_many_it_functions():
"""Test detection with many it functions."""
# Build code with 100 it functions
code_lines = [f"it('test {i}', () => {{}})" for i in range(100)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.91μs -> 1.61μs (80.4% faster)
def test_code_with_alternating_test_and_it_functions():
"""Test detection with alternating test and it functions."""
# Build code with 100 alternating test and it functions
code_lines = []
for i in range(50):
code_lines.append(f"test('test {i}', () => {{}})")
code_lines.append(f"it('it {i}', () => {{}})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.93μs -> 1.64μs (78.7% faster)
def test_code_with_many_non_matching_similar_patterns():
"""Test performance with many similar but non-matching patterns."""
# Build code with 500 similar patterns that don't match
code_lines = [f"test{i}('name', () => {{}})" for i in range(500)]
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 64.3μs -> 62.9μs (2.27% faster)
def test_large_code_with_test_at_end():
"""Test detection when test function is at end of large code."""
# Build code with 500 lines and test at the end
code_lines = [f"const var{i} = {i};" for i in range(500)]
code_lines.append("test('test at end', () => {})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 41.4μs -> 40.3μs (2.65% faster)
def test_large_code_with_it_at_end():
"""Test detection when it function is at end of large code."""
# Build code with 500 lines and it at the end
code_lines = [f"const var{i} = {i};" for i in range(500)]
code_lines.append("it('it at end', () => {})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 41.5μs -> 40.3μs (3.00% faster)
def test_large_code_with_test_at_beginning():
"""Test detection when test function is at beginning of large code."""
# Build code with test at beginning and 500 lines after
code_lines = ["test('test at beginning', () => {})"]
code_lines.extend([f"const var{i} = {i};" for i in range(500)])
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 3.09μs -> 1.90μs (62.2% faster)
def test_large_code_with_it_at_beginning():
"""Test detection when it function is at beginning of large code."""
# Build code with it at beginning and 500 lines after
code_lines = ["it('it at beginning', () => {})"]
code_lines.extend([f"const var{i} = {i};" for i in range(500)])
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 3.02μs -> 1.81μs (67.1% faster)
def test_code_with_multiple_tests_scattered():
"""Test detection with multiple test functions scattered throughout large code."""
# Build code with 20 test functions scattered among 480 non-test lines
code_lines = []
for i in range(500):
if i % 25 == 0:
code_lines.append(f"test('scattered test {i}', () => {{}})")
else:
code_lines.append(f"const var{i} = {i};")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.86μs -> 1.79μs (59.8% faster)
def test_code_with_very_large_test_name():
"""Test performance with very long test name."""
# Create a test with name of 10000 characters
long_name = "x" * 10000
code = f"test('{long_name}', () => {{}})"
codeflash_output = _has_test_functions(code) # 2.99μs -> 1.84μs (62.1% faster)
def test_code_with_deeply_nested_structures():
"""Test detection in deeply nested code structures."""
# Build nested structure with test at bottom
code = "const nested = { level1: { level2: { level3: { level4: { " * 50
code += "test('nested test', () => {})"
code += " } } } } };" * 50
codeflash_output = _has_test_functions(code) # 14.6μs -> 13.5μs (8.56% faster)
def test_code_with_many_whitespace_variations():
"""Test detection with many different whitespace patterns."""
code_lines = []
for i in range(100):
if i % 4 == 0:
code_lines.append(f"test('test {i}', () => {{}})")
elif i % 4 == 1:
code_lines.append(f"test ('test {i}', () => {{}})")
elif i % 4 == 2:
code_lines.append(f"test ('test {i}', () => {{}})")
else:
code_lines.append(f"test\t('test {i}', () => {{}})")
code = "\n".join(code_lines)
codeflash_output = _has_test_functions(code) # 2.92μs -> 1.71μs (70.7% faster)
def test_code_return_type_is_boolean():
"""Test that return value is always boolean regardless of input size."""
# Various test inputs
test_inputs = [
"",
"test",
"test('name', () => {})",
"const x = 1;" * 100,
"test('name', () => {})" + "const x = 1;" * 100,
]
for test_input in test_inputs:
codeflash_output = _has_test_functions(test_input)
result = codeflash_output # 11.9μs -> 9.34μs (27.0% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
```
</details>
To edit these changes `git checkout
codeflash/optimize-pr2247-2026-01-25T08.57.25` and push.
[](https://codeflash.ai)
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: Kevin Turcios <106575910+KRRT7@users.noreply.github.com>
## Summary
- Fix CST tree corruption issues that caused 'NoneType' object has no
attribute 'visit' errors
- Consolidate testgen postprocessing into a single pipeline with
tuple-based pattern
- Improve markdown code extraction to prefer filepath-annotated blocks
- Add diagnostic context to optimization failure logs
## Changes
- Handle empty `SimpleStatementLine` and `StatementHandler` body to
prevent malformed CST
- Add trace_id logging to optimization and import failure paths
- Refactor testgen postprocessing into consolidated pipeline
- Fix code extraction for LLM responses with multiple code blocks
## Test plan
- [x] Added integration tests for full testgen pipeline
- [x] Added tests for markdown extraction with filepath preference
- [x] Existing tests pass
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
- Add language parameter to split_markdown_code and group_code for JS/TS support
- Fix callable type annotation in instrument_javascript.py
- Update testgen_javascript.py to use ChatCompletionMessageParam types
- Add None checks before parse_python_version calls
- Add missing None assertions in test files
- Apply ruff auto-fixes for formatting and unused imports
- Accept consolidated markdown utilities from common module
- Use wrap_code_in_markdown with language parameter for language support
- Remove duplicate split_markdown_code implementation
- Add validation for python_version before parsing
## Summary
- Removes `profanity_regex` and `profanity_words` from
`postprocess_constants.py`
- Removes `remove_profanity_from_explanation` from the optimization
pipeline
- Removes associated test
## Summary
- Add forward reference detection and automatic fix with `from
__future__ import annotations`
- Handle aliased imports and chained calls in test instrumentation
- Fix import resolution from correct module in multi-context testgen
- Allow ellipsis in Protocol/abstract method bodies
- Add dataclass constructor notes for LLM about required/positional
arguments
- Add logging to silent exception handlers
## Test plan
- [x] Unit tests added for forward reference detection
- [x] Unit tests added for dataclass constructor notes
- [x] Unit tests added for ellipsis handling in AST
- [x] Unit tests added for chained call instrumentation
- [x] Unit tests extended for add_missing_imports
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Remove unicode quote sanitization from test code validation
- Rely on individual test validation to filter out tests with syntax
errors (including unicode characters)
## Test plan
- [x] Existing tests pass
- [x] Tests with unicode quote syntax errors are correctly filtered out
during individual validation
Make get_referenced_names_from_source scope-aware by reusing
UndefinedNameCollector, preventing invalid imports like `i` and `v`
from loop variables in AI-generated tests.
## Summary
- Add `normalize_code` helper in `tests/conftest.py` for comparing code
while ignoring quote style differences
- Update test assertions to use `normalize_code()` wrapper
- Add unit tests for comprehension instrumentation cases
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Skip instrumentation for target function calls inside list/set/dict
comprehensions and generator expressions
- Fixes NameError when AI-generated tests use comprehensions like
`[func(x) for x in items]`
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
