## Summary
- Fix CST tree corruption issues that caused 'NoneType' object has no
attribute 'visit' errors
- Consolidate testgen postprocessing into a single pipeline with
tuple-based pattern
- Improve markdown code extraction to prefer filepath-annotated blocks
- Add diagnostic context to optimization failure logs
## Changes
- Handle empty `SimpleStatementLine` and `StatementHandler` body to
prevent malformed CST
- Add trace_id logging to optimization and import failure paths
- Refactor testgen postprocessing into consolidated pipeline
- Fix code extraction for LLM responses with multiple code blocks
## Test plan
- [x] Added integration tests for full testgen pipeline
- [x] Added tests for markdown extraction with filepath preference
- [x] Existing tests pass
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
- Add language parameter to split_markdown_code and group_code for JS/TS support
- Fix callable type annotation in instrument_javascript.py
- Update testgen_javascript.py to use ChatCompletionMessageParam types
- Add None checks before parse_python_version calls
- Add missing None assertions in test files
- Apply ruff auto-fixes for formatting and unused imports
- Accept consolidated markdown utilities from common module
- Use wrap_code_in_markdown with language parameter for language support
- Remove duplicate split_markdown_code implementation
- Add validation for python_version before parsing
## Summary
- Removes `profanity_regex` and `profanity_words` from
`postprocess_constants.py`
- Removes `remove_profanity_from_explanation` from the optimization
pipeline
- Removes associated test
## Summary
- Add forward reference detection and automatic fix with `from
__future__ import annotations`
- Handle aliased imports and chained calls in test instrumentation
- Fix import resolution from correct module in multi-context testgen
- Allow ellipsis in Protocol/abstract method bodies
- Add dataclass constructor notes for LLM about required/positional
arguments
- Add logging to silent exception handlers
## Test plan
- [x] Unit tests added for forward reference detection
- [x] Unit tests added for dataclass constructor notes
- [x] Unit tests added for ellipsis handling in AST
- [x] Unit tests added for chained call instrumentation
- [x] Unit tests extended for add_missing_imports
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Remove unicode quote sanitization from test code validation
- Rely on individual test validation to filter out tests with syntax
errors (including unicode characters)
## Test plan
- [x] Existing tests pass
- [x] Tests with unicode quote syntax errors are correctly filtered out
during individual validation
Make get_referenced_names_from_source scope-aware by reusing
UndefinedNameCollector, preventing invalid imports like `i` and `v`
from loop variables in AI-generated tests.
## Summary
- Add `normalize_code` helper in `tests/conftest.py` for comparing code
while ignoring quote style differences
- Update test assertions to use `normalize_code()` wrapper
- Add unit tests for comprehension instrumentation cases
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Skip instrumentation for target function calls inside list/set/dict
comprehensions and generator expressions
- Fixes NameError when AI-generated tests use comprehensions like
`[func(x) for x in items]`
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
## Summary
- Preserve nested functions in test instrumentation (skip instrumenting
calls inside nested function definitions)
- Prevent LLM from embedding markdown code fences in generated tests
(updated system prompts)
- Handle optional filename in python code blocks (e.g.,
`\`\`\`python:filename.py`)
## Test plan
- [x] Unit tests added for nested function preservation
- [x] Unit test for optional filename parsing
- [ ] Manual testing with generated tests
## Summary
- Include private symbols when resolving undefined names in generated
tests
- Resolve symbols imported by source module in generated tests
- Add fallback for symbols referenced but not defined in source snippet
- Ensure LocalDefinitionRemover only removes top-level definitions
(preserves nested classes/functions)
- Performance optimization for `get_symbols_from_source_code` function
## Test plan
- [x] Unit tests added for all new functionality
- [ ] Manual testing with generated tests
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
gpt-5-mini is misinterpreting the template format in execute_user_prompt.md. When it sees:
# function to test
{function_code}
It's outputting a nested code block with markdown path syntax (python:unstructured/staging/label_box.py) instead of the actual function code inline. It never reaches the # unit tests section where it should generate def test_* functions.
Fixes CF-690
Pull Request Checklist
Description
- Description of PR:
Privacy Mode Feature for Pro Users
This PR introduces a Privacy Mode feature that allows paid users
(Pro/Enterprise) to control how their code is stored during the
optimization review process.
Key Changes:
1. Privacy Mode Toggle in Sidebar
- Added a new toggle in the dashboard sidebar for Privacy Mode
- Only available for Pro/Enterprise users (disabled with upgrade prompt
for free users)
- Persists user preference in database with localStorage fallback for
fast UI
2. Storage Strategy Based on Privacy Setting
- Privacy Mode ON: Code is stored exclusively in GitHub branches (never
cached in database)
- Privacy Mode OFF: Code is temporarily cached in database for faster
loading, cleaned up when PR is created
3. Database Changes
- Added privacy_mode boolean field to users table (default: false)
- Added staging_storage_type field to track storage method
4. API Updates
- cf-api now checks user's privacy mode when determining storage
strategy
- StagingStorageStrategyFactory considers privacy mode alongside tier
eligibility
- Added getUserPrivacyMode, setUserPrivacyMode, and isUserPaid functions
User Experience
- Free users see the toggle disabled with "Upgrade to Pro" messaging
- Tooltip explains the trade-off: privacy vs. loading speed
- Toggle state syncs between localStorage (for instant UI) and database
(for persistence)
Storage Flow
User Request → Check Privacy Mode
├─ Privacy ON + Paid + Valid Repo → Git Branch Storage (GitHub only)
└─ Privacy OFF or Free → Plain Text Storage (Database cache)
---------
Co-authored-by: ali <mohammed18200118@gmail.com>
Co-authored-by: Mohamed Ashraf <ashraf@codeflash.ai>
- Do not use Mock objects for domain classes (pickling issues)
- Use correct constructor signatures from context
- Use concrete subclasses when base class is abstract
The optimization eliminates a significant performance bottleneck by moving the computation of `set(dir(builtins))` from runtime to module load time.
**What changed:**
- Introduced a module-level constant `_BUILTIN_NAMES = frozenset(dir(builtins))`
- Replaced the repeated `builtin_names = set(dir(builtins))` call with direct reference to `_BUILTIN_NAMES`
- Used `frozenset` instead of `set` for the constant (immutable and slightly more efficient for lookups)
**Why this is faster:**
The original code calls `dir(builtins)` and creates a new set from it on every invocation of `get_undefined_names()`. The line profiler shows this single line consumed 73.3% of the function's execution time (1.68ms out of 2.29ms total).
Python's `dir(builtins)` returns a list of ~150 builtin names, and converting this to a set requires:
1. Calling `dir()` to introspect the builtins module
2. Iterating through the list
3. Allocating a new set and hashing each name
Since builtin names are constant for a Python process, this work is entirely redundant. By computing it once at module import time, each call to `get_undefined_names()` saves ~21.8μs per invocation (from the line profiler: 21799.7ns per hit).
**Performance impact:**
- **184% speedup** overall (1.09ms → 383μs)
- The optimized version shows 100% of time now spent on the actual set operations (name filtering), not builtin lookup
- Test results show 1350-2450% improvement for simple cases with few names, and 22-126% improvement even for large-scale tests with 1000+ names
- The optimization is particularly effective when `get_undefined_names()` is called repeatedly (as shown by the 77 hits in the profiler and multiple test invocations)
**Test case effectiveness:**
- Most effective for tests with repeated calls to `get_undefined_names()` on the same or different collector instances
- Benefits all test patterns since the builtin check happens on every call regardless of the number of user-defined names
- Even large-scale tests with 1000+ names show 22-35% improvements because the builtin lookup overhead is eliminated
Instead of just adding missing imports, now also:
1. Detects classes/functions defined locally that exist in source module
2. Removes those local redefinitions
3. Adds imports for them
This ensures tests use the real classes from the source module.
When the LLM generates test code, it sometimes redefines classes locally
(like Element, ChunkingOptions) but forgets to import or define others
(like PreChunk), causing NameError at runtime.
This adds a postprocessing step that:
1. Detects undefined names in the generated test code
2. Checks if those names are defined in the source module
3. Adds the missing imports automatically
This is more reliable than relying on prompt instructions which the LLM
sometimes ignores.
Replace logger.error() and debug_log_sensitive_data() with
logging.exception() in exception handlers to ensure full stack traces
are logged to console/logs, not just to Sentry.
Affected endpoints:
- /optimize: Added exception logging
- /rank: Added logging import and exception logging
- /explanations: Added logging import and exception logging
- /workflow-gen: Changed logger.error to logger.exception
Remove the print_messages function and its usage since observability
tooling is now in place. This eliminates verbose system prompt output
during test generation in non-production environments.
Generated tests were failing isinstance() checks because LLM created
mock class definitions instead of importing real ones.
Added prompt instruction to import classes from their actual modules
when file paths are shown in the context.
Added base class dependency tracking as a secondary safeguard.
When a class inherits from another, the parent is now marked as
a dependency, preventing incorrect removal.
delete_top_def_nodes was removing helper classes even when they were
actually used in test code, causing NameError at runtime.
Now collects all names referenced in the module and protects them from
deletion. This ensures base classes and instantiated classes are kept.
## Add Optimization Usage Display
### Technical Changes
- **New Component**: Created `OptimizationUsageCard` component
(`js/cf-webapp/src/components/dashboard/OptimizationUsageCard.tsx`)
- Reusable component matching existing dashboard card styling
- Uses shadcn/ui `Progress` component for visual progress indicator
- Responsive design with mobile-friendly layouts
- Color-coded progress bar (green/yellow < 80%, yellow 80-95%,
orange/red ≥ 95%)
- **Server Action**: Added `getSubscriptionData` server action in
`dashboard/action.ts`
- Properly handles Prisma calls on server-side (fixes browser
PrismaClient error)
- Fetches subscription data with lazy period reset logic
- **Dashboard Integration**: Added usage card to main dashboard page
- Displays at top of metrics section for individual users
- Only shows for personal accounts (hidden for organization views)
- Integrated with existing data fetching pipeline
<img width="1891" height="991" alt="image"
src="https://github.com/user-attachments/assets/12c4f22b-51e7-439a-8614-644b06a423b2"
/>
- **Billing Page Integration**: Added usage card to billing page
- Positioned above subscription card for easy visibility
- Uses existing subscription data from page props
<img width="1891" height="982" alt="image"
src="https://github.com/user-attachments/assets/9c603227-750f-47be-ac10-36cfec3e4b45"
/>
### User-Facing Features
Users can now see their optimization usage in a clear, visual format:
- **Usage Metrics**: Displays formatted numbers showing "X / Y"
optimizations used
- **Remaining Count**: Shows how many optimizations are still available
- **Visual Progress Bar**: Color-coded progress indicator that changes
based on usage percentage
- **Plan Information**: Badge showing current plan type (free/pro)
- **Reset Date**: Displays when the usage period resets (if available)
- **Percentage**: Shows exact percentage of usage
The component appears on both the Dashboard and Billing pages, giving
users easy access to their usage information wherever they need it.
---------
Co-authored-by: Aseem Saxena <aseem.bits@gmail.com>
Co-authored-by: HeshamHM28 <HeshamMohamedFathy@outlook.com>
Co-authored-by: Sarthak Agarwal <sarthak.saga@gmail.com>
Co-authored-by: Kevin Turcios <106575910+KRRT7@users.noreply.github.com>
fixes CF-932
# Pull Request Checklist
## Description
- [ ] **Description of PR**: Clear and concise description of what this
PR accomplishes
- [ ] **Breaking Changes**: Document any breaking changes (if
applicable)
- [ ] **Related Issues**: Link to any related issues or tickets
## Testing
- [ ] **Test cases Attached**: All relevant test cases have been
added/updated
- [ ] **Manual Testing**: Manual testing completed for the changes
## Monitoring & Debugging
- [ ] **Logging in place**: Appropriate logging has been added for
debugging user issues
- [ ] **Sentry will be able to catch errors**: Error handling ensures
Sentry can capture and report errors
- [ ] **Avoid Dev based/Prisma logging**: No development-only or
Prisma-specific logging in production code
## Configuration
- [ ] **Env variables newly added**: Any new environment variables are
documented in .env.example file or mentioned in description
---
## Additional Notes
<!-- Add any additional context, screenshots, or notes for reviewers
here -->
---------
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: ali <mohammed18200118@gmail.com>
Co-authored-by: Kevin Turcios <106575910+KRRT7@users.noreply.github.com>
