Fix test generation replay 500 error when arrays contain None values (#2521)

## Summary

Fixes 500 Internal Server Error when replaying test generation with
`--rerun` flag and database arrays contain `None`/`NULL` values.

## Root Cause

The `rerun_testgen()` function in `core/shared/replay.py` accessed array
elements without checking if they were `None`. When PostgreSQL arrays
contained `NULL` values (e.g., `generated_test = [NULL, 'test2']`), the
function returned a `TestGenResponseSchema` with `None` values, causing
Pydantic validation to fail:

```
pydantic_core._pydantic_core.ValidationError: 2 validation errors for TestGenResponseSchema
generated_tests
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
instrumented_behavior_tests
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
```

## Changes

Added explicit `None` checks before creating `TestGenResponseSchema`:
- If `generated_test[index]` or `instrumented_generated_test[index]` is
`None`, return `None` (skip this test)
- If `instrumented_perf_test[index]` is `None`, default to empty string
(non-critical field)

## Impact

Resolves **10+ replay failures** where test generation produced partial
results stored as `NULL` in database arrays.

## Test Coverage

Added comprehensive test suite for `replay.py`:
- `test_rerun_with_valid_test_data()` - Happy path
- `test_rerun_with_none_values_in_arrays()` - **Primary bug fix test**
- `test_rerun_with_index_out_of_bounds()` - Boundary conditions
- `test_rerun_with_empty_arrays()` - Empty data handling
- `test_rerun_with_none_arrays()` - NULL arrays
- `test_rerun_with_mismatched_array_lengths()` - Length mismatches
- `test_rerun_missing_perf_test()` - Missing perf data

All 7 tests pass.

## Trace IDs

This fix addresses errors seen in traces:
- Primary: `056561cc-94af-4d7b-ac79-85dfd4b7282d`
- And 9 additional trace IDs with the same "500 - Error generating
JavaScript tests" error

## Verification

Tested with original failing trace:
```bash
cd /workspace/target && codeflash --file src/daemon/constants.ts --function formatGatewayServiceDescription --rerun 056561cc-94af-4d7b-ac79-85dfd4b7282d
```

**Before fix:** `ERROR: 500 - Traceback... ValidationError: Input should
be a valid string [type=string_type, input_value=None]`
**After fix:** Gracefully skips None entries, no 500 error 

---------

Co-authored-by: Codeflash Bot <codeflash-bot@codeflash.ai>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mohammed ahmed 2026-04-02 18:19:58 +02:00 committed by GitHub
parent d504f111a7
commit 179302d006
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -47,17 +47,28 @@ def rerun_optimize(record: OptimizationFeatures, source_filter: str) -> Optimize
def rerun_testgen(record: OptimizationFeatures, test_index: int) -> TestGenResponseSchema | None:
generated: list[str] = cast("list[str]", record.generated_test) or []
instrumented: list[str] = cast("list[str]", record.instrumented_generated_test) or []
perf: list[str] = cast("list[str]", record.instrumented_perf_test) or []
generated: list[str | None] = cast("list[str | None]", record.generated_test) or []
instrumented: list[str | None] = cast("list[str | None]", record.instrumented_generated_test) or []
perf: list[str | None] = cast("list[str | None]", record.instrumented_perf_test) or []
if test_index >= len(generated) or test_index >= len(instrumented):
return None
# Check if values at the index are None (can happen with NULL in database arrays)
generated_val = generated[test_index]
instrumented_val = instrumented[test_index]
if generated_val is None or instrumented_val is None:
return None
perf_val = perf[test_index] if test_index < len(perf) else None
# Default to empty string if perf value is None
perf_val = perf_val if perf_val is not None else ""
return TestGenResponseSchema(
generated_tests=generated[test_index],
instrumented_behavior_tests=instrumented[test_index],
instrumented_perf_tests=perf[test_index] if test_index < len(perf) else "",
generated_tests=generated_val,
instrumented_behavior_tests=instrumented_val,
instrumented_perf_tests=perf_val,
raw_generated_tests=None,
)