mirror of
https://github.com/codeflash-ai/codeflash-internal.git
synced 2026-05-04 18:25:18 +00:00
56 lines
2 KiB
Markdown
56 lines
2 KiB
Markdown
|
# Test Generation Pipeline
|
||
|
|
|
||
|
|
Flow from test generation request to instrumented test output.
|
||
|
|
|
||
|
|
## 1. Request Entry (`core/shared/testgen_router.py`)
|
||
|
|
|
||
|
|
The `testgen_api` NinjaAPI router receives a POST request and dispatches by `data.language`:
|
||
|
|
- `"javascript"` / `"typescript"` → `core.languages.js_ts.testgen.generate_tests_javascript`
|
||
|
|
- `"java"` → `core.languages.java.testgen.generate_tests_java`
|
||
|
|
- Default → `core.languages.python.testgen.testgen.generate_tests_python`
|
||
|
|
|
||
|
|
## 2. Python Testgen (`core/languages/python/testgen/testgen.py`)
|
||
|
|
|
||
|
|
### `build_prompt()`
|
||
|
|
|
||
|
|
Jinja2-based prompt builder for test generation:
|
||
|
|
- Constructs system and user prompts from context
|
||
|
|
- Handles async/sync function variants
|
||
|
|
- Includes source code, dependency code, and test context
|
||
|
|
|
||
|
|
### `instrument_tests()`
|
||
|
|
|
||
|
|
Applies instrumentation to generated tests:
|
||
|
|
- Behavior instrumentation (captures return values, stdout)
|
||
|
|
- Performance instrumentation (timing loops)
|
||
|
|
|
||
|
|
### `LLMOutputParseError`
|
||
|
|
|
||
|
|
Custom exception for LLM response parsing failures — includes raw output for debugging.
|
||
|
|
|
||
|
|
## 3. Instrumentation (`testgen/instrumentation/instrument_new_tests.py`)
|
||
|
|
|
||
|
|
### Framework Detection
|
||
|
|
|
||
|
|
- `detect_frameworks_from_code()` — parses imports to identify ML frameworks (PyTorch, TensorFlow, JAX) and their aliases
|
||
|
|
- Used to add GPU sync calls in timing blocks
|
||
|
|
|
||
|
|
### Device Sync
|
||
|
|
|
||
|
|
- `_create_device_sync_precompute_statements()` — pre-computes device sync checks (CUDA, MPS) outside timing blocks
|
||
|
|
- Ensures accurate timing for GPU-accelerated code
|
||
|
|
|
||
|
|
## 4. Postprocessing (`testgen/postprocessing/`)
|
||
|
|
|
||
|
|
- Import management: `add_missing_imports.py` adds `from __future__ import annotations`
|
||
|
|
- Test validation and cleanup
|
||
|
|
|
||
|
|
## Key Files
|
||
|
|
|
||
|
|
| File | Role |
|
||
|
|
|------|------|
|
||
|
|
| `core/shared/testgen_router.py` | Language dispatch |
|
||
|
|
| `core/languages/python/testgen/testgen.py` | Testgen flow |
|
||
|
|
| `core/languages/python/testgen/instrumentation/instrument_new_tests.py` | Test instrumentation |
|
||
|
|
| `core/languages/python/testgen/postprocessing/` | Import management, cleanup |
|