codeflash/tiles/codeflash-docs/evals/scenario-1/task.md
Kevin Turcios 869fbe1766 chore: add eval scenarios for codeflash-docs tile
5 scenarios testing: code serialization format, candidate lifecycle/DAG,
deterministic patches, effort levels/selection criteria, and function
representation/concurrency model.
2026-02-14 21:29:22 -05:00

1.3 KiB

Format Code for AI Service Request

Context

You are working on the codeflash optimization engine. The AI service accepts optimization requests with source code and dependency context. A function calculate_total in analytics/metrics.py needs to be optimized. It calls a helper normalize_values in the same file (both modifiable), and imports BaseMetric from analytics/base.py (not modifiable, just for reference).

# analytics/metrics.py
from analytics.base import BaseMetric

def normalize_values(data: list[float]) -> list[float]:
    max_val = max(data)
    return [x / max_val for x in data]

def calculate_total(metrics: list[BaseMetric]) -> float:
    values = [m.value for m in metrics]
    normalized = normalize_values(values)
    return sum(normalized)
# analytics/base.py
class BaseMetric:
    def __init__(self, name: str, value: float):
        self.name = name
        self.value = value

Task

Write a Python function prepare_optimization_payload that constructs the code payload for an AI service optimization request for calculate_total. It should properly format the source code and dependency code, and include a function to parse the AI service response back into structured code objects.

Expected Outputs

  • A Python file payload_builder.py with the payload construction and response parsing logic