Enhance documentation with improved formatting and additional details on configuration options, async function support, and Codeflash setup process. Update keywords for better searchability and add visual placeholders for clarity.
This commit is contained in:
Mohamed Ashraf
parent
8a09ee9eea
commit
d32e18ce5d
6 changed files with 160 additions and 114 deletions
|
|
@ -17,7 +17,7 @@ Codeflash scans your codebase to identify all available functions. It locates ex
|
|||
#### What kind of functions can Codeflash optimize?
|
||||
|
||||
Codeflash works best with self-contained functions that have minimal side effects (like communicating with external systems or sending network requests). Codeflash optimizes a group of functions - consisting of an entry point function and any other functions it directly calls.
|
||||
Currently, Codeflash cannot optimize async functions.
|
||||
Codeflash supports optimizing async functions.
|
||||
|
||||
#### Test Discovery
|
||||
|
||||
|
|
|
|||
|
|
@ -3,7 +3,15 @@ title: "Manual Configuration"
|
|||
description: "Configure Codeflash for your project with pyproject.toml settings and advanced options"
|
||||
icon: "gear"
|
||||
sidebarTitle: "Manual Configuration"
|
||||
keywords: ["configuration", "pyproject.toml", "setup", "settings", "pytest", "formatter"]
|
||||
keywords:
|
||||
[
|
||||
"configuration",
|
||||
"pyproject.toml",
|
||||
"setup",
|
||||
"settings",
|
||||
"pytest",
|
||||
"formatter",
|
||||
]
|
||||
---
|
||||
|
||||
# Manual Configuration
|
||||
|
|
@ -12,12 +20,13 @@ Codeflash is installed and configured on a per-project basis.
|
|||
`codeflash init` should guide you through the configuration process, but if you need to manually configure Codeflash or set advanced settings, you can do so by editing the `pyproject.toml` file in the root directory of your project.
|
||||
|
||||
## Configuration Options
|
||||
|
||||
Codeflash config looks like the following
|
||||
|
||||
```toml
|
||||
[tool.codeflash]
|
||||
module-root = "my_module"
|
||||
tests-root = "tests"
|
||||
test-framework = "pytest"
|
||||
formatter-cmds = ["black $file"]
|
||||
# optional configuration
|
||||
benchmarks-root = "tests/benchmarks" # Required when running with --benchmark
|
||||
|
|
@ -25,25 +34,33 @@ ignore-paths = ["my_module/build/"]
|
|||
pytest-cmd = "pytest"
|
||||
disable-imports-sorting = false
|
||||
disable-telemetry = false
|
||||
git-remote = "origin"
|
||||
override-fixtures = false
|
||||
```
|
||||
|
||||
All file paths are relative to the directory of the `pyproject.toml` file.
|
||||
|
||||
Required Options:
|
||||
|
||||
- `module-root`: The Python module you want Codeflash to optimize going forward. Only code under this directory will be optimized. It should also have an `__init__.py` file to make the module importable.
|
||||
- `tests-root`: The directory where your tests are located. Codeflash will use this directory to discover existing tests as well as generate new tests.
|
||||
- `test-framework`: The test framework you use for your project. Codeflash supports `pytest` and `unittest`.
|
||||
|
||||
Optional Configuration:
|
||||
|
||||
- `benchmarks-root`: The directory where your benchmarks are located. Codeflash will use this directory to discover existing benchmarks. Note that this option is required when running with `--benchmark`.
|
||||
- `ignore-paths`: A list of paths withing the `module-root` to ignore when optimizing code. Codeflash will not optimize code in these paths. Useful for ignoring build directories or other generated code. You can also leave this empty if not needed.
|
||||
- `ignore-paths`: A list of paths within the `module-root` to ignore when optimizing code. Codeflash will not optimize code in these paths. Useful for ignoring build directories or other generated code. You can also leave this empty if not needed.
|
||||
- `pytest-cmd`: The command to run your tests. Defaults to `pytest`. You can specify extra commandline arguments here for pytest.
|
||||
- `formatter-cmds`: The command line to run your code formatter or linter. Defaults to `["black $file"]`. In the command line `$file` refers to the current file being optimized. The assumption with using tools here is that they overwrite the same file and returns a zero exit code. You can also specify multiple tools here that run in a chain as a toml array. You can also disable code formatting by setting this to `["disabled"]`.
|
||||
- `ruff` - A recommended way to run ruff linting and formatting is `["ruff check --exit-zero --fix $file", "ruff format $file"]`. To make `ruff check --fix` return a 0 exit code please add a `--exit-zero` argument.
|
||||
- `disable-imports-sorting`: By default, codeflash uses isort to organize your imports before creating suggestions. You can disable this by setting this field to `true`. This could be useful if you don't sort your imports or while using linters like ruff that sort imports too.
|
||||
- `disable-telemetry`: Disable telemetry data collection. Defaults to `false`. Set this to `true` to disable telemetry data collection. Codeflash collects anonymized telemetry data to understand how users are using Codeflash and to improve the product. Telemetry does not collect any code data.
|
||||
- `git-remote`: The git remote to use for pull requests. Defaults to `"origin"`.
|
||||
- `override-fixtures`: Override pytest fixtures during optimization. Defaults to `false`.
|
||||
|
||||
## Example Configuration
|
||||
|
||||
Here's an example project with the following structure:
|
||||
|
||||
```text
|
||||
acme-project/
|
||||
|- foo_module/
|
||||
|
|
@ -57,10 +74,10 @@ acme-project/
|
|||
```
|
||||
|
||||
Here's a sample `pyproject.toml` file for the above project:
|
||||
|
||||
```toml
|
||||
[tool.codeflash]
|
||||
module-root = "foo_module"
|
||||
tests-root = "tests"
|
||||
test-framework = "pytest" # or "unittest"
|
||||
ignore-paths = []
|
||||
```
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ Before installing Codeflash, ensure you have:
|
|||
3. **Project dependencies installed** in your virtual environment
|
||||
|
||||
Good to have (optional):
|
||||
|
||||
1. **Unit Tests** that Codeflash uses to ensure correctness of the optimizations
|
||||
|
||||
<Warning>
|
||||
|
|
@ -27,11 +28,13 @@ source venv/bin/activate # On Linux/Mac
|
|||
# or
|
||||
venv\Scripts\activate # On Windows
|
||||
```
|
||||
|
||||
</Warning>
|
||||
<Steps>
|
||||
<Step title="Install Codeflash">
|
||||
|
||||
You can install Codeflash locally for a project by running the following command in the project's virtual environment:
|
||||
|
||||
```bash
|
||||
pip install codeflash
|
||||
```
|
||||
|
|
@ -51,6 +54,7 @@ uv add --dev codeflash
|
|||
```bash poetry
|
||||
poetry add codeflash@latest --group dev
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
</Tip>
|
||||
</Step>
|
||||
|
|
@ -69,20 +73,24 @@ If you don't have a pyproject.toml file yet, the codeflash init command will ask
|
|||
|
||||
`pyproject.toml` is a configuration file that is used to specify build and tool settings for Python projects.
|
||||
`pyproject.toml` is the modern replacement for setup.py and requirements.txt files.
|
||||
|
||||
</Info>
|
||||
|
||||
When running `codeflash init`, you will see the following prompts:
|
||||
|
||||
```text
|
||||
1. Enter your Codeflash API key:
|
||||
2. Install the GitHub app.
|
||||
3. Which Python module do you want me to optimize going forward? (e.g. my_module)
|
||||
4. Where are your tests located? (e.g. tests/)
|
||||
5. Which test framework do you use? (pytest/unittest)
|
||||
6. Install GitHub actions for Continuous optimization?
|
||||
1. Enter your Codeflash API key (or login with Codeflash)
|
||||
2. Which Python module do you want me to optimize going forward? (e.g. my_module)
|
||||
3. Where are your tests located? (e.g. tests/)
|
||||
4. Which code formatter do you use? (black/ruff/other/disabled)
|
||||
5. Which git remote should Codeflash use for Pull Requests? (if multiple remotes exist)
|
||||
6. Help us improve Codeflash by sharing anonymous usage data?
|
||||
7. Install the GitHub app
|
||||
8. Install GitHub actions for Continuous optimization?
|
||||
```
|
||||
|
||||
After you have answered these questions, the Codeflash configuration will be saved in the `pyproject.toml` file.
|
||||
|
||||
</Step>
|
||||
|
||||
<Step title="Generate a Codeflash API Key">
|
||||
|
|
@ -96,6 +104,7 @@ Codeflash uses cloud-hosted AI models and integrations with GitHub. If you haven
|
|||
**Free Tier Available**
|
||||
|
||||
Codeflash offers a **free tier** with a limited number of optimizations. Perfect for trying it out on small projects!
|
||||
|
||||
</Note>
|
||||
</Step>
|
||||
|
||||
|
|
@ -108,14 +117,13 @@ The Codeflash GitHub App allows access to your repository to the codeflash-ai bo
|
|||
Please [install the Codeflash GitHub
|
||||
app](https://github.com/apps/codeflash-ai/installations/select_target) by choosing the repository you want to install
|
||||
Codeflash on.
|
||||
</Step>
|
||||
|
||||
</Step>
|
||||
|
||||
</Steps>
|
||||
|
||||
To understand the configuration options, and set more advanced options, see the [Manual Configuration](/configuration) page.
|
||||
|
||||
|
||||
## Try It Out!
|
||||
|
||||
<Tabs>
|
||||
|
|
@ -154,12 +162,8 @@ cd optimize-me
|
|||
</Step>
|
||||
|
||||
<Step title="Set Up Environment">
|
||||
```bash
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
pip install codeflash
|
||||
```
|
||||
```bash python -m venv .venv source .venv/bin/activate pip install -r
|
||||
requirements.txt pip install codeflash ```
|
||||
</Step>
|
||||
|
||||
<Step title="Run Codeflash">
|
||||
|
|
@ -221,7 +225,6 @@ codeflash --all # optimize the entire repo
|
|||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
|
||||
### Next Steps
|
||||
|
||||
- Learn about [Codeflash Concepts](/codeflash-concepts/how-codeflash-works)
|
||||
|
|
|
|||
|
|
@ -3,10 +3,21 @@ title: "Optimize Performance Benchmarks with every Pull Request"
|
|||
description: "Configure and use pytest-benchmark integration for performance-critical code optimization"
|
||||
icon: "chart-line"
|
||||
sidebarTitle: Setup Benchmarks to Optimize
|
||||
keywords: ["benchmarks", "CI", "pytest-benchmark", "performance testing", "github actions", "benchmark mode"]
|
||||
keywords:
|
||||
[
|
||||
"benchmarks",
|
||||
"CI",
|
||||
"pytest-benchmark",
|
||||
"performance testing",
|
||||
"github actions",
|
||||
"benchmark mode",
|
||||
]
|
||||
---
|
||||
|
||||
<Info>
|
||||
**Performance-critical optimization** - Define benchmarks for your most important code sections and let Codeflash optimize and measure the real-world impact of every optimization on your performance metrics.
|
||||
**Performance-critical optimization** - Define benchmarks for your most
|
||||
important code sections and let Codeflash optimize and measure the real-world
|
||||
impact of every optimization on your performance metrics.
|
||||
</Info>
|
||||
|
||||
Benchmark mode is an easy way to define workflows that are performance-critical and need to be optimized and run fast.
|
||||
|
|
@ -20,12 +31,12 @@ It will then try to optimize the new code for the benchmark and calculate the im
|
|||
Create a directory for benchmarks if it does not already exist.
|
||||
|
||||
In your pyproject.toml, add the path to the 'benchmarks-root' section.
|
||||
```yaml
|
||||
|
||||
```toml
|
||||
[tool.codeflash]
|
||||
# All paths are relative to this pyproject.toml's directory.
|
||||
module-root = "inference"
|
||||
tests-root = "tests"
|
||||
test-framework = "pytest"
|
||||
benchmarks-root = "tests/benchmarks" # add your benchmarks root dir here
|
||||
ignore-paths = []
|
||||
formatter-cmds = ["disabled"]
|
||||
|
|
@ -49,7 +60,6 @@ It will then try to optimize the new code for the benchmark and calculate the im
|
|||
|
||||
The pytest-benchmark format is simply used as an interface. The plugin is actually not used - Codeflash will run these benchmarks with its own pytest plugin
|
||||
|
||||
|
||||
3. **Run and Test Codeflash:**
|
||||
|
||||
Run Codeflash with the `--benchmark` flag. Note that benchmark mode cannot be used with `--all`.
|
||||
|
|
@ -64,7 +74,6 @@ It will then try to optimize the new code for the benchmark and calculate the im
|
|||
codeflash --file test_file.py --benchmark --benchmarks-root path/to/benchmarks
|
||||
```
|
||||
|
||||
|
||||
4. **Run Codeflash :**
|
||||
|
||||
Benchmark mode is best used together with Codeflash as a GitHub Action. This way,
|
||||
|
|
@ -79,23 +88,16 @@ It will then try to optimize the new code for the benchmark and calculate the im
|
|||
codeflash --benchmark
|
||||
```
|
||||
|
||||
|
||||
|
||||
## How it works
|
||||
|
||||
1. Codeflash identifies benchmarks in the benchmarks-root directory.
|
||||
|
||||
|
||||
2. The benchmarks are run so that runtime statistics and inputs can be recorded.
|
||||
|
||||
|
||||
3. Replay tests are generated so the performance of optimization candidates on the exact inputs used in the benchmarks can be measured.
|
||||
|
||||
|
||||
4. If an optimization candidate is verified to be correct, the speedup of the optimization is calculated for each benchmark.
|
||||
|
||||
|
||||
5. Codeflash then reports the impact of the optimization on each benchmark.
|
||||
|
||||
|
||||
Using Codeflash with benchmarks is a great way to find optimizations that really matter.
|
||||
|
|
|
|||
|
|
@ -3,7 +3,14 @@ title: "Optimize a Single Function"
|
|||
description: "Target and optimize individual Python functions for maximum performance gains"
|
||||
icon: "bullseye"
|
||||
sidebarTitle: "Optimize Single Function"
|
||||
keywords: ["function optimization", "single function", "class methods", "performance", "targeted optimization"]
|
||||
keywords:
|
||||
[
|
||||
"function optimization",
|
||||
"single function",
|
||||
"class methods",
|
||||
"performance",
|
||||
"targeted optimization",
|
||||
]
|
||||
---
|
||||
|
||||
Codeflash is essentially a function optimizer. When asked to optimize a function, Codeflash will analyze the function,
|
||||
|
|
|
|||
|
|
@ -3,7 +3,15 @@ title: "Trace & Optimize E2E Workflows"
|
|||
description: "End-to-end optimization of entire Python workflows with execution tracing"
|
||||
icon: "route"
|
||||
sidebarTitle: "Optimize E2E Workflows"
|
||||
keywords: ["tracing", "workflow optimization", "replay tests", "end-to-end", "script optimization", "context manager"]
|
||||
keywords:
|
||||
[
|
||||
"tracing",
|
||||
"workflow optimization",
|
||||
"replay tests",
|
||||
"end-to-end",
|
||||
"script optimization",
|
||||
"context manager",
|
||||
]
|
||||
---
|
||||
|
||||
Codeflash can optimize an entire Python script end-to-end by tracing the script's execution and generating Replay Tests.
|
||||
|
|
@ -30,8 +38,15 @@ The generated replay tests and the trace file are for the immediate optimization
|
|||
|
||||
## Codeflash optimize 1 min demo
|
||||
|
||||
<iframe width="640" height="400" src="https://www.youtube.com/embed/_nwliGzRIug" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
|
||||
|
||||
<iframe
|
||||
width="640"
|
||||
height="400"
|
||||
src="https://www.youtube.com/embed/_nwliGzRIug"
|
||||
title="YouTube video player"
|
||||
frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen
|
||||
></iframe>
|
||||
|
||||
## What is the codeflash optimize command?
|
||||
|
||||
|
|
@ -42,7 +57,7 @@ We call these generated test cases "Replay Tests" because they replay the inputs
|
|||
These replay tests are representative of the real-world usage of your functions.
|
||||
|
||||
Using Replay Tests, Codeflash can verify that the optimized functions produce the same output as the original function and also measure the performance gains of the optimized function on the real-world inputs.
|
||||
This way you can be *sure* that the optimized function causes no changes of behavior for the traced workflow and also, that it is faster than the original function. To get more confidence on the correctness of the code, we also generate several LLM generated test cases and discover any existing unit cases you may have.
|
||||
This way you can be _sure_ that the optimized function causes no changes of behavior for the traced workflow and also, that it is faster than the original function. To get more confidence on the correctness of the code, we also generate several LLM generated test cases and discover any existing unit cases you may have.
|
||||
|
||||
## Using codeflash optimize
|
||||
|
||||
|
|
@ -63,7 +78,7 @@ Codeflash script optimizer can be used in three ways:
|
|||
```
|
||||
|
||||
The above command should suffice in most situations.
|
||||
To customize the trace file location you can specify it like `codeflash optimize -o trace_file_path.trace`. Otherwise, it defaults to `codeflash.trace` in the current working directory.
|
||||
To customize the trace file location you can specify it like `codeflash optimize --output trace_file_path.trace`. Otherwise, it defaults to `codeflash.trace` in the current working directory.
|
||||
|
||||
2. **Trace and optimize as two separate steps**
|
||||
|
||||
|
|
@ -72,7 +87,7 @@ Codeflash script optimizer can be used in three ways:
|
|||
To create just the trace file first, run
|
||||
|
||||
```bash
|
||||
codeflash optimize -o trace_file.trace --trace-only path/to/your/file.py --your_options
|
||||
codeflash optimize --output trace_file.trace --trace-only path/to/your/file.py --your_options
|
||||
```
|
||||
|
||||
This will create a replay test file. To optimize with the replay test, run the
|
||||
|
|
@ -82,7 +97,9 @@ Codeflash script optimizer can be used in three ways:
|
|||
```
|
||||
|
||||
More Options:
|
||||
- `--tracer-timeout`: The maximum time in seconds to trace the entire workflow. Default is indefinite. This is useful while tracing really long workflows.
|
||||
|
||||
- `--timeout`: The maximum time in seconds to trace the entire workflow. Default is indefinite. This is useful while tracing really long workflows.
|
||||
|
||||
3. **As a Context Manager -**
|
||||
|
||||
To trace only specific sections of your code, You can also use the Codeflash Tracer as a context manager.
|
||||
|
|
|
|||
Loading…
Reference in a new issue