Commit graph

3514 commits

Author SHA1 Message Date
RD
d35208ac7a
Merge pull request #1225 from codeflash-ai/mypy-gha
mypy GHA, with allowlist of currently passing functions
2024-11-14 09:52:06 -08:00
RD
cbd1b5b97a
Merge branch 'main' into mypy-gha 2024-11-13 15:35:48 -08:00
Saurabh Misra
4327189373
Merge pull request #1256 from codeflash-ai/remove-crosshair-as-a-dependency
remove crosshair as a dependency
2024-11-13 14:39:35 -08:00
Saurabh Misra
4a2681f8b9 remove crosshair as a dependency 2024-11-13 14:34:40 -08:00
Alvin Ryanputra
6f6e836126
Merge pull request #1255 from codeflash-ai/env_routing
"gpt-4o-2" and "gpt-4o
2024-11-13 12:21:29 -08:00
Alvin Ryanputra
bbc2fdc6b7
Merge branch 'main' into env_routing 2024-11-13 12:09:28 -08:00
Sarthak Agarwal
4e01932148
seeding random (#1165)
### **User description**

Closes #
https://linear.app/codeflash-ai/issue/CF-175/account-for-numpyrandom-calls-by-setting-a-seed


___

### **PR Type**
Enhancement, Tests


___

### **Description**
- Enhanced the handling of random number generator (RNG) modules by
adding a dictionary to map modules to their seed-setting functions.
- Improved the visitor pattern to detect RNG imports and set seeds
accordingly in the test instrumentation process.
- Updated existing test cases and added new ones to cover various
scenarios of RNG usage and seed setting.
- Adjusted the indentation and added a caution comment in the optimizer
code regarding loop index handling.



___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>optimizer.py</strong><dd><code>Adjust indentation and
add caution comment in optimizer</code>&nbsp; &nbsp; </dd></summary>
<hr>

cli/codeflash/optimization/optimizer.py

<li>Adjusted the indentation of <code>total_timing</code>
assignment.<br> <li> Added a caution comment about loop index
handling.<br>


</details>


  </td>
<td><a
href="https://github.com/codeflash-ai/codeflash/pull/1165/files#diff-9bbfdc23f4fbf008e5c7d80d4ac3dbb36e757835ef7f868b42d86aad0b29a77b">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>instrument_new_tests.py</strong><dd><code>Enhance RNG
handling and seed setting in test instrumentation</code></dd></summary>
<hr>

django/aiservice/testgen/instrumentation/instrument_new_tests.py

<li>Added a dictionary for RNG modules and their seed functions.<br>
<li> Enhanced the visitor pattern to handle RNG imports and set
seeds.<br> <li> Implemented methods to create seed statements and
process statements <br>recursively.<br>


</details>


  </td>
<td><a
href="https://github.com/codeflash-ai/codeflash/pull/1165/files#diff-b174c49c3e090435474a9548d4d2522d0ac073fc1a2658d6ca97c4c85e9c5adc">+131/-81</a></td>

</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>

<summary><strong>test_instrument_generated_tests.py</strong><dd><code>Update
and expand test cases for RNG seed handling</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>


django/aiservice/testgen/instrumentation/tests/test_instrument_generated_tests.py

<li>Updated test cases to use new seed setting logic.<br> <li> Added
multiple test cases for different RNG import scenarios.<br> <li>
Modified assertions to reflect changes in seed handling.<br>


</details>


  </td>
<td><a
href="https://github.com/codeflash-ai/codeflash/pull/1165/files#diff-27560015a80c8a9bdf30c40c8e57b65b82312f452a51c11b65ffe2b38ac4bb79">+1113/-8</a></td>

</tr>
</table></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull
request to receive relevant information

---------

Co-authored-by: Saurabh Misra <misra.saurabh1@gmail.com>
2024-11-13 15:08:33 -05:00
Alvin Ryanputra
621f0a0879 "gpt-4o" by default. if OPENAI_API_TYPE is azure, use "gpt-4o-2" for higher rate limits 2024-11-13 14:37:49 -05:00
RD
f94feb6a27 Merge branch 'main' of github.com:codeflash-ai/codeflash into mypy-gha 2024-11-13 00:57:08 -08:00
RD
3b9669ab78
Merge pull request #1215 from codeflash-ai/typed-signature-filtering
Typed Signature Filtering
2024-11-13 00:55:17 -08:00
RD
4d4c8b1e51 Merge remote-tracking branch 'origin/typed-signature-filtering' into typed-signature-filtering 2024-11-13 00:30:42 -08:00
RD
2c8bc063ae Merge branch 'main' of github.com:codeflash-ai/codeflash into typed-signature-filtering 2024-11-13 00:30:17 -08:00
RD
4cfc82c295
Merge pull request #1163 from codeflash-ai/Concolicity
Concolic testing forever
2024-11-13 00:24:10 -08:00
RD
609d3ab904 Conflictacide 2024-11-12 22:59:47 -08:00
RD
49b37b9e52 Merge branch 'main' of github.com:codeflash-ai/codeflash into Concolicity
# Conflicts:
#	cli/codeflash/optimization/optimizer.py
2024-11-12 22:55:49 -08:00
Saurabh Misra
bc67f5356b
Merge pull request #1223 from codeflash-ai/optimize-merge-test-results
Optimize joining two Test results + fix matching the results across two TestResults objects + filter out @property functions
2024-11-12 22:43:06 -08:00
RD
b9fdf8be66 Merge branch 'main' of github.com:codeflash-ai/codeflash into mypy-gha 2024-11-12 22:36:25 -08:00
RD
acb2fdd61c This is promising. 2024-11-12 22:32:08 -08:00
RD
fb3256f317 This is promising. 2024-11-12 22:27:56 -08:00
RD
76ba89c94c Giving up on uv pip 2024-11-12 22:22:00 -08:00
RD
4ea1ecb3a0 Trying uv pip 2024-11-12 22:15:42 -08:00
RD
0f5dac57c0 Trying uv pip 2024-11-12 22:00:46 -08:00
RD
0dd1e5d8f0 Trying uv pip 2024-11-12 21:35:44 -08:00
RD
ddfac5e4f0 Trying uv pip 2024-11-12 21:31:50 -08:00
RD
2c98b0c4d5 Typo. 2024-11-12 20:56:44 -08:00
RD
23ff57117b Typo. 2024-11-12 20:55:01 -08:00
RD
827e28ac71 Trying uv. 2024-11-12 20:50:34 -08:00
Saurabh Misra
e327dafa63
Merge pull request #1238 from codeflash-ai/codeflash/optimize-pr1223-2024-11-13T02.45.49
️ Speed up method `TestResults.add` by 40% in PR #1223 (`optimize-merge-test-results`)
2024-11-12 19:17:24 -08:00
Saurabh Misra
fe88f3972d
Update cli/codeflash/verification/test_results.py 2024-11-12 19:16:56 -08:00
codeflash-ai[bot]
2670ef04e4
️ Speed up method TestResults.add by 40% in PR #1223 (optimize-merge-test-results)
Here is a more optimized version of your Python program.



### Changes Applied for Optimization.
1. Removed unnecessary imports and streamlined import statements.
2. Removed redundant casting and optimized type checking.
3. Set recursion limit conditionally only when needed.
4. Removed redundant calculation of `unique_invocation_loop_id` by caching it where necessary.
5. Added a helper function `_compare_results` for clean comparison logic to improve readability and performance reuse.
6. Optimized the `__eq__` comparison to create a dictionary of `other`'s test results for faster lookups.
7. Added an `__init__` method in `TestResults` to properly initialize `test_results` and `test_result_idx`.

These changes aim to reduce redundancy, simplify code, and increase overall performance.
2024-11-13 02:45:53 +00:00
RD
3ae9fc4b0a aiservice is good, cli/codeflash is a mystery. 2024-11-12 18:44:22 -08:00
RD
b30f5f7de4 Revert "aiservice is good, cli/codeflash is a mystery."
This reverts commit 106ec17cd2.
2024-11-12 18:42:36 -08:00
RD
106ec17cd2 aiservice is good, cli/codeflash is a mystery. 2024-11-12 18:32:24 -08:00
Saurabh Misra
d1ed637a2c
Merge branch 'main' into optimize-merge-test-results 2024-11-12 18:05:00 -08:00
RD
c4b9402c26 Revert "aiservice is good, cli/codeflash has pyproject.toml in cli."
This reverts commit 0a7bba81a2.
2024-11-12 17:35:13 -08:00
Saurabh Misra
d14e72928b
Merge pull request #1233 from codeflash-ai/update-gpt-4o-deployment
update the deployment to higher TPS model
2024-11-12 17:30:50 -08:00
RD
0a7bba81a2 aiservice is good, cli/codeflash has pyproject.toml in cli. 2024-11-12 17:26:44 -08:00
Saurabh Misra
46ef7b09f8 not raise an exception, but log a warning 2024-11-12 17:24:32 -08:00
Saurabh Misra
2219fa0794 update the deployment to higher TPS model 2024-11-12 17:16:41 -08:00
RD
6f99ea305a Installing everything 2024-11-12 16:49:25 -08:00
Saurabh Misra
ababe8fb3c fix the loop_index in the case there is a test failure with unittests 2024-11-12 16:47:05 -08:00
RD
07df7c6ee7 One step at a time 2024-11-12 16:45:43 -08:00
RD
c3299cd7d0 Poetry lies 2024-11-12 16:25:08 -08:00
RD
02915622b6 In poetry we trust 2024-11-12 16:22:19 -08:00
RD
52705d6e32 Apparently we need mypy to test mypy. 2024-11-12 16:12:18 -08:00
RD
8f8b2381df mypy GHA 2024-11-12 15:47:02 -08:00
Saurabh Misra
4cd9aeadb1 don't attempt to optimize functions that are a property 2024-11-12 15:33:43 -08:00
Saurabh Misra
37367c12ed forgot a file 2024-11-12 14:48:24 -08:00
Saurabh Misra
7248d4c5dc remove the concept of cached-tests, not needed anymore
make sure that the generated test file path is robust
2024-11-12 14:44:20 -08:00
Saurabh Misra
722c267d6c Merge branch 'refs/heads/main' into optimize-merge-test-results 2024-11-12 14:07:15 -08:00