Agents · real repositories · reference
SWE-bench Verified
Real GitHub issues from popular Python projects, checked by people to be solvable. The model writes a patch, and the project’s own tests decide whether it worked.
- Models
- 34
- By makers
- 34
- Latest
- Aug 25
% of issues resolved. v marks a score the model’s maker reported. Prices are FullsendAI’s per million tokens, blended 3 input : 1 output.
About the data
From SWE-bench. The leaderboard data (swe-bench.github.io) is CC BY-NC 4.0, non-commercial; the harness and dataset are MIT.