Score by category
short
0.520
w/o8/8
w/8/8
medium
0.522
w/o8/8
w/8/8
long
0.638
w/o8/8
w/8/8
Tokens - miner vs baseline
MinerBaselineTotal weighted112.15M of 265.11M↓ 57.7%
Input26.15M of 30.86M↓ 15.3%
Cache583.27M of 1745.73M↓ 66.6%
Output9.23M of 19.89M↓ 53.6%
JEV inputBaseline never calls JEV0
Competition tasks
| Task | Result (w/o → w/) | Runs | |||
|---|---|---|---|---|---|
| pipecat-ai__pipecat-5084short | 23/2323/23 | 0.386 | 23 | 8.02M | |
| hynek__structlog-350short | 20/2020/20 | 0.423 | 20 | 4.06M | |
| dpkp__kafka-python-2548short | 32/3232/32 | 0.518 | 32 | 9.40M | |
| dask__dask-12416short | 20/2020/20 | 0.373 | 20 | 5.89M | |
| andialbrecht__sqlparse-323short | 23/2323/23 | 0.269 | 23 | 8.52M | |
| benoitc__gunicorn-3550short | 33/3333/33 | 0.855 | 33 | 6.24M | |
| getsentry__responses-485short | 33/3333/33 | 0.713 | 33 | 8.62M | |
| facebookresearch__xformers-147short | 33/3333/33 | 0.591 | 33 | 13.10M | |
| amazon-science__chronos-forecasting-407medium | 34/3434/34 | 0.424 | 34 | 16.98M | |
| cornellius-gp__gpytorch-2677medium | 27/2727/27 | 0.382 | 27 | 14.72M | |
| elastic__elasticsearch-py-2030medium | 22/2222/22 | 0.712 | 22 | 6.85M | |
| geopandas__geopandas-3363medium | 28/2828/28 | 0.544 | 28 | 10.21M | |
| livekit__agents-6585medium | 26/2626/26 | 0.698 | 26 | 9.29M | |
| kernc__backtesting.py-1366medium | 24/2424/24 | 0.547 | 24 | 10.61M | |
| tobymao__sqlglot-7717medium | 21/2121/21 | 0.393 | 21 | 14.68M | |
| sympy__sympy-29719medium | 29/2929/29 | 0.307 | 29 | 29.17M | |
| vega__altair-4019long | 27/2727/27 | 0.403 | 27 | 30.89M | |
| element-hq__synapse-19468long | 20/2016/20 | 0.762 | 20 | 10.95M | |
| xonsh__xonsh-6413long | 29/2926/29 | 0.464 | 29 | 26.89M | |
| 567-labs__instructor-2301long | 35/3532/35 | 0.553 | 35 | 32.42M | |
| conan-io__conan-20139long | 26/2624/26 | 0.555 | 26 | 39.32M | |
| gaogaotiantian__viztracer-423long | 28/2928/29 | 0.660 | 29 | 38.70M | |
| simonw__llm-1479long | 22/2221/22 | 1.000 | 22 | 22.21M | |
| pymc-devs__pymc-8302long | 29/2927/29 | 0.699 | 29 | 115.65M |