AI Coding New Leaderboard Tests Whether Code Can Be Merged, Opus 5.5 Still Fails 40% of Attempts

动察 Beating AI News: Artificial Analysis has launched the FrontierCode 1.1 coding leaderboard, focusing on evaluating whether AI-generated code can meet the merge standards of open-source project maintainers. Claude Opus 5.5 ranked first with a composite score of 52.2%, but only 58% of its attempts passed all hard checks, meaning over 40% still fell short.

This evaluation was developed by Cognition in collaboration with maintainers of 36 open-source projects, with each problem refined for over 40 hours. It does not just check whether code runs; it also requires that newly added tests can detect issues in old code, that unrelated files are not arbitrarily modified, and that existing functionality is not broken. Artificial Analysis selected 100 of the harder private tasks, had all models use the same Agent framework, and ran each problem 3 times. For 9 of these problems, no tested model passed successfully.

The cost gap is even larger than the score gap. GPT-6.1 Sol scored 49.1%, only 3.1 percentage points lower than Opus 5.5, but its average cost per problem was only $0.81, while the latter reached $10.15, a difference of more than 12 times. Xiaomi MiMo-V2.6-Pro led open-source models with 45%, at a cost of only $0.18 per problem; DeepSeek V4.1 Flash and Kimi K3 both scored 44%.

It should be noted that 52.2% is a composite score, not a real PR merge rate. The evaluation uses automated checks and review by the GPT-6 Luna model, and is not actually handed to maintainers for item-by-item review. All models using a unified Agent framework also means this ranking cannot directly represent the actual performance of each company's official coding tools.