← Benchmarks
SWE-bench Multilingual
A multilingual benchmark for issue resolving in software engineering that covers Java, TypeScript, JavaScript, Go, Rust, C, and C++. Contains 1,632 high-quality instances carefully annotated from 2,456 candidates by 68 expert annotators, designed to evaluate Large Language Models across diverse software ecosystems beyond Python.
id swe-bench-multilingual · max 1 · 46 models reported
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 5.5 Anthropic | 0.94 |
| 2 | Claude Mythos Preview Anthropic | 0.87 |
| 3 | Claude Opus 4.8 Anthropic | 0.84 |
| 4 | Hy4 preview Tencent · open | 0.83 |
| 5 | Qwen3.8 Flash Alibaba Cloud / Qwen Team | 0.81 |
| 6 | Qwen3.8-Flash-Next Alibaba Cloud / Qwen Team · open | 0.81 |
| 7 | Laguna S 2.1 Poolside · open | 0.79 |
| 8 | Claude Sonnet 5 Anthropic | 0.78 |
| 9 | Qwen3.7 Max Alibaba Cloud / Qwen Team | 0.78 |
| 10 | Claude Opus 4.6 Anthropic | 0.78 |
| 11 | Kimi K2.6 Moonshot AI · open | 0.77 |
| 12 | MiniMax M2.7 MiniMax · open | 0.77 |
| 13 | DeepSeek-V4-Pro-Max DeepSeek · open | 0.76 |
| 14 | Hy3 Tencent · open | 0.76 |
| 15 | Qwen3.7-Plus Alibaba Cloud / Qwen Team | 0.76 |
| 16 | Qwen3.6 Plus Alibaba Cloud / Qwen Team | 0.74 |
| 17 | DeepSeek-V4-Flash-Max DeepSeek · open | 0.73 |
| 18 | Kimi K2.5 Moonshot AI · open | 0.73 |
| 19 | MiniMax M2.1 MiniMax · open | 0.72 |
| 20 | Ling 3.0 Flash InclusionAI | 0.72 |
| 21 | MiMo-V2-Flash Xiaomi · open | 0.72 |
| 22 | MiMo-V2-Pro Xiaomi | 0.72 |
| 23 | Qwen3.6-27B Alibaba Cloud / Qwen Team · open | 0.71 |