← Benchmarks
MGSM
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
id mgsm · max 1 · 31 models reported
| # | Model | Score |
|---|---|---|
| 1 | Llama 4 Maverick Meta · open | 0.92 |
| 2 | o3-mini OpenAI | 0.92 |
| 3 | Claude 3.5 Sonnet Anthropic | 0.92 |
| 4 | Claude 3.5 Sonnet Anthropic | 0.92 |
| 5 | Llama 3.3 70B Instruct Meta · open | 0.91 |
| 6 | o1-preview OpenAI | 0.91 |
| 7 | Claude 3 Opus Anthropic | 0.91 |
| 8 | Llama 4 Scout Meta · open | 0.91 |
| 9 | GPT-4o OpenAI | 0.91 |
| 10 | o1 OpenAI | 0.89 |