Qwen/Qwen2.5-Omni-3B
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| DocVQA | multimodal | 95.2 | DocVQA<sub>test</sub> | medium | | DocVQA<sub>test</sub> | 95.2 | 93.3 | 93.5 | **95.7** | - | |
| GSM8K | math | 88.7 | GSM8K | medium | | GSM8K | 88.7 | 82.6 | **91.6** | 86.7 | 85.7 | 84.5 | 76.7 | |
| ChartQA | multimodal | 85.3 | ChartQA<sub>test Avg</sub> | medium | | ChartQA<sub>test Avg</sub> | 85.3 | 82.8 | 84.9 | **87.3** | - | |
| MMBench | multimodal | 81.8 | MMBench-V1.1-EN<sub>test</sub> | medium | | MMBench-V1.1-EN<sub>test</sub> | 81.8 | 77.8 | 80.5 | **82.6** | 76.0 | |
| HumanEval | coding | 78.7 | HumanEval | medium | | HumanEval | 78.7 | 70.7 | **84.8** | 74.4 | 79.9 | 72.6 | 68.9 | |
| MBPP | coding | 73.2 | MBPP | medium | | MBPP | 73.2 | 70.4 | **79.2** | 72.7 | 67.2 | 69.6 | 74.9 | |
| MATH | math | 71.5 | MATH | medium | | MATH | 71.5 | 63.6 | **75.5** | 65.9 | 52.9 | 51.9 | 44.3 | |
| MMLU | knowledge | 71.0 | MMLU-redux | medium | | MMLU-redux | 71.0 | 60.9 | **75.4** | 64.4 | 67.3 | 67.2 | 72.8 | |
| MathVista | multimodal | 67.9 | MathVista<sub>testmini</sub> | medium | | MathVista<sub>testmini</sub> | 67.9 | 59.4 | **71.9** | 68.2 | 52.5 | |
| MMMU | multimodal | 59.2 | MMMU<sub>val</sub> | medium | | MMMU<sub>val</sub> | 59.2 | 53.1 | 53.9 | 58.6 | **60.0** | |
| MMLU-Pro | knowledge | 47.0 | MMLU-Pro | medium | | MMLU-Pro | 47.0 | 40.4 | **56.3** | 43.7 | 44.1 | 48.3 | 52.1 | |
| MMMU | multimodal | 36.6 | MMMU-Pro<sub>overall</sub> | medium | | MMMU-Pro<sub>overall</sub> | 36.6 | 29.7 | - | **38.3** | 37.6 | |
| GPQA | reasoning | 30.8 | GPQA | medium | | GPQA | 30.8 | 34.3 | **36.4** | 30.3 | 34.3 | 32.8 | 32.8 | |
| LiveCodeBench | coding | 24.6 | LiveCodeBench<sub>2305-2409</sub> | medium | | LiveCodeBench<sub>2305-2409</sub> | 24.6 | 16.5 | **28.7** | 19.9 | 23.9 | 8.3 | 18.9 | |
| MT-Bench | chat | 6.0 | MM-MT-Bench | medium | | MM-MT-Bench | 6.0 | 5.0 | - | **6.3** | - | |
none scraped.