inclusionAI/Ming-Lite-Omni
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| DocVQA | multimodal | 93.0 | DocVQA_TEST | medium | | DocVQA_TEST | 93 | <b>95.7</b> | |
| ChartQA | multimodal | 85.1 | ChartQA_TEST | medium | | ChartQA_TEST | 85.1 | <b>87.3</b> | |
| MMBench | multimodal | 80.8 | MMBench_TEST_V11 | medium | | MMBench_TEST_V11 | 80.8 | <b>82.8</b> | 82.0 | |
| MathVista | multimodal | 68.1 | MathVista | medium | | MathVista | <b>71.6</b> | 68.1 | 67.9 | |
| OpenBookQA | knowledge | 61.98 | high | | Ming-lite-omni | <b>4.34</b> | <b>4.63</b> | <b>4.06</b> | 58.84 | 47.53 | 61.98 | 58.36 | 99.04 | | |
| IFEval | instruction-following | 58.36 | high | | Ming-lite-omni | <b>4.34</b> | <b>4.63</b> | <b>4.06</b> | 58.84 | 47.53 | 61.98 | 58.36 | 99.04 | | |
| MMMU | multimodal | 56.3 | MMMU | medium | | MMMU | 56.3 | <b>56.6</b> | 54.8 | |
| MMMU | multimodal | 46.3 | Video-MMMU | medium | | Video-MMMU | 46.3 | <b>47.4</b> | |
| WER VoxPopuli | speech-asr | 5.8 | high | | Ming-lite-omni | 1.47 | **2.55** | **2.52** | 6.31 | 2.96 | 5.95 | 5.46 | 1.44 | 2.80 | **4.15** | **6.89** | **3.39** | **5.80** | | |
| AlpacaEval | chat | 3.69 | low | | Qwen2-Audio-chat | 3.545 | 3.69 | 3.40 | 35.35 | 35.43 | 49.01 | 22.57 | 98.85 | | |
| WER LibriSpeech | speech-asr | 1.44 | high | | Ming-lite-omni | 1.47 | **2.55** | **2.52** | 6.31 | 2.96 | 5.95 | 5.46 | 1.44 | 2.80 | **4.15** | **6.89** | **3.39** | **5.80** | |
none scraped.