unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| DocVQA | multimodal | 94.08 | low | | **Small 3.1 24B Instruct** | 64.00% | **49.25%** | **68.91%** | 86.24% | **94.08%** | **93.72%** | **7.3** | | |
| RULER | long-context | 93.96 | RULER 32K | RULER 128K | low | | **Small 3.1 24B Instruct** | **37.18%** | **93.96%** | 81.20% | |
| HumanEval | coding | 88.41 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| ChartQA | multimodal | 86.24 | low | | **Small 3.1 24B Instruct** | 64.00% | **49.25%** | **68.91%** | 86.24% | **94.08%** | **93.72%** | **7.3** | | |
| MMLU | knowledge | 81.01 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | low | | **Small 3.1 24B Base** | **81.01%** | **56.03%** | 80.50% | **37.50%** | **59.27%** | |
| MMLU | knowledge | 80.62 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| TriviaQA | knowledge | 80.5 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | low | | **Small 3.1 24B Base** | **81.01%** | **56.03%** | 80.50% | **37.50%** | **59.27%** | |
| MBPP | coding | 74.71 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| MATH | math | 69.3 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| MathVista | multimodal | 68.91 | low | | **Small 3.1 24B Instruct** | 64.00% | **49.25%** | **68.91%** | 86.24% | **94.08%** | **93.72%** | **7.3** | | |
| MMLU-Pro | knowledge | 66.76 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| MMMU | multimodal | 64.0 | low | | **Small 3.1 24B Instruct** | 64.00% | **49.25%** | **68.91%** | 86.24% | **94.08%** | **93.72%** | **7.3** | | |
| MMMU | multimodal | 59.27 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | low | | **Small 3.1 24B Base** | **81.01%** | **56.03%** | 80.50% | **37.50%** | **59.27%** | |
| MMLU-Pro | knowledge | 56.03 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | low | | **Small 3.1 24B Base** | **81.01%** | **56.03%** | 80.50% | **37.50%** | **59.27%** | |
| GPQA Diamond | reasoning | 45.96 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| GPQA | reasoning | 44.42 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| GPQA | reasoning | 37.5 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | low | | **Small 3.1 24B Base** | **81.01%** | **56.03%** | 80.50% | **37.50%** | **59.27%** | |
| LongBench | long-context | 37.18 | RULER 32K | RULER 128K | low | | **Small 3.1 24B Instruct** | **37.18%** | **93.96%** | 81.20% | |
| RULER | long-context | 32.0 | medium | RULER 32 | |
| SimpleQA | knowledge | 10.43 | MMLU Pro (5-shot CoT) | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | low | | **Small 3.1 24B Instruct** | 80.62% | 66.76% | 69.30% | **44.42%** | **45.96%** | 74.71% | **88.41%** | **10.43%** | |
| MT-Bench | chat | 7.3 | low | | **Small 3.1 24B Instruct** | 64.00% | **49.25%** | **68.91%** | 86.24% | **94.08%** | **93.72%** | **7.3** | |
none scraped.