HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Orgs/mistralai/mistralai/Mistral-Small-3.1-24B-Base-2503

mistralai/Mistral-Small-3.1-24B-Base-2503

family mistral · 8 variants · unknown

provider
mistralai
released
—
huggingface
https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Base-2503
best scores
MMLU: 81.01MMLU-Pro: 66.76TriviaQA: 80.5GPQA: 44.42MMMU: 64.0MATH: 69.3GPQA Diamond: 45.96MBPP: 74.71HumanEval: 88.99SimpleQA: 10.43MathVista: 68.91ChartQA: 86.24DocVQA: 94.08MT-Bench: 7.3LongBench: 37.18RULER: 93.96WildBench: 55.6Chatbot Arena: 19.56SWE-bench Verified: 72.2Terminal-Bench: 32.6

Best scores per benchmark

benchmarkcategorybest score
DocVQAmultimodal94.08
RULERlong-context93.96
HumanEvalcoding88.99
ChartQAmultimodal86.24
MMLUknowledge81.01
TriviaQAknowledge80.5
MBPPcoding74.71
SWE-bench Verifiedswe72.2
MATHmath69.3
MathVistamultimodal68.91
MMLU-Proknowledge66.76
MMMUmultimodal64.0
WildBenchchat55.6
GPQA Diamondreasoning45.96
GPQAreasoning44.42
LongBenchlong-context37.18
Terminal-Benchagentic-coding32.6
Chatbot Arenachat19.56
SimpleQAknowledge10.43
MT-Benchchat7.3

Variants (8)

variantkindquantbitsvRAMdlorgyear
Mistral-Small-3.1-24B-Instruct-2503originalbf1616.0271,011mistralai2025
Mistral-Small-3.2-24B-Instruct-2506originalbf1616.0261,400mistralai2025
Devstral-Small-2-24B-Instruct-2512fp8fp88.0251,726mistralai2026
Mistral-Small-3.1-24B-Base-2503originalbf1616.01,126mistralai2025
sarvam-m-GGUFggufgguf16.0450Mungert2025
Mistral-Small-3.1-24B-Instruct-2503-GGUFggufgguf16.0293Mungert2025
Dans-PersonalityEngine-V1.3.0-24b-GGUFggufgguf16.0120Mungert2025
Mistral-Small-3.2-24B-Instruct-2506-GGUFggufgguf16.0100Mungert2025
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.