HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Orgs/mistralai/mistralai/Mistral-Small-3.1-24B-Instruct-2503

mistralai/Mistral-Small-3.1-24B-Instruct-2503

family mistral · 15 variants · unknown

provider
mistralai
released
—
huggingface
https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503
best scores
SWE-bench Verified: 53.6DeepSWE: 42.2AIME 2024: 73.59AIME 2025: 64.95GPQA Diamond: 71.46LiveCodeBench: 59.36MMLU: 81.01MMLU-Pro: 66.76TriviaQA: 80.5GPQA: 44.42MMMU: 64.0MATH: 69.3MBPP: 74.71HumanEval: 88.41SimpleQA: 10.43MathVista: 68.91ChartQA: 86.24DocVQA: 94.08MT-Bench: 7.3LongBench: 37.18RULER: 93.96

Best scores per benchmark

benchmarkcategorybest score
DocVQAmultimodal94.08
RULERlong-context93.96
HumanEvalcoding88.41
ChartQAmultimodal86.24
MMLUknowledge81.01
TriviaQAknowledge80.5
MBPPcoding74.71
AIME 2024math73.59
GPQA Diamondreasoning71.46
MATHmath69.3
MathVistamultimodal68.91
MMLU-Proknowledge66.76
AIME 2025math64.95
MMMUmultimodal64.0
LiveCodeBenchcoding59.36
SWE-bench Verifiedswe53.6
GPQAreasoning44.42
DeepSWEswe42.2
LongBenchlong-context37.18
SimpleQAknowledge10.43
MT-Benchchat7.3

Variants (15)

variantkindquantbitsvRAMdlorgyear
Devstral-Small-2507originalbf1616.087,172mistralai2025
Magistral-Small-2506originalbf1616.025,921mistralai2025
Devstral-Small-2505originalbf1616.04,813mistralai2025
mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUFggufgguf2.04,072bartowski2025
Mistral-Small-3.1-24B-Instruct-2503-GGUFggufgguf16.03,356unsloth2025
Mistral-Small-3.1-24B-Instruct-2503-GGUFggufgguf3.02,160lmstudio-community2025
Mistral-Small-3.1-24B-Instruct-2503-bnb-4bitbnb-4bit4bit907unsloth2025
Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bitbnb-4bit4bit526unsloth2025
Magistral-Small-2507originalbf1616.0513mistralai2025
Devstral-Small-2507originalbf1616.0365unsloth2025
Mistral-Small-3.1-24B-Instruct-2503originalbf1616.0347unsloth2025
Mistral-Small-3.1-24B-Instruct-2503-GGUFggufgguf2.0202ggml-org2025
Magistral-Small-2506-GGUFggufgguf16.0152Mungert2025
mistral-small-3.1-24b-instruct-2503-hforiginalbf1616.0147mrfakename2025
Harbinger-24B-GGUFggufgguf16.0114Mungert2025
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.