HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks

Pixtral-12B-2409

mistralai/Pixtral-12B-2409

kind
original
quant
bf16 (16-bit)
family
mistral
modality
—
root model
mistralai/Pixtral-12B-Base-2409
provider
mistralai
license
apache-2.0
language
en, fr, de, es, it, pt, ru, zh, ja
context
—
library
vllm
vRAM (GB)
—
size on disk
—
downloads
6,542
likes
692
created
—
last modified
2026-06-02T04:44:58+00:00 (2026)
huggingface
https://huggingface.co/mistralai/Pixtral-12B-2409
tags
vllmmistral-commonenfrdeesitptruzhjabase_model:mistralai/Pixtral-12B-Base-2409base_model:finetune:mistralai/Pixtral-12B-Base-2409license:apache-2.0region:usdeploy:sagemaker

Benchmark scores (10)

benchmarkcategoryscorecontextconfevidence
DocVQAmultimodal90.7**DocVQA** *(ANLS)*medium| **DocVQA** *(ANLS)* | 90.7 | <ins>**94.5**</ins> | 90.5 | 84.9 | 74.4 |
MMLUknowledge68.5**MMLU** *(5-shot)*medium| **MMLU** *(5-shot)* | <ins>**69.2**</ins> | 68.5 | 67.9 | 63.5 | 63.6 |
MathVistamultimodal54.4**Mathvista** *(CoT)*medium| **Mathvista** *(CoT)* | <ins>**58.0**</ins> | 54.4 | 36.1 | 36.4 | 39.3 |
IFEvalinstruction-following50.1**Text IF-Eval**medium| **Text IF-Eval** | <ins>**61.3**</ins> | 50.1 | 51.4 | 50.9 | 47.4 |
MMMUmultimodal47.6**MMMU** *(CoT)*medium| **MMMU** *(CoT)* | <ins>**52.5**</ins> | 47.6 | 45.1 | 40.3 | 38.3 |
IFEvalinstruction-following38.9**MM IF-Eval**medium| **MM IF-Eval** | <ins>**52.7**</ins> | 38.9 | 42.5 | 41.2 | 31.4 |
ChartQAmultimodal38.6**ChartQA** *(CoT)*medium| **ChartQA** *(CoT)* | <ins>**81.8**</ins> | 38.6 | 67.1 | 72.0 | 67.7 |
MATHmath27.8**Math** *(Pass@1)*medium| **Math** *(Pass@1)* | <ins>**48.1**</ins> | 27.8 | 38.6 | 29.2 | 28.4 |
MT-Benchchat6.41**Text MT-Bench**medium| **Text MT-Bench** | <ins>**7.68**</ins> | 6.41 | 6.94 | 6.27 | 6.31 |
MT-Benchchat5.43**MM MT-Bench**medium| **MM MT-Bench** | <ins>**6.05**</ins> | 5.43 | 4.12 | 3.70 | 4.46 |

Discussions (0)

none scraped.

Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.