HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Orgs/google/google/gemma-2-2b

google/gemma-2-2b

family gemma · 4 variants · unknown

provider
google
released
—
huggingface
https://huggingface.co/google/gemma-2-2b
best scores
MMLU: 51.3HellaSwag: 73.0PIQA: 77.8BoolQ: 72.5WinoGrande: 70.9TriviaQA: 59.4HumanEval: 17.7MBPP: 29.6GSM8K: 23.9MATH: 15.0AGIEval: 30.6DROP: 52.0BigBench: 41.9TruthfulQA: 43.72

Best scores per benchmark

benchmarkcategorybest score
PIQAknowledge77.8
HellaSwagknowledge73.0
BoolQknowledge72.5
WinoGrandeknowledge70.9
TriviaQAknowledge59.4
DROPreasoning52.0
MMLUknowledge51.3
TruthfulQAsafety43.72
BigBenchreasoning41.9
AGIEvalreasoning30.6
MBPPcoding29.6
GSM8Kmath23.9
HumanEvalcoding17.7
MATHmath15.0

Variants (4)

variantkindquantbitsvRAMdlorgyear
gemma-2-2b-itoriginalbf1616.0604,952google2024
gemma-2-2boriginalbf1616.0222,596google2024
gemma-2-2b-it-bnb-4bitbnb-4bit4bit32,547unsloth2024
gemma-2-2b-bnb-4bitbnb-4bit4bit2,698unsloth2024
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.