HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Orgs/google/google/gemma-1.1-2b-it

google/gemma-1.1-2b-it

family gemma · 6 variants · unknown

provider
google
released
—
huggingface
https://huggingface.co/google/gemma-1.1-2b-it
best scores
MMLU: 42.3HellaSwag: 71.4PIQA: 77.3WinoGrande: 65.4CommonsenseQA: 65.3OpenBookQA: 47.8TriviaQA: 53.2HumanEval: 22.0MBPP: 29.2GSM8K: 17.7MATH: 11.8AGIEval: 24.2BigBench: 35.2TruthfulQA: 44.84

Best scores per benchmark

benchmarkcategorybest score
PIQAknowledge77.3
HellaSwagknowledge71.4
WinoGrandeknowledge65.4
CommonsenseQAknowledge65.3
TriviaQAknowledge53.2
OpenBookQAknowledge47.8
TruthfulQAsafety44.84
MMLUknowledge42.3
BigBenchreasoning35.2
MBPPcoding29.2
AGIEvalreasoning24.2
HumanEvalcoding22.0
GSM8Kmath17.7
MATHmath11.8

Variants (6)

variantkindquantbitsvRAMdlorgyear
gemma-1.1-2b-itoriginalbf1616.0207,713google2024
gemma-1.1-2b-it-GGUFggufgguf3.0893lmstudio-community2024
gemma-1.1-2b-it-GGUFggufgguf1.0668bartowski2024
gemma-1.1-2b-it-Q8_0-GGUFggufgguf8.0110ggml-org2024
gemma-1.1-2b-it-bnb-4bitbnb-4bit4bit82unsloth2024
gemma-1.1-2b-itbnb-4bit4bit54unsloth2024
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.