HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Orgs/google/google/gemma-2b-it-pytorch

google/gemma-2b-it-pytorch

family gemma · 11 variants · unknown

provider
google
released
—
huggingface
https://huggingface.co/google/gemma-2b-it-pytorch
best scores
MMLU: 42.3HellaSwag: 71.4PIQA: 77.3WinoGrande: 65.4CommonsenseQA: 65.3OpenBookQA: 47.8TriviaQA: 53.2HumanEval: 22.0MBPP: 29.2GSM8K: 17.7MATH: 11.8AGIEval: 24.2BigBench: 35.2TruthfulQA: 44.84

Best scores per benchmark

benchmarkcategorybest score
PIQAknowledge77.3
HellaSwagknowledge71.4
WinoGrandeknowledge65.4
CommonsenseQAknowledge65.3
TriviaQAknowledge53.2
OpenBookQAknowledge47.8
TruthfulQAsafety44.84
MMLUknowledge42.3
BigBenchreasoning35.2
MBPPcoding29.2
AGIEvalreasoning24.2
HumanEvalcoding22.0
GSM8Kmath17.7
MATHmath11.8

Variants (11)

variantkindquantbitsvRAMdlorgyear
gemma-2bggufgguf97,106google2024
gemma-2b-itggufgguf57,240google2024
gemma-2b-itoriginalbf1616.016,802unsloth2024
gemma-2b-AWQawqawq3,916google2024
gemma-2boriginalbf1616.03,347unsloth2024
gemma-2b-bnb-4bitbnb-4bit4bit3,038unsloth2024
gemma-2b-it-bnb-4bitbnb-4bit4bit2,835unsloth2024
gemma-2b-GGUFggufgguf2.0830mlabonne2024
gemma-2boriginalbf1616.0277alpindale2024
gemma-2b-it-GGUFggufgguf2.0274mlabonne2024
gemma-2b-it-pytorchoriginalbf1616.065google2024
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.