HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Orgs/google/google/gemma-1.1-7b-it

google/gemma-1.1-7b-it

family gemma · 3 variants · unknown

provider
google
released
—
huggingface
https://huggingface.co/google/gemma-1.1-7b-it
best scores
MMLU: 42.3HellaSwag: 71.4PIQA: 77.3WinoGrande: 65.4CommonsenseQA: 65.3OpenBookQA: 47.8TriviaQA: 53.2HumanEval: 22.0MBPP: 29.2GSM8K: 17.7MATH: 11.8AGIEval: 24.2BigBench: 35.2TruthfulQA: 44.84

Best scores per benchmark

benchmarkcategorybest score
PIQAknowledge77.3
HellaSwagknowledge71.4
WinoGrandeknowledge65.4
CommonsenseQAknowledge65.3
TriviaQAknowledge53.2
OpenBookQAknowledge47.8
TruthfulQAsafety44.84
MMLUknowledge42.3
BigBenchreasoning35.2
MBPPcoding29.2
AGIEvalreasoning24.2
HumanEvalcoding22.0
GSM8Kmath17.7
MATHmath11.8

Variants (3)

variantkindquantbitsvRAMdlorgyear
gemma-1.1-7b-itoriginalbf1616.025,287google2024
gemma-1.1-7b-it-GGUFggufgguf3.0880bartowski2024
gemma-1.1-7b-it-Q4_K_M-GGUFggufgguf4.0237ggml-org2024
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.