gemma-2-27b

Home/Models/google/gemma-2-27b

google/gemma-2-27b

kind: original
quant: bf16 (16-bit)
family: gemma
modality: text-generation
root model: google/gemma-2-27b
provider: google
license: gemma
language: —
context: ,
library: transformers
vRAM (GB): —
size on disk: —
downloads: 5,933
likes: 209
created: —
last modified: 2024-08-07T18:26:09+00:00 (2024)
huggingface: https://huggingface.co/google/gemma-2-27b
tags: transformerssafetensorsgemma2text-generationarxiv:2009.03300arxiv:1905.07830arxiv:1911.11641arxiv:1904.09728arxiv:1905.10044arxiv:1907.10641arxiv:1811.00937arxiv:1809.02789arxiv:1911.01547arxiv:1705.03551arxiv:2107.03374arxiv:2108.07732arxiv:2110.14168arxiv:2009.11462arxiv:2101.11718arxiv:2110.08193arxiv:1804.09301arxiv:2109.07958arxiv:1804.06876arxiv:2103.03874

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	84.2	[BoolQ][]	medium	`\| [BoolQ][boolq] \| 0-shot \| 84.2 \| 84.8 \|`
HellaSwag	knowledge	81.9	[HellaSwag][]	medium	`\| [HellaSwag][hellaswag] \| 10-shot \| 81.9 \| 86.4 \|`
PIQA	knowledge	81.7	[PIQA][]	medium	`\| [PIQA][piqa] \| 0-shot \| 81.7 \| 83.2 \|`
WinoGrande	knowledge	80.6	[WinoGrande][]	medium	`\| [WinoGrande][winogrande] \| partial score \| 80.6 \| 83.7 \|`
TriviaQA	knowledge	76.6	[TriviaQA][]	medium	`\| [TriviaQA][triviaqa] \| 5-shot \| 76.6 \| 83.7 \|`
MMLU	knowledge	71.3	[MMLU][]	medium	`\| [MMLU][mmlu] \| 5-shot, top-1 \| 71.3 \| 75.2 \|`
GSM8K	math	68.6	[GSM8K][]	medium	`\| [GSM8K][gsm8k] \| 5-shot, maj@1 \| 68.6 \| 74.0 \|`
BigBench	reasoning	68.2	[BIG-Bench][big-bench]	medium	`\| [BIG-Bench][big-bench] \| 3-shot, CoT \| 68.2 \| 74.9 \|`
AGIEval	reasoning	52.8	[AGIEval][]	medium	`\| [AGIEval][agieval] \| 3-5-shot \| 52.8 \| 55.1 \|`
MBPP	coding	52.4	[MBPP][]	medium	`\| [MBPP][mbpp] \| 3-shot \| 52.4 \| 62.6 \|`
TruthfulQA	safety	50.27	[TruthfulQA][]	medium	`\| [TruthfulQA][truthfulqa] \| \| 50.27 \| 51.60 \|`
HumanEval	coding	40.2	[HumanEval][]	medium	`\| [HumanEval][humaneval] \| pass@1 \| 40.2 \| 51.8 \|`
MATH	math	36.6	[MATH][]	medium	`\| [MATH][math] \| 4-shot \| 36.6 \| 42.3 \|`

none scraped.