Gemmalpaca-2B

Home/Models/mlabonne/Gemmalpaca-2B

mlabonne/Gemmalpaca-2B

kind: original
quant: bf16 (16-bit)
family: gemma
modality: text-generation
root model: google/gemma-2b
provider: mlabonne
license: other
language: —
context: 8k
library: transformers
vRAM (GB): —
size on disk: —
downloads: 84
likes: 14
created: —
last modified: 2024-03-04T15:18:57+00:00 (2024)
huggingface: https://huggingface.co/mlabonne/Gemmalpaca-2B
tags: transformerssafetensorsgemmatext-generationdataset:vicgalle/alpaca-gpt4base_model:google/gemma-2bbase_model:finetune:google/gemma-2blicense:othermodel-indextext-generation-inferenceendpoints_compatibleregion:us

benchmark	category	score	context	conf	evidence
HellaSwag	knowledge	71.36	HellaSwag (10-Shot)	medium	`\| HellaSwag (10-Shot) \| 71.36 \|`
WinoGrande	knowledge	65.59	Winogrande (5-shot)	medium	`\| Winogrande (5-shot) \| 65.59 \|`
TruthfulQA	safety	47.02		low	`\| [mlabonne/Gemmalpaca-2B](https://huggingface.co/mlabonne/Gemmalpaca-2B) [📄](https://gist.github.com/mlabonne/4b638752fc3227df566f9562064cb864) \| 38.39 \| 24.48 \| 51.22 \| 47.02 \| 3`
TruthfulQA	safety	41.24	TruthfulQA (0-shot)	medium	`\| TruthfulQA (0-shot) \| 41.24 \|`
MMLU	knowledge	36.3	MMLU (5-Shot)	medium	`\| MMLU (5-Shot) \| 36.30 \|`
BigBench	reasoning	30.85		low	`\| [mlabonne/Gemmalpaca-2B](https://huggingface.co/mlabonne/Gemmalpaca-2B) [📄](https://gist.github.com/mlabonne/4b638752fc3227df566f9562064cb864) \| 38.39 \| 24.48 \| 51.22 \| 47.02 \| 3`
AGIEval	reasoning	24.48		low	`\| [mlabonne/Gemmalpaca-2B](https://huggingface.co/mlabonne/Gemmalpaca-2B) [📄](https://gist.github.com/mlabonne/4b638752fc3227df566f9562064cb864) \| 38.39 \| 24.48 \| 51.22 \| 47.02 \| 3`
GSM8K	math	10.69	GSM8k (5-shot)	medium	`\| GSM8k (5-shot) \| 10.69 \|`

none scraped.