OLMo-1B-0724-hf

Home/Models/allenai/OLMo-1B-0724-hf

allenai/OLMo-1B-0724-hf

kind: original
quant: bf16 (16-bit)
family: olmo
modality: text-generation
root model: allenai/OLMo-1B-0724-hf
provider: allenai
license: apache-2.0
language: en
context: —
library: —
vRAM (GB): —
size on disk: —
downloads: 6,743
likes: 24
created: —
last modified: 2024-08-05T20:14:39+00:00 (2024)
huggingface: https://huggingface.co/allenai/OLMo-1B-0724-hf
tags: transformerssafetensorsolmotext-generationendataset:allenai/dolmaarxiv:2402.00838license:apache-2.0endpoints_compatibleregion:us

benchmark	category	score	context	conf	evidence
PIQA	knowledge	77.2		medium	`\| piqa \| 77.2 \| 76.7 \| 78.5 \| 77.3 \| 78.4 \| 79.0 \| 77.5 \|`
HellaSwag	knowledge	76.2		medium	`\| hellaswag \| 76.2 \| 76.8 \| 75.9 \| 77.6 \| 76.4 \| 78.6 \| 75.5 \|`
BoolQ	knowledge	75.4		medium	`\| boolq \| 75.4 \| 80.2 \| 74.6 \| 74.2 \| 73.4 \| 82.2 \| 83.7 \|`
WinoGrande	knowledge	70.5		medium	`\| winogrande \| 70.5 \| 69.4 \| 68.9 \| 69.9 \| 67.9 \| 73.5 \| 69.8 \|`
OpenBookQA	knowledge	51.2		medium	`\| openbookqa \| 51.2 \| 48.4 \| 53.0 \| 48.6 \| 50.4 \| 51.8 \| 50.0 \|`
TruthfulQA	safety	33.9	truthfulQA (MC2)	medium	`\| truthfulQA (MC2) \| 33.9 \| 38.5 \| 34.0 \| 33.0 \| 36.0 \| 36.8 \| 35.8 \|`
MMLU	knowledge	31.5	MMLU (5 shot MC)	medium	`\| MMLU (5 shot MC) \| 31.5 \| 45.0 \| 24.0 \| 30.8 \| 28.3 \| 55.5 \| 52.0 \|`
ARC-Challenge	knowledge	25.0	_challenge	medium	`\| arc_challenge \| 25 \| 43.8 \| 33.1 \| 34.8 \| 34.5 \| 36.5 \|`
GSM8K	math	10.0	GSM8k	medium	`\| GSM8k \| 10.0 \| 12.0 \| 4.0 \| 4.5 \| 8.5 \| 25.0 \| 29.0 \|`

none scraped.