OLMoE-1B-7B-0125-SFT

Home/Models/allenai/OLMoE-1B-7B-0125-SFT

allenai/OLMoE-1B-7B-0125-SFT

kind: original
quant: bf16 (16-bit)
family: misc
modality: text-generation
root model: allenai/OLMoE-1B-7B-0125
provider: allenai
license: apache-2.0
language: en
context: —
library: transformers
vRAM (GB): —
size on disk: —
downloads: 166
likes: 4
created: —
last modified: 2025-02-04T16:45:57+00:00 (2025)
huggingface: https://huggingface.co/allenai/OLMoE-1B-7B-0125-SFT
tags: transformerspytorcholmoetext-generationconversationalendataset:allenai/tulu-3-sft-olmo-2-mixturearxiv:2409.02060arxiv:2411.15124base_model:allenai/OLMoE-1B-7B-0125base_model:finetune:allenai/OLMoE-1B-7B-0125license:apache-2.0endpoints_compatibleregion:us

benchmark	category	score	context	conf	evidence
GSM8K	math	72.4	GSM8K	medium	`\| GSM8K \| 72.40 \| 47.38 \| 64.59 \| 55.72 \| 42.46 \|`
IFEval	instruction-following	66.36	IFEval	medium	`\| IFEval \| 66.36 \| 45.29 \| 65.62 \| 56.56 \| 41.22 \|`
HumanEval	coding	62.3	HumanEval	medium	`\| HumanEval \| 62.30 \| 63.04 \| 61.92 \| 62.58 \| 63.72 \|`
MMLU	knowledge	55.08	MMLU (CoT)	medium	`\| MMLU (CoT) \| 55.08 \| 54.57 \| 54.93 \| 55.26 \| 54.32 \|`
HumanEval	coding	54.37	HumanEval+	medium	`\| HumanEval+ \| 54.37 \| 58.93 \| 57.61 \| 55.67 \| 57.40 \|`
TruthfulQA	safety	50.56	TruthfulQA	medium	`\| TruthfulQA \| 50.56 \| 49.14 \| 49.99 \| 45.48 \| 44.66 \|`
DROP	reasoning	47.87	DROP	medium	`\| DROP \| 47.87 \| 34.48 \| 48.38 \| 48.57 \| 34.71 \|`
BBH	reasoning	38.61	BigBenchHard (CoT)	medium	`\| BigBenchHard (CoT) \| 38.61 \| 36.78 \| 37.37 \| 37.31 \| 36.55 \|`
MATH	math	21.41	MATH (Flex)	medium	`\| MATH (Flex) \| 21.41 \| 8.16 \| 20.36 \| 21.38 \| 8.15 \|`
PopQA	knowledge	19.75	PopQA	medium	`\| PopQA \| 19.75 \| 20.56 \| 19.65 \| 20.12 \| 21.01 \|`
AlpacaEval	chat	17.99	AlpacaEval	medium	`\| AlpacaEval \| 17.99 \| 7.54 \| 19.50 \| 5.83 \| 6.38 \|`

none scraped.