NeuralHermes-2.5-Mistral-7B-laser

Home/Models/mlabonne/NeuralHermes-2.5-Mistral-7B-laser

NeuralHermes-2.5-Mistral-7B-laser

mlabonne/NeuralHermes-2.5-Mistral-7B-laser

kind: original
quant: bf16 (16-bit)
family: mistral
modality: text-generation
root model: teknium/OpenHermes-2.5-Mistral-7B
provider: mlabonne
license: apache-2.0
language: en
context: —
library: —
vRAM (GB): —
size on disk: —
downloads: 223
likes: 16
created: —
last modified: 2024-03-04T15:17:29+00:00 (2024)
huggingface: https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B-laser
tags: transformerssafetensorsmistraltext-generationinstructfinetunechatmlgpt4synthetic datadistillationdporlhflaserconversationalendataset:mlabonne/chatml_dpo_pairsarxiv:2312.13558base_model:teknium/OpenHermes-2.5-Mistral-7Bbase_model:finetune:teknium/OpenHermes-2.5-Mistral-7Blicense:apache-2.0model-indextext-generation-inferenceendpoints_compatibleregion:us

Benchmark scores (41)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	86.7		medium	`\| boolq \| 1 \| acc \| 86.70 \| ± \| 0.59 \|`
HellaSwag	knowledge	85.09	HellaSwag (10-Shot)	medium	`\| HellaSwag (10-Shot) \| 85.09 \|`
PIQA	knowledge	81.94		medium	`\| piqa \| 0 \| acc \| 81.94 \| ± \| 0.90 \|`
WinoGrande	knowledge	78.14	Winogrande (5-shot)	medium	`\| Winogrande (5-shot) \| 78.14 \|`
WinoGrande	knowledge	75.22		medium	`\| winogrande \| 0 \| acc \| 75.22 \| ± \| 1.21 \|`
AGIEval	reasoning	74.27	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 74.27 \| ± \| 3.05 \|`
BigBench	reasoning	70.72	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 70.72 \| ± \| 3.39 \|`
BigBench	reasoning	65.72	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 65.72 \| ± \| 1.51 \|`
BigBench	reasoning	65.31	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 65.31 \| ± \| 2.48 \|`
BigBench	reasoning	65.0	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 65.00 \| ± \| 1.07 \|`
MMLU	knowledge	63.43	MMLU (5-Shot)	medium	`\| MMLU (5-Shot) \| 63.43 \|`
HellaSwag	knowledge	63.15		medium	`\| hellaswag \| 0 \| acc \| 63.15 \| ± \| 0.48 \|`
AGIEval	reasoning	58.36	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 58.36 \| ± \| 3.01 \|`
ARC-Challenge	knowledge	57.76	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 57.76 \| ± \| 1.44 \|`
GSM8K	math	55.72	GSM8k (5-shot)	medium	`\| GSM8k (5-shot) \| 55.72 \|`
TruthfulQA	safety	55.26		low	`\| [NeuralHermes-2.5-Mistral-7B-laser](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B-laser) \| 43.54 \| 73.44 \| 55.26 \| 42.24 \| 53.62 \|`
TruthfulQA	safety	54.95	TruthfulQA (0-shot)	medium	`\| TruthfulQA (0-shot) \| 54.95 \|`
BigBench	reasoning	53.16	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 53.16 \| ± \| 3.63 \|`
BigBench	reasoning	50.0	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 50.00 \| ± \| 1.58 \|`
AGIEval	reasoning	48.82	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 48.82 \| ± \| 2.22 \|`
BigBench	reasoning	48.33	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 48.33 \| ± \| 2.89 \|`
BigBench	reasoning	48.33	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 48.33 \| ± \| 2.89 \|`
BigBench	reasoning	46.21	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 46.21 \| ± \| 2.36 \|`
AGIEval	reasoning	43.69	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 43.69 \| ± \| 3.46 \|`
AGIEval	reasoning	43.54		low	`\| [NeuralHermes-2.5-Mistral-7B-laser](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B-laser) \| 43.54 \| 73.44 \| 55.26 \| 42.24 \| 53.62 \|`
BigBench	reasoning	42.24		low	`\| [NeuralHermes-2.5-Mistral-7B-laser](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B-laser) \| 43.54 \| 73.44 \| 55.26 \| 42.24 \| 53.62 \|`
BigBench	reasoning	41.4	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 41.40 \| ± \| 2.20 \|`
AGIEval	reasoning	39.32	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 39.32 \| ± \| 1.92 \|`
TruthfulQA	safety	37.7	_mc	medium	`\| truthfulqa_mc \| 1 \| mc1 \| 37.70 \| ± \| 1.70 \|`
AGIEval	reasoning	37.27	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 37.27 \| ± \| 3.27 \|`
OpenBookQA	knowledge	34.4		medium	`\| openbookqa \| 0 \| acc \| 34.40 \| ± \| 2.13 \|`
BigBench	reasoning	34.11	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 34.11 \| ± \| 2.96 \|`
BigBench	reasoning	30.4	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 30.40 \| ± \| 1.46 \|`
BigBench	reasoning	27.8	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 27.80 \| ± \| 2.01 \|`
BigBench	reasoning	27.25	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 27.25 \| ± \| 1.41 \|`
BigBench	reasoning	27.02	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 27.02 \| ± \| 2.35 \|`
AGIEval	reasoning	25.65	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 25.65 \| ± \| 2.89 \|`
BigBench	reasoning	22.56	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 22.56 \| ± \| 1.18 \|`
AGIEval	reasoning	21.26	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 21.26 \| ± \| 2.57 \|`
BigBench	reasoning	19.86	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 19.86 \| ± \| 1.51 \|`
BigBench	reasoning	17.09	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 17.09 \| ± \| 0.90 \|`

Discussions (0)

none scraped.