Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF

Home/Models/TheBloke/Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF

Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF

TheBloke/Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF

kind: gguf
quant: gguf (2-bit)
family: mistral
modality: —
root model: NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
provider: TheBloke
license: apache-2.0
language: en
context: 32768,
library: —
vRAM (GB): —
size on disk: —
downloads: 2,000
likes: 66
created: —
last modified: 2024-01-16T09:01:46+00:00 (2024)
huggingface: https://huggingface.co/TheBloke/Nous-Hermes-2-Mixtral-8x7B-DPO-GGUF
tags: transformersggufmixtralMixtralinstructfinetunechatmlDPORLHFgpt4synthetic datadistillationenbase_model:NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPObase_model:quantized:NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPOlicense:apache-2.0region:usconversational

Benchmark scores (32)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	87.83		medium	`\| boolq \| 1 \| acc \| 0.8783 \| ± \| 0.0057 \|`
PIQA	knowledge	83.24		medium	`\| piqa \| 0 \| acc \| 0.8324 \| ± \| 0.0087 \|`
AGIEval	reasoning	80.1	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 0.8010 \| ± \| 0.0279 \|`
WinoGrande	knowledge	76.16		medium	`\| winogrande \| 0 \| acc \| 0.7616 \| ± \| 0.0120 \|`
BigBench	reasoning	74.0	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 0.7400 \| ± \| 0.0139 \|`
BigBench	reasoning	72.93	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 0.7293 \| ± \| 0.0331 \|`
BigBench	reasoning	71.82	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 0.7182 \| ± \| 0.0235 \|`
BigBench	reasoning	69.0	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 0.6900 \| ± \| 0.0103 \|`
BigBench	reasoning	67.44	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 0.6744 \| ± \| 0.0149 \|`
HellaSwag	knowledge	66.61		medium	`\| hellaswag \| 0 \| acc \| 0.6661 \| ± \| 0.0047 \|`
AGIEval	reasoning	66.17	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 0.6617 \| ± \| 0.0289 \|`
BigBench	reasoning	63.17	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 0.6317 \| ± \| 0.0228 \|`
BigBench	reasoning	61.05	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 0.6105 \| ± \| 0.0355 \|`
ARC-Challenge	knowledge	59.9	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 0.5990 \| ± \| 0.0143 \|`
BigBench	reasoning	57.36	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 0.5736 \| ± \| 0.0308 \|`
AGIEval	reasoning	55.49	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 0.5549 \| ± \| 0.0220 \|`
BigBench	reasoning	52.0	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 0.5200 \| ± \| 0.0289 \|`
BigBench	reasoning	52.0	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 0.5200 \| ± \| 0.0289 \|`
BigBench	reasoning	50.0	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 0.5000 \| ± \| 0.0158 \|`
AGIEval	reasoning	49.09	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 0.4909 \| ± \| 0.0338 \|`
AGIEval	reasoning	48.06	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 0.4806 \| ± \| 0.0349 \|`
BigBench	reasoning	45.96	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 0.4596 \| ± \| 0.0263 \|`
AGIEval	reasoning	41.17	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 0.4117 \| ± \| 0.0193 \|`
BigBench	reasoning	35.4	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 0.3540 \| ± \| 0.0214 \|`
BigBench	reasoning	35.0	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 0.3500 \| ± \| 0.0214 \|`
OpenBookQA	knowledge	34.4		medium	`\| openbookqa \| 0 \| acc \| 0.3440 \| ± \| 0.0213 \|`
BigBench	reasoning	25.35	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 0.2535 \| ± \| 0.0138 \|`
BigBench	reasoning	25.0	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 0.2500 \| ± \| 0.0164 \|`
AGIEval	reasoning	24.02	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 0.2402 \| ± \| 0.0269 \|`
AGIEval	reasoning	23.48	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 0.2348 \| ± \| 0.0280 \|`
BigBench	reasoning	21.76	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 0.2176 \| ± \| 0.0117 \|`
BigBench	reasoning	15.43	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 0.1543 \| ± \| 0.0086 \|`

Discussions (0)

none scraped.