Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF

Home/Models/TheBloke/Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF

Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF

TheBloke/Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF

kind: gguf
quant: gguf (2-bit)
family: mistral
modality: —
root model: NousResearch/Nous-Hermes-2-Mixtral-8x7B-SFT
provider: TheBloke
license: apache-2.0
language: en
context: 32768,
library: —
vRAM (GB): —
size on disk: —
downloads: 517
likes: 25
created: —
last modified: 2024-01-16T20:39:50+00:00 (2024)
huggingface: https://huggingface.co/TheBloke/Nous-Hermes-2-Mixtral-8x7B-SFT-GGUF
tags: transformersggufmixtralMixtralinstructfinetunechatmlDPORLHFgpt4synthetic datadistillationenbase_model:NousResearch/Nous-Hermes-2-Mixtral-8x7B-SFTbase_model:quantized:NousResearch/Nous-Hermes-2-Mixtral-8x7B-SFTlicense:apache-2.0region:usconversational

Benchmark scores (32)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	87.83		medium	`\| boolq \| 1 \| acc \| 0.8783 \| ± \| 0.0057 \|`
PIQA	knowledge	83.24		medium	`\| piqa \| 0 \| acc \| 0.8324 \| ± \| 0.0087 \|`
AGIEval	reasoning	79.61	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 0.7961 \| ± \| 0.0281 \|`
WinoGrande	knowledge	75.69		medium	`\| winogrande \| 0 \| acc \| 0.7569 \| ± \| 0.0121 \|`
BigBench	reasoning	71.54	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 0.7154 \| ± \| 0.0235 \|`
BigBench	reasoning	71.27	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 0.7127 \| ± \| 0.0337 \|`
BigBench	reasoning	68.6	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 0.6860 \| ± \| 0.0147 \|`
BigBench	reasoning	66.65	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 0.6665 \| ± \| 0.0105 \|`
AGIEval	reasoning	66.17	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 0.6617 \| ± \| 0.0289 \|`
HellaSwag	knowledge	65.92		medium	`\| hellaswag \| 0 \| acc \| 0.6592 \| ± \| 0.0047 \|`
BigBench	reasoning	65.92	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 0.6592 \| ± \| 0.0151 \|`
BigBench	reasoning	63.17	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 0.6317 \| ± \| 0.0228 \|`
ARC-Challenge	knowledge	59.04	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 0.5904 \| ± \| 0.0144 \|`
BigBench	reasoning	57.89	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 0.5789 \| ± \| 0.0359 \|`
BigBench	reasoning	53.88	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 0.5388 \| ± \| 0.0311 \|`
AGIEval	reasoning	53.53	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 0.5353 \| ± \| 0.0221 \|`
BigBench	reasoning	52.33	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 0.5233 \| ± \| 0.0289 \|`
BigBench	reasoning	52.33	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 0.5233 \| ± \| 0.0289 \|`
BigBench	reasoning	50.0	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 0.5000 \| ± \| 0.0158 \|`
AGIEval	reasoning	48.18	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 0.4818 \| ± \| 0.0338 \|`
AGIEval	reasoning	47.57	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 0.4757 \| ± \| 0.0349 \|`
BigBench	reasoning	46.8	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 0.4680 \| ± \| 0.0264 \|`
AGIEval	reasoning	40.25	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 0.4025 \| ± \| 0.0192 \|`
BigBench	reasoning	37.0	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 0.3700 \| ± \| 0.0216 \|`
OpenBookQA	knowledge	34.0		medium	`\| openbookqa \| 0 \| acc \| 0.3400 \| ± \| 0.0212 \|`
BigBench	reasoning	32.6	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 0.3260 \| ± \| 0.0210 \|`
BigBench	reasoning	25.05	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 0.2505 \| ± \| 0.0137 \|`
BigBench	reasoning	24.43	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 0.2443 \| ± \| 0.0163 \|`
AGIEval	reasoning	24.41	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 0.2441 \| ± \| 0.0270 \|`
AGIEval	reasoning	23.91	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 0.2391 \| ± \| 0.0282 \|`
BigBench	reasoning	22.0	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 0.2200 \| ± \| 0.0117 \|`
BigBench	reasoning	15.03	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 0.1503 \| ± \| 0.0085 \|`

Discussions (0)

none scraped.