OpenHermes-2-Mistral-7B-GGUF

Home/Models/TheBloke/OpenHermes-2-Mistral-7B-GGUF

OpenHermes-2-Mistral-7B-GGUF

TheBloke/OpenHermes-2-Mistral-7B-GGUF

kind: gguf
quant: gguf (2-bit)
family: mistral
modality: —
root model: teknium/OpenHermes-2-Mistral-7B
provider: TheBloke
license: apache-2.0
language: en
context: —
library: —
vRAM (GB): —
size on disk: —
downloads: 1,861
likes: 77
created: —
last modified: 2023-10-16T20:26:01+00:00 (2023)
huggingface: https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GGUF
tags: transformersggufmistralinstructfinetunechatmlgpt4synthetic datadistillationenbase_model:teknium/OpenHermes-2-Mistral-7Bbase_model:quantized:teknium/OpenHermes-2-Mistral-7Blicense:apache-2.0region:us

Benchmark scores (34)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	86.88		medium	`\| boolq \| 1 \| acc \| 0.8688 \| ± \| 0.0059 \|`
PIQA	knowledge	80.9		medium	`\| piqa \| 0 \| acc \| 0.8090 \| ± \| 0.0092 \|`
WinoGrande	knowledge	74.66		medium	`\| winogrande \| 0 \| acc \| 0.7466 \| ± \| 0.0122 \|`
BigBench	reasoning	74.03	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 0.7403 \| ± \| 0.0327 \|`
AGIEval	reasoning	72.82	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 0.7282 \| ± \| 0.0311 \|`
BigBench	reasoning	68.15	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 0.6815 \| ± \| 0.0104 \|`
BigBench	reasoning	66.94	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 0.6694 \| ± \| 0.0245 \|`
BigBench	reasoning	66.63	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 0.6663 \| ± \| 0.0150 \|`
HellaSwag	knowledge	62.05		medium	`\| hellaswag \| 0 \| acc \| 0.6205 \| ± \| 0.0048 \|`
BigBench	reasoning	57.89	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 0.5789 \| ± \| 0.0359 \|`
AGIEval	reasoning	57.25	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 0.5725 \| ± \| 0.0302 \|`
ARC-Challenge	knowledge	54.52	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 0.5452 \| ± \| 0.0146 \|`
BigBench	reasoning	50.1	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 0.5010 \| ± \| 0.0158 \|`
AGIEval	reasoning	45.15	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 0.4515 \| ± \| 0.0348 \|`
AGIEval	reasoning	44.51	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 0.4451 \| ± \| 0.0220 \|`
BigBench	reasoning	43.0	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 0.4300 \| ± \| 0.0286 \|`
BigBench	reasoning	43.0	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 0.4300 \| ± \| 0.0286 \|`
BigBench	reasoning	42.19	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 0.4219 \| ± \| 0.0234 \|`
BigBench	reasoning	38.76	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 0.3876 \| ± \| 0.0304 \|`
BigBench	reasoning	38.3	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 0.3830 \| ± \| 0.0154 \|`
BigBench	reasoning	37.6	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 0.3760 \| ± \| 0.0256 \|`
AGIEval	reasoning	37.2	AGI Eval	medium	`\| AGI Eval \| 37.20 \| 35.56 \| 39.77 \| +2.57 \| +4.21 \|`
AGIEval	reasoning	35.0	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 0.3500 \| ± \| 0.0322 \|`
OpenBookQA	knowledge	34.8		medium	`\| openbookqa \| 0 \| acc \| 0.3480 \| ± \| 0.0213 \|`
AGIEval	reasoning	34.72	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 0.3472 \| ± \| 0.0187 \|`
TruthfulQA	safety	33.9	_mc	medium	`\| truthfulqa_mc \| 1 \| mc1 \| 0.3390 \| ± \| 0.0166 \|`
BigBench	reasoning	31.4	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 0.3140 \| ± \| 0.0208 \|`
BigBench	reasoning	28.8	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 0.2880 \| ± \| 0.0203 \|`
AGIEval	reasoning	24.35	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 0.2435 \| ± \| 0.0284 \|`
AGIEval	reasoning	23.23	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 0.2323 \| ± \| 0.0265 \|`
BigBench	reasoning	21.68	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 0.2168 \| ± \| 0.0117 \|`
BigBench	reasoning	20.57	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 0.2057 \| ± \| 0.0153 \|`
BigBench	reasoning	16.93	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 0.1693 \| ± \| 0.0119 \|`
BigBench	reasoning	15.49	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 0.1549 \| ± \| 0.0087 \|`

Discussions (0)

none scraped.