OpenHermes-2.5-Mistral-7B-AWQ

Home/Models/TheBloke/OpenHermes-2.5-Mistral-7B-AWQ

OpenHermes-2.5-Mistral-7B-AWQ

TheBloke/OpenHermes-2.5-Mistral-7B-AWQ

kind: awq
quant: awq
family: mistral
modality: text-generation
root model: teknium/OpenHermes-2.5-Mistral-7B
provider: TheBloke
license: apache-2.0
language: en
context: —
library: —
vRAM (GB): —
size on disk: —
downloads: 24,599
likes: 21
created: —
last modified: 2023-11-09T18:16:14+00:00 (2023)
huggingface: https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-AWQ
tags: transformerssafetensorsmistraltext-generationinstructfinetunechatmlgpt4synthetic datadistillationenbase_model:teknium/OpenHermes-2.5-Mistral-7Bbase_model:quantized:teknium/OpenHermes-2.5-Mistral-7Blicense:apache-2.0text-generation-inference4-bitawqdeploy:azureregion:us

Benchmark scores (34)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	86.57		medium	`\| boolq \| 1 \| acc \| 0.8657 \| ± \| 0.0060 \|`
PIQA	knowledge	81.45		medium	`\| piqa \| 0 \| acc \| 0.8145 \| ± \| 0.0091 \|`
AGIEval	reasoning	74.76	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 0.7476 \| ± \| 0.0303 \|`
WinoGrande	knowledge	74.35		medium	`\| winogrande \| 0 \| acc \| 0.7435 \| ± \| 0.0123 \|`
BigBench	reasoning	67.96	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 0.6796 \| ± \| 0.0348 \|`
BigBench	reasoning	66.67	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 0.6667 \| ± \| 0.0246 \|`
BigBench	reasoning	66.3	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 0.6630 \| ± \| 0.0106 \|`
BigBench	reasoning	64.91	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 0.6491 \| ± \| 0.0152 \|`
HellaSwag	knowledge	63.1		medium	`\| hellaswag \| 0 \| acc \| 0.6310 \| ± \| 0.0048 \|`
AGIEval	reasoning	59.11	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 0.5911 \| ± \| 0.0300 \|`
ARC-Challenge	knowledge	56.23	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 0.5623 \| ± \| 0.0145 \|`
BigBench	reasoning	53.16	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 0.5316 \| ± \| 0.0363 \|`
AGIEval	reasoning	50.59	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 0.5059 \| ± \| 0.0222 \|`
BigBench	reasoning	50.0	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 0.5000 \| ± \| 0.0158 \|`
BigBench	reasoning	48.0	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 0.4800 \| ± \| 0.0289 \|`
BigBench	reasoning	48.0	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 0.4800 \| ± \| 0.0289 \|`
AGIEval	reasoning	44.17	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 0.4417 \| ± \| 0.0347 \|`
BigBench	reasoning	42.41	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 0.4241 \| ± \| 0.0234 \|`
AGIEval	reasoning	38.71	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 0.3871 \| ± \| 0.0191 \|`
AGIEval	reasoning	37.73	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 0.3773 \| ± \| 0.0328 \|`
BigBench	reasoning	36.2	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 0.3620 \| ± \| 0.0215 \|`
TruthfulQA	safety	35.99	_mc	medium	`\| truthfulqa_mc \| 1 \| mc1 \| 0.3599 \| ± \| 0.0168 \|`
AGIEval	reasoning	35.56	AGI Eval	medium	`\| AGI Eval \| 35.56 \| 39.77 \| 43.07 \| +7.51 \| +3.33 \|`
OpenBookQA	knowledge	34.6		medium	`\| openbookqa \| 0 \| acc \| 0.3460 \| ± \| 0.0213 \|`
BigBench	reasoning	34.11	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 0.3411 \| ± \| 0.0296 \|`
BigBench	reasoning	28.6	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 0.2860 \| ± \| 0.0202 \|`
BigBench	reasoning	28.0	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 0.2800 \| ± \| 0.0142 \|`
AGIEval	reasoning	25.22	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 0.2522 \| ± \| 0.0287 \|`
AGIEval	reasoning	23.23	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 0.2323 \| ± \| 0.0265 \|`
BigBench	reasoning	22.85	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 0.2285 \| ± \| 0.0133 \|`
BigBench	reasoning	21.45	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 0.2145 \| ± \| 0.0217 \|`
BigBench	reasoning	20.86	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 0.2086 \| ± \| 0.0154 \|`
BigBench	reasoning	20.72	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 0.2072 \| ± \| 0.0115 \|`
BigBench	reasoning	16.91	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 0.1691 \| ± \| 0.0090 \|`

Discussions (0)

none scraped.