Redmond-Hermes-Coder-GPTQ

Home/Models/TheBloke/Redmond-Hermes-Coder-GPTQ

Redmond-Hermes-Coder-GPTQ

TheBloke/Redmond-Hermes-Coder-GPTQ

kind: gptq
quant: gptq
family: misc
modality: text-generation
root model: —
provider: TheBloke
license: gpl
language: en
context: —
library: —
vRAM (GB): —
size on disk: —
downloads: 56
likes: 10
created: —
last modified: 2023-08-21T02:08:38+00:00 (2023)
huggingface: https://huggingface.co/TheBloke/Redmond-Hermes-Coder-GPTQ
tags: transformerssafetensorsgpt_bigcodetext-generationstarcoderwizardcodercodeself-instructdistillationenlicense:gpltext-generation-inference4-bitgptqregion:us

Benchmark scores (24)

benchmark	category	score	context	conf	evidence
PIQA	knowledge	66.0		medium	`\| piqa \| 0 \| acc \| 0.6600 \| ± \| 0.0111 \|`
BoolQ	knowledge	54.98		medium	`\| boolq \| 1 \| acc \| 0.5498 \| ± \| 0.0087 \|`
WinoGrande	knowledge	53.43		medium	`\| winogrande \| 0 \| acc \| 0.5343 \| ± \| 0.0140 \|`
BigBench	reasoning	52.49	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 0.5249 \| ± \| 0.0372 \|`
BigBench	reasoning	52.3	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 0.5230 \| ± \| 0.0260 \|`
BigBench	reasoning	51.58	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 0.5158 \| ± \| 0.0364 \|`
BigBench	reasoning	48.3	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 0.4830 \| ± \| 0.0158 \|`
BigBench	reasoning	48.17	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 0.4817 \| ± \| 0.0159 \|`
BigBench	reasoning	41.5	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 0.4150 \| ± \| 0.0110 \|`
HellaSwag	knowledge	38.14		medium	`\| hellaswag \| 0 \| acc \| 0.3814 \| ± \| 0.0048 \|`
BigBench	reasoning	37.33	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 0.3733 \| ± \| 0.0280 \|`
BigBench	reasoning	37.33	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 0.3733 \| ± \| 0.0280 \|`
BigBench	reasoning	32.95	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 0.3295 \| ± \| 0.0293 \|`
BigBench	reasoning	32.0	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 0.3200 \| ± \| 0.0209 \|`
BigBench	reasoning	29.26	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 0.2926 \| ± \| 0.0144 \|`
ARC-Challenge	knowledge	28.58	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 0.2858 \| ± \| 0.0132 \|`
BigBench	reasoning	27.0	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 0.2700 \| ± \| 0.0140 \|`
BigBench	reasoning	22.6	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 0.2260 \| ± \| 0.0187 \|`
BigBench	reasoning	21.43	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 0.2143 \| ± \| 0.0194 \|`
OpenBookQA	knowledge	19.6		medium	`\| openbookqa \| 0 \| acc \| 0.1960 \| ± \| 0.0178 \|`
BigBench	reasoning	19.57	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 0.1957 \| ± \| 0.0150 \|`
BigBench	reasoning	18.64	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 0.1864 \| ± \| 0.0110 \|`
BigBench	reasoning	13.49	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 0.1349 \| ± \| 0.0082 \|`
BigBench	reasoning	10.03	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 0.1003 \| ± \| 0.0159 \|`

Discussions (0)

none scraped.