Redmond-Puffin-13B-GPTQ

Home/Models/TheBloke/Redmond-Puffin-13B-GPTQ

Redmond-Puffin-13B-GPTQ

TheBloke/Redmond-Puffin-13B-GPTQ

kind: gptq
quant: gptq
family: misc
modality: text-generation
root model: NousResearch/Redmond-Puffin-13B
provider: TheBloke
license: - mit
language: eng
context: 4096
library: —
vRAM (GB): —
size on disk: —
downloads: 69
likes: 26
created: —
last modified: 2023-09-27T12:44:50+00:00 (2023)
huggingface: https://huggingface.co/TheBloke/Redmond-Puffin-13B-GPTQ
tags: transformerssafetensorsllamatext-generationllama-2sftengdataset:LDJnr/Puffinbase_model:NousResearch/Redmond-Puffin-13Bbase_model:quantized:NousResearch/Redmond-Puffin-13Blicense:mittext-generation-inference4-bitgptqregion:us

Benchmark scores (32)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	81.5		medium	`\| boolq \| 1 \| acc \| 0.8150 \| ± \| 0.0068 \|`
PIQA	knowledge	79.54		medium	`\| piqa \| 0 \| acc \| 0.7954 \| ± \| 0.0094 \|`
WinoGrande	knowledge	72.45		medium	`\| winogrande \| 0 \| acc \| 0.7245 \| ± \| 0.0126 \|`
BigBench	reasoning	71.27	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 0.7127 \| ± \| 0.0236 \|`
HellaSwag	knowledge	61.32		medium	`\| hellaswag \| 0 \| acc \| 0.6132 \| ± \| 0.0049 \|`
AGIEval	reasoning	58.74	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 0.5874 \| ± \| 0.0344 \|`
BigBench	reasoning	54.3	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 0.5430 \| ± \| 0.0111 \|`
BigBench	reasoning	53.68	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 0.5368 \| ± \| 0.0363 \|`
BigBench	reasoning	50.83	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 0.5083 \| ± \| 0.0373 \|`
BigBench	reasoning	50.0	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 0.5000 \| ± \| 0.0158 \|`
ARC-Challenge	knowledge	49.83	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 0.4983 \| ± \| 0.0146 \|`
BigBench	reasoning	49.7	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 0.4970 \| ± \| 0.0159 \|`
BigBench	reasoning	44.42	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 0.4442 \| ± \| 0.0235 \|`
AGIEval	reasoning	43.87	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 0.4387 \| ± \| 0.0303 \|`
AGIEval	reasoning	42.23	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 0.4223 \| ± \| 0.0345 \|`
BigBench	reasoning	42.0	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 0.4200 \| ± \| 0.0285 \|`
BigBench	reasoning	42.0	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 0.4200 \| ± \| 0.0285 \|`
OpenBookQA	knowledge	35.6		medium	`\| openbookqa \| 0 \| acc \| 0.3560 \| ± \| 0.0214 \|`
AGIEval	reasoning	33.92	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 0.3392 \| ± \| 0.0210 \|`
AGIEval	reasoning	33.64	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 0.3364 \| ± \| 0.0319 \|`
BigBench	reasoning	32.6	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 0.3260 \| ± \| 0.0148 \|`
BigBench	reasoning	30.23	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 0.3023 \| ± \| 0.0286 \|`
BigBench	reasoning	29.0	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 0.2900 \| ± \| 0.0203 \|`
AGIEval	reasoning	27.8	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 0.2780 \| ± \| 0.0176 \|`
BigBench	reasoning	25.2	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 0.2520 \| ± \| 0.0194 \|`
AGIEval	reasoning	23.48	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 0.2348 \| ± \| 0.0280 \|`
AGIEval	reasoning	22.83	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 0.2283 \| ± \| 0.0264 \|`
BigBench	reasoning	21.36	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 0.2136 \| ± \| 0.0116 \|`
BigBench	reasoning	20.74	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 0.2074 \| ± \| 0.0128 \|`
BigBench	reasoning	17.43	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 0.1743 \| ± \| 0.0143 \|`
BigBench	reasoning	13.26	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 0.1326 \| ± \| 0.0081 \|`
BigBench	reasoning	10.03	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 0.1003 \| ± \| 0.0159 \|`

Discussions (0)

none scraped.