Tulpar-7B-v0-GGUF

Home/Models/TheBloke/Tulpar-7B-v0-GGUF

Tulpar-7B-v0-GGUF

TheBloke/Tulpar-7B-v0-GGUF

kind: gguf
quant: gguf (2-bit)
family: misc
modality: —
root model: HyperbeeAI/Tulpar-7b-v0
provider: TheBloke
license: llama2
language: en
context: —
library: transformers
vRAM (GB): —
size on disk: —
downloads: 516
likes: 3
created: —
last modified: 2023-09-27T12:48:39+00:00 (2023)
huggingface: https://huggingface.co/TheBloke/Tulpar-7B-v0-GGUF
tags: transformersggufllamaenbase_model:HyperbeeAI/Tulpar-7b-v0base_model:quantized:HyperbeeAI/Tulpar-7b-v0license:llama2region:us

Benchmark scores (28)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	83.06		medium	`\| boolq \| acc \| 0.8306 \|`
PIQA	knowledge	79.05		medium	`\| piqa \| acc \| 0.7905 \|`
HellaSwag	knowledge	79.01	**	medium	`\| hellaswag \| acc_norm \| 0.7901 \|`
WinoGrande	knowledge	71.59		medium	`\| winogrande \| acc \| 0.7159 \|`
BigBench	reasoning	67.54	_hyperbaton	medium	`\| bigbench_hyperbaton \| multiple_choice_grade \| 0.6754 \|`
BigBench	reasoning	64.23	_date_understanding	medium	`\| bigbench_date_understanding \| multiple_choice_grade \| 0.6423 \|`
BigBench	reasoning	61.05	_causal_judgement	medium	`\| bigbench_causal_judgement \| multiple_choice_grade \| 0.6105 \|`
BigBench	reasoning	60.22	_snarks	medium	`\| bigbench_snarks \| multiple_choice_grade \| 0.6022 \|`
BigBench	reasoning	57.5	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| multiple_choice_grade \| 0.5750 \|`
ARC-Challenge	knowledge	56.14	_challenge	medium	`\| arc_challenge \| acc_norm \| 0.5614 \|`
MMLU	knowledge	52.42	**	medium	`\| mmlu \| acc_norm \| 0.5242 \|`
TruthfulQA	safety	51.6	_mc	medium	`\| truthfulqa_mc \| mc2 \| 0.5160 \|`
BigBench	reasoning	51.22	_sports_understanding	medium	`\| bigbench_sports_understanding \| multiple_choice_grade \| 0.5122 \|`
BigBench	reasoning	50.02	_formal_fallacies_syllogisms_negation	medium	`\| bigbench_formal_fallacies_syllogisms_negation \| multiple_choice_grade \| 0.5002 \|`
BigBench	reasoning	50.0	_navigate	medium	`\| bigbench_navigate \| multiple_choice_grade \| 0.5000 \|`
BigBench	reasoning	41.33	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| multiple_choice_grade \| 0.4133 \|`
BigBench	reasoning	41.33	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| multiple_choice_grade \| 0.4133 \|`
BigBench	reasoning	36.43	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| multiple_choice_grade \| 0.3643 \|`
OpenBookQA	knowledge	35.6		medium	`\| openbookqa \| acc \| 0.356 \|`
BigBench	reasoning	32.81	_ruin_names	medium	`\| bigbench_ruin_names \| multiple_choice_grade \| 0.3281 \|`
BigBench	reasoning	30.0	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| multiple_choice_grade \| 0.3000 \|`
BigBench	reasoning	29.76	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| multiple_choice_grade \| 0.2976 \|`
BigBench	reasoning	27.0	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| multiple_choice_grade \| 0.2700 \|`
BigBench	reasoning	20.0	_dyck_languages	medium	`\| bigbench_dyck_languages \| multiple_choice_grade \| 0.2000 \|`
BigBench	reasoning	19.76	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| multiple_choice_grade \| 0.1976 \|`
BigBench	reasoning	19.29	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| multiple_choice_grade \| 0.1929 \|`
BigBench	reasoning	14.5	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| multiple_choice_grade \| 0.1450 \|`
BigBench	reasoning	14.4	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| multiple_choice_grade \| 0.1440 \|`

Discussions (0)

none scraped.