Marcoro14-7B-slerp

Home/Models/mlabonne/Marcoro14-7B-slerp

Marcoro14-7B-slerp

mlabonne/Marcoro14-7B-slerp

kind: merge
quant: merge
family: misc
modality: text-generation
root model: AIDC-ai-business/Marcoroni-7B-v3
provider: mlabonne
license: cc-by-nc-4.0
language: —
context: —
library: —
vRAM (GB): —
size on disk: —
downloads: 245
likes: 30
created: —
last modified: 2024-03-04T15:18:12+00:00 (2024)
huggingface: https://huggingface.co/mlabonne/Marcoro14-7B-slerp
tags: transformerssafetensorsmistraltext-generationmergemergekitlazymergekitAIDC-ai-business/Marcoroni-7B-v3EmbeddedLLM/Mistral-7B-Merge-14-v0.1license:cc-by-nc-4.0model-indextext-generation-inferenceendpoints_compatibleregion:us

Benchmark scores (41)

benchmark	category	score	context	conf	evidence
BoolQ	knowledge	88.56		medium	`\| boolq \| 1 \| acc \| 88.56 \| ± \| 0.56 \|`
HellaSwag	knowledge	87.13	HellaSwag (10-Shot)	medium	`\| HellaSwag (10-Shot) \| 87.13 \|`
PIQA	knowledge	82.59		medium	`\| piqa \| 0 \| acc \| 82.59 \| ± \| 0.88 \|`
WinoGrande	knowledge	81.61	Winogrande (5-shot)	medium	`\| Winogrande (5-shot) \| 81.61 \|`
AGIEval	reasoning	79.61	_sat_en	medium	`\| agieval_sat_en \| 0 \| acc \| 79.61 \| ± \| 2.81 \|`
WinoGrande	knowledge	78.53		medium	`\| winogrande \| 0 \| acc \| 78.53 \| ± \| 1.15 \|`
BigBench	reasoning	73.63	_sports_understanding	medium	`\| bigbench_sports_understanding \| 0 \| multiple_choice_grade \| 73.63 \| ± \| 1.40 \|`
BigBench	reasoning	72.38	_snarks	medium	`\| bigbench_snarks \| 0 \| multiple_choice_grade \| 72.38 \| ± \| 3.33 \|`
BigBench	reasoning	72.15	_reasoning_about_colored_objects	medium	`\| bigbench_reasoning_about_colored_objects \| 0 \| multiple_choice_grade \| 72.15 \| ± \| 1.00 \|`
GSM8K	math	70.89	GSM8k (5-shot)	medium	`\| GSM8k (5-shot) \| 70.89 \|`
HellaSwag	knowledge	67.31		medium	`\| hellaswag \| 0 \| acc \| 67.31 \| ± \| 0.47 \|`
BigBench	reasoning	66.4	_date_understanding	medium	`\| bigbench_date_understanding \| 0 \| multiple_choice_grade \| 66.40 \| ± \| 2.46 \|`
MMLU	knowledge	65.11	MMLU (5-Shot)	medium	`\| MMLU (5-Shot) \| 65.11 \|`
TruthfulQA	safety	64.15		high	`\| Marcoro14-7B-slerp \| 44.66 \| 76.24 \| 64.15 \| 45.64 \| 57.67 \|`
ARC-Challenge	knowledge	63.91	_challenge	medium	`\| arc_challenge \| 0 \| acc \| 63.91 \| ± \| 1.40 \|`
TruthfulQA	safety	63.54	TruthfulQA (0-shot)	medium	`\| TruthfulQA (0-shot) \| 63.54 \|`
AGIEval	reasoning	62.83	_lsat_rc	medium	`\| agieval_lsat_rc \| 0 \| acc \| 62.83 \| ± \| 2.95 \|`
BigBench	reasoning	56.32	_causal_judgement	medium	`\| bigbench_causal_judgement \| 0 \| multiple_choice_grade \| 56.32 \| ± \| 3.61 \|`
BigBench	reasoning	53.9	_navigate	medium	`\| bigbench_navigate \| 0 \| multiple_choice_grade \| 53.90 \| ± \| 1.58 \|`
BigBench	reasoning	52.46	_ruin_names	medium	`\| bigbench_ruin_names \| 0 \| multiple_choice_grade \| 52.46 \| ± \| 2.36 \|`
BigBench	reasoning	52.33	_logical_deduction_three_objects	medium	`\| bigbench_logical_deduction_three_objects \| 0 \| multiple_choice_grade \| 52.33 \| ± \| 2.89 \|`
BigBench	reasoning	52.33	_tracking_shuffled_objects_three_objects	medium	`\| bigbench_tracking_shuffled_objects_three_objects \| 0 \| multiple_choice_grade \| 52.33 \| ± \| 2.89 \|`
AGIEval	reasoning	50.0	_lsat_lr	medium	`\| agieval_lsat_lr \| 0 \| acc \| 50.00 \| ± \| 2.22 \|`
TruthfulQA	safety	46.88	_mc	medium	`\| truthfulqa_mc \| 1 \| mc1 \| 46.88 \| ± \| 1.75 \|`
BigBench	reasoning	45.7	_temporal_sequences	medium	`\| bigbench_temporal_sequences \| 0 \| multiple_choice_grade \| 45.70 \| ± \| 1.58 \|`
BigBench	reasoning	45.64		high	`\| Marcoro14-7B-slerp \| 44.66 \| 76.24 \| 64.15 \| 45.64 \| 57.67 \|`
BigBench	reasoning	45.35	_disambiguation_qa	medium	`\| bigbench_disambiguation_qa \| 0 \| multiple_choice_grade \| 45.35 \| ± \| 3.11 \|`
AGIEval	reasoning	45.15	_sat_en_without_passage	medium	`\| agieval_sat_en_without_passage \| 0 \| acc \| 45.15 \| ± \| 3.48 \|`
AGIEval	reasoning	44.66		high	`\| Marcoro14-7B-slerp \| 44.66 \| 76.24 \| 64.15 \| 45.64 \| 57.67 \|`
BigBench	reasoning	39.2	_movie_recommendation	medium	`\| bigbench_movie_recommendation \| 0 \| multiple_choice_grade \| 39.20 \| ± \| 2.19 \|`
AGIEval	reasoning	38.25	_logiqa_en	medium	`\| agieval_logiqa_en \| 0 \| acc \| 38.25 \| ± \| 1.91 \|`
OpenBookQA	knowledge	36.4		medium	`\| openbookqa \| 0 \| acc \| 36.40 \| ± \| 2.15 \|`
AGIEval	reasoning	33.18	_sat_math	medium	`\| agieval_sat_math \| 0 \| acc \| 33.18 \| ± \| 3.18 \|`
BigBench	reasoning	30.0	_logical_deduction_five_objects	medium	`\| bigbench_logical_deduction_five_objects \| 0 \| multiple_choice_grade \| 30.00 \| ± \| 2.05 \|`
AGIEval	reasoning	26.38	_aqua_rat	medium	`\| agieval_aqua_rat \| 0 \| acc \| 26.38 \| ± \| 2.77 \|`
BigBench	reasoning	25.75	_salient_translation_error_detection	medium	`\| bigbench_salient_translation_error_detection \| 0 \| multiple_choice_grade \| 25.75 \| ± \| 1.38 \|`
AGIEval	reasoning	24.35	_lsat_ar	medium	`\| agieval_lsat_ar \| 0 \| acc \| 24.35 \| ± \| 2.84 \|`
BigBench	reasoning	23.44	_tracking_shuffled_objects_five_objects	medium	`\| bigbench_tracking_shuffled_objects_five_objects \| 0 \| multiple_choice_grade \| 23.44 \| ± \| 1.20 \|`
BigBench	reasoning	21.43	_logical_deduction_seven_objects	medium	`\| bigbench_logical_deduction_seven_objects \| 0 \| multiple_choice_grade \| 21.43 \| ± \| 1.55 \|`
BigBench	reasoning	20.33	_geometric_shapes	medium	`\| bigbench_geometric_shapes \| 0 \| multiple_choice_grade \| 20.33 \| ± \| 2.13 \|`
BigBench	reasoning	18.51	_tracking_shuffled_objects_seven_objects	medium	`\| bigbench_tracking_shuffled_objects_seven_objects \| 0 \| multiple_choice_grade \| 18.51 \| ± \| 0.93 \|`

Discussions (0)

none scraped.