unsloth/Llama-3.1-Tulu-3-8B-GGUF
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| HumanEval | coding | 86.2 | **HumanEval (pass@10)** | medium | | **HumanEval (pass@10)** | 86.2 | 83.9 | 83.9 | 86.3 | **93.1** | 75.4 | 71.7 | 91.0 | |
| HumanEval | coding | 81.4 | **HumanEval+ (pass@10)** | medium | | **HumanEval+ (pass@10)** | 81.4 | 78.6 | 79.2 | 82.9 | **89.7** | 69.1 | 67.0 | 88.5 | |
| GSM8K | math | 76.2 | **GSM8K (8 shot, CoT)** | medium | | **GSM8K (8 shot, CoT)** | 76.2 | 84.3 | **87.6** | 83.4 | 83.8 | 61.2 | 79.7 | 80.0 | |
| IFEval | instruction-following | 72.8 | **IFEval (prompt loose)** | medium | | **IFEval (prompt loose)** | 72.8 | 81.1 | **82.4** | 80.6 | 74.7 | 38.8 | 69.9 | 56.4 | |
| BBH | reasoning | 67.9 | **BigBenchHard (3 shot, CoT)** | medium | | **BigBenchHard (3 shot, CoT)** | **67.9** | 65.8 | 66.0 | 62.8 | 21.7 | 0.9 | 2.5 | 56.2 | |
| MMLU | knowledge | 65.9 | **MMLU (0 shot, CoT)** | medium | | **MMLU (0 shot, CoT)** | 65.9 | 68.7 | 68.2 | 71.2 | **76.6** | 62.0 | 74.6 | 68.5 | |
| DROP | reasoning | 61.3 | **DROP (3 shot)** | medium | | **DROP (3 shot)** | 61.3 | 62.5 | **62.6** | 61.5 | 54.4 | 49.4 | 58.8 | 56.2 | |
| TruthfulQA | safety | 46.8 | **TruthfulQA (6 shot)** | medium | | **TruthfulQA (6 shot)** | 46.8 | 56.1 | 55.0 | 55.1 | **63.1** | 57.0 | 61.4 | 55.5 | |
| MATH | math | 31.5 | **MATH (4 shot CoT, Flex)** | medium | | **MATH (4 shot CoT, Flex)** | 31.5 | 42.0 | **43.7** | 42.5 | 14.8 | 5.1 | 29.8 | 40.0 | |
| PopQA | knowledge | 29.3 | **PopQA (15 shot)** | medium | | **PopQA (15 shot)** | **29.3** | 29.3 | 29.1 | 20.2 | 18.1 | 22.5 | 28.3 | 20.2 | |
| AlpacaEval | chat | 12.4 | **AlpacaEval 2 (LC % win)** | medium | | **AlpacaEval 2 (LC % win)** | 12.4 | 33.5 | 34.5 | 24.2 | 29.0 | **49.0** | 43.7 | 31.4 | |
none scraped.