tiiuae/Falcon-H1-1.5B-Deep-Instruct-GGUF
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| IFEval | instruction-following | 83.5 | IFEval | medium | | IFEval | **83.5** | 70.77 | 45.33 | 61.48 | 55.34 | 54.26 | |
| GSM8K | math | 82.34 | GSM8k | medium | | GSM8k | **82.34** | 69.83 | 57.47 | 42.38 | 44.28 | 44.05 | |
| MATH-500 | math | 77.8 | MATH-500 | medium | | MATH-500 | **77.8** | 73.0 | 48.4 | 45.4 | 13.2 | 19.8 | |
| HumanEval | coding | 73.78 | HumanEval | medium | | HumanEval | **73.78** | 67.68 | 56.1 | 40.85 | 34.15 | 22.56 | |
| HumanEval | coding | 68.9 | HumanEval+ | medium | | HumanEval+ | **68.9** | 60.96 | 50.61 | 37.2 | 29.88 | 20.73 | |
| MBPP | coding | 68.25 | MBPP | medium | | MBPP | **68.25** | 58.73 | 64.81 | 57.67 | 33.6 | 20.63 | |
| MMLU | knowledge | 67.3 | MMLU-stem | medium | | MMLU-stem | **67.3** | 54.3 | 54.04 | 35.39 | 39.16 | 39.64 | |
| MMLU | knowledge | 66.11 | MMLU | medium | | MMLU | **66.11** | 57.04 | 59.76 | 40.87 | 45.93 | 46.1 | |
| HellaSwag | knowledge | 65.54 | HellaSwag | medium | | HellaSwag | **65.54** | 49.27 | 62.23 | 42.24 | 55.3 | 58.53 | |
| MBPP | coding | 56.61 | MBPP+ | medium | | MBPP+ | **56.61** | 49.74 | 56.08 | 50.0 | 29.37 | 17.2 | |
| BBH | reasoning | 54.43 | BBH | medium | | BBH | **54.43** | 35.18 | 42.41 | 35.86 | 33.21 | 34.47 | |
| TruthfulQA | safety | 50.48 | TruthfulQA | medium | | TruthfulQA | **50.48** | 49.39 | 47.05 | 42.17 | 42.08 | 42.31 | |
| MMLU-Pro | knowledge | 41.89 | MMLU-Pro | medium | | MMLU-Pro | **41.89** | 23.54 | 28.35 | 14.46 | 16.2 | 18.49 | |
| GPQA Diamond | reasoning | 40.57 | GPQA_Diamond | medium | | GPQA_Diamond | **40.57** | 33.33 | 25.59 | 21.55 | 25.08 | 31.31 | |
| GPQA | reasoning | 33.22 | GPQA | medium | | GPQA | **33.22** | 27.68 | 26.26 | 28.19 | 26.59 | 26.76 | |
| AlpacaEval | chat | 27.12 | Alpaca-Eval | medium | | Alpaca-Eval | **27.12** | 21.89 | 9.54 | 17.87 | 9.38 | 6.98 | |
| LiveCodeBench | coding | 23.87 | LiveCodeBench | medium | | LiveCodeBench | **23.87** | 14.87 | 12.52 | 5.09 | 2.35 | 0.78 | |
| MT-Bench | chat | 8.53 | MTBench | medium | | MTBench | **8.53** | 7.61 | 7.1 | 7.03 | 6.37 | 6.03 | |
none scraped.