microsoft/bitnet-b1.58-2B-4T-gguf
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| PIQA | knowledge | 74.21 | PIQA | medium | | PIQA | 74.21 | 71.93 | 76.12 | 76.12 | 76.66 | **77.09** | |
| BoolQ | knowledge | 64.65 | BoolQ | medium | | BoolQ | 64.65 | 74.22 | 78.04 | 75.78 | **80.67** | 80.18 | |
| IFEval | instruction-following | 62.71 | IFEval | medium | | IFEval | 62.71 | **66.67** | 50.12 | 57.91 | 36.81 | 53.48 | |
| HellaSwag | knowledge | 60.8 | HellaSwag | medium | | HellaSwag | 60.80 | 57.69 | 68.28 | **71.71** | 70.81 | 68.44 | |
| WinoGrande | knowledge | 59.51 | WinoGrande | medium | | WinoGrande | 59.51 | 58.48 | 62.83 | 68.98 | 61.80 | **71.90** | |
| CommonsenseQA | knowledge | 58.48 | CommonsenseQA | medium | | CommonsenseQA | 58.48 | 42.10 | **76.41** | 63.55 | 71.74 | 71.58 | |
| MMLU | knowledge | 45.58 | MMLU | medium | | MMLU | 45.58 | 39.91 | **60.25** | 49.24 | 51.82 | 53.17 | |
| TruthfulQA | safety | 43.8 | TruthfulQA | medium | | TruthfulQA | 43.80 | 38.66 | **46.67** | 39.90 | 41.41 | 45.31 | |
| GSM8K | math | 38.21 | GSM8K | medium | | GSM8K | 38.21 | 31.16 | 56.79 | 45.11 | 4.40 | **58.38** | |
| ARC-Challenge | knowledge | 37.8 | ARC-Challenge | medium | | ARC-Challenge | 37.80 | 38.40 | 46.67 | 43.52 | 44.80 | **49.91** | |
| TriviaQA | knowledge | 37.6 | TriviaQA | medium | | TriviaQA | 37.60 | 23.49 | 38.37 | **45.97** | 34.13 | 33.57 | |
| OpenBookQA | knowledge | 34.8 | OpenbookQA | medium | | OpenbookQA | 34.80 | 38.80 | 40.80 | **46.00** | 40.20 | 41.60 | |
| HumanEval | coding | 31.1 | HumanEval+ | medium | | HumanEval+ | 31.10 | 37.20 | **50.60** | 28.00 | 43.90 | 38.40 | |
| MATH-500 | math | 23.0 | MATH-500 | medium | | MATH-500 | 23.00 | 42.00 | **53.00** | 17.60 | 14.80 | 43.40 | |
| MT-Bench | chat | 5.43 | MT-bench | medium | | MT-bench | 5.43 | 6.40 | 6.12 | 5.50 | **6.57** | 5.85 | |
none scraped.