microsoft/Phi-3-small-8k-instruct
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| ARC-Challenge | knowledge | 90.7 | ARC Challenge<br>10-shot | medium | | ARC Challenge<br>10-shot | 90.7 | 78.3 | 87.3 | 82.8 | 87.4 | 88.3 | 95.6 | |
| GSM8K | math | 89.6 | GSM8K Chain of Thought<br>8-shot | medium | | GSM8K Chain of Thought<br>8-shot | 89.6 | 59.8 | 64.7 | 77.4 | 78.1 | 80.4 | 94.2 | |
| OpenBookQA | knowledge | 88.0 | OpenBookQA<br>10-shot | medium | | OpenBookQA<br>10-shot | 88.0 | 78.6 | 85.8 | 82.6 | 86.0 | 86.4 | 93.4 | |
| PIQA | knowledge | 86.9 | PIQA<br>5-shot | medium | | PIQA<br>5-shot | 86.9 | 78.1 | 86.0 | 75.7 | 86.6 | 86.2 | 90.1 | |
| BoolQ | knowledge | 84.8 | BoolQ<br>2-shot | medium | | BoolQ<br>2-shot | 84.8 | 66.0 | 76.6 | 80.9 | 79.1 | 86.4 | 91.3 | |
| WinoGrande | knowledge | 81.5 | WinoGrande<br>5-shot | medium | | WinoGrande<br>5-shot | 81.5 | 55.6 | 62.0 | 65.0 | 68.8 | 72.2 | 86.7 | |
| CommonsenseQA | knowledge | 80.0 | CommonsenseQA<br>10-shot | medium | | CommonsenseQA<br>10-shot | 80.0 | 76.2 | 78.1 | 79.0 | 79.6 | 81.8 | 86.7 | |
| BBH | reasoning | 79.1 | BigBench Hard<br>3-shot | medium | | BigBench Hard<br>3-shot | 79.1 | 59.6 | 69.7 | 51.5 | 68.3 | 75.6 | 87.7 | |
| HellaSwag | knowledge | 77.0 | HellaSwag<br>5-shot | medium | | HellaSwag<br>5-shot | 77.0 | 49.8 | 70.4 | 71.1 | 78.8 | 76.2 | 88.3 | |
| MMLU | knowledge | 75.7 | MMLU<br>5-shot | medium | | MMLU<br>5-shot | 75.7 | 63.6 | 70.5 | 66.5 | 71.4 | 66.7 | 84.0 | |
| MBPP | coding | 71.7 | MBPP<br>3-shot | medium | | MBPP<br>3-shot | 71.7 | 51.5 | 60.2 | 67.7 | 77.8 | 73.2 | 86.7 | |
| TruthfulQA | safety | 70.2 | TruthfulQA (MC2)<br>10-shot | medium | | TruthfulQA (MC2)<br>10-shot | 70.2 | 52.1 | 60.1 | 63.2 | 67.7 | 72.6 | 85.2 | |
| MedQA | domain | 65.4 | MedQA<br>2-shot | medium | | MedQA<br>2-shot | 65.4 | 49.6 | 62.2 | 60.5 | 63.4 | 58.2 | 83.7 | |
| HumanEval | coding | 61.0 | HumanEval<br>0-shot | medium | | HumanEval<br>0-shot | 61.0 | 34.1 | 37.8 | 60.4 | 62.2 | 64.4 | 79.9 | |
| TriviaQA | knowledge | 58.1 | TriviaQA<br>5-shot | medium | | TriviaQA<br>5-shot | 58.1 | 72.3 | 82.2 | 67.7 | 85.8 | 80.2 | 73.3 | |
| MATH | math | 51.6 | Math | medium | | Math | 51.6 | 35.8 | 40.3 | 41.1 | 52.8 | 50.9 | 67.1 | |
| AGIEval | reasoning | 45.1 | AGI Eval<br>5-shot | medium | | AGI Eval<br>5-shot | 45.1 | 42.1 | 45.2 | 42.0 | 48.4 | 49.0 | 59.6 | |
none scraped.