microsoft/Phi-3-medium-4k-instruct
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| ARC-Challenge | knowledge | 91.6 | ARC Challenge<br>10-shot | medium | | ARC Challenge<br>10-shot | 91.6 | 86.9 | 91.3 | 93.0 | 87.4 | 88.3 | 95.6 | |
| GSM8K | math | 91.0 | GSM8K Chain of Thought<br>8-shot | medium | | GSM8K Chain of Thought<br>8-shot | 91.0 | 78.3 | 83.8 | 93.5 | 78.1 | 80.4 | 94.2 | |
| PIQA | knowledge | 87.9 | PIQA<br>5-shot | medium | | PIQA<br>5-shot | 87.9 | 86.4 | 85.0 | 85.3 | 86.6 | 86.2 | 90.1 | |
| OpenBookQA | knowledge | 87.4 | OpenBookQA<br>10-shot | medium | | OpenBookQA<br>10-shot | 87.4 | 86.8 | 88.6 | 91.8 | 86.0 | 86.4 | 93.4 | |
| BoolQ | knowledge | 86.5 | BoolQ<br>2-shot | medium | | BoolQ<br>2-shot | 86.5 | 86.1 | 82.7 | 89.1 | 79.1 | 86.4 | 91.3 | |
| CommonsenseQA | knowledge | 82.8 | CommonsenseQA<br>10-shot | medium | | CommonsenseQA<br>10-shot | 82.8 | 82.0 | 82.0 | 84.4 | 79.6 | 81.8 | 86.7 | |
| HellaSwag | knowledge | 82.4 | HellaSwag<br>5-shot | medium | | HellaSwag<br>5-shot | 82.4 | 78.0 | 79.0 | 82.6 | 78.8 | 76.2 | 88.3 | |
| WinoGrande | knowledge | 81.5 | WinoGrande<br>5-shot | medium | | WinoGrande<br>5-shot | 81.5 | 77.0 | 75.3 | 83.3 | 68.8 | 72.2 | 86.7 | |
| BBH | reasoning | 81.4 | BigBench Hard<br>3-shot | medium | | BigBench Hard<br>3-shot | 81.4 | 74.1 | 81.8 | 80.4 | 68.3 | 75.6 | 87.7 | |
| MMLU | knowledge | 78.0 | MMLU<br>5-shot | medium | | MMLU<br>5-shot | 78.0 | 73.8 | 76.2 | 80.2 | 71.4 | 66.7 | 84.0 | |
| MBPP | coding | 75.2 | MBPP<br>3-shot | medium | | MBPP<br>3-shot | 75.2 | 68.9 | 70.7 | 81.3 | 77.8 | 73.2 | 86.7 | |
| TruthfulQA | safety | 75.1 | TruthfulQA (MC2)<br>10-shot | medium | | TruthfulQA (MC2)<br>10-shot | 75.1 | 57.8 | 67.4 | 81.9 | 67.7 | 72.6 | 85.2 | |
| TriviaQA | knowledge | 73.9 | TriviaQA<br>5-shot | medium | | TriviaQA<br>5-shot | 73.9 | 82.8 | 84.5 | 78.5 | 85.8 | 80.2 | 73.3 | |
| MedQA | domain | 69.9 | MedQA<br>2-shot | medium | | MedQA<br>2-shot | 69.9 | 59.2 | 67.9 | 78.5 | 63.4 | 58.2 | 83.7 | |
| HumanEval | coding | 62.2 | HumanEval<br>0-shot | medium | | HumanEval<br>0-shot | 62.2 | 61.6 | 39.6 | 78.7 | 62.2 | 64.4 | 79.9 | |
| MATH | math | 52.8 | Math | medium | | Math | 52.8 | 45.3 | 52.5 | 59.7 | 52.8 | 50.9 | 67.1 | |
| AGIEval | reasoning | 50.2 | AGI Eval<br>5-shot | medium | | AGI Eval<br>5-shot | 50.2 | 50.1 | 54.0 | 56.9 | 48.4 | 49.0 | 59.6 | |
none scraped.