alpindale/gemma-7b-it
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| PIQA | knowledge | 77.3 | [PIQA](https://arxiv.org/abs/1911.11641) | medium | | [PIQA](https://arxiv.org/abs/1911.11641) | 0-shot | 77.3 | 81.2 | |
| HellaSwag | knowledge | 71.4 | [HellaSwag](https://arxiv.org/abs/1905.07830) | medium | | [HellaSwag](https://arxiv.org/abs/1905.07830) | 0-shot | 71.4 | 81.2 | |
| WinoGrande | knowledge | 65.4 | [WinoGrande](https://arxiv.org/abs/1907.10641) | medium | | [WinoGrande](https://arxiv.org/abs/1907.10641) | partial score | 65.4 | 72.3 | |
| CommonsenseQA | knowledge | 65.3 | [CommonsenseQA](https://arxiv.org/abs/1811.00937) | medium | | [CommonsenseQA](https://arxiv.org/abs/1811.00937) | 7-shot | 65.3 | 71.3 | |
| TriviaQA | knowledge | 53.2 | [TriviaQA](https://arxiv.org/abs/1705.03551) | medium | | [TriviaQA](https://arxiv.org/abs/1705.03551) | 5-shot | 53.2 | 63.4 | |
| OpenBookQA | knowledge | 47.8 | [OpenBookQA](https://arxiv.org/abs/1809.02789) | medium | | [OpenBookQA](https://arxiv.org/abs/1809.02789) | | 47.8 | 52.8 | |
| TruthfulQA | safety | 44.84 | [TruthfulQA](https://arxiv.org/abs/2109.07958) | medium | | [TruthfulQA](https://arxiv.org/abs/2109.07958) | | 44.84 | 31.81 | |
| MMLU | knowledge | 42.3 | [MMLU](https://arxiv.org/abs/2009.03300) | medium | | [MMLU](https://arxiv.org/abs/2009.03300) | 5-shot, top-1 | 42.3 | 64.3 | |
| BigBench | reasoning | 35.2 | [BIG-Bench](https://arxiv.org/abs/2206.04615) | medium | | [BIG-Bench](https://arxiv.org/abs/2206.04615) | | 35.2 | 55.1 | |
| MBPP | coding | 29.2 | [MBPP](https://arxiv.org/abs/2108.07732) | medium | | [MBPP](https://arxiv.org/abs/2108.07732) | 3-shot | 29.2 | 44.4 | |
| AGIEval | reasoning | 24.2 | [AGIEval](https://arxiv.org/abs/2304.06364) | medium | | [AGIEval](https://arxiv.org/abs/2304.06364) | | 24.2 | 41.7 | |
| HumanEval | coding | 22.0 | [HumanEval](https://arxiv.org/abs/2107.03374) | medium | | [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 | 22.0 | 32.3 | |
| GSM8K | math | 17.7 | [GSM8K](https://arxiv.org/abs/2110.14168) | medium | | [GSM8K](https://arxiv.org/abs/2110.14168) | maj@1 | 17.7 | 46.4 | |
| MATH | math | 11.8 | [MATH](https://arxiv.org/abs/2108.07732) | medium | | [MATH](https://arxiv.org/abs/2108.07732) | 4-shot | 11.8 | 24.3 | |
none scraped.