google/gemma-2-2b-it
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| PIQA | knowledge | 77.8 | [PIQA][] | medium | | [PIQA][piqa] | 0-shot | 77.8 | 81.7 | 83.2 | |
| HellaSwag | knowledge | 73.0 | [HellaSwag][] | medium | | [HellaSwag][hellaswag] | 10-shot | 73.0 | 81.9 | 86.4 | |
| BoolQ | knowledge | 72.5 | [BoolQ][] | medium | | [BoolQ][boolq] | 0-shot | 72.5 | 84.2 | 84.8 | |
| WinoGrande | knowledge | 70.9 | [WinoGrande][] | medium | | [WinoGrande][winogrande] | partial score | 70.9 | 80.6 | 83.7 | |
| TriviaQA | knowledge | 59.4 | [TriviaQA][] | medium | | [TriviaQA][triviaqa] | 5-shot | 59.4 | 76.6 | 83.7 | |
| DROP | reasoning | 52.0 | [DROP][] | medium | | [DROP][drop] | 3-shot, F1 | 52.0 | 69.4 | 72.2 | |
| MMLU | knowledge | 51.3 | [MMLU][] | medium | | [MMLU][mmlu] | 5-shot, top-1 | 51.3 | 71.3 | 75.2 | |
| TruthfulQA | safety | 43.72 | [TruthfulQA][] | medium | | [TruthfulQA][truthfulqa] | | 43.72 | 50.27 | 51.60 | |
| BigBench | reasoning | 41.9 | [BIG-Bench][big-bench] | medium | | [BIG-Bench][big-bench] | 3-shot, CoT | 41.9 | 68.2 | 74.9 | |
| AGIEval | reasoning | 30.6 | [AGIEval][] | medium | | [AGIEval][agieval] | 3-5-shot | 30.6 | 52.8 | 55.1 | |
| MBPP | coding | 29.6 | [MBPP][] | medium | | [MBPP][mbpp] | 3-shot | 29.6 | 52.4 | 62.6 | |
| GSM8K | math | 23.9 | [GSM8K][] | medium | | [GSM8K][gsm8k] | 5-shot, maj@1 | 23.9 | 68.6 | 74.0 | |
| HumanEval | coding | 17.7 | [HumanEval][] | medium | | [HumanEval][humaneval] | pass@1 | 17.7 | 40.2 | 51.8 | |
| MATH | math | 15.0 | [MATH][] | medium | | [MATH][math] | 4-shot | 15.0 | 36.6 | 42.3 | |
none scraped.