google/t5gemma-2-1b-1b
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| PIQA | knowledge | 66.5 | [PIQA] | medium | | [PIQA] | 0-shot | 66.5 | 74.3 | 79.2 | |
| BoolQ | knowledge | 57.4 | [BoolQ] | medium | | [BoolQ] | 0-shot | 57.4 | 68.5 | 79.3 | |
| RULER | long-context | 57.3 | [Ruler 32K] | medium | | [Ruler 32K] | 57.3 | 69.2 | 81.7 | |
| WinoGrande | knowledge | 53.9 | [WinoGrande] | medium | | [WinoGrande] | 5-shot | 53.9 | 60.8 | 71.6 | |
| DocVQA | multimodal | 41.6 | [DocVQA] (val) | medium | | [DocVQA] (val) | 41.6 | 66.6 | 74.7 | |
| HellaSwag | knowledge | 41.1 | [HellaSwag] | medium | | [HellaSwag] | 10-shot | 41.1 | 62.9 | 77.4 | |
| DROP | reasoning | 39.4 | [DROP] | medium | | [DROP] | 1-shot | 39.4 | 51.2 | 66.7 | |
| RULER | long-context | 32.0 | medium | Ruler 32 | |
| ChartQA | multimodal | 29.2 | [ChartQA] | medium | | [ChartQA] | 29.2 | 50.2 | 66.0 | |
| RULER | long-context | 25.5 | [Ruler 128K] | medium | | [Ruler 128K] | 25.5 | 35.1 | 57.6 | |
| MMLU | knowledge | 23.5 | [Global-MMLU-Lite] | medium | | [Global-MMLU-Lite] | 23.5 | 33.1 | 53.5 | |
| BBH | reasoning | 22.9 | [BIG-Bench Hard] | medium | | [BIG-Bench Hard] | few-shot | 22.9 | 28.5 | 43.7 | |
| MMMU | multimodal | 22.7 | [MMMU] (pt) | medium | | [MMMU] (pt) | 22.7 | 28.4 | 39.4 | |
| AGIEval | reasoning | 20.9 | [AGIEval] | medium | | [AGIEval] | 3-5-shot | 20.9 | 23.8 | 41.1 | |
| TriviaQA | knowledge | 14.8 | [TriviaQA] | medium | | [TriviaQA] | 5-shot | 14.8 | 29.0 | 53.1 | |
| MMLU | knowledge | 10.6 | [MMLU] (Pro CoT) | medium | | [MMLU] (Pro CoT) | 5-shot | 10.6 | 16.1 | 33.2 | |
| GPQA | reasoning | 9.6 | [GPQA] | medium | | [GPQA] | 5-shot | 9.6 | 10.5 | 17.6 | |
| MBPP | coding | 5.6 | [MBPP] | medium | | [MBPP] | 3-shot | 5.6 | 25.2 | 44.8 | |
| HumanEval | coding | 4.3 | [HumanEval] | medium | | [HumanEval] | 0-shot | 4.3 | 15.2 | 30.5 | |
| MGSM | math | 1.8 | [MGSM] | medium | | [MGSM] | 1.8 | 8.9 | 42.1 | |
| GSM8K | math | 1.7 | [GSM8K] | medium | | [GSM8K] | 8-shot | 1.7 | 9.1 | 44.0 | |
| MATH | math | 1.5 | [MATH] | medium | | [MATH] | 4-shot | 1.5 | 4.5 | 21.2 | |
none scraped.