google/gemma-3-270m-qat-q4_0-unquantized
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| IFEval | instruction-following | 80.2 | [IFEval][ifeval] | medium | | [IFEval][ifeval] | 0-shot | 80.2 | 90.2 | 88.9 | 90.4 | |
| DocVQA | multimodal | 75.8 | [DocVQA][] | medium | | [DocVQA][docvqa] | 75.8 | 87.1 | 86.6 | |
| DocVQA | multimodal | 72.8 | [DocVQA][] (val) | medium | | [DocVQA][docvqa] (val) | 72.8 | 82.3 | 85.6 | |
| ChartQA | multimodal | 68.8 | [ChartQA][] | medium | | [ChartQA][chartqa] | 68.8 | 75.7 | 78.0 | |
| PIQA | knowledge | 67.7 | [PIQA][] | medium | | [PIQA][piqa] | 0-shot | 67.7 | |
| GSM8K | math | 62.8 | [GSM8K][] | medium | | [GSM8K][gsm8k] | 0-shot | 62.8 | 89.2 | 94.4 | 95.9 | |
| BoolQ | knowledge | 61.4 | [BoolQ][] | medium | | [BoolQ][boolq] | 0-shot | 61.4 | |
| MMLU | knowledge | 59.6 | [MMLU][] | medium | | [MMLU][mmlu] | 5-shot | 59.6 | 74.5 | 78.6 | |
| WinoGrande | knowledge | 52.0 | [WinoGrande][] | medium | | [WinoGrande][winogrande] | 5-shot | 52.0 | |
| IFEval | instruction-following | 51.2 | [IF Eval][ifeval] | medium | | [IF Eval][ifeval] | 0-shot | 51.2 | |
| MathVista | multimodal | 50.0 | [MathVista][] (testmini) | medium | | [MathVista][mathvista] (testmini) | 50.0 | 62.9 | 67.6 | |
| MMMU | multimodal | 48.8 | [MMMU][] (val) | medium | | [MMMU][mmmu] (val) | 48.8 | 59.6 | 64.9 | |
| MATH | math | 48.0 | [Math][] | medium | | [Math][math] | 0-shot | 48.0 | 75.6 | 83.8 | 89.0 | |
| DROP | reasoning | 42.4 | [DROP][] | medium | | [DROP][drop] | 1-shot | 42.4 | 60.1 | 72.2 | 77.2 | |
| AGIEval | reasoning | 42.1 | [AGIEval][] | medium | | [AGIEval][agieval] | 3-5-shot | 42.1 | 57.4 | 66.2 | |
| HumanEval | coding | 41.5 | [HumanEval][] | medium | | [HumanEval][humaneval] | 0-shot | 41.5 | 71.3 | 85.4 | 87.8 | |
| HellaSwag | knowledge | 40.9 | [HellaSwag][] | medium | | [HellaSwag][hellaswag] | 10-shot | 40.9 | |
| MMMU | multimodal | 39.2 | [MMMU][] (pt) | medium | | [MMMU][mmmu] (pt) | 39.2 | 50.3 | 56.1 | |
| MBPP | coding | 35.2 | [MBPP][] | medium | | [MBPP][mbpp] | 3-shot | 35.2 | 63.2 | 73.0 | 74.4 | |
| MMLU | knowledge | 34.2 | [Global-MMLU-Lite][global--lite] | medium | | [Global-MMLU-Lite][global-mmlu-lite] | 0-shot | 34.2 | 54.5 | 69.5 | 75.1 | |
| MMLU | knowledge | 29.2 | [MMLU][] (Pro COT) | medium | | [MMLU][mmlu] (Pro COT) | 5-shot | 29.2 | 45.3 | 52.2 | |
| BBH | reasoning | 26.7 | [BIG-Bench Hard][] | medium | | [BIG-Bench Hard][bbh] | few-shot | 26.7 | |
| MATH | math | 24.2 | [MATH][] | medium | | [MATH][math] | 4-shot | 24.2 | 43.3 | 50.0 | |
| GPQA | reasoning | 19.2 | [GPQA][] Diamond | medium | | [GPQA][gpqa] Diamond | 0-shot | 19.2 | 30.8 | 40.9 | 42.4 | |
| TriviaQA | knowledge | 15.4 | [TriviaQA][] | medium | | [TriviaQA][triviaqa] | 5-shot | 15.4 | |
| GPQA | reasoning | 15.0 | [GPQA][] | medium | | [GPQA][gpqa] | 5-shot | 15.0 | 25.4 | 24.3 | |
| MMLU | knowledge | 14.7 | [MMLU][] (Pro) | medium | | [MMLU][mmlu] (Pro) | 0-shot | 14.7 | 43.6 | 60.6 | 67.5 | |
| BigBench | reasoning | 7.2 | [BIG-Bench Extra Hard][bbeh] | medium | | [BIG-Bench Extra Hard][bbeh] | 0-shot | 7.2 | 11.0 | 16.3 | 19.3 | |
| SimpleQA | knowledge | 2.2 | [SimpleQA][] | medium | | [SimpleQA][simpleqa] | 0-shot | 2.2 | 4.0 | 6.3 | 10.0 | |
| MGSM | math | 2.04 | [MGSM][] | medium | | [MGSM][mgsm] | 2.04 | 34.7 | 64.3 | 74.3 | |
| LiveCodeBench | coding | 1.9 | [LiveCodeBench][lcb] | medium | | [LiveCodeBench][lcb] | 0-shot | 1.9 | 12.6 | 24.6 | 29.7 | |
none scraped.