HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks

gemma-4-12B-it

google/gemma-4-12B-it

kind
original
quant
bf16 (16-bit)
family
gemma
modality
any-to-any
root model
google/gemma-4-12B
provider
google
license
apache-2.0
language
—
context
128K
library
transformers
vRAM (GB)
—
size on disk
—
downloads
2,998,017
likes
1,358
created
—
last modified
2026-07-20T16:42:08+00:00 (2026)
huggingface
https://huggingface.co/google/gemma-4-12B-it
tags
transformerssafetensorsgemma4_unifiedimage-text-to-textany-to-anyarxiv:2607.02770base_model:google/gemma-4-12Bbase_model:finetune:google/gemma-4-12Blicense:apache-2.0eval-resultsendpoints_compatibleregion:us

Benchmark scores (10)

benchmarkcategoryscorecontextconfevidence
AIME 2026math89.2AIME 2026 no toolsmedium| AIME 2026 no tools | 89.2% | 88.3% | 77.5% | 42.5% | 37.5% | 20.8% |
MATHmath85.6MATH-Visionmedium| MATH-Vision | 85.6% | 82.4% | 79.7% | 59.5% | 52.4% | 46.0% |
MMLU-Proknowledge85.2MMLU Promedium| MMLU Pro | 85.2% | 82.6% | 77.2% | 69.4% | 60.0% | 67.6% |
GPQA Diamondreasoning84.3GPQA Diamondmedium| GPQA Diamond | 84.3% | 82.3% | 78.8% | 58.6% | 43.4% | 42.4% |
LiveCodeBenchcoding80.0LiveCodeBench v6medium| LiveCodeBench v6 | 80.0% | 77.1% | 72.0% | 52.0% | 44.0% | 29.1% |
MMMUmultimodal76.9MMMU Promedium| MMMU Pro | 76.9% | 73.8% | 69.1% | 52.6% | 44.2% | 49.7% |
BigBenchreasoning74.4BigBench Extra Hardmedium| BigBench Extra Hard | 74.4% | 64.8% | 53.0% | 33.1% | 21.9% | 19.3% |
Humanity's Last Examknowledge26.5HLE with searchmedium| HLE with search | 26.5% | 17.2% | - | - | - | - |
Humanity's Last Examknowledge19.5HLE no toolsmedium| HLE no tools | 19.5% | 8.7% | 5.2% | - | - | - |
WER FLEURSspeech-asr8.0FLEURS (lower is better)medium| FLEURS (lower is better) | - | - | 0.069<sup>*</sup> | 0.08 | 0.09 | - |

Discussions (48)

titleauthorstatuscommentstype
Demo for this model on Spacesmultimodalartclosed0disc
Update READMEMaartenGrmerged0disc
Raise per-image vision soft-token budget from 280 to 1120lucianommartinsopen0disc
Gemma 4 12B Unified (encoder-free audio): audio comprehension collapses past ~450–500 tokens of absolute position — vision on the same checkpoint is unaffected, E2B/E4B (Conformer) are unaffectedfrandagostinoopen0disc
SAE for this model?joggybackopen0disc
Add response_template to tokenizer_config.jsonRocketknight1merged0disc
https://share.gemini.google/7T1rHMOMMfPHDIGITALLEADERSHIPopen0disc
gemma-4-12B-it: deterministic "thought\n thought\n …" degenerate loop on long agent prompts (~60% reproducer, 4-bit, repros across temperatures)Raullenopen0disc
Fairly Uselessyano2mchopen0disc
Gemma 4 12b bias affraid meAlektoopen0disc
Chat template may re-inject prior-turn reasoning during multi-turn tool use → repetition loopsManniX-ITAopen0disc
fdsfsdfsdfdfjiwang1open0disc
asfdjiwang1open0disc
fix: chat template — null handling, reasoning preservation, turn-tag balance, input validationlucianommartinsmerged0disc
Gemma 4 124b, please!slavczaysopen0disc
prompt templatemaymoclosed0disc
vllm support?mohamedemamopen0disc
gemma 4 12b falsesly claims yearTralalabsopen0disc
Gemma4-12B Audio Deep EvaluationPeterchanCNopen0disc
How to use an assistent model for MTP in llama.cpp?Regrinopen0disc
Tool Calling and Tags and Context awareness.JanjanJeanopen0disc
MLX 4-bit conversion of gemma-4-12B-it — with vision + audio embedder weights preservedjokerniftyopen0disc
Gemma 4 124B DENSE + Nvidia N1x Laptop87455aopen0disc
What languages are supported in audio input?J22open0disc
it is speaking Cantonese while user prompt is Simplified Chineseleiyang88open0disc
Can't deploy on huggingface...p-christopen0disc
NVFP4 Quantberkerdoooopen0disc
Update max_position_embeddings to full context lengthdougreidmerged0disc
👋 Training support for transformers/megatron backendsstudy-hjtopen0disc
NVFP4 / FP8 Quantizationsvincentzed-hfopen0disc
Dimension mismatch crashes the engine at startupCallShaulopen0disc
Gemma 4 124bFusionCowopen0disc
Thank you Google teamjreokaopen0disc
Thanks again!JanjanJeanopen0disc
MTP Assistant Modelpatrickdeanbrownopen0disc
I've been having a bunch of issues trying to do translations from Chinese to Englishunlambdaclosed0disc
Preserve the correct ordering of content and tool calls when rendering OpenAI Chat Completionsyzong-rhopen0disc
fix max_position_embeddings 128Ki --> 256Kiddh0closed0disc
Wrong context size in config.json?coder543closed0disc
Good, Googledof-studio-orgopen0disc
File path upper case issuesleepyeldraziclosed0disc
Add HLE evaluation resultSaylorTwiftopen0disc
Add AIME 2026 evaluation resultSaylorTwiftopen0disc
Add MMMU Pro evaluation resultSaylorTwiftopen0disc
Add MMLU-Pro evaluation resultSaylorTwiftopen0disc
Add GPQA Diamond evaluation resultSaylorTwiftopen0disc
This is an awesome model proposition/idea for users with few VRAM.nesymerp1open0disc
Gemma 4:124bseamon67open0disc
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.