HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks

gemma-3-1b-it

google/gemma-3-1b-it

kind
original
quant
bf16 (16-bit)
family
gemma
modality
text-generation
root model
google/gemma-3-1b-pt
provider
google
license
gemma
language
—
context
—
library
transformers
vRAM (GB)
—
size on disk
—
downloads
3,187,943
likes
1,067
created
—
last modified
2025-04-04T13:12:40+00:00 (2025)
huggingface
https://huggingface.co/google/gemma-3-1b-it
tags
transformerssafetensorsgemma3_texttext-generationconversationalarxiv:1905.07830arxiv:1905.10044arxiv:1911.11641arxiv:1904.09728arxiv:1705.03551arxiv:1911.01547arxiv:1907.10641arxiv:1903.00161arxiv:2009.03300arxiv:2304.06364arxiv:2103.03874arxiv:2110.14168arxiv:2311.12022arxiv:2108.07732arxiv:2107.03374arxiv:2210.03057arxiv:2106.03193arxiv:1910.11856arxiv:2502.12404

Benchmark scores (20)

benchmarkcategoryscorecontextconfevidence
PIQAknowledge73.8[PIQA][]medium| [PIQA][piqa] | 0-shot | 73.8 | 79.6 | 81.8 | 83.3 |
DocVQAmultimodal72.8[DocVQA][] (val)medium| [DocVQA][docvqa] (val) | 72.8 | 82.3 | 85.6 |
ChartQAmultimodal63.6[ChartQA][]medium| [ChartQA][chartqa] | 63.6 | 74.7 | 76.3 |
BoolQknowledge63.2[BoolQ][]medium| [BoolQ][boolq] | 0-shot | 63.2 | 72.3 | 78.8 | 82.4 |
HellaSwagknowledge62.3[HellaSwag][]medium| [HellaSwag][hellaswag] | 10-shot | 62.3 | 77.2 | 84.2 | 85.6 |
MMLUknowledge59.6[MMLU][]medium| [MMLU][mmlu] | 5-shot | 59.6 | 74.5 | 78.6 |
WinoGrandeknowledge58.2[WinoGrande][]medium| [WinoGrande][winogrande] | 5-shot | 58.2 | 64.7 | 74.3 | 78.8 |
MBPPcoding46.0[MBPP][]medium| [MBPP][mbpp] | 3-shot | 46.0 | 60.4 | 65.6 |
DROPreasoning42.4[DROP][]medium| [DROP][drop] | 1-shot | 42.4 | 60.1 | 72.2 | 77.2 |
AGIEvalreasoning42.1[AGIEval][]medium| [AGIEval][agieval] | 3-5-shot | 42.1 | 57.4 | 66.2 |
TriviaQAknowledge39.8[TriviaQA][]medium| [TriviaQA][triviaqa] | 5-shot | 39.8 | 65.8 | 78.2 | 85.5 |
MMMUmultimodal39.2[MMMU][] (pt)medium| [MMMU][mmmu] (pt) | 39.2 | 50.3 | 56.1 |
GSM8Kmath38.4[GSM8K][]medium| [GSM8K][gsm8k] | 8-shot | 38.4 | 71.0 | 82.6 |
HumanEvalcoding36.0[HumanEval][]medium| [HumanEval][humaneval] | 0-shot | 36.0 | 45.7 | 48.8 |
MMLUknowledge29.2[MMLU][] (Pro COT)medium| [MMLU][mmlu] (Pro COT) | 5-shot | 29.2 | 45.3 | 52.2 |
BBHreasoning28.4[BIG-Bench Hard][]medium| [BIG-Bench Hard][bbh] | few-shot | 28.4 | 50.9 | 72.6 | 77.7 |
MMLUknowledge24.9[Global-MMLU-Lite][global--lite]medium| [Global-MMLU-Lite][global-mmlu-lite] | 24.9 | 57.0 | 69.4 | 75.7 |
MATHmath24.2[MATH][]medium| [MATH][math] | 4-shot | 24.2 | 43.3 | 50.0 |
GPQAreasoning15.0[GPQA][]medium| [GPQA][gpqa] | 5-shot | 15.0 | 25.4 | 24.3 |
MGSMmath2.04[MGSM][]medium| [MGSM][mgsm] | 2.04 | 34.7 | 64.3 | 74.3 |

Discussions (45)

titleauthorstatuscommentstype
Update generation_config.jsonerickhjopen0disc
Gemma-3-1B runs beautifully on Snapdragon 8653morixdopen0disc
TemporalMesh Transformer: 29.4 PPL at 48% compute — dynamic graph attention + adaptive exit gates (open-source, 226 tests)vigneshwar234open0disc
Add EvalEval community eval resultsEvalEvalBotopen0disc
Update README.mdMew22closed0disc
Request: DOIMartaAguilarMorcilloopen0disc
Request: DOIBehnambagiopen0disc
Personalzak90zakarezohradraft0disc
Request: DOIzak90zakarezohraclosed0disc
Multilingual Translation?anshuKropen0disc
Request: DOIrabiulhassanopen0disc
GraiLLMCaesaropapismdraft0disc
Cannot load model or tokenizerIJSDerbashiclosed0disc
How to finetune this and make it work using mediapipe on kotlin with GPU delegate support like litert-community/Gemma3-1B-IT ?andromedaztopen0disc
Request: DOINajin06open0disc
Add chat_template.jsonraphaelmerxopen0disc
Is it able to select languages after training for generating text?DuongLeVanopen0disc
Get AutoProcessor workingmamousavidraft0disc
[Gemma-3-1B] Gibberish outputs after instruction fine-tuningrazumeloopen0disc
Honest Reviewkalashshah19open0disc
Update README.mdkrooneropen0disc
Request: DOIfrdfdopen0disc
TRAINING DATAamanpreet7open0disc
GemmaKairosEtpdraft0disc
OSError: ./gemma-3-1b-it does not appear to have a file named preprocessor_config.json.zjnylyclosed0disc
OSError: Can't load tokenizer for 'google/gemma-3-1b-it'.JeffMIIopen0disc
RuntimeError: value cannot be converted to type uint8_t without overflowTyJaJaopen0disc
Request: DOIS22-22open0disc
Why are vocab_size and tokenizer different length?choco9966open0disc
Update config.jsonGIronManclosed0disc
Update README.mdmervemerged0disc
GUI pythonsportcmaneiroopen0disc
Update README.mdGopiUpparimerged0disc
Serving on vLLM creates nonsense responsescahmetcanopen0disc
Remove development branch of transformersfarzadabopen0disc
Use the model as a sequence classifierA123hmedopen0disc
Transformers Pipeline Error: AttributeError: 'NoneType' object has no attribute 'apply_chat_template'steve122192open0disc
AttributeError: 'HybridCache' object has no attribute 'float'naruto-soopopen0disc
Remove processor class from tokenizer_config.jsonXenovaopen0disc
What transformers version can this be deployed with?Khalizoopen0disc
Update EOS tokenspcuenqmerged0disc
Update chat_templateXenovamerged0disc
Add code snippet, change pipeline tag and library namemervemerged0disc
Final conversion test for evaluationpcuenqclosed0disc
Final conversion test for evaluationpcuenqclosed0disc
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.