HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks

Qwen3-4B-Instruct-2507

Qwen/Qwen3-4B-Instruct-2507

kind
original
quant
bf16 (16-bit)
family
qwen
modality
text-generation
root model
Qwen/Qwen3-4B-Instruct-2507
provider
Qwen
license
apache-2.0
language
—
context
262,144
library
transformers
vRAM (GB)
—
size on disk
—
downloads
3,506,956
likes
905
created
—
last modified
2025-09-17T06:56:53+00:00 (2025)
huggingface
https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
tags
transformerssafetensorsqwen3text-generationconversationalarxiv:2505.09388license:apache-2.0eval-resultstext-generation-inferenceendpoints_compatibledeploy:azureregion:usdeploy:sagemaker

Benchmark scores (9)

benchmarkcategoryscorecontextconfevidence
MMLUknowledge80.2MMLU-Reduxmedium| MMLU-Redux | 80.2 | 84.1 | 77.3 | **84.2** |
IFEvalinstruction-following74.5IFEvalmedium| IFEval | 74.5 | **83.7** | 81.2 | 83.4 |
MMLU-Proknowledge62.8MMLU-Promedium| MMLU-Pro | 62.8 | 69.1 | 58.0 | **69.6** |
MMLU-Proknowledge56.2MMLU-ProXmedium| MMLU-ProX | 56.2 | **65.1** | 49.6 | 61.6 |
BFCL (function calling)agentic53.0BFCL-v3medium| BFCL-v3 | 53.0 | 58.6 | 57.6 | **61.9** |
GPQAreasoning50.3GPQAmedium| GPQA | 50.3 | 54.8 | 41.7 | **62.0** |
LiveCodeBenchcoding31.5LiveCodeBench v6 (25.02-25.05)medium| LiveCodeBench v6 (25.02-25.05) | 31.5 | 29.0 | 26.4 | **35.1** |
AIME 2025math22.7AIME25medium| AIME25 | 22.7 | 21.6 | 19.1 | **47.4** |
Chatbot Arenachat15.9Arena-Hard v2*medium| Arena-Hard v2* | 15.9 | 24.8 | 9.5 | **43.4** |

Discussions (29)

titleauthorstatuscommentstype
Qwen3 4B Instruct 2507 — the best mobile model of 2026?3morixdopen0disc
Listed on OpenModelMapduola15open0disc
TemporalMesh Transformer: 29.4 PPL at 48% compute — beats Mamba, new open-source architecturevigneshwar234open0disc
Opened by mistake, closing this PR.Jennnyclosed0disc
Opened by mistake, closing this PR.Jennnyclosed0disc
Add MMLU-Pro evaluation resultburtenshawopen0disc
Update README.mdVishva007closed0disc
Add GPQA evaluation resultburtenshawopen0disc
注意:此模型仅支持非思考模式,不会在其输出中生成 <think></think> 块。Jay-v2open0disc
Safety Audit: GAE Score 40.13% (FAIL)GAE-Auditordraft0disc
Add 12 evaluation results to model-indexburtenshawclosed0disc
Discrepancy in benchmark score (BFCL-v3)mmrbulbulopen0disc
The model doesn't know about itselfsakazakiMGJopen0disc
Not working with structured jsonshubhamapolloclosed0disc
vRAM needed ?Ashish18110open0disc
Please support vietnamese more and more in the models.DuongLeVanopen0disc
Adding mention of Tinker and TRL supportclemopen0disc
<tool_call> generated even with no tools or asked fordipta007open0disc
Release training token statsjquessadaopen0disc
Remove `<think></think>` blocks from chat templatemamousavidraft0disc
No other 2507 modelsSipOfSpikeopen0disc
Sampling parameters to tau2-bench?lewtunopen0disc
1.7b 2507?CHNtentesopen0disc
Training completed @ 2025-08-09 11:36:54enziiclosed0disc
Upload folder using huggingface_hubenziiclosed0disc
Why is <think></think> required in history messages?giangndmopen0disc
Terrible instruction followingdenisalpinoopen0disc
4b model with an 84.2 MMLU-Redux score?phil111open0disc
when 32B?AaronFeng753open0disc
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.