deepseek-ai/DeepSeek-V4-Flash
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| GPQA Diamond | reasoning | 91.3 | GPQA Diamond (Pass@1) | medium | | GPQA Diamond (Pass@1) | 91.3 | 93.0 | **94.3** | 90.5 | 86.2 | 90.1 | |
| GSM8K | math | 91.1 | GSM8K (EM) | medium | | GSM8K (EM) | 8-shot | 91.1 | 90.8 | **92.6** | |
| C-Eval | knowledge | 90.4 | C-Eval (EM) | medium | | C-Eval (EM) | 5-shot | 90.4 | 92.1 | **93.1** | |
| CMMLU | knowledge | 88.9 | CMMLU (EM) | medium | | CMMLU (EM) | 5-shot | 88.9 | 90.4 | **90.8** | |
| LiveCodeBench | coding | 88.8 | LiveCodeBench (Pass@1) | medium | | LiveCodeBench (Pass@1) | 88.8 | - | 91.7 | 89.6 | - | **93.5** | |
| DROP | reasoning | 88.2 | DROP (F1) | medium | | DROP (F1) | 1-shot | 88.2 | 88.6 | **88.7** | |
| MMLU | knowledge | 87.8 | MMLU (EM) | medium | | MMLU (EM) | 5-shot | 87.8 | 88.7 | **90.1** | |
| BBH | reasoning | 87.6 | BBH (EM) | medium | | BBH (EM) | 3-shot | **87.6** | 86.9 | 87.5 | |
| MMLU | knowledge | 87.5 | MMLU-Redux (EM) | medium | | MMLU-Redux (EM) | 5-shot | 87.5 | 89.4 | **90.8** | |
| HellaSwag | knowledge | 86.4 | HellaSwag (EM) | medium | | HellaSwag (EM) | 0-shot | 86.4 | 85.7 | **88.0** | |
| BrowseComp | agentic | 83.7 | BrowseComp (Pass@1) | medium | | BrowseComp (Pass@1) | 83.7 | 82.7 | **85.9** | 83.2 | 79.3 | 83.4 | |
| TriviaQA | knowledge | 83.3 | TriviaQA (EM) | medium | | TriviaQA (EM) | 5-shot | 83.3 | 82.8 | **85.6** | |
| MGSM | math | 81.3 | MGSM (EM) | medium | | MGSM (EM) | 8-shot | 81.3 | **85.7** | 84.4 | |
| AGIEval | reasoning | 80.1 | AGIEval (EM) | medium | | AGIEval (EM) | 0-shot | 80.1 | 82.6 | **83.1** | |
| WinoGrande | knowledge | 78.9 | WinoGrande (EM) | medium | | WinoGrande (EM) | 0-shot | 78.9 | 79.5 | **81.5** | |
| SimpleQA | knowledge | 76.4 | Chinese-SimpleQA (Pass@1) | medium | | Chinese-SimpleQA (Pass@1) | 76.4 | 76.8 | **85.9** | 75.9 | 75.0 | 84.4 | |
| MMLU-Pro | knowledge | 65.5 | MMLU-Pro (EM) | medium | | MMLU-Pro (EM) | 5-shot | 65.5 | 68.3 | **73.5** | |
| Terminal-Bench 2.0 | agentic-coding | 65.4 | Terminal Bench 2.0 (Acc) | medium | | Terminal Bench 2.0 (Acc) | 65.4 | **75.1** | 68.5 | 66.7 | 63.5 | 67.9 | |
| BigCodeBench | coding | 63.9 | BigCodeBench (Pass@1) | medium | | BigCodeBench (Pass@1) | 3-shot | **63.9** | 56.8 | 59.2 | |
| HumanEval | coding | 62.8 | HumanEval (Pass@1) | medium | | HumanEval (Pass@1) | 0-shot | 62.8 | 69.5 | **76.8** | |
| MATH | math | 60.5 | MATH (EM) | medium | | MATH (EM) | 4-shot | 60.5 | 57.4 | **64.5** | |
| Humanity's Last Exam | knowledge | 53.1 | HLE w/ tools (Pass@1) | medium | | HLE w/ tools (Pass@1) | 53.1 | 52.0 | 51.6 | **54.0** | 50.4 | 48.2 | |
| SimpleQA | knowledge | 46.2 | SimpleQA-Verified (Pass@1) | medium | | SimpleQA-Verified (Pass@1) | 46.2 | 45.3 | **75.6** | 36.9 | 38.1 | 57.9 | |
| LongBench | long-context | 40.2 | LongBench-V2 (EM) | medium | | LongBench-V2 (EM) | 1-shot | 40.2 | 44.7 | **51.5** | |
| Humanity's Last Exam | knowledge | 40.0 | HLE (Pass@1) | medium | | HLE (Pass@1) | 40.0 | 39.8 | **44.4** | 36.4 | 34.7 | 37.7 | |
| SimpleQA | knowledge | 28.3 | Simple-QA verified (EM) | medium | | Simple-QA verified (EM) | 25-shot | 28.3 | 30.1 | **55.2** | |
| Terminal-Bench | agentic-coding | 2.0 | medium | Terminal Bench 2.0 |