nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| ARC-Challenge | knowledge | 93.26 | ARC Challenge | medium | | ARC Challenge | **93.26** | | 90.70 | 93.09 | 90.44 | |
| GSM8K | math | 91.66 | GSM8K CoT | medium | | GSM8K CoT | **91.66** | | 91.36 | 84.00 | 74.45 | |
| RULER | long-context | 84.74 | RULER-128K | medium | | RULER-128K | **84.74** | | 82.22 | \- | 80.70 | |
| HellaSwag | knowledge | 84.0 | HellaSwag | medium | | HellaSwag | 84.00 | | 79.90 | 79.75 | **84.15** | |
| MATH | math | 83.54 | Math | medium | | Math | **83.54** | | 80.50 | 55.40 | 42.40 | |
| PIQA | knowledge | 82.54 | PIQA | medium | | PIQA | **82.54** | | 81.83 | 79.43 | 82.10 | |
| WinoGrande | knowledge | 79.24 | WinoGrande | medium | | WinoGrande | 79.24 | | 75.30 | 75.93 | **79.95** | |
| MMLU | knowledge | 78.24 | MMLU | medium | | MMLU | **78.24** | | 74.53 | 76.44 | 73.61 | |
| AGIEval | reasoning | 68.03 | AGIEval English CoT | medium | | AGIEval English CoT | **68.03** | | 65.28 | 59.54 | 51.69 | |
| MATH | math | 67.61 | MATH Level 5 | medium | | MATH Level 5 | **67.61** | | **63.64** | 29.91 | 17.71 | |
| MMLU-Pro | knowledge | 63.98 | MMLU-Pro 5-shot | medium | | MMLU-Pro 5-shot | **63.98** | | 59.43 | 56.27 | 45.12 | |
| MBPP | coding | 61.55 | MBPP+ Pass@1 | medium | | MBPP+ Pass@1 | **61.55** | | 58.95 | 58.56 | 51.73 | |
| HumanEval | coding | 61.03 | HumanEval+ Pass@1 | medium | | HumanEval+ Pass@1 | **61.03** | | 58.50 | 57.55 | 36.68 | |
| AIME 2024 | math | 56.67 | AIME 2024 avg@32 | medium | | AIME 2024 avg@32 | **56.67** | | 30.00 | 20.00 | 16.67 | |
| OpenBookQA | knowledge | 46.0 | OpenBookQA | medium | | OpenBookQA | **46.00** | | 44.80 | 42.00 | **46.00** | |
none scraped.