nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| ARC-Challenge | knowledge | 94.45 | ARC-Challenge (25-shot, acc_norm) | medium | | ARC-Challenge (25-shot, acc_norm) | **94.45** | 91.89 | |
| GSM8K | math | 89.01 | GSM8K (8-shot, acc) | medium | | GSM8K (8-shot, acc) | 89.01 | **92.34** | |
| HellaSwag | knowledge | 83.14 | HellaSwag (10-shot, acc_norm) | medium | | HellaSwag (10-shot, acc_norm) | 83.14 | **85.56** | |
| MGSM | math | 82.53 | MGSM (8-shot, avg acc) | medium | | MGSM (8-shot, avg acc) | 82.53 | **83.00** | |
| MMLU | knowledge | 81.07 | MMLU (5-shot, acc) | medium | | MMLU (5-shot, acc) | **81.07** | 78.56 | |
| PIQA | knowledge | 81.01 | PIQA (0-shot, acc_norm) | medium | | PIQA (0-shot, acc_norm) | 81.01 | **84.33** | |
| WinoGrande | knowledge | 78.22 | WinoGrande (5-shot, acc) | medium | | WinoGrande (5-shot, acc) | 78.22 | **79.64** | |
| MMLU | knowledge | 76.84 | MMLU Global Lite (5-shot, avg acc) | medium | | MMLU Global Lite (5-shot, avg acc) | **76.84** | 74.47 | |
| RULER | long-context | 75.44 | RULER (256K, 0-shot, acc) | medium | | RULER (256K, 0-shot, acc) | Not Supported | **75.44** | |
| MBPP | coding | 73.15 | MBPP-Sanitized (3-shot) | medium | | MBPP-Sanitized (3-shot) | 73.15 | **75.49** | |
| HumanEval | coding | 70.73 | HumanEval (0-shot) | medium | | HumanEval (0-shot) | 70.73 | **78.05** | |
| RULER | long-context | 70.56 | RULER (512K, 0-shot, acc) | medium | | RULER (512K, 0-shot, acc) | Not Supported | **70.56** | |
| RULER | long-context | 63.55 | RULER (64K, 0-shot, acc) | medium | | RULER (64K, 0-shot, acc) | 63.55 | **87.50** | |
| AGIEval | reasoning | 63.12 | AGIEval-En (3/5-shot, CoT acc) | medium | | AGIEval-En (3/5-shot, CoT acc) | 63.12 | **68.32** | |
| MMLU-Pro | knowledge | 61.71 | MMLU-Pro (5-shot, CoT EM) | medium | | MMLU-Pro (5-shot, CoT EM) | 61.71 | **65.05** | |
| MATH | math | 61.14 | MATH (4-shot, acc) | medium | | MATH (4-shot, acc) | 61.14 | **82.88** | |
| RULER | long-context | 60.69 | RULER (128K, 0-shot, acc) | medium | | RULER (128K, 0-shot, acc) | 60.69 | **82.92** | |
| MATH-500 | math | 55.08 | MATH-500 (4-shot, avg@32) | medium | | MATH-500 (4-shot, avg@32) | 55.08 | **78.63** | |
| OpenBookQA | knowledge | 44.8 | OpenBookQA (0-shot, acc_norm) | medium | | OpenBookQA (0-shot, acc_norm) | 44.80 | **46.20** | |
none scraped.