HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks

NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16

nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16

kind
original
quant
bf16 (16-bit)
family
nemotron
modality
text-generation
root model
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16
provider
nvidia
license
other
language
en, fr, es, it, de, pt, ja, ko, hi, ar, zh, he
context
1M
library
transformers
vRAM (GB)
—
size on disk
—
downloads
801
likes
29
created
—
last modified
2026-06-05T17:57:11+00:00 (2026)
huggingface
https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16
tags
transformerssafetensorsnemotron_htext-generationnvidiapytorchnemotron-3latent-moemtpenfresitdeptjakohiarzhhelicense:otherendpoints_compatibleregion:us

Benchmark scores (21)

benchmarkcategoryscorecontextconfevidence
ARC-Challengeknowledge97.35ARC-Challengemedium| ARC-Challenge | *25-shot, acc_norm* | **97.35** | 95.22 | 97.27 | 95.82 | 96.59 |
RULERlong-context95.3RULER 64Kmedium| RULER 64K | *0-shot* | **95.30** | 93.30 | 90.11 | 93.79 | 16.12 |
RULERlong-context92.49RULER 128Kmedium| RULER 128K | *0-shot* | **92.49** | 91.88 | 55.77 | 88.61 | 0.00 |
HellaSwagknowledge90.51HellaSwagmedium| HellaSwag | *10-shot, acc_norm* | 90.51 | 89.44 | 88.88 | **90.92** | 90.17 |
MMLUknowledge90.13MMLU Global Litemedium| MMLU Global Lite | *5-shot, avg* | **90.13** | 85.59 | 87.34 | 85.63 | 85.81 |
MMLUknowledge89.08MMLUmedium| MMLU | *5-shot, acc* | **89.08** | 87.82 | 87.35 | 87.60 | 86.50 |
GSM8Kmath88.1GSM8Kmedium| GSM8K | *8-shot, CoT EM* | 88.10 | 84.38 | **91.21** | 91.05 | 85.37 |
MGSMmath87.73MGSMmedium| MGSM | *8-shot, native CoT avg* | **87.73** | 82.33 | 82.93 | 85.20 | 81.27 |
RULERlong-context86.22RULER 256Kmedium| RULER 256K | *0-shot* | **86.22** | -- | 35.50 | -- | -- |
MBPPcoding85.97MBPP-Sanitizedmedium| MBPP-Sanitized | *3-shot pass@1 n=32, EvalPlus sanitized* | **85.97** | 58.66 | 84.08 | 72.14 | 76.69 |
RULERlong-context84.54RULER 512Kmedium| RULER 512K | *0-shot* | **84.54** | -- | -- | -- | -- |
HumanEvalcoding83.84HumanEvalmedium| HumanEval | *sampled pass@1 n=32, EvalPlus sanitized* | **83.84** | 61.85 | 66.71 | 78.20 | 78.16 |
PIQAknowledge83.79PIQAmedium| PIQA | *0-shot, acc_norm* | 83.79 | 85.09 | 84.82 | **85.47** | 85.09 |
MATHmath82.0MATHmedium| MATH | *4-shot, EM* | **82.00** | 60.12 | 62.88 | 68.40 | 57.58 |
WinoGrandeknowledge79.32WinoGrandemedium| WinoGrande | *5-shot, acc* | 79.32 | 83.43 | 82.08 | 84.21 | **85.24** |
MMLU-Proknowledge79.07MMLU-Promedium| MMLU-Pro | *5-shot, CoT EM* | **79.07** | 63.26 | 67.42 | 69.15 | 65.78 |
AGIEvalreasoning78.73AGIEval-Enmedium| AGIEval-En | *3/5-shot, CoT EM* | **78.73** | 70.13 | 69.30 | 72.55 | 70.06 |
RULERlong-context76.83RULER 1Mmedium| RULER 1M | *0-shot* | **76.83** | -- | -- | -- | -- |
RULERlong-context64.0mediumRULER 64
GPQAreasoning50.0GPQAmedium| GPQA | *5-shot, CoT EM* | **50.00** | 31.82 | 34.85 | 43.43 | 34.85 |
OpenBookQAknowledge48.6OpenBookQAmedium| OpenBookQA | *0-shot, acc_norm* | 48.60 | 48.20 | **51.40** | 50.80 | 49.60 |

Discussions (0)

none scraped.

Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.