nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16
| benchmark | category | score | context | conf | evidence |
|---|---|---|---|---|---|
| QuALITY | knowledge | 100.0 | Quality retained (% of AR baseline) | medium | | Quality retained (% of AR baseline) | 100% | **98.7%** | |
| GSM8K | math | 92.49 | GSM8K (8-shot, acc) | medium | | GSM8K (8-shot, acc) | 92.49 | 90.14 | |
| ARC-Challenge | knowledge | 91.72 | ARC-Challenge (25-shot, acc_norm) | medium | | ARC-Challenge (25-shot, acc_norm) | 91.72 | 92.66 | |
| MATH-500 | math | 84.4 | MATH-500 (4-shot) | medium | | MATH-500 (4-shot) | 84.40 | 80.60 | |
| MGSM | math | 80.8 | MGSM (8-shot, avg acc) | medium | | MGSM (8-shot, avg acc) | 80.80 | 80.40 | |
| HumanEval | coding | 79.27 | HumanEval (0-shot) | medium | | HumanEval (0-shot) | 79.27 | 75.58 | |
| MMLU | knowledge | 78.56 | MMLU (5-shot, acc) | medium | | MMLU (5-shot, acc) | 78.56 | 78.24 | |
| WinoGrande | knowledge | 76.09 | WinoGrande (5-shot, acc) | medium | | WinoGrande (5-shot, acc) | 76.09 | 76.09 | |
| MBPP | coding | 74.71 | MBPP-Sanitized (3-shot) | medium | | MBPP-Sanitized (3-shot) | 74.71 | 74.28 | |
| MMLU | knowledge | 73.97 | MMLU Global Lite (5-shot, avg acc) | medium | | MMLU Global Lite (5-shot, avg acc) | 73.97 | 73.94 | |
| MMLU-Pro | knowledge | 62.59 | MMLU-Pro (5-shot, CoT EM) | medium | | MMLU-Pro (5-shot, CoT EM) | 62.59 | 60.93 | |
none scraped.