HF Model & Benchmark Matrix

24,078 variants · 14,984 scores · 120 benchmarks
Home/Benchmarks/RULER

RULER

long-context · ruler · 109 scores

https://huggingface.co/spaces/ruler

rankvariantrootorgscoreconfcontext
1NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.79mediumRULER-500 @ 128k (500 samples per task)
2NVIDIA-Nemotron-3-Super-120B-A12B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.79mediumRULER-500 @ 128k (500 samples per task)
3NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.79mediumRULER-500 @ 128k (500 samples per task)
4NVIDIA-Nemotron-3-Super-120B-A12B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.79mediumRULER-500 @ 128k (500 samples per task)
5NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.6mediumRULER-500 @ 256k (500 samples per task)
6NVIDIA-Nemotron-3-Super-120B-A12B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.6mediumRULER-500 @ 256k (500 samples per task)
7NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.6mediumRULER-500 @ 256k (500 samples per task)
8NVIDIA-Nemotron-3-Super-120B-A12B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.6mediumRULER-500 @ 256k (500 samples per task)
9NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.3mediumRULER @ 256k
10NVIDIA-Nemotron-3-Super-120B-A12B-GGUFnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.3mediumRULER @ 256k
11NVIDIA-Nemotron-3-Super-120B-A12B-BF16-GGUFnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16Mungert96.3mediumRULER @ 256k
12NVIDIA-Nemotron-3-Super-120B-A12Bnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.3mediumRULER @ 256k
13NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.09mediumRULER-500 @ 512k (500 samples per task)
14NVIDIA-Nemotron-3-Super-120B-A12B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia96.09mediumRULER-500 @ 512k (500 samples per task)
15NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.09mediumRULER-500 @ 512k (500 samples per task)
16NVIDIA-Nemotron-3-Super-120B-A12B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth96.09mediumRULER-500 @ 512k (500 samples per task)
17NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia95.67mediumRULER @ 512k
18NVIDIA-Nemotron-3-Super-120B-A12B-GGUFnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth95.67mediumRULER @ 512k
19NVIDIA-Nemotron-3-Super-120B-A12B-BF16-GGUFnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16Mungert95.67mediumRULER @ 512k
20NVIDIA-Nemotron-3-Super-120B-A12Bnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth95.67mediumRULER @ 512k
21NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia95.3mediumRULER 64K
22NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia95.1mediumRULER @ 256k
23NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia95.1mediumRULER @ 256k
24NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia95.1mediumRULER @ 256k
25NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16nvidia94.7mediumRULER (1M)
26NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16nvidia94.7mediumRULER 1M
27NVIDIA-Nemotron-3-Ultra-550B-A55B-GGUFnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16unsloth94.7mediumRULER (1M)
28NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16unsloth94.7mediumRULER (1M)
29NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia94.2mediumRULER @ 512k
30NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia94.2mediumRULER @ 512k
31NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia94.2mediumRULER @ 512k
32Mistral-Small-3.1-24B-Instruct-2503mistralai/Mistral-Small-3.1-24B-Base-2503mistralai93.96lowRULER 32K | RULER 128K
33Mistral-Small-3.1-24B-Instruct-2503-GGUFmistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth93.96lowRULER 32K | RULER 128K
34Mistral-Small-3.1-24B-Instruct-2503-bnb-4bitmistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth93.96lowRULER 32K | RULER 128K
35Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bitmistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth93.96lowRULER 32K | RULER 128K
36Mistral-Small-3.1-24B-Instruct-2503mistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth93.96lowRULER 32K | RULER 128K
37Mistral-Small-3.1-24B-Instruct-2503-GGUFmistralai/Mistral-Small-3.1-24B-Base-2503Mungert93.96lowRULER 32K | RULER 128K
38NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia93.17mediumRULER 64K
39NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia92.9mediumRULER-100@256k
40Nemotron-3-Nano-30B-A3B-GGUFnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16unsloth92.9mediumRULER-100@256k
41Nemotron-3-Nano-30B-A3Bnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16unsloth92.9mediumRULER-100@256k
42NVIDIA-Nemotron-3-Nano-30B-A3B-BF16-GGUFnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Mungert92.9mediumRULER-100@256k
43NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia92.49mediumRULER 128K
44NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia92.2mediumRULER @ 1M
45NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-FP8nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia92.2mediumRULER @ 1M
46NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia92.2mediumRULER @ 1M
47NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16nvidia91.75mediumRULER @ 1M
48NVIDIA-Nemotron-3-Super-120B-A12B-GGUFnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth91.75mediumRULER @ 1M
49NVIDIA-Nemotron-3-Super-120B-A12B-BF16-GGUFnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16Mungert91.75mediumRULER @ 1M
50NVIDIA-Nemotron-3-Super-120B-A12Bnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16unsloth91.75mediumRULER @ 1M
51NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia91.3mediumRULER-100@512k
52Nemotron-3-Nano-30B-A3B-GGUFnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16unsloth91.3mediumRULER-100@512k
53Nemotron-3-Nano-30B-A3Bnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16unsloth91.3mediumRULER-100@512k
54NVIDIA-Nemotron-3-Nano-30B-A3B-BF16-GGUFnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Mungert91.3mediumRULER-100@512k
55NVIDIA-Nemotron-3-Nano-4B-BF16nvidia/NVIDIA-Nemotron-Nano-9B-v2nvidia91.1mediumRULER (128k)
56NVIDIA-Nemotron-3-Nano-4B-FP8nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16nvidia91.1mediumRULER (128k)
57NVIDIA-Nemotron-3-Nano-4B-GGUFnvidia/NVIDIA-Nemotron-3-Nano-4B-FP8nvidia91.1mediumRULER (128k)
58NVIDIA-Nemotron-3-Nano-4Bnvidia/NVIDIA-Nemotron-3-Nano-4B-BF16unsloth91.1mediumRULER (128k)
59NVIDIA-Nemotron-3-Nano-4B-GGUFnvidia/NVIDIA-Nemotron-3-Nano-4B-BF16unsloth91.1mediumRULER (128k)
60NVIDIA-Nemotron-3-Nano-4B-FP8nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16unsloth91.1mediumRULER (128k)
61NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia89.0mediumRULER 128K
62NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia86.3mediumRULER-100@1M
63Nemotron-3-Nano-30B-A3B-GGUFnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16unsloth86.3mediumRULER-100@1M
64Nemotron-3-Nano-30B-A3Bnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16unsloth86.3mediumRULER-100@1M
65NVIDIA-Nemotron-3-Nano-30B-A3B-BF16-GGUFnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Mungert86.3mediumRULER-100@1M
66NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia86.22mediumRULER 256K
67NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia86.18mediumRULER 256K
68NVIDIA-Nemotron-Nano-9B-v2-Basenvidia/NVIDIA-Nemotron-Nano-9B-v2-Basenvidia84.74mediumRULER-128K
69NVIDIA-Nemotron-Nano-12B-v2-Basenvidia/NVIDIA-Nemotron-Nano-12B-v2-Basenvidia84.74mediumRULER-128K
70NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia84.54mediumRULER 512K
71Llama-3_1-Nemotron-Ultra-253B-CPT-v1nvidia/Llama-3_1-Nemotron-Ultra-253B-CPT-v1nvidia83.21mediumRULER
72NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia82.16mediumRULER 512K
73NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia76.83mediumRULER 1M
74NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia75.44mediumRULER (256K, 0-shot, acc)
75NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia74.39mediumRULER 1M
76NVIDIA-Nemotron-Nano-9B-v2nvidia/NVIDIA-Nemotron-Nano-12B-v2-Basenvidia74.1mediumRULER (128K)
77NVIDIA-Nemotron-Nano-9B-v2-FP8nvidia/NVIDIA-Nemotron-Nano-9B-v2nvidia74.1mediumRULER (128K)
78NVIDIA-Nemotron-Nano-9B-v2-NVFP4nvidia/NVIDIA-Nemotron-Nano-9B-v2nvidia74.1mediumRULER (128K)
79NVIDIA-Nemotron-Nano-9B-v2nvidia/NVIDIA-Nemotron-Nano-9B-v2unsloth74.1mediumRULER (128K)
80NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia70.56mediumRULER (512K, 0-shot, acc)
81Qwen3-8B-DMS-8xQwen/Qwen3-8Bnvidia69.2mediumRULER 64K
82SmolLM3-3B-GGUFHuggingFaceTB/SmolLM3-3Bggml-org64.0medium
83NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16nvidia64.0medium
84SmolLM3-3B-GGUFHuggingFaceTB/SmolLM3-3Bunsloth64.0medium
85SmolLM3-3B-128K-GGUFHuggingFaceTB/SmolLM3-3Bunsloth64.0medium
86SmolLM3-3BHuggingFaceTB/SmolLM3-3Bunsloth64.0medium
87NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16nvidia64.0medium
88SmolLM3-3B-BaseHuggingFaceTB/SmolLM3-3B-Baseunsloth64.0medium
89Qwen3-8B-DMS-8xQwen/Qwen3-8Bnvidia64.0medium
90SmolLM3-3B-bnb-4bitHuggingFaceTB/SmolLM3-3Bunsloth64.0medium
91SmolLM3-3B-GGUFHuggingFaceTB/SmolLM3-3BMungert64.0medium
92NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia63.55mediumRULER (64K, 0-shot, acc)
93NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16nvidia60.69mediumRULER (128K, 0-shot, acc)
94t5gemma-2-270m-270mgoogle/t5gemma-2-270m-270mgoogle57.3medium[Ruler 32K]
95t5gemma-2-4b-4bgoogle/t5gemma-2-4b-4bgoogle57.3medium[Ruler 32K]
96t5gemma-2-1b-1bgoogle/t5gemma-2-1b-1bgoogle57.3medium[Ruler 32K]
97Mistral-Small-3.1-24B-Instruct-2503mistralai/Mistral-Small-3.1-24B-Base-2503mistralai32.0medium
98t5gemma-2-270m-270mgoogle/t5gemma-2-270m-270mgoogle32.0medium
99t5gemma-2-4b-4bgoogle/t5gemma-2-4b-4bgoogle32.0medium
100t5gemma-2-1b-1bgoogle/t5gemma-2-1b-1bgoogle32.0medium
101Mistral-Small-3.1-24B-Instruct-2503-GGUFmistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth32.0medium
102Mistral-Small-3.1-24B-Instruct-2503-bnb-4bitmistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth32.0medium
103Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bitmistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth32.0medium
104Mistral-Small-3.1-24B-Instruct-2503mistralai/Mistral-Small-3.1-24B-Instruct-2503unsloth32.0medium
105Mistral-Small-3.1-24B-Instruct-2503-GGUFmistralai/Mistral-Small-3.1-24B-Base-2503Mungert32.0medium
106t5gemma-2-270m-270mgoogle/t5gemma-2-270m-270mgoogle25.5medium[Ruler 128K]
107t5gemma-2-4b-4bgoogle/t5gemma-2-4b-4bgoogle25.5medium[Ruler 128K]
108t5gemma-2-1b-1bgoogle/t5gemma-2-1b-1bgoogle25.5medium[Ruler 128K]
109Qwen3-8B-DMS-8xQwen/Qwen3-8Bnvidia25.0mediumRULER 128K
Hugging Face model & benchmark data scraped & matrix-ified. 24,078 variants · 14,984 scores · 120 benchmarks.