quantizer · 15 variants · 0 roots
| variant | kind | quant | vRAM | dl | root |
|---|---|---|---|---|---|
| Llama-3.2-1B-Instruct-Q8_0-GGUF | gguf | gguf | 402,271 | meta-llama/Llama-3.2-1B-Instruct | |
| gemma-2-9b-it-AWQ-INT4 | awq | awq | 283,782 | google/gemma-2-9b-it | |
| Meta-Llama-3.1-70B-Instruct-AWQ-INT4 | awq | awq | 258,512 | meta-llama/Meta-Llama-3.1-70B | |
| Meta-Llama-3.1-8B-Instruct-AWQ-INT4 | awq | awq | 188,386 | meta-llama/Meta-Llama-3.1-8B | |
| Llama-3.2-1B-Instruct-Q4_K_M-GGUF | gguf | gguf | 118,996 | meta-llama/Llama-3.2-1B-Instruct | |
| Llama-3.2-3B-Instruct-Q4_K_M-GGUF | gguf | gguf | 23,295 | meta-llama/Llama-3.2-3B-Instruct | |
| Mixtral-8x7B-Instruct-v0.1-AWQ-INT4 | awq | awq | 15,496 | mistralai/Mixtral-8x7B-Instruct-v0.1 | |
| Meta-Llama-3.1-8B-Instruct-GPTQ-INT4 | gptq | gptq | 9,264 | meta-llama/Meta-Llama-3.1-8B | |
| Meta-Llama-3.1-70B-Instruct-GPTQ-INT4 | gptq | gptq | 3,551 | meta-llama/Meta-Llama-3.1-70B | |
| Llama-3.2-3B-Instruct-Q8_0-GGUF | gguf | gguf | 1,260 | meta-llama/Llama-3.2-3B-Instruct | |
| Meta-Llama-3.1-405B-Instruct-AWQ-INT4 | awq | awq | 1,216 | meta-llama/Meta-Llama-3.1-405B | |
| Meta-Llama-3.1-8B-Instruct-BNB-NF4 | bnb-4bit | 4bit | 368 | meta-llama/Meta-Llama-3.1-8B-Instruct | |
| Meta-Llama-3.1-405B-BNB-NF4-BF16 | bnb-4bit | 4bit | 144 | meta-llama/Meta-Llama-3.1-405B | |
| Meta-Llama-3.1-405B-Instruct-GPTQ-INT4 | gptq | gptq | 137 | meta-llama/Meta-Llama-3.1-405B | |
| Meta-Llama-3.1-405B-Instruct-BNB-NF4 | bnb-4bit | 4bit | 42 | meta-llama/Meta-Llama-3.1-405B-Instruct |