Which AI models can the NVIDIA RTX PRO 6000 Blackwell run?
Every model we track, checked against the NVIDIA RTX PRO 6000 Blackwell at 8K context. The largest that runs well is gpt-oss 120B.
VRAM96 GBNVIDIA
Bandwidth1792GB/s
Runs well57/69models at Q4_K_M or better
Fastest model~717tok/s · Qwen3 0.6B
Every model on the NVIDIA RTX PRO 6000 Blackwell
Bars show memory used out of 96 GB. Pick a row for the full breakdown.
57 Runs well
NameMemory usedBest quantSpeed
gpt-oss 120B117BQ5_K_M · 84 GB~296 tok/s
Llama 4 Scout 17B-16E109BQ6_K · 91.6 GB~77 tok/s
GLM-4.5-Air 106B-A12B106BQ6_K · 89.1 GB~109 tok/s
Qwen3 Coder Next 80B-A3B80BQ8_0 · 85.5 GB~337 tok/s- Qwen2.5 72B73BQ8_0 · 80.5 GB~14 tok/s
- Llama 3.1 70B71BQ8_0 · 78.3 GB~14 tok/s
- Llama 3.3 70B71BQ8_0 · 78.3 GB~14 tok/s
DeepSeek R1 Distill Llama 70B71BQ8_0 · 78.3 GB~14 tok/s
Mixtral 8x7B47BFP16 · 95.1 GB~42 tok/s- Qwen3.6 35B-A3B36BFP16 · 72.8 GB~179 tok/s
- Qwen2.5 32B33BFP16 · 68.3 GB~16 tok/s
- DeepSeek R1 Distill Qwen 32B33BFP16 · 68.3 GB~16 tok/s
- Qwen3 32B33BFP16 · 68.3 GB~16 tok/s
- QwQ 32B33BFP16 · 68.3 GB~16 tok/s
- Qwen2.5 Coder 32B33BFP16 · 68.3 GB~16 tok/s
- Nemotron 3 Nano 30B-A3B32BFP16 · 63.9 GB~154 tok/s
Gemma 4 31B31BFP16 · 64.4 GB~17 tok/s- GLM-4.7 Flash 30B-A3B31BFP16 · 63.4 GB~179 tok/s
- Qwen3 30B-A3B31BFP16 · 62.4 GB~163 tok/s
- Qwen3 Coder 30B-A3B31BFP16 · 62.4 GB~163 tok/s
- Qwen3.8 27B28BFP16 · 56.7 GB~19 tok/s
- Qwen3.6 27B28BFP16 · 56.7 GB~19 tok/s
- Gemma 3 27B27BFP16 · 56.5 GB~20 tok/s
- Gemma 2 27B27BFP16 · 57.3 GB~20 tok/s
- Gemma 4 26B-A4B26BFP16 · 52.5 GB~141 tok/s
- Mistral Small 3.2 24B24BFP16 · 49.9 GB~22 tok/s
- Mistral Small 24B24BFP16 · 49.1 GB~23 tok/s
- Devstral Small 24B24BFP16 · 49.1 GB~23 tok/s
- Magistral Small 24B24BFP16 · 49.1 GB~23 tok/s
- gpt-oss 20B21BFP16 · 42.6 GB~149 tok/s
- Qwen3 14B15BFP16 · 31.5 GB~36 tok/s
- Qwen2.5 Coder 14B15BFP16 · 31.8 GB~36 tok/s
- DeepSeek R1 Distill Qwen 14B15BFP16 · 31.8 GB~36 tok/s
- Qwen2.5 14B15BFP16 · 31.6 GB~37 tok/s
Phi-4 14B15BFP16 · 31.7 GB~37 tok/s- Phi-4 Reasoning 14B15BFP16 · 31.7 GB~37 tok/s
- Mistral Nemo 12B12BFP16 · 26.3 GB~44 tok/s
- Gemma 3 12B12BFP16 · 25.9 GB~44 tok/s
- Gemma 4 12B12BFP16 · 24.9 GB~45 tok/s
- Gemma 2 9B9.2BFP16 · 21.2 GB~58 tok/s
- Qwen3 8B8.2BFP16 · 18.2 GB~66 tok/s
- DeepSeek R1 0528 Qwen3 8B8.2BFP16 · 18.2 GB~66 tok/s
Granite 3.3 8B8.2BFP16 · 18.3 GB~66 tok/s- Llama 3.1 8B8BFP16 · 17.7 GB~67 tok/s
- Qwen2.5 7B7.6BFP16 · 16.3 GB~71 tok/s
- Qwen2.5 Coder 7B7.6BFP16 · 16.3 GB~71 tok/s
- DeepSeek R1 Distill Qwen 7B7.6BFP16 · 16.3 GB~71 tok/s
- Mistral 7B v0.37.3BFP16 · 16.2 GB~74 tok/s
- Gemma 3 4B4.3BFP16 · 9.5 GB~125 tok/s
- Qwen3 4B4BFP16 · 9.8 GB~134 tok/s
- Phi-4 Mini 3.8B3.8BFP16 · 9.4 GB~140 tok/s
- Phi-3.5 Mini 3.8B3.8BFP16 · 11.5 GB~141 tok/s
- Llama 3.2 3B3.2BFP16 · 8 GB~167 tok/s
- Qwen3 1.7B2BFP16 · 5.6 GB~265 tok/s
- Llama 3.2 1B1.2BFP16 · 3.3 GB~434 tok/s
- Gemma 3 1B1BFP16 · 2.6 GB~538 tok/s
- Qwen3 0.6B0.8BFP16 · 3 GB~717 tok/s
2 Fits with heavy quantization
NameMemory usedBest quantSpeed
10 Too big
NameMemory usedBest quantSpeed
Kimi K3.1 Pro 2.8T-A104B2800Bneeds 1698.3 GB96 GB free- Qwen3.8 2.4T-A95B2446Bneeds 1484.3 GB96 GB free
- DeepSeek V4 Pro 1.6T-A49B1600Bneeds 971.2 GB96 GB free
- Mistral Large 4 1T-A52B1000Bneeds 607.5 GB96 GB free
- DeepSeek V4.1 Flash763Bneeds 463.4 GB96 GB free
- GLM-5.3753Bneeds 457.8 GB96 GB free
- GLM-5 744B-A40B744Bneeds 452.4 GB96 GB free
MiniMax M3 428B-A23B427Bneeds 260.5 GB96 GB free- GLM-5.3 Flash 320B-A18B321Bneeds 195.3 GB96 GB free
- DeepSeek V4 Flash 284B-A13B284Bneeds 173 GB96 GB free
Run it in the cloud
Need more than 96 GB? Rent a bigger GPU by the hour.
Rent a cloud GPU by the hour. No upfront cost, stop any time.
Apple M4 Max (64GB) 64 GB