Can the AMD Radeon RX 7900 XTX run Mistral 7B v0.3?
Yes
QuantFP16Full quality
Memory16.2 GBof 24 GB
Speed~40tokens/s
Yes. The AMD Radeon RX 7900 XTX can run Mistral 7B v0.3 fully in VRAM at FP16 (full quality), using about 16.2 GB of the 24 GB available, at roughly 40 tokens/second.
Try other context lengths in the calculator →Memory needed by quantization
At 8,192 tokens of context.
| Quant | Quality | Memory | Fits? | Speed |
|---|---|---|---|---|
| FP16 | Full quality | 16.2 GB | Fits | ~40 tok/s |
| Q8_0 | Near-lossless | 9.4 GB | Fits | ~75 tok/s |
| Q6_K | Excellent | 7.6 GB | Fits | ~97 tok/s |
| Q5_K_M | Very good | 6.8 GB | Fits | ~112 tok/s |
| Q4_K_M | Good, most popular | 6.1 GB | Fits | ~131 tok/s |
| Q3_K_M | Noticeably degraded | 5.2 GB | Fits | ~163 tok/s |
| Q2_K | Heavily degraded | 4.4 GB | Fits | ~212 tok/s |
How to run Mistral 7B v0.3 on the AMD Radeon RX 7900 XTX
With Ollama installed, one command downloads and starts it:
ollama run mistral:7bOllama's default download is Q4_K_M (about 6.1 GB here). You have room for FP16, which is full quality: pick that GGUF in LM Studio or from Hugging Face.
GPUs that run Mistral 7B v0.3 comfortably
- NVIDIA RTX 4060 (8 GB)
- NVIDIA RTX 3070 (8 GB)
- NVIDIA RTX 3080 10GB
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4070 (12 GB)
Models that run well on the AMD Radeon RX 7900 XTX
Run it in the cloud
Don't want to buy hardware? Run Mistral 7B v0.3 on a rented GPU today.
Rent a cloud GPU by the hour. No upfront cost, cancel any time.