Can the NVIDIA RTX 2080 Ti run DeepSeek R1 Distill Llama 70B?
- FP16
- Q8_0
- Q6_K
- Q5_K_M
- Q4_K_M
- Q3_K_M
- Q2_K
No. Even at Q4_K_M, DeepSeek R1 Distill Llama 70B needs about 46.1 GB, but the NVIDIA RTX 2080 Ti has 11 GB usable. You could offload some layers to system RAM, but it will be very slow.
Run it on a rented GPU
On a cloud GPU with enough memory, the same command you'd use locally works:
ollama run deepseek-r1:70bAt Q4_K_M it needs about 46.1 GB, so pick a GPU with at least 48 GB.
New to this? Step-by-step setup for the RTX 2080 Ti
Run it in the cloud
Too big for the NVIDIA RTX 2080 Ti? Run DeepSeek R1 Distill Llama 70B on a rented GPU instead.
Rent a cloud GPU by the hour. No upfront cost, stop any time. Not sure whether to rent or buy? Compare the cost.
Memory needed by quantization
At 8,192 tokens of context. The black line marks the 11 GB the NVIDIA RTX 2080 Ti can use.
| Quant | Quality | Memory | Fits? | Speed | |
|---|---|---|---|---|---|
| FP16 | Full quality | 144.5 GB | Too big | — | |
| Q8_0 | Near-lossless | 78.3 GB | Too big | — | |
| Q6_K | Excellent | 61.2 GB | Too big | — | |
| Q5_K_M | Very good | 53.5 GB | Too big | — | |
| Q4_K_M | Good, most popular | 46.1 GB | Too big | — | |
| Q3_K_M | Noticeably degraded | 37.8 GB | Too big | — | |
| Q2_K | Heavily degraded | 29.8 GB | Too big | — |
Measured speed from real RTX 2080 Ti owners
Tokens per second that people measured on their own NVIDIA RTX 2080 Ti, next to our estimate.
No one has measured DeepSeek R1 Distill Llama 70B on the NVIDIA RTX 2080 Ti yet. The speeds above are estimates. Run the benchmark and yours becomes the first real number on this page.
Have an RTX 2080 Ti? Add your real speed
One command times DeepSeek R1 Distill Llama 70B in your own Ollama, shows you exactly what it would send, and asks before sending anything. No sign-up, no personal data.
macOS and Linux
curl -fsSL https://gpufit.dev/bench.sh | sh -s -- deepseek-r1:70bWindows (PowerShell)
$env:GPUFIT_MODEL="deepseek-r1:70b"; irm https://gpufit.dev/bench.ps1 | iexGPUs that run DeepSeek R1 Distill Llama 70B comfortably
Models that run well on the NVIDIA RTX 2080 Ti
Frequently asked questions
Can NVIDIA RTX 2080 Ti run DeepSeek R1 Distill Llama 70B?
No. Even at Q4_K_M, DeepSeek R1 Distill Llama 70B needs about 46.1 GB, but the NVIDIA RTX 2080 Ti has 11 GB usable. You could offload some layers to system RAM, but it will be very slow.
How much VRAM does DeepSeek R1 Distill Llama 70B need?
About 46.1 GB at Q4_K_M with 8,192 tokens of context, and about 144.5 GB at full FP16 precision.
How fast is DeepSeek R1 Distill Llama 70B on the NVIDIA RTX 2080 Ti?
It does not fit in the NVIDIA RTX 2080 Ti's memory, so most layers would run from system RAM and generation would be very slow.