Llama 3.2 1B hardware requirements
1.24B parameters. Needs about 1.6 GB at Q4_K_M with 8K context. Model card.
Memory needed at each quantization
| Quant | Quality | 8K context | 32K context |
|---|---|---|---|
| FP16 | Full quality | 3.3 GB | 4.2 GB |
| Q8_0 | Near-lossless | 2.2 GB | 3 GB |
| Q6_K | Excellent | 1.9 GB | 2.7 GB |
| Q5_K_M | Very good | 1.8 GB | 2.6 GB |
| Q4_K_M | Good, most popular | 1.6 GB | 2.4 GB |
| Q3_K_M | Noticeably degraded | 1.5 GB | 2.3 GB |
| Q2_K | Heavily degraded | 1.3 GB | 2.1 GB |
Run it with Ollama: ollama run llama3.2:1b (downloads Q4_K_M by default).
Run it in the cloud
Don't want to buy hardware? Run Llama 3.2 1B on a rented GPU today.
Rent a cloud GPU by the hour. No upfront cost, cancel any time.
Every GPU and Mac checked
| GPU | Runs? | Best fit |
|---|---|---|
| NVIDIA RTX 3060 12GB | Yes | FP16 · ~3.3 GB · ~87 tok/s |
| NVIDIA RTX 3070 | Yes | FP16 · ~3.3 GB · ~108 tok/s |
| NVIDIA RTX 3080 10GB | Yes | FP16 · ~3.3 GB · ~184 tok/s |
| NVIDIA RTX 3090 | Yes | FP16 · ~3.3 GB · ~226 tok/s |
| NVIDIA RTX 4060 | Yes | FP16 · ~3.3 GB · ~66 tok/s |
| NVIDIA RTX 4060 Ti 16GB | Yes | FP16 · ~3.3 GB · ~70 tok/s |
| NVIDIA RTX 5060 Ti 16GB | Yes | FP16 · ~3.3 GB · ~108 tok/s |
| NVIDIA RTX 4070 | Yes | FP16 · ~3.3 GB · ~122 tok/s |
| NVIDIA RTX 4070 Ti Super | Yes | FP16 · ~3.3 GB · ~163 tok/s |
| NVIDIA RTX 4080 | Yes | FP16 · ~3.3 GB · ~173 tok/s |
| NVIDIA RTX 4090 | Yes | FP16 · ~3.3 GB · ~244 tok/s |
| NVIDIA RTX 5070 | Yes | FP16 · ~3.3 GB · ~163 tok/s |
| NVIDIA RTX 5070 Ti | Yes | FP16 · ~3.3 GB · ~217 tok/s |
| NVIDIA RTX 5080 | Yes | FP16 · ~3.3 GB · ~232 tok/s |
| NVIDIA RTX 5090 | Yes | FP16 · ~3.3 GB · ~434 tok/s |
| AMD Radeon RX 7800 XT | Yes | FP16 · ~3.3 GB · ~151 tok/s |
| AMD Radeon RX 7900 XTX | Yes | FP16 · ~3.3 GB · ~232 tok/s |
| Apple M4 (16GB) | Yes | FP16 · ~3.3 GB · ~29 tok/s |
| Apple M1 (16GB) | Yes | FP16 · ~3.3 GB · ~16 tok/s |
| Apple M2 Pro (32GB) | Yes | FP16 · ~3.3 GB · ~48 tok/s |
| Apple M3 Max (64GB) | Yes | FP16 · ~3.3 GB · ~97 tok/s |
| Apple M4 Pro (48GB) | Yes | FP16 · ~3.3 GB · ~66 tok/s |
| Apple M4 Max (128GB) | Yes | FP16 · ~3.3 GB · ~132 tok/s |
| Apple M2 Ultra (192GB) | Yes | FP16 · ~3.3 GB · ~194 tok/s |