Rent a GPU

Kimi K3.1 Pro 2.8T-A104B hardware requirements

2800B parameters (104B active, mixture of experts). Needs about 1698.3 GB at Q4_K_M with 8K context.

Parameters2800B104B active (MoE)
Q4_K_M · 8K1698.3 GBthe usual download
FP16 · 8K5600.8 GBfull precision
Runs well on0/51GPUs and Macs we track

Memory needed at each quantization

Weights, KV cache and runtime overhead, at 8K and 32K tokens of context.

QuantQuality8K context32K context
FP16Full quality5600.8 GB5601.5 GB
Q8_0Near-lossless2975.8 GB2976.5 GB
Q6_KExcellent2296.8 GB2297.5 GB
Q5_K_MVery good1992.3 GB1993 GB
Q4_K_MGood, most popular1698.3 GB1699 GB
Q3_K_MNoticeably degraded1369.3 GB1370 GB
Q2_KHeavily degraded1050.8 GB1051.5 GB

Model card

moonshotai/Kimi-K3.1-Pro on Hugging Face: licence, prompt format and GGUF builds.

Run it in the cloud

Run Kimi K3.1 Pro 2.8T-A104B on a rented GPU today.

It needs about 1698.3 GB at Q4_K_M: a rented 8× B300 runs it for about $71.92 an hour. Compare with Claude and APIs or see how it scores against them.

RunPodRent a GPU on RunPodVast.aiRent a GPU on Vast.ai

Every GPU and Mac checked

Sorted by memory. Bars show how much of each card Kimi K3.1 Pro 2.8T-A104B fills at the best quant that fits.

51 Too big

NameMemory usedBest quantSpeed