What is the largest LLM the NVIDIA RTX 3090 Ti can run?

Qwen3.6 35B-A3B, at Q4_K_M using about 22.6 GB of the 24 GB available, at roughly 333 tokens per second (8K context).

How many local AI models run well on the NVIDIA RTX 3090 Ti?

48 of the 69 models we track run fully in its 24 GB of VRAM at Q4_K_M or better.

Generation speed follows memory bandwidth. With 1008 GB/s, expect about 38 tokens per second on an 8B model.

Every model we track, checked against the NVIDIA RTX 3090 Ti at 8K context. The largest that runs well is Qwen3.6 35B-A3B.

VRAM24 GBNVIDIA

Bandwidth1008GB/s

Runs well48/69models at Q4_K_M or better

Fastest model~403tok/s · Qwen3 0.6B

Bars show memory used out of 24 GB. Pick a row for the full breakdown.

NameMemory usedBest quantSpeed

NameMemory usedBest quantSpeed

NameMemory usedBest quantSpeed

Run it in the cloud

Rent a cloud GPU by the hour. No upfront cost, stop any time.