Hey all,

Looking for some real-world advice on GPU choices for running the new dense models — mainly Qwen 3.5 27B and Gemma 4 31B.

What I’m targeting

• Context: 64K+ (ideally higher later)
• Speed: 30+ tok/s @ tg128 minimum
• Power: not critical, but lower is a bonus

From what I’ve seen, these dense models are way more demanding than MoE.

Why not MoE?

I’m already running MoE just fine on P40s:

• Gemma 4 26B MoE
• ~32K ctx
• ~42+ tok/s @ tg128

So now I want to move to dense models for better quality / reasoning.

Budget

• ~2500 AUD (~$1800 USD)
• GPU only (already have CPU / RAM / board)
• Ignore PCIe lane limits for now

Options I’m considering

A. 2× 9070 XT (16GB)
B. 1× R9 9700 (32GB)
C. 2× 7900 XTX (24GB)
D. 1× RTX Pro 4000 (24GB)

N. 1× Intel Arc Pro B70 (32GB, maybe future option, but not now)

My current understanding (please correct me)

• 16GB cards → basically forced into pipeline parallel, so per-GPU compute matters a lot
• 2× 7900 XTX should have the best raw throughput
• RTX Pro 4000 maybe similar class, but VRAM limits context flexibility
• 32GB single card (R9 9700) is attractive for KV cache / long ctx, BUT:
  • perf ≈ 9070 XT?
  • price = ~2× 9070 XT + extra GPU…
• 2× 9070 XT might be best “budget parallel” option

Concerns (based on what I’ve seen here)

• KV cache is brutal on Gemma 4 31B“massive KV cache… biggest drawback”
• Even people with large VRAM struggle with higher quants / context
• 24GB seems like the minimum viable tier for 31B dense
• Long context scaling is still very hardware-sensitive
• Multi-GPU scaling (esp PCIe) seems very inconsistent depending on backend

What I want to know

If you’ve actually run Qwen3.5 27B / Gemma 4 31B (dense):

• What GPU are you using?
• What real tok/s are you getting (esp @ 64K+)
• Does multi-GPU actually scale well or just look good on paper?
• Is 32GB single GPU > dual 16/24GB in practice?
• Any regrets / “don’t buy this” advice?

Bonus question

If you had ~$1800 today, would you:

• go multi-GPU AMD (cheap + raw compute)
• or single high-VRAM card (simpler + better ctx)

Appreciate any real benchmarks / configs 🙏