QWEN3.6 + ik_llama is fast af
running qwen3.6 UD_Q_4_K_M on 16GB vram + 32GB ram with 200k cw @50+ tok/s submitted by /u/_BigBackClock [link] [comments]
LocalLLaMA
LocalLLaMA
QWEN3.6 + ik_llama is fast af
running qwen3.6 UD_Q_4_K_M on 16GB vram + 32GB ram with 200k cw @50+ tok/s submitted by /u/_BigBackClock [link] [comments]
LocalLLaMA
Qwen3.6 GGUF is so good for debugging.
using unsloth dynamic quant on 16GB vram + 32GB dram. 200k q8_0 kv cache (context window) submitted by /u/_BigBackClock [link] [comments]