Personal Eval follow-up: Gemma4 26B MoE (Q8) vs Qwen3.5 27B Dense vs Gemma4 31B Dense Compared

This is a follow-up update to my previous post comparing Qwen 3.6 35B vs Gemma 4 26B.

I wanted to particularly follow-up with the following: 1. Gemma 4 26B could've suffered the quantization tax and perform drastically better with an 8-bit quant. So I wanted to put that to the test with UD's Q8_K_XL this time 2. A lot of people (including myself) were curious to see how the Qwen 3.5 27B dense would perform in these tests. 3. Speaking of dense models, I also wanted to include the Gemma 4 31B to see how it performs.

Sharing results consolidated with previous run for a complete comparison

1. Test Results

2. Token Usage

3. Tool Calls

4. Timing & Efficiency

5. Model & Server Configuration

Key Observations

• Gemma 4 26B's performance remains in the same ballpark even with Q8. It performed slightly worse than Q4 in this run but that variance is likely noise. I'll stick with my Q4_K_XL quant
• Both Qwen 3.5 27B and Gemma 4 31B aced the test. The dense models are in a different league from the MoE ones. (Especially the Gemma 31B)
• Gemma 4 31B is the most efficient when it comes to tool calling. It fixed all issues in 100 error-free tool calls
• Qwen 3.5 27B is the most token-efficient expending an average of 16k tokens per fix.
• Gemma 4 31B also exhibited extremely low inference speeds for some reason and ran for 10 hours and 29 minutes due to the abysmally slow speeds. DRAM also bloated upto 70GB even with -cram and -ctkcp flags. I'm not sure if this is expected.

I'd say Gemma4 31B is objectively the most capable in my tests but it's also the slowest of the bunch with my setup. Qwen 3.5 27B follows up with comparable performance at a lot more tolerable speeds. Qwen 3.6 35B remains the speed-to-performance champ and will remain being my daily driver for the same reason.