Hello everyone. Finally I found a way to fix ssm_conv1d tensor drift in quantized GGUF models via Wasserstein metric (W1). It's a lot better than Kullback Leibler for detecting numerical instability and drift in tensors.
All three are ssm_conv1d.weight layers – recurrent state transition layers responsible for long‑context memory. It appears the Qwen team may not be aware of this specific drift issue in the SSM layers. I found the same bug in quants from Unsloth.
| Tensor | α | D (log‑ratio) | W1 before | W1 after |
|---|---|---|---|---|
| blk.36.ssm_conv1d.weight | 0.5765 | 0.553 | 0.0038 | 0.0009 |
| blk.37.ssm_conv1d.weight | 0.5768 | 0.725 | 0.0040 | 0.0009 |
| blk.38.ssm_conv1d.weight | 0.6533 | 0.649 | 0.0026 | 0.0006 |
Other tensors in model are healthy.
Here fixed model: https://huggingface.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF
Model is based on this one: https://huggingface.co/HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive . Thanks to HauhauCS for amazing job.
System prompt: https://pastebin.com/pU25DVnB
Chat template: https://pastebin.com/Dy2fmmpN
Reccomended quant: Q4_K_P
Recommended Settings (LM Studio):
| Parameter | Value |
|---|---|
| Temperature | 0.7 |
| Top K Sampling | 20 |
| Presence Penalty | 1.5 |
| Repeat Penalty | Disabled |
| Top P Sampling | 0.8 |
| Min P Sampling | 0 |
| Seed | 42 |
Model features:
- It talks almost like human. Short and consize.
- Fully uncensored.
- Programming works fine.
I tested long context window in model via roleplay with my System Prompt. According to my taste I didn't find any problems in following character.
Enjoy ^_^
[link] [comments]