vLLM Just Merged TurboQuant Fix for Qwen 3.5+
Previously it was throwing a 'Not Implemented' error due to Mamba layers. Going to test it now! https://github.com/vllm-project/vllm/pull/39931 submitted by /u/havenoammo [link] [comments]
LocalLLaMA
Previously it was throwing a 'Not Implemented' error due to Mamba layers. Going to test it now! https://github.com/vllm-project/vllm/pull/39931 submitted by /u/havenoammo [link] [comments]