Hi guys, We’ve implemented a one-click app for OpenClaw with Local Models built in. It includes TurboQuant caching, a large context window, and proper tool calling. It runs on mid-range devices. Free and Open source. The biggest challenge was ena…
Gemma 4 31B takes an incredible 3rd place on FoodTruck Bench, beating GLM 5, Qwen 3.5 397B and all Claude Sonnets! I'm looking forward to how they'll explain the result. Based on the previous models that failed to finish the run, it would…
Hey r/LocalLLaMA, I just uploaded Harmonic-9B, my latest Qwen3.5-9B fine-tune aimed at agent use. Current status: • Stage 1 (heavy reasoning training) is complete • Stage 2 (light tool-calling / agent fine-tune) is still training right now The plan is …
I have always considered the term RAG to be a hype term. to me Retrieval Augmented Generation just means the model retrieves the data, interprets it based on what you requested and responds with the data in context, meaning any agentic system that has …
This post was written in my own words, but AI assistance. I own two DGX Sparks myself, and the lack of NVFP4 has been a real pain in the ass. The reason the product made sense in the first place was the Blackwell + NVFP4 combo on a local AI machine wit…
I know that artificial analysis is not everyone's favorite benchmarking site but it's a bullet point. I was particularly interested in how well Gemma 4 E4B performs against comparable models for hallucination rate and intelligence/output …
I always add instructions to find an open one but if I forget it kills processes that I had up for a reason 🤦♂️ submitted by /u/bs6 [link] [comments]
Disclaimer: sorry if this post comes out weirdly worded, English is not my main language. I’m a bit confused by how people use the term RAG. I thought the basic idea was: use an embedding model / retriever to find relevant chunks maybe rerank them pas…
The benchmarks may not show it but it's a substantial improvement over 3.5 for real world tasks. This model is performing better than GLM-5.1 and Kimi-k2.5 for me, and the biggest area of improvement has been reliability. It feels as reliable as c…
I am currently utilizing qwen3.5 and Gemma 4 model. Realized Gemma 4 requires 2x ram for same context length. As far as I understand, what turbo quant gives is quantizing kv cache into about 4 bit and minimize the loses But Q8 still not lose the cont…