LocalLLaMA

submitted by /u/BAZfp [link] [comments]

Edit: "it admits that it does not know" (sorry for the TYPO!) Although Qwen3.5 is a great series of models, it is prone to make very broad assumptions/hallucinate stuff and it does it with a great confidence, so you may believe what it says. …

submitted by /u/Ryoiki-Tokuiten [link] [comments]

Typically, models in the 26B-class range are difficult to run on 16GB macs because any GPU acceleration requires the accelerated layers to sit entirely within wired memory. It's possible with aggressive quants (2 bits, or maybe a very lightweight I…

Hi guys, We’ve implemented a one-click app for OpenClaw with Local Models built in. It includes TurboQuant caching, a large context window, and proper tool calling. It runs on mid-range devices. Free and Open source. The biggest challenge was ena…

Gemma 4 31B takes an incredible 3rd place on FoodTruck Bench, beating GLM 5, Qwen 3.5 397B and all Claude Sonnets! I'm looking forward to how they'll explain the result. Based on the previous models that failed to finish the run, it would…

Hey r/LocalLLaMA, I just uploaded Harmonic-9B, my latest Qwen3.5-9B fine-tune aimed at agent use. Current status: • Stage 1 (heavy reasoning training) is complete • Stage 2 (light tool-calling / agent fine-tune) is still training right now The plan is …

I have always considered the term RAG to be a hype term. to me Retrieval Augmented Generation just means the model retrieves the data, interprets it based on what you requested and responds with the data in context, meaning any agentic system that has …

This post was written in my own words, but AI assistance. I own two DGX Sparks myself, and the lack of NVFP4 has been a real pain in the ass. The reason the product made sense in the first place was the Blackwell + NVFP4 combo on a local AI machine wit…

I know that artificial analysis is not everyone's favorite benchmarking site but it's a bullet point. I was particularly interested in how well Gemma 4 E4B performs against comparable models for hallucination rate and intelligence/output …