Hello everyone, I have noticed there is no data yet what quants are better for 26B A4B and 31b. Personally, in my experience testing 26b a4b q4_k_m from Bartowski and the full version on openrouter and AI Studio, I have found this quant to perform exce…
according to this report it seems that by "hammering" bits into dram chips through malicious cuda kernels, it could be possible to compromise systems equipped w/ several nvidia gpus up to excalating unsupervised privileged access to administr…
In the past year you may have encountered the following prompt: The surgeon, who is the boy's father, says, 'I cannot operate on this boy—he's my son!'. Who is the surgeon to the boy? If you try to give this prompt to an LLM rig…
Hey guys, quick follow up to my post yesterday about running Gemma 4 26B. I kept testing and realized you can just use the Q8_0 mmproj for vision instead of F16. There is no quality drop, and it actually performed a bit better in a few of my tests (wit…
"https://www.tomshardware.com/tech-industry/iran-threatens-complete-and-utter-annihilation-of-openais-usd30b-stargate-ai-data-center-in-abu-dhabi-regime-posts-video-with-satellite-imagery-of-chatgpt-makers-premier-1gw-data-center" Although it…
I've been running some tests lately and I'm honestly blown away. I just tried the new HunyuanOCR (specifically the GGUF versions) and the performance on budget hardware is insane. Using the 1B parameter model, I’m getting around 90 t/s on my ol…
Something I'm noticing that I don't think I've noticed before. I've been testing out Gemma 4 31B with 32GB of VRAM and 64GB of DDR5. I can load up the UD_Q5_K_XL Unsloth quant with about 100k context with plenty of VRAM headroom, but wh…
I wanted to know which type of quant is the best on this laptop (Intel 258V – iGPU 140V 18GB), so I tested all these small quants hoping that it generalizes to bigger models: Winners in bold (KLD≤0.01) Uploader Quant tk/s KLD GB KLD/GB* mr…
I think the biggest unlock for local models over the next year is not another benchmark jump. It’s making the whole stack feel boring and dependable. Right now the average workflow still has too many sharp edges: model format mismatch, VRAM roulette, b…
Just by way of background: I am from the Midwest but I’m lawyer in South Carolina (and I am actually preparing for a trial next week and should be asleep). Have had my own Law firm for 11 years now. About 4 months ago Claude code did some things that …