Quick specs, this is a workstation that was morphed into something LocalLLaMa friendly over time: 3950x 96GB DDR4 (dual channel, running at 3000mhz) w6800 + Rx6800 (48GB of VRAM at ~512GB/s) most tests done with ~20k context; kv-cache at q8_0 llama cp…
If yes, which quantized model are you using abe what’s your vllm serve command? I’ve been struggling getting that model up and running on my dgx spark gb10. I tried the intel int4 quant for the 31B and it seems to be working well but way too slow. Any…
If yes, which quantized model are you using abe what’s your vllm serve command? I’ve been struggling getting that model up and running on my dgx spark gb10. I tried the intel int4 quant for the 31B and it seems to be working well but way too slow. Any…
The previous post was probably automoded or something, so I'll give you the TL;DR and point you to search for the model card yourself. Tbh, it's sad that bot posts / posts made by an AI gets prompted, while human made one gets banned. I t…
Supposedly we’ve reached AGI according to Jensen Huang and Marc Andreessen. What a load of shit. I tried to get Claude code with Opus 4.6 max plan to play Elden Ring. Couldn’t even get past the first room. It made it past the character creator, but cou…
Good day. I am looking for best local model for coding agent. I might've missed something or some model which is not that widely used so I cam here for the help. Currently I have following models I found useful in agentic coding via Google's tu…
After seeing Karpathy's tweet about using LLMs to build personal wikis from research documents, I realized I'd already been using something similar like this internally for our R&D. So I cleaned it up and open-sourced it. What it does: You …
Been experimenting with it, first on my buddy's compute he let me borrow, and then with the Gemini SDK so that I don't need to keep stealing his macbook from 600 miles away. Originally my home agent was run through Gemini-3-Flash because no oth…
Hey everybody, I have a strong interest in offloading work to small, specialized models that I can parallelize – this lets me scale work significantly (plus, I am less dependent on proprietary APIs) Some time ago, I saw a blog post from Wiz about…
I posted about my APEX quantization of QWEN Coder 80B Next yesterday and got a ton of great questions. Some people loved it, some people were skeptical, and one person asked "what exactly is the point of this when K quants already do mixed precisi…