Google published Ironwood inference benchmarks in their AI-Hypercomputer/tpu-recipes repo. Nvidia has InferenceMAX numbers for B200. Nobody has compared them head-to-head under identical conditions. Ironwood skipped MLPerf v6.0, so there's no…
The 3.5 122b model already is fantastic at 4-bit. Really the best model I ever ran on my 4×3090, but from what I read how 35B 3.6 is doing, the 3.6 122b model would be an absolute value banger. Are we going to get it? submitted by /u/Mr_M…
I keep presenting Local and Huge cloud models with the same challenge: "Two paratroopers land on an infinite 1D numeric axis at distinct, unknown integer coordinates. They both execute the exact same deterministic program. They have no inter…
Quick demo of KV cache compression on Qwen 3.6 at 1M context. In this run: KV cache: 10.74 GB → 6.92 GB V cache: 5.37 GB → 1.55 GB (~3.5× reduction) Still seeing near-zero PPL change in early tests (3 seeds), but focusing mainly on memory + long-…
submitted by /u/B89983ikei [link] [comments]
I have a personal eval harness: A repo with around 30k lines of code that has 37 intentional issues for LLMs to debug and address through an agentic setup (I use OpenCode) A subset of the harness also has the LLM extract key information from reasonably…
One thing I absolutely love about the paid platforms is the deep research system. Is there a good one on local? I have SearXNG set up, and it's ok, it doesn't seem to pull back many google results but the resutls it can pull back are ok. I…
I recently came across the Granite 4.1 8B dense model on Hugging Face, but I haven’t been able to find any further details about it online. https://huggingface.co/ibm-granite/granite-4.1-8b /u/ibm could you please provide some information on this? …
https://preview.redd.it/dfqed57qgsvg1.png?width=1706&format=png&auto=webp&s=3859209698d2e844e2731326e355d60928658f8a The most fun part was reasoning, here is a gist: https://gist.github.com/anzax/5f06716c66180013cd715f6c2e5848df There…
Qwen 3.6 dropped yesterday and I wanted to see if hybrid offloading actually earns its keep on this hardware. My box is two RTX 5060 Ti (32GB VRAM total) with 64GB system RAM. Not a workstation card in sight. I ran the same bench harness across three c…