https://preview.redd.it/j0wyntj1hdvg1.png?width=2480&format=png&auto=webp&s=aa4fd94712bb95cd201a3f654b14a7a11b896033 Given the increasingly closed-source nature of the U.S. AI ecosystem, it is now more important than ever to push for …
I wrote a book that implements modern LLM architectures from scratch. The part most relevant to this sub: Chapter 3 takes GPT-2 and swaps exactly 4 things to get Llama 3.2-3B: LayerNorm → RMSNorm Learned positional encodings → RoPE GELU → SwiGLU Multi…
I'm building a real-time speech translator (STT → LLM translation → TTS) and spent a couple weeks benchmarking every TTS engine I could find — cloud and local. Running on MacBook Air M4, 24GB RAM. Some findings were… not what I expected. Sh…
I am running unsloth/gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf with llama-server (with reasoning enabled). Is it possible to disable reasoning for some requests only? If yes, how? I want to leave reasoning on by default, but in some o…
I've been digging into how AI coding agents actually handle memory — not what the marketing says, but what the code and benchmarks show. Here's what I found. TL;DR: Every agent memory system in 2026 is either too simple (can't search), too …
It has been hyped ±2 weeks ago and I remember seeing some pull requests into llama.cpp, but what is the current status after the hype faded away? submitted by /u/kickerua [link] [comments]
Use the following system prompt to allow Gemma (and most open source models) to talk about anything you wish. Add or remove from the list of allowed content as needed. Derived from the GPT-OSS jailbreak. Works with both GGUF and MLX variants. _________…
Oobabooga recently published 5 detailed benchmark reports on GGUF performance for Gemma 4 26B-A4B, Gemma 4 E4B, Qwen3.5-35-A3B, and Qwen3.5-27B, covering releases from Unsloth, Bartowski, LM Studio, GGML, Mradermacher, AesSedai, and Ubergarm. The…
Been running a few models locally at different quant levels and honestly the jump from Q5 to Q4 sometimes feels like nothing and other times it completely tanks coherence on longer outputs. is there a general rule for where the cliff is, or does it jus…
Funny image, but also I'd like to add that I love how much freedom and honesty I can finetune the model to. No glazing, no censorship, no data harvesting. I can discuss and analyze personal stuff with ease of mind knowing that it stays in my …