LocalLLaMA

submitted by /u/moneyspirit25 [link] [comments]

I think the 26B A4B MoE model is superior for 16 GB. I tested many quantizations, but if you want to keep the vision, I think the best one currently is: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/blob/main/gemma-4-26B-A4B-it-UD-IQ4_XS.gguf …

Working on building live Closed Captions for Discord calls for my TTRPG group. With Gemma being able to do voice transcription and translation, does it still make sense to run Whisper + a smaller model for translation? Is it better, faster, or has some…

over the course of the arc of local model history (the past six weeks) we have reached a plateau with models and quantization that would have left our ancient selves (back in the 2025 dark ages) stunned and gobsmacked at the progress we currently enjoy…

Kreuzberg v4.7.0 is here. Kreuzberg is a Rust-core document intelligence library that works with Python, TypeScript/Node.js, Go, Ruby, Java, C#, PHP, Elixir, R, C, and WASM. We’ve added several features, integrated OpenWEBUI, and made a big improvemen…

I got a 64gb memory mac about a month ago and I've been trying to find a model that is reasonably quick, decently good at coding, and doesn't overload my system. My test I've been running is having it create a doom style raycaster in html a…

Just because I've seen a couple of "I want this on Android" questions, PocketPal got updated a few hours ago, and runs Gemma 4 2B and 4B fine. At least on my hardware (crappy little moto g84 workhorse phone). Love an app that gets regular…

Hello everyone. I'm at a startup of a team of less than 10 ppl. Everyone in our team wants to use AI to speed up their work and iron out issues faster, which LLMs can be used for. The purposes we use LLMs can be coding, sales presentations, pitch p…

Hello Peeps Salman, Shuguang and Adil here from Katanemo Labs (a DigitalOcean company). Wanted to introduce our latest research on agentic systems called Signals. If you've been building agents, you've probably noticed that there are far …

after long research, finding best alternative for Using a local LLM in OpenCode with llama.cpp to use totally local environment for coding tasks I found this article How to connect Claude Code CLI to a local llama.cpp server how to disable telemetry an…