Thought I'd share some benchmark numbers from my local setup. Hardware: Dual NVIDIA RTX 3090s Model: Gemma 4 (MoE architecture) Performance: ~120 Tokens Per Second The efficiency of this MoE implementation is unreal. Even with a heavy load, the thr…
YESSS LLAMA.CPP IS UPDATED AND IT DOESN'T TAKE UP PETABYTES OF VRAM submitted by /u/FusionCow [link] [comments]
i dont know what im doing with my life submitted by /u/redilaify [link] [comments]
I made a program that converts any llama2 large language model into a minecraft datapack, and you can run inference right inside the game. It's still semi-finished, Currently I've only implemented argmax sampling, so the output tends to s…
I've been using both side by side over this evening working on a project. Basically I'd paste a chunk of creative text into chat and tell it to dismantle it thesis-by-thesis, then I'd see if the criticism is actually sound, and submit the n…
I took the official benchmarks for Qwen 3.5 and Gemma 4 and compiled them into a neck-and-neck comparison here. The Benchmark Table Benchmark Qwen 2B Gemma E2B Qwen 4B Gemma E4B Qwen 27B Gemma 31B Qwen 35B (MoE) Gemma 26B (MoE) MMLU-Pro 66.5% 60.0…
I'm into HPC, and C++ static, zero allocation and zero dependancy software. I was studying BPE tokenizers, how do they work, so decided to build that project. I hardcoded qwen tokenizer for LLMs developers. I really know that whole Tokenizati…
I am still processing this lol. I gave both Gemini 3 Deepthink and Gemma 4 (31B) the exact same complex security puzzle (which was secretly an unwinnable paradox). Gemini completely fell for the trap. It spit out this incredibly professional-look…
source: https://x.com/osanseviero/status/2040105484061954349 https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4 submitted by /u/jacek2023 [link] [comments]
I mean, I have 40GB of Vram and I still cannot fit the entire Unsloth Gemma-4-31B-it-UD-Q8 (35GB) even at 2K context size unless I quantize KV to Q4 with 2K context size? WTF? For comparison, I can fit the entire UD-Q8 Qwen3.5-27B at full context witho…