Hey all, this is an update! A few days ago I posted to show the performance of a Raspberry Pi5 when using a SSD to let larger models run. Rightfully so, a few brought to my attention that the PCIe is faster than the USB3 connection I was using, s…
Wanted to share one of my personal projects, since similar work has been shared here. TLDR is that I trained an LLM from scratch on pre-1900 text to see if it could come up with quantum mechanics and relativity. The model was too small to do mean…
Been working on custom Triton kernels for LLM inference for a while. My latest project: a fused MoE dispatch pipeline that handles the full forward pass in 5 kernel launches instead of 24+ in the naive approach. Results on Mixtral-8x7B (A100): Tokens…
Sure you can't do agentic coding with the Gemma 4 E2B, but this model is a game-changer for people learning a new language. Imagine a few years from now that people can run this locally on their phones. They can point their camera at objects…
Gemma 4 dropped this week so I fine-tuned E4B for a specific task: extracting structured JSON (doc type, obligations, key fields) from technical and regulatory documents. https://preview.redd.it/v7yg80prpetg1.png?width=1026&format=png&aut…
Support seems non-existent and the last proper release was over a month ago. Comparing with llama.cpp, they are just miles different in activity and support. Is there an alternative or should just use llama.cpp for my macbook? submitted by …
Yes, Google stole my proprietary model size (31B). Yes, I plan to tune all the Gemma 4 models. Join us, and support the mission! Thank you all for the love <3 submitted by /u/TheLocalDrummer [link] [comments]
Gemma 4 Uncensored — all 4 models, MoE expert abliteration, automated research loop Released uncensored versions of all four Gemma 4 models. bf16 + GGUF for each. Collection: https://huggingface.co/collections/TrevorJS/gemma-4-uncensored-69d2885d…
Not sure if it was there. As far as I know it was only open for the api. Qwen 3.5 max preview is in there as well but I am not sure if it was there before. submitted by /u/ba2sYd [link] [comments]