sllm is an experiment in sharing GPUs between developers. I think everyone at some point in their agentic development journey thinks about hosting their own LLM. And if you can afford it, great, but I looked pretty deep into the economics and it's actually incredibly wasteful.
Most of the time your GPU is sitting idle. So I built sllm to see if it's possible to share a single LLM node between hundreds of developers and give everyone unlimited tokens at a flat rate. Honestly, I'm not sure how well this will work. But if it does, it means developers get unlimited tokens for roughly 1/400th the cost of running their own node and way cheaper than per-token providers like OpenAI.
What do you guys think? Anyone here have experience with this before I shoot myself in the foot with a huge bill lol
[link] [comments]