Sup, I'm Crownelius, I made that popular opus distill dataset.
TODAY YOU ARE INTRODUCED TO SHARD a 40m parameter mal-formed LLM.
Right now I'm working on a series of tiny LLM's, with a goal to run a coherent model for IoT tasks. I've researched atomic models, and while doing that I came across a project called Compact AI. Since joining them, I've learned a lot and even made my own model from scratch.
The model is available here: CompactAI-O[HF Organization]
About my model named "Shard"-I call it Scamp.
[link] [comments]