r/AllenAI is cooking with MolmoAct2, a 5B vision-language-action model for robot control. They keep releasing new fine-tunes on different kinds of robotics datasets, including (but not limited to, and they keep releasing new ones):
https://huggingface.co/allenai/MolmoAct2-LIBERO - general robotics tasks
https://huggingface.co/allenai/MolmoAct2-DROID - interactive robotics tasks
https://huggingface.co/allenai/MolmoAct2-BimanualYAM - absolute joint-pose control
https://huggingface.co/allenai/MolmoAct2-SO100_101 - also absolute joint-pose control
AllenAI has released these as fully open source models, publishing not only their weights but also their complete training datasets (including pretraining), their training software source code, and technical papers describing the theory, training, and assessments of these models.
If anyone is fiddling with robots controlled via LLM inference, you should give MolmoAct2 models a look.
[link] [comments]