 | SenseNova dropped SenseNova-U1 on the last day of April and I’ve only found like one other mostly ignored post on this sub talking about it. It seems like a really exciting novel architecture to me. It appears to be exceptional at text-to-infographics as one of its major high points, as well as being good at image editing, generation, and visual understanding. Supposedly it’s not the traditional mash-up (no VAE) types of multimodal models that we’ve seen before. The following is from their Hugging Face: ——— SenseNova U1 is a new series of native multimodal models that unifies multimodal understanding, reasoning, and generation within a monolithic architecture. It marks a fundamental paradigm shift in multimodal AI: from modality integration to true unification. Rather than relying on adapters to translate between modalities, SenseNova U1 models think-and-act across language and vision natively. The unification of visual understanding and generation opens tremendous possibilities. SenseNova U1 sits in the stage of Data-driven Learning (like ChatGPT), yet gestures toward the next stage, that is, Agentic Learning (like OpenClaw) and thinking in a natively multimodal way. Key Pillars: At the core of SenseNova U1 is NEO-Unify, a novel architecture designed from the first principles for multimodal AI: It eliminates both Visual Encoder (VE) and Variational Auto-Encoder (VAE) where pixel-word information are inherently and deeply correlated. Several important features are as follows: - Model language and visual information end-to-end as a unified compound. - Open-source SoTA in both understanding and generation: SenseNova U1 sets a new standard for unified multimodal understanding and generation, achieving state-of-the-art performance among open-source models across a wide range of understanding, reasoning, and generation benchmarks. - Native interleaved image-text generation: SenseNova U1 can generate coherent interleaved text and images in a single flow with one model, enabling use cases such as practical guides and travel diaries that combine clear communication with vivid storytelling and transform complex information into intuitive visuals. - High-density information rendering: SenseNova U1 demonstrates strong capabilities in dense visual communication, generating richly structured layouts for knowledge illustrations, posters, presentations, comics, resumes, and other information-rich formats. Beyond Multimodality: - Vision–Language–Action (VLA) They also released several agent skills to plug the model into Agents like Hermes. Here’s their skills repo: https://github.com/OpenSenseNova/SenseNova-Skills The skills are likely set up to drive traffic to their hosted APIs, but I’m sure it’ll be pretty easy to mod them to point to local endpoints instead. (I’m working on this now for myself). Just curious to see if anyone has tested this and if it’s living up to the hype or not. [link] [comments] |