SenseTime has published SenseNova-Loop-9B-Preview on Hugging Face, a multimodal model that takes text and images as input and generates text, designed for long-context workflows, code generation and external tool use. What sets it apart is its architecture: instead of adding parameters to increase computational depth, the model runs the same block of layers twice.
At its core is a decoder with 32 distinct layers. Layers 4 to 27 form a block executed twice in sequence, with shared weights, while layers 0-3 and 28-31 handle input and output. The result is 56 layer executions per forward pass, without duplicating the parameters of the repeated block. Total parameters are about 9 billion, inference parameters about 14.6 billion.
Image: sensenova — https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview
The second component is hybrid attention: layers alternate in groups of four, three linear-attention layers followed by one full-attention layer. The loop boundary always falls on a complete group, so the pattern stays identical across both passes. On top of this, two-stream manifold-constrained hyper-connections (mHC) let learned mappings decide how each layer reads the streams, mixes them and writes its output back. Sinkhorn normalization constrains the mixing map, with 20 iterations in the documented configuration.
The documentation reports a maximum training sequence length of 262,144 tokens, with data covering text, images, code and tool-use trajectories. Accelerated inference via MTP is available, while the Dspark and Dflash2 methods are still in training. Practical speedups depend on workload, hardware and serving configuration.
Image: OpenSenseNova — https://github.com/OpenSenseNova/torchtitan/tree/main/examples/train/SenseNova-Loop-9B-Preview
In practice, the model loads with Transformers 5.18.0 and PyTorch 2.11.0 with CUDA 13.0 and requires trust_remote_code because it uses a custom implementation. The repository also includes a fine-tuning recipe with mHC on a single node with eight GPUs, built on torchtitan. The technical report is not yet published and the model card lists it as to be announced.
The release arrives in a crowded season of open models built to run locally, from Qwen3.8-27B to NVIDIA's Nemotron family. The difference here is in approach: not just more parameters, but more computational depth obtained by reusing layers already trained.
No comments yet. Be the first!