Intern-S2-Mobius shares model memory and shortens visible reasoning

Most inference improvements try to run the same model faster: better kernels, quantization, caches, or speculative prediction. Intern-S2-Mobius, introduced by Shanghai AI Laboratory, makes a different bet. It reorganizes where the model stores knowledge and lets several reasoning blocks query a common memory. The aim is not to make every operation cheaper, but to need fewer sequential operations and less intermediate text to reach an answer.

The release deserves attention because it is more than a diagram. There is an ungated 35-billion-parameter checkpoint under the Apache 2.0 license, with configuration and custom inference code. Its five BF16 shards total about 68 GiB. This is an artifact others can run and inspect, not merely an architectural promise.

Its central claim also requires careful reading. “Nearly four times faster” sounds as though Mobius processes each token at one quarter of the cost. The published data do not show that. They measure completed requests per second, and most of the advantage comes from the model generating much shorter reasoning chains. That could matter more than a faster kernel, but it is a different improvement and calls for different evidence.

One memory for several reasoners

In a conventional Transformer, each layer combines attention—which relates elements of the context—with a feed-forward network often understood as part of the model's knowledge store. Information moves through that sequence. If an answer requires recombining something that was not activated in time, the model can use new tokens as another trip through all the layers.

Mobius decouples those functions. It gathers feed-forward experts into a global sparse Memory and has several attention-based “Reasoners” query it while recurrent hidden states are refined. This does not mean that the Memory contains readable fact cards or that attention is reasoning in the human sense. It is a functional description of learned matrices. The novelty is that knowledge is no longer tied to a single position in the stack and the latent state can iterate before several tokens are decoded.

The released model was not trained from scratch. The team converted Qwen3.5-35B-A3B—a sparse model that activates roughly 3 billion of its 35 billion parameters per token—continued pretraining it, and applied supervised fine-tuning and reinforcement learning. That makes the idea testable at a useful scale, but it complicates attributing the differences solely to architecture: the data and post-training also changed.

The authors include a more controlled experiment with two 7-billion-parameter models trained from scratch. Mobius reached the Transformer's final MMLU score using 62.6% of its data. That is an interesting compression signal, not yet a law of data efficiency. A single general-knowledge benchmark cannot show whether language, code, robustness, or harder-to-score capabilities were preserved equally well.

The 4× is an end-to-end result

Across nine general evaluations, Intern-S2-Mobius reports an average of 67.88 versus 65.05 for Qwen3.5-35B. It wins seven but trails on UGD hard and Humanity's Last Exam. Its lead is far larger on three scientific datasets; without an ablation separating architecture, continued pretraining, and post-training, those numbers do not establish shared memory as the cause.

The throughput chart supplies the crucial qualification. Averaged over five reasoning benchmarks, Mobius completes 2.9 times as many requests per second at batch 16 and 4.6 times as many at batch 256. On some datasets the advantage grows sharply with batch size; on others, the Transformer retains an advantage at some points. The paper itself acknowledges that querying the large sparse Memory adds pressure and makes each pass less efficient. The end-to-end balance improves because Mobius emits fewer visible tokens and because its multi-token prediction head can accept several at once.

Important details are missing before that result can travel. The report does not identify alongside the test the hardware, serving engine, software version, input lengths, exact use of speculative prediction, or a latency distribution. Nor does it publish the OpenCompass configurations used. Without them, “4×” is not yet a deployment capacity budget or an independently reproduced comparison.

Brevity raises another question. In the two worked examples, Mobius reaches the same correct answer while avoiding repeated checks. But two cases cannot show whether it removes useless redundancy or steps that caught errors on harder problems. The authors themselves say that the exact mechanism is not established. A stronger test would compare many runs at equal quality, measure latency tails and energy cost, and look for failures that increase when visible reasoning is compressed.

Open weights enable a better test

The model core can now be downloaded and served through Transformers, LMDeploy, vLLM, or SGLang. Its configuration declares a native 262,144-position context and a vision tower, although the paper reports only text and science results; it does not demonstrate a new multimodal capability here. This is not a small model either: the BF16 weights occupy almost 68 GiB before caches and runtime memory, and the repository does not yet provide an official quantization.

The guide requires trust_remote_code, a practical reason to review and pin that code before running it. The project repository currently contains documentation and examples, while the checkpoint hosts the model implementation. The training set, full recipe, logs, and code needed to repeat the conversion and training are not published. “Open weights” lets others evaluate the result; it does not automatically reproduce the path to it.

My reading is that the most interesting contribution is not a new benchmark summit. It is turning latent reasoning—previously explored by work such as COCONUT—into a practical-scale architecture and shipping the artifact. If independent labs reproduce both quality and end-to-end advantage, the cost of reasoning would depend less on a long token-by-token transcript. That would change service design: the number of tokens a model avoids would matter as much as how quickly it generates each one.

The tradeoff is not free. A shorter trace costs less and may contain less explanatory theater, but it also leaves less surface area for inspecting a failure. Chains of thought were never faithful audits of internal state; hiding more of the process makes it even more important to evaluate outcomes, calibration, and behavior under variation instead of mistaking a fluent explanation for evidence.

Mobius has made the architectural question testable, not settled the industrial answer. The next useful step is easy to state and hard to perform: same hardware, same engine, same questions, equal accuracy, and a complete latency distribution. If the advantage survives that test, “thinking less out loud” will have moved from a style preference to a material infrastructure improvement.

Sources

• Intern-S2-Mobius Team, Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning, preprint submitted August 14, 2026.
• Shanghai AI Laboratory, Intern-S2-Mobius checkpoint, configuration, and model card, accessed August 17, 2026.
• InternLM, Intern-S2-Mobius repository, accessed August 17, 2026.
• Qwen, Qwen3.5-35B-A3B model card, the starting model and Transformer reference.
• Hao et al., Training Large Language Models to Reason in a Continuous Latent Space, the COCONUT work on continuous latent reasoning.