OpenAI is reportedly racing to finish its next flagship model, which uses the same looped transformer technique as GPT-6 Astra. According to employees cited by The Information, the company will likely release "one or two new models" before the end of the year, though it is unclear whether the major upgraded model it is working on will be among them.
The new flagship could still be some way off. The model needs alignment work so that its goals match what OpenAI's researchers and customers expect, and the company has to shore up the testing infrastructure around it. After leadership is satisfied with safety evaluations, there is one more hurdle: under an agreement governing evaluations of advanced AI systems, OpenAI has agreed to give the US government access to new models up to 30 days before they go to outside partners, and longer still before a public launch.
How looped transformers work
The technique, which OpenAI already runs in production under the name "Recurrent Depth", passes a token through the same layers multiple times before outputting the next word, instead of once. The result is stronger performance in math and coding and lower costs, since a smaller model can punch above its weight.
The trade-off is that part of the model's "thinking" moves into internal numerical representations that human reviewers cannot read. That undermines chain-of-thought monitoring, in which safety researchers trace a model's reasoning to spot bad behavior. For this reason, according to The Information, OpenAI capped how many loops Astra runs in order to keep its chain of thought readable.
More compute for safety
According to a person familiar with the figures, OpenAI has moved roughly 10% of its total compute toward alignment and safety. The Information reports that this amounts to hundreds of megawatts of data center capacity, but it is unclear how much went to safety before or what the current split looks like. AI researcher Noam Brown also said in a mid-September podcast that about ten percent of his team works on security and safety.
The detail Microsoft let slip
How OpenAI actually uses the technique came to light through a slip-up at Microsoft. The major investor, which has access to OpenAI's intellectual property, published a technical write-up on GPT-6.1 Sol, a cheaper Astra alternative that OpenAI launched earlier this month. In a section since deleted, Microsoft described the model as using "looping inference", in which the same problem is worked through in multiple passes before an answer is produced. A person with direct knowledge confirmed the write-up matched what OpenAI had told Microsoft in private. Looped transformers remain a sensitive part of OpenAI's model development, both in training and at inference.
OpenAI chief scientist Jakub Pachocki has already warned that chain-of-thought monitoring is growing "fragile" and "unfortunately trending in a negative direction". He also cautioned against an industry-wide race toward models without readable reasoning.
No comments yet. Be the first!