OpenAI has published an evaluation focused on low-resource programming languages, meaning languages with little code and documentation available for training models. Among them are COBOL, Fortran and Visual Basic, still used in banks, public bodies and large companies, where many critical systems still run on code written decades ago.
The core of the announcement is MultiLangBench, a benchmark that asks the model to build from scratch a SQLite-compatible database engine in 20 different languages. The key figure is that GPT-6 Astra reaches an average test pass rate of 87.4%, against 26.8% for GPT-5.6 Sol. The improvement covers all 20 languages tested. The benchmark has been published on GitHub, so anyone can reproduce it.
COBOL migration
The second test covers translating COBOL programs into C++, Java, Kotlin, Python and Rust, a task companies face when they want to modernize their systems without losing business logic built up over years. Here GPT-6.1 Sol reaches 86.6% accuracy with low reasoning effort, while GPT-5.6 Sol scores 5.4% under the same conditions. With high reasoning effort the result rises to 97.9%. According to OpenAI, the newer model also costs 39% less than the previous one.
The lighter GPT-6 Luna outperforms GPT-5.6 Sol in every language, with an average of 40.3% and an estimated cost of about $0.77 per run, against $3.48 for the previous model.
Why it matters for businesses
According to OpenAI, COBOL still supports more than 40% of online banking systems worldwide, 80% of in-person credit card transactions and 95% of ATM transactions. The issue is therefore not academic: companies that must modernize these systems want tools that understand the logic of old code without changing its behavior. The GPT-6 models already available are the starting point for these tests.
OpenAI itself notes that the results come from its own tests and that expert review and thorough checks are needed before using such tools in production. The company is also asking developers and businesses for feedback on which languages and use cases to add to future evaluations.
No comments yet. Be the first!