China's AI Benchmark Crisis: When Near-Perfect Scores Stop Meaning Anything
For years, the fastest way to judge a large language model was to look at its score on a public benchmark. In October 2026, that habit is breaking down. According to the third-party evaluation aggregator BenchLM, of the 710 benchmarks it tracks as of October 26, 295 are percentage-based tests with meaningful coverage — and 105 of them, roughly 36%, are now effectively saturated, with the top-ranked model scoring above 90 on every one.
When every leading model crowds the ceiling, a one- or two-point gap no longer tells a buyer which system is stronger. The old leaderboards are approaching full marks just as newer, more practical evaluations are producing a different picture: models that ace the textbook questions collectively fail the hands-on tests. The result is a credit reassessment of AI rankings themselves, as evaluation turns into a business and companies choosing a model start calculating cost per task instead of chasing a rank.
The technical gap behind that shift is visible in fresh research. A paper jointly produced by Tsinghua University and Zhipu AI, with Tang Jie as corresponding author, introduces a model called GLM-Omni. It reports 54% accuracy on image generation but only 14.57% on text question answering — a split that shows how a single system can look strong on one leaderboard and fragile on another. Social media users quickly linked the paper to a possible GLM-5.5 release, though the paper does not confirm any version mapping.
Zhipu is one of China’s best-known AI model developers, spun out of Tsinghua research and listed in Hong Kong. The GLM-Omni results suggest the next round of competition will not be won by topping a single chart. Unified models that handle vision and language together are being judged on whether they can actually complete work: reading a document, generating a usable image, answering a customer question without hallucinating.
The same logic is pushing the evaluation industry itself toward harder tests. Benchmarks built on multiple-choice questions are cheap to run and easy to game through training-data overlap; agentic tasks that require a model to plan, use tools and recover from errors are expensive and slow but far closer to what enterprises pay for. The 36% saturation rate is a warning that the easy measurements have been used up.
For global buyers, the practical consequence is a change in procurement. Instead of asking which model ranks first, teams are increasingly running their own small trials on real workflows and comparing total cost, including inference fees and retries. Chinese model makers, facing intense domestic price competition, have an incentive to publish results on these harder evaluations — but also to cherry-pick the ones they win.
The next signal will be whether independent evaluators can keep pace. If saturated benchmarks keep dominating headlines while practical tests stay fragmented, the gap between reputation and delivery will widen — and the companies that build their own evaluation harnesses will be the ones making the better bets.