Microsoft CEO: In the era of superintelligence, we cannot rely on AI "black boxes"; we need to establish external control and audit systems.

PANews, October 11 news: Microsoft CEO Satya Nadella posted that as the capabilities of Super Intelligence systems rapidly improve, the industry needs to rebuild the AI trust architecture and cannot place the responsibility for the actions performed by AI on model providers. Over the past few decades, when traditional software systems were widely deployed, people could trace the specific code paths corresponding to program behavior. But today's frontier AI models, although their capabilities have surpassed those of traditional software systems, cannot clearly attribute model outputs to specific training data or model weight configurations.

PANews, October 11 news: Microsoft CEO Satya Nadella posted that as the capabilities of Super Intelligence systems rapidly advance, the industry needs to rebuild the AI trust architecture and must not place responsibility for the consequences of AI execution behavior on model providers.

Over the past few decades, when traditional software systems were widely deployed, people could trace the specific code paths corresponding to program behavior. But today's frontier AI models, although their capabilities already exceed those of traditional software systems, cannot clearly attribute model outputs to specific training data or model weight configurations. Enterprises are currently exposing complex AI agents to sensitive data and granting them the ability to perform critical tasks, so Super Intelligence systems cannot be treated as incomprehensible "nested black boxes," simply accepting or rejecting their suggestions; instead, controllable systems must be built that can observe behavior, test boundaries, and restrict actions at any time.

Regarding governance principles, Nadella proposed that Super Intelligence systems need to have:

- Model diversity: avoid relying on a single model as the sole dependency for critical decisions;

- Comprehensive observability: all important AI behavior should leave tamper-proof and human-readable records;

- Continuous verification capability: continuously test system performance in the face of failures, attacks, and anomalies;

- Independent control and audit: models should not simultaneously control system behavior and the evidence verifying their own behavior;

- Risk isolation mechanisms: assume by default that models may fail, and provide the ability to pause or shut down at any time;

- Incident disclosure mechanisms: promptly disclose causes and improvement measures when failures or security incidents occur.

Nadella also emphasized that chain-of-thought (CoT) transparency is an important foundation for AI governance, but CoT transparency alone is not enough, because it is still impossible to guarantee that model outputs always have reliable interpretability. The most trustworthy Super Intelligence systems in the future are not the "most trustworthy models," but systems that can still operate safely even if the models cannot be fully trusted.