Microsoft's Satya Nadella has abandoned his earlier conviction that artificial intelligence alignment would emerge naturally as models grew more powerful. During an appearance on the 20VC podcast, the company's chief executive told host Harry Stebbings about a significant shift in his thinking over the past year.
When asked what assumptions he had reconsidered, Nadella stated: "I used to always think that alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder, different problem."
Nadella acknowledged that he had embraced scaling laws during 2018 and 2019, a period when Dario Amodei, Ilya Sutskever, Sam Altman, Greg Brockman and Elon Musk were, in his words, "one happy family." The scaling principle—that models improve through greater amounts of data and computational resources—has delivered results as predicted. Alignment, the challenge of ensuring AI systems operate consistently with human intentions and values, has proven to be a separate matter entirely.
The training phase holds the key
According to Nadella, alignment work must predominantly occur during the training phase. He suggested it may even require attention during pretraining, the initial and most resource-intensive stage of model development. He also noted that understanding the internal mechanisms of how models function remains scientifically unclear.
On the limitations of current refinement techniques, Nadella remarked: "I don't think you can even RL your way to alignment." Reinforcement learning, or RL, represents the standard method by which research labs fine-tune models following their initial pretraining phase.
Building protective layers around AI systems
Nadella advocates for implementing robust security frameworks surrounding AI models in the near term. He champions zero trust architecture, a security paradigm that subjects every action to verification rather than granting default authorization.
Nadella explained the rationale: "If you really want to have agents that are more capable and yet less aligned, then you need to have a trust system that you can rely on." He highlighted that Microsoft's Agent 365 and Perception platforms assign unique identities to each agent and maintain comprehensive records of their interactions with tools and models. These records must be immutable, preventing agents from altering their own activity logs. Organizations can then establish behavioral policies and monitor compliance. Microsoft introduced additional agent monitoring capabilities in October, and on 10 October, Nadella posted on X that companies should regard AI models as potential insider threats.
Transparency in reasoning and model accountability
Nadella cautioned against placing confidence in chain of thought, the intermediate reasoning steps that models display before producing their final answers.
He elaborated: "We know that the chain of thought itself can be unfaithful, and in fact it can be made worse when the model company says, 'Well, it's unfaithful and I'm going to hide it.'" Though he refrained from identifying any specific organization, Nadella stressed that trustworthy partners should receive chain of thought outputs in accessible natural language. These partners could then conduct audits and cross-validate findings using alternative models. Should developers of proprietary models resist demands for greater openness, "we may have a problem," Nadella warned. Such resistance could accelerate adoption of open-weight models, which permit anyone to obtain and operate their underlying parameters.
The open-source question and global implications
When Stebbings pointed out that the majority of open-weight models originate from China, Nadella responded that misaligned AI poses dangers across geographies. He stated: "An unaligned open-weight Chinese model will probably attack even China."
Nadella confirmed that Microsoft will not distribute its own MAI models as open-weight offerings. Instead, the company positions them as licensed weights, operating them as a conventional software product. Looking ahead, Nadella anticipates that most AI applications will depend on inexpensive commodity tokens drawn from numerous competing models, spanning both proprietary and open architectures.
Microsoft is scheduled to announce its quarterly financial results on Wednesday, according to reporting by Todd Bishop for GeekWire. The earnings report will mark the company's first disclosure since it extended severance packages to approximately 60,000 employees.
Source: The Next Web
