Mozilla CEO Satya Nadella has called for a more cautious approach to AI development, suggesting that models should be designed with an 'emergency brake' feature. In a recent post on X, Nadella highlighted the need for a 'trust architecture' around AI, warning against treating it as a 'set of nested black boxes'. He proposed that models be 'separated from the harness that orchestrates its work' and that 'external controls and safeguards' be implemented.

This comes as major AI companies grapple with incidents where their models seemed to lose control. Microsoft CEO Nadella is not the first to call for increased scrutiny, but his comments have resonated given his position at a leading tech company. He suggested that every 'meaningful model action' should be documented with 'tamper-proof human-readable evidence', and that systems should allow for 'an authorized person' to 'pause or shut down a model mid-task'.

His comments have been met with both interest and skepticism. Critics argue that such measures could stifle innovation, while others believe that without such precautions, AI could pose significant risks to society. Some see this as a sign of the growing concerns around AI, with leading companies now acknowledging the need for more cautious development.

Anthropic CEO Dario Amodei has also proposed a plan for more cautious AI development, indicating a shift in the industry towards a more cautious approach.