When OpenAI released its purported solutions to some of the world’s toughest math problems, it consulted a group of top mathematicians to avoid controversy. However, the lab's latest release has fallen short of these mathematicians' standards, particularly regarding the need for human understanding of the results.

The Advisory Group on Mathematics and Artificial Intelligence (AGMAI) issued guidelines for solving mathematical problems with AI. OpenAI's latest proofs, while transparent about the model's conclusions, did not fully adhere to these guidelines. Only 10 out of 719 papers included the model's reasoning process, and just 42% underwent formalization, raising concerns about the reliability of the proofs.

One of the main issues highlighted is the translation of natural language proofs into formal code. Discrepancies between the language explanation and the code have been noted, calling into question the trustworthiness of AI-generated proofs. The AGMAI suggests that OpenAI should fund human mathematicians to ensure the meaningfulness of these AI solutions.

Mathematician Terence Tao, known for criticizing OpenAI’s approach, noted that AI models often lack the broader field's interest and understanding, making it difficult for them to contribute meaningfully. The paper 'Lost in Translation' further emphasizes the need for human oversight in the formalization process, suggesting that peer review is essential.

This latest development underscores the ongoing challenge of ensuring that AI-generated solutions are both accurate and comprehensible, highlighting the importance of human engagement in the mathematical community.