Blog

A pair of AI systems developed by Google DeepMind made history this week, achieving a score equivalent to a silver medal at the International Mathematical Olympiad (IMO). AlphaProof and AlphaGeometry 2, the two systems, collaborated to tackle six challenging problems from the prestigious competition.

Notably, the combined system solved four out of six problems, earning 28 out of a possible 42 points, just one short of the golden threshold.

Notably, the AI ​​achieved a perfect score on the competition’s most challenging problem, which only five human contestants solved. The remarkable achievement places DeepMind’s AI among the best young mathematical minds in the world.

“What it does is far beyond what a purely brute-force search would be capable of, so there’s clearly something interesting going on when it works,” said Prof Timothy Gowers, a Fields Medalist and former gold winner of the IMO who scored AI on answers.

Different approaches to problem solving

The two systems use different approaches. AlphaProof, a language model combined with reinforcement learning, tackled two algebra problems and one number theory problem. It uses “formal mathematics” to write verifiable mathematical proofs as programs, allowing the system to learn and improve.

On the other hand, AlphaGeometry 2 focuses on geometry questions and amazingly solved its problem in just 16 seconds. His solution involved a creative approach that even surprised human experts, demonstrating AI’s ability to think outside the box.

“There are some legendary examples of [computer-aided] evidence that is longer than Wikipedia. It wasn’t that: we’re talking about a very short human-style score,” added Prof Gowers.

Achievements and limitations

While the AI ​​excelled in some areas, it struggled in others. For two of the six questions, the systems failed to make any progress. Additionally, Google’s AI systems take varying amounts of time to solve problems, ranging from minutes to up to three days.

For reference, while human contestants were given a time limit of nine hours, DeepMind’s AI took three days to solve one particularly difficult problem.

Prof. Gowers, while acknowledging the achievement as “far beyond what automatic theorem provers could do before,” pointed out several important qualifications.

“The main qualification is that the program needed much more time than the human contestants,” he said. “If human contestants were allowed that kind of time per problem, they would undoubtedly have a higher score.”

He also emphasized human involvement in translating issues into official language.

“Are we close to the point where mathematicians are redundant? It’s hard to say. I would guess we’re still a break or two short of that,” Gowers concluded.

Future potential

Despite the limitations, Google DeepMind’s achievement represents a significant step forward in AI’s mathematical reasoning capabilities.

Developing AI systems that can handle complex mathematical problems could have far-reaching implications for fields ranging from scientific research to education.