AI’s ‘Last Exam’: Google Gemini Nears Human-Level Scores on Tough New Test

The pursuit of artificial general intelligence (AGI) – machines capable of understanding, learning, and applying knowledge across a wide range of tasks at a human level – has long been a central goal in the field of artificial intelligence. A newly published benchmark, dubbed “Humanity’s Last Exam” (HLE), is offering a challenging new yardstick for measuring progress toward that goal. Created jointly by the Center for AI Safety and Scale AI, the exam isn’t designed to celebrate success, but to rigorously assess the limits of current AI models and, perhaps, signal how far we are from truly intelligent machines.

Launched in January 2025, HLE consists of 2,500 questions spanning over 100 subjects, drawing on expertise from more than 1,000 subject-matter experts across 50 countries. The test isn’t about recalling facts readily available online. questions are specifically designed to be “unambiguous and easily verifiable but cannot be quickly answered by internet retrieval.” This focus on genuine understanding, rather than information access, sets it apart from many existing AI benchmarks.

A Benchmark Built to Stump AI

The origins of HLE can be traced back to concerns that existing AI benchmarks were becoming too easily mastered by increasingly powerful language models. As Stanford HAI’s AI Index 2025 Annual Report noted, many popular benchmarks had reached “saturation,” meaning they no longer effectively differentiated between models. Dan Hendrycks, director of the Center for AI Safety, was reportedly inspired to create a more demanding test after a conversation with Elon Musk, who felt existing benchmarks were too lenient. The resulting exam is a deliberate attempt to push the boundaries of AI capabilities.

The creation process itself was rigorous. Initial questions were screened by AI models; those answered correctly were discarded. The remaining questions – those that stumped the AI – were then vetted by human experts in a two-round review process. A bug bounty program further refined the dataset, identifying and correcting errors. The final 2,500 questions represent a formidable challenge, generally requiring PhD-level expertise or knowledge of highly specific topics.

The subject matter is diverse, encompassing mathematics (41%), physics (9%), biology/medicine (11%), humanities/social science (9%), computer science/artificial intelligence (10%), engineering (4%), chemistry (7%), and other fields (9%). Example questions range from trivia – “In Greek mythology, who was Jason’s maternal great-grandfather?” – to complex physics problems involving forces, and motion. This breadth is intentional, aiming to assess a model’s general knowledge rather than specialized skills.

Current Performance and the Pursuit of AGI

Initial testing in February 2026 revealed that even the most advanced AI models struggle with HLE. OpenAI’s GPT-4o and o1 models, Google’s Gemini 1.5 Pro, Anthropic’s Claude 3.5 Sonnet, and DeepSeek R1 were among the first to be evaluated. OpenAI’s o1 system achieved the highest score at launch, but still only managed 8.3%. However, progress is rapid. As of February 12, 2026, Google’s Gemini 3 Deep Think has achieved a score of 48.4%, demonstrating significant improvement. Human experts, by comparison, typically score around 90% in their respective fields.

Despite this progress, the creators of HLE are cautious about interpreting these results as evidence of imminent AGI. They emphasize that a high score on the exam demonstrates expert-level performance on verifiable questions, but does not necessarily indicate autonomous research capabilities or true general intelligence. Manuel Schottdorf, a neuroscientist at the University of Delaware, echoed this sentiment, stating that solving the questions is a necessary, but not sufficient, condition for achieving true intelligence.

Beyond Memorization: Addressing the Limitations of Existing Benchmarks

HLE distinguishes itself from other benchmarks, such as the Massive Multitask Language Understanding (MMLU) dataset, by focusing on genuine understanding rather than memorization or simple web searches. MMLU, while valuable, tends to concentrate on a narrower range of expert-level knowledge, particularly in coding and mathematics. Even benchmarks like Francois Chollet’s ARC-AGI suite, designed to be more challenging, can be susceptible to models exploiting memorization and searchability. Gemini’s Deep Think, for example, achieved a high score on ARC-AGI-2 but still fell short of 50% on HLE.

The exam’s design specifically addresses these limitations by prioritizing questions that require deep reasoning and cannot be easily answered through information retrieval. This focus on higher-order cognitive skills is crucial for assessing progress toward AGI.

What Does This Mean for the Future of AI?

Humanity’s Last Exam provides a valuable tool for tracking the evolution of AI capabilities. While it doesn’t offer a definitive answer to when AGI might be achieved, it does offer a more rigorous and comprehensive assessment of current models than many existing benchmarks. The exam’s creators hope it will serve as a common reference point for scientists and policymakers, facilitating more informed discussions about the development and governance of AI.

The ongoing development and refinement of HLE – including the recent release of a dynamic fork version, HLE-Rolling – demonstrate a commitment to maintaining its relevance and challenge. As AI models continue to improve, the exam will likely evolve to keep pace, pushing the boundaries of what’s possible and providing a clearer picture of the path toward artificial general intelligence.

Looking Ahead: The Center for AI Safety and Scale AI are continuing to analyze the results of HLE and gather feedback from the AI community. Further research will focus on identifying the specific types of questions that pose the greatest challenges for AI models, and on developing new benchmarks that address emerging capabilities. The ongoing process of evaluation and refinement is essential for ensuring that benchmarks remain relevant and informative as the field of AI continues to advance.