Not Even Generative AI’s Developers Fully Understand How Their Models Work


In a futuristic lab or control room, a diverse group of frustrated scientists and developers in lab coats are gathered around a table with laptops, gesturing in confusion. Behind them, a large holographic screen prominently displays "GENERATIVE AI MODEL: UNKNOWABLE COMPLEXITY, INTERNAL LOGIC: BLACK BOX" overlaid on a glowing neural network. Numerous red question marks and "ACCESS DENIED" messages highlight their inability to fully comprehend the AI's workings. Image (and typos) generated by Nano Banana.
Groundbreaking research has unveiled a startling truth: even the developers of generative AI models do not fully comprehend the intricate inner workings of their own creations. This image vividly portrays a team of scientists grappling with the “black box” phenomenon of advanced AI, highlighting the profound challenge of understanding systems whose complexity surpasses human intuition and complete analysis. Image (and typos) generated by Nano Banana.

Source

The Irish Times

Summary

John Thornhill examines the paradox at the heart of the artificial intelligence boom: even the developers of generative AI systems cannot fully explain how their models function. Despite hundreds of billions being invested in the race toward artificial general intelligence (AGI), experts remain divided on what AGI means or whether it is achievable. While industry leaders such as OpenAI and Google DeepMind pursue it with near-religious zeal, critics warn of existential risks and call for restraint. At a Royal Society conference, scholars argued for redirecting research toward tangible, transparent goals and prioritising safety over hype in AI’s relentless expansion.

Key Points

  • Massive investment continues despite no shared understanding of AGI’s meaning or feasibility.
  • Industry figures frame AGI as imminent, while most academics consider it unlikely.
  • Experts highlight safety, transparency, and regulation as neglected priorities.
  • Alan Kay and Shannon Vallor urge shifting focus from “intelligence” to demonstrable utility.
  • Thornhill concludes that humanity’s true “superhuman intelligence” remains science itself.

Keywords

URL

https://www.irishtimes.com/business/2025/10/10/not-even-generative-ais-developers-fully-understand-how-their-models-work/

Summary generated by ChatGPT 5