How to test GenAI’s impact on learning


 In a futuristic classroom or lab, a large holographic screen prominently displays a glowing human brain at its center, surrounded by various metrics like "GENAI IMPACT ASSESSMENT," "CREATIVITY INDEX," and "CRITICAL THINKING SCORE." Several individuals, some wearing VR/AR headsets, are engaged with individual holographic desks showing similar data, actively analyzing GenAI's effects on learning. Image (and typos) generated by Nano Banana.
As generative AI becomes more prevalent, understanding its true impact on student learning is paramount. This image envisions a sophisticated approach to assessing GenAI’s influence, utilising advanced metrics and holographic displays to quantify and analyse its effects on creativity, critical thinking, and overall educational outcomes. Image (and typos) generated by Nano Banana.

Source

Times Higher Education

Summary

Thibault Schrepel argues against speculation and for empirical classroom experiments to measure how generative AI truly affects student learning. He outlines simple, scalable experimental designs—e.g. groups forbidden from AI, groups using it without guidance, groups trained in prompting and critique—to compare outcomes in recall, writing quality, and reasoning. Schrepel also suggests activities like having students build AI research assistants, comparing human and AI summaries, and using AI as a Socratic tutor. He emphasises that AI won’t uniformly help or hurt; its impact depends on how it’s used, taught, and assessed.

Key Points

  • Use controlled classroom experiments with different levels of AI access/training to reveal real effects.
  • Recall or rote learning may not change much; AI’s effects show more in reasoning, argumentation and writing quality.
  • Activities like comparing AI vs human summaries or having AI play the role of interlocutor can highlight strengths and limitations.
  • Prompting, critique, and metacognitive reflection are central to converting AI from crutch to tool.
  • Banning AI outright is less useful than enabling pedagogical experimentation and shared insight across faculty.

Keywords

URL

https://www.timeshighereducation.com/campus/how-test-genais-impact-learning

Summary generated by ChatGPT 5