Date
AI Overview
Why it matters
Generative AI tools provide synthesized, ready-to-use information, raising critical questions about whether these tools impact student learning and whether their effects vary across learning tasks of different cognitive complexities.
Study at a glance
- Question: How do generative AI, search engines, and e-textbooks influence students' learning performance and long-term retention across different cognitive tasks?
- Setting and data: 123 undergraduate information systems students at a private university.
- Method: Randomized controlled laboratory experiment with pretests, immediate quizzes, and a three-week retention post-test.
- AI tools: ChatGPT
Key findings
- On immediate lower-order recall tasks, the ChatGPT group scored significantly higher than the e-textbook and control groups, with mean differences of 16.60 and 22.39 points.
- On immediate applying tasks, both the ChatGPT and Google groups scored significantly higher than the control group, with mean differences of 20.15 and 18.81 points.
- Three weeks later, ChatGPT and Google groups experienced retention drops on recall tasks, aligning with e-textbook scores, while no significant tool differences emerged on higher-order tasks.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
The rise of Generative AI (GenAI) tools, such as ChatGPT, has transformed how
students access and engage with information, raising questions about their
impact on learning outcomes and retention. This study investigates how GenAI
(ChatGPT), search engines (Google), and e-textbooks influence student
performance across tasks of varying cognitive complexity, based on Bloom's
Taxonomy. Using a sample of 123 students, we examined performance in three
tasks: [1] knowing and understanding, [2] applying, and [3] synthesizing,
evaluating, and creating. Results indicate that ChatGPT and Google groups
outperformed the control group in immediate assessments for lower-order
cognitive tasks, benefiting from quick access to structured information.
However, their advantage diminished over time, with retention test scores
aligning with those of the e-textbook group. For higher-order cognitive tasks,
no significant differences were observed among groups, with the control group
demonstrating the highest retention. These findings suggest that while
AI-driven tools facilitate immediate performance, they do not inherently
reinforce long-term retention unless supported by structured learning
strategies. The study highlights the need for balanced technology integration
in education, ensuring that AI tools are paired with pedagogical approaches
that promote deep cognitive engagement and knowledge retention.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
