Date
Publisher
arXiv
The increasing availability and use of artificial intelligence (AI) tools in
educational settings has raised concerns about students' overreliance on these
technologies. Overreliance occurs when individuals accept incorrect
AI-generated recommendations, often without critical evaluation, leading to
flawed problem solutions and undermining learning outcomes. This study
investigates potential factors contributing to patterns of AI reliance among
undergraduate students, examining not only overreliance but also appropriate
reliance (correctly accepting helpful and rejecting harmful recommendations)
and underreliance (incorrectly rejecting helpful recommendations). Our approach
combined pre- and post-surveys with a controlled experimental task where
participants solved programming problems with an AI assistant that provided
both accurate and deliberately incorrect suggestions, allowing direct
observation of students' reliance patterns when faced with varying AI
reliability. We find that appropriate reliance is significantly related to
students' programming self-efficacy, programming literacy, and need for
cognition, while showing negative correlations with post-task trust and
satisfaction. Overreliance showed significant correlations with post-task trust
and satisfaction with the AI assistant. Underreliance was negatively correlated
with programming literacy, programming self-efficacy, and need for cognition.
Overall, the findings provide insights for developing targeted interventions
that promote appropriate reliance on AI tools, with implications for the
integration of AI in curriculum and educational technologies.
Study at a glance
- Question: What factors influence how undergraduate students rely on AI suggestions when solving problems?
- Setting and data: Undergraduate students solving programming problems.
- Method: Controlled experiment using problem-solving tasks with an AI assistant, combined with pre- and post-surveys.
Why it matters
Increasing use of AI tools in education raises concerns that students will overrely on these technologies. Accepting incorrect AI recommendations without critical evaluation can lead to flawed problem solutions and undermine learning outcomes.
Key findings
- Among undergraduate programming students, appropriately accepting or rejecting AI suggestions was linked to higher coding skills and confidence, but lower AI trust.
- Accepting incorrect AI recommendations was significantly correlated with higher post-task trust in and satisfaction with the AI assistant.
- Incorrectly rejecting helpful AI suggestions was linked to lower coding skills, lower confidence, and less desire for complex thinking.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
