Search and Filter

Help Or Hype? Students' Engagement And Perception Of Using Ai To Solve Physics Problems

Date

AI Overview

Why it matters

Generative AI can personalize learning and solve physics problems, but it frequently struggles with calculation errors and inconsistent outputs. Educators need insight into how students interact with AI tools to support conceptual learning rather than replacing it.

Study at a glance
  • Question: How do introductory physics students interact with ChatGPT during problem solving, and what are their perceptions of using AI?
  • Setting and data: Undergraduate students in a calculus-based introductory physics course at a northeastern U.S. community college, Fall 2024 and Spring 2025 semesters, 49 students.
  • Method: Mixed methods analysis comparing assessment scores before and after AI access, combined with qualitative analysis of prompt transcripts and reflections.
  • AI tools: ChatGPT
Key findings
  • Students who engaged with AI tools improved by an average of 0.43 points, whereas those who chose not to use AI saw an average decrease of 1.8 points.
  • Students who provided detailed problem context and asked conceptual questions received more accurate AI assistance and achieved larger gains than students seeking procedural problem-solving steps.
  • About 20 percent of students chose not to use ChatGPT, primarily citing confidence in their own understanding or unfamiliarity with the tool.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

With the rise of large language models such as ChatGPT, interest has grown in understanding how these tools influence learning in STEM education, including physics. This study explores how students use ChatGPT during a physics problem-solving task embedded in a formal assessment. We analyzed patterns of AI usage and their relationship to student performance. Findings indicate that students who engaged with ChatGPT generally performed better than those who did not. Particularly, students who provided more complete and contextual prompts experienced greater benefits. Further, students who demonstrated overall positive gains collectively asked more conceptual questions than those who exhibited overall negative gains. However, the presence of incorrect AI-generated responses also underscores the importance of critically evaluating AI output. These results suggest that while AI can be a valuable aid in problem solving, its effectiveness depends significantly on how students use it, reinforcing the need to incorporate structured AI-literacy into STEM education.