Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

From Self-Crafted to Engineered Prompts: Student Evaluations of AI-Generated Feedback in Introductory Physics

Authors
Amogh Sirnoorkar,
N. Sanjay Rebello
Date
Publisher
arXiv
Link
http://arxiv.org/pdf/2508.09825v1
The abilities of Generative-Artificial Intelligence (AI) to produce real-time, sophisticated responses across diverse contexts has promised a huge potential in physics education, particularly in providing customized feedback. In this study, we investigate around 1200 introductory students' preferences about AI-feedback generated from three distinct prompt types: (a) self-crafted, (b) entailing foundational prompt-engineering techniques, and (c) entailing foundational prompt-engineering techniques along with principles of effective-feedback. The results highlight an overwhelming fraction of students preferring feedback generated using structured prompts, with those entailing combined features of prompt engineering and effective feedback to be favored most. However, the popular choice also elicited stronger preferences with students either liking or disliking the feedback. Students also ranked the feedback generated using their self-crafted prompts as the least preferred choice. Students' second preferences given their first choice and implications of the results such as the need to incorporate prompt engineering in introductory courses are discussed.
Study at a glance
  • Question: What types of AI-generated feedback prompts do introductory physics students prefer?
  • Setting and data: Around 1,200 introductory physics students.
  • Method: Student evaluations and rankings of AI-generated feedback created from three different prompt types.
Why it matters

Generative AI promises huge potential in physics education, particularly in providing real-time, customized feedback. Understanding that potential requires examining student preferences for feedback generated from self-crafted prompts versus foundational prompt-engineering techniques.

Key findings
  • Among around 1,200 introductory physics students, an overwhelming fraction preferred AI feedback generated from structured prompts combining prompt engineering and effective feedback principles.
  • The most popular feedback choice also elicited stronger preferences, with students either liking or disliking the feedback.
  • Students ranked AI feedback generated from their own self-crafted prompts as their least preferred choice.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

What is the application?
Teaching – Assessment and Feedback
Who is the user?
Student
Who age?
Post-Secondary
Why use AI?
Outcomes – Other Academic,
Outcomes – Differentiation
Study design
Quantitative – Others
Download PDF