Date
Publisher
arXiv
Evolving technologies have traditionally influenced pedagogical practices and Generative AI is one such technology that promises to transform higher education. In this study, we investigate the association between introductory students' preferences for chatbot behavior and their epistemological beliefs surrounding physics. Our context involves a custom built online module on waves containing simulations integrated with a chatbot. While students' chatbot preferences were captured through three provided options (guided-inquiry, direct answer, and a combination of inquiry and answer), their epistemological beliefs were captured through the standardized Epistemological Beliefs Assessment for Physical Sciences (EBAPS) survey. Results highlight that students who preferred chatbots that initially engage them in guided-inquiry but provide answers when explicitly sought (`Combination'), demonstrated sophisticated epistemological beliefs than those who preferred answer-providing chatbots. Notably, we did not observe any association between the EBAPS' total scores among students who preferred guided-inquiry and those who preferred answer-oriented chatbots. Furthermore, the observed differences did not remain statistically significant after applying a Bonferroni-adjusted significance level. Implications of these results for the design and instructional use of chatbots in physics education are discussed.
Study at a glance
- Question: How do introductory physics students' preferences for chatbot behavior relate to their epistemological beliefs?
- Setting and data: Introductory physics course at a US midwestern university, Spring 2026, 1,048 students with overlapping survey and activity responses.
- Method: Survey analysis comparing group differences across preference choices (Kruskal-Wallis tests and post-hoc Dunn's tests).
Why it matters
While students increasingly use artificial intelligence for coursework, little is known about whether they prefer guided-inquiry or direct-answer chatbots, or how these preferences connect to their epistemological beliefs about knowledge.
Key findings
- Most students preferred combination chatbots (52%) over direct-answer chatbots (39.6%) or guided questions and answers alone (7.6%).
- At standard significance levels, students preferring combination chatbots had higher physics epistemological belief scores than those preferring direct answers, particularly regarding evolving knowledge and learning ability.
- The observed differences across chatbot preference groups were not statistically significant after applying a more conservative Bonferroni correction across multiple tests.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
