Date
Publisher
arXiv
As Artificial Intelligence (AI) tools become increasingly embedded in higher
education, understanding how students interact with these systems is essential
to supporting effective learning. This study examines how students' AI literacy
and prior exposure to AI technologies shape their perceptions of Socratic Mind,
an interactive AI-powered formative assessment tool. Drawing on
Self-Determination Theory and user experience research, we analyze
relationships among AI literacy, perceived usability, satisfaction, engagement,
and perceived learning effectiveness. Data from 309 undergraduates in Computer
Science and Business courses were collected through validated surveys. Partial
least squares structural equation modeling showed that AI literacy - especially
self-efficacy, conceptual understanding, and application skills - significantly
predicts usability, satisfaction, and engagement. Usability and satisfaction,
in turn, strongly predict perceived learning effectiveness, while prior AI
exposure showed no significant effect. These findings highlight that AI
literacy, rather than exposure alone, shapes student experiences. Designers
should integrate adaptive guidance and user-centered features to support
diverse literacy levels, fostering inclusive, motivating, and effective
AI-based learning environments.
Study at a glance
- Question: How do students' AI literacy and prior AI exposure influence their experiences using an AI assessment tool?
- Setting and data: Higher education setting with 309 undergraduate students in computer science and business courses.
- Method: Validated surveys analyzed using structural equation modeling (partial least squares structural equation modeling).
Why it matters
As artificial intelligence tools become increasingly embedded in higher education, understanding how students interact with these systems is essential to supporting effective learning.
Key findings
- Among 309 undergraduate business and computer science students, higher AI literacy predicted greater perceived usability, satisfaction, and engagement with an AI assessment tool.
- Students' perceived usability and satisfaction with the tool strongly predicted how effectively they felt it supported their learning.
- Prior exposure to AI technologies did not significantly influence students' satisfaction, engagement, or perceived usability of the tool.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
