Date
Publisher
arXiv
Generative artificial intelligence (GenAI) tools such as ChatGPT have attracted growing attention in higher education, particularly in relation to how students perceive their usefulness, usability, and educational value. This study develops an exploratory Monte Carlo simulation framework for quantifying perception-based student success in the context of GenAI use. A PRISMA-informed structured literature search in Scopus identified nineteen empirical studies published between 2023 and 2025, of which six reported item-level means and standard deviations suitable for probabilistic modelling. One coherent 10-item, 5-point Likert-scale usability-oriented instrument was selected as a canonical proof-of-concept dataset and used to parameterise an inverse-variance-weighted Monte Carlo simulation generating 10,000 synthetic observations. The results show that the weighting structure substantially influences the simulated outcome, with System Efficiency and Learning Burden receiving the largest inverse-variance weight and therefore the strongest influence on the composite score. The study offers a transparent, reproducible, and privacy-preserving proof-of-concept framework linking structured literature search, item-level summary statistics, and probabilistic modelling.
Study at a glance
- Question: How can computer simulations use published survey data to measure students' perceived success with generative AI?
- Setting and data: 10,000 synthetic observations simulated from published survey statistics of undergraduate health sciences students (Veras et al., 2024).
- Method: Structured literature search and Monte Carlo simulation with inverse-variance weighting.
Why it matters
Research on student perceptions of generative AI in higher education is largely descriptive. The field lacks structured quantitative frameworks to translate perception data into composite indicators under uncertainty.
Key findings
- The simulated perception-based student success score averaged 4.07 on a 5-point scale, indicating generally favorable student evaluations of ChatGPT.
- System efficiency and learning burden had the largest influence on the overall score because it had the lowest variance among the evaluated themes.
- Ease of use, learnability, and system efficiency each averaged 4.12, while perceived complexity and integration scored lower at 3.71.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
