Aligning Learners' Expectations and Performance by Learning Analytics System with a Predictive Model

Sasa Brdnik, Bostjan Sumak, Vili Podgorelec

Date

11/2022

AI Overview

Why it matters

Higher education institutions in Slovenia have lagged in learning analytics adoption, and existing dashboard research rarely integrates student expectations into user-centered solutions designed to support self-regulated learning.

Study at a glance

Question: What do students expect from learning analytics, and how can those expectations shape a predictive learning dashboard?
Setting and data: University of Maribor in Slovenia; 276 survey respondents, 19 focus group students, and course records from 106 engineering undergraduates (fall 2021/22).
Method: Mixed methods combining a questionnaire, focus groups, machine learning model development, and a 10-student usability test.

Key findings

A Random Forest classification model predicted students at risk of failing with 98% precision after the first month of the course.
A Decision Tree Regressor predicted students' exact final grades with a mean absolute error of 11.2 points on a 100-point scale after month one.
Survey and focus group results revealed students placed the highest value on data privacy and security, followed by service features such as peer comparisons.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Learning analytics (LA) is data collection, analysis, and representation of data about learners in order to improve their learning and performance. Furthermore, LA opens the door to opportunities for self-regulated learning in higher education, a circular process in which learners activate and sustain behaviours that are oriented toward their personal learning goals. The potentials of LA and self-regulated learning are huge; however, they are not yet widely applied in higher education institutions. Slovenian higher education institutions have lagged behind other European countries in LA adoption. Our research aims to fill this gap by using a qualitatively and quantitatively led workflow for building a requirement-oriented LA solution, consisting of empirically gathering the students' expectations of LA and presenting a dashboard solution. Translated Student Expectations of Learning Analytics Questionnaire and focus groups were used to gather expectations from learners. Based on them, a user interface utilizing LA and grade prediction with an AI model was implemented for a selected course. The interface includes early grade prediction, peer comparison, and historical data overview. Grade prediction is based on a machine learning model built on users' interaction in the virtual learning environment, demographic data and lab grades. First, classification is used to determine students at risk of failing - its precision reaches 98% after the first month of the course. Second, the exact grade is predicted with the Decision Tree Regressor, reaching a mean absolute error of 11.2grade points (on a 100 points scale) after the first month. The proposed system's main benefit is the support for self-regulation of the learning process during the semester, possibly motivating students to adjust their learning strategies to prevent failing the course. Initial student evaluation showed positive results.