Design and implementation of an Al-enabled visual report tool as formative assessment to promote learning achievement and self-regulated learning: An experimental study

Xiaofang Liao, Central China Normal University

Xuedi Zhang, South China Normal University

Zhifeng Wang, Central China Normal University

Heng Luo, Central China Normal University

Date

01/2024

AI Overview

Why it matters

Formative assessment is essential for improving teaching and learning, and artificial intelligence and visualization techniques provide great potential for its design and delivery.

Study at a glance

Question: How do AI-enabled visual formative assessment reports affect high school biology students' achievement, anxiety, and self-efficacy?
Setting and data: 125 ninth-grade biology students; high school classroom; dates not stated.
Method: Experimental study comparing students receiving AI visual reports to those receiving teacher oral feedback.

Key findings

Learning achievement showed a significant interaction effect between the intervention and time.
Students receiving the AI visual reports experienced increases in self-efficacy (Cohen's d = 1.793) and learning anxiety (Cohen's d = 0.203) over time.
Comparing AI visual reports to teacher feedback revealed no significant differences on measures except for increased self-efficacy (Cohen's d = 0.046).

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

Formative assessment is essential for improving teaching and learning, and AI and visualization techniques provide great potential for its design and delivery. Using NLP, cognitive diagnostic and visualization techniques designed to analyse and present students' monthly exam data, we developed an AI-enabled visual report tool comprising six modules and conducted an empirical study of its effectiveness in a high school biology classroom. A total of 125 students in a ninth-grade biology course were assigned to a treatment group (n63) receiving AI-enabled visual reports as the intervention and a control group (n62) receiving overall oral feedback from the teacher. We present the main statistical results of the within-subjects design and the between-subjects design respectively, to better capture the main findings. Repeated measures ANOVA revealed a significant interaction effect of intervention and time on learning achievement, and the paired-sample Wilcoxon test indicated that the treatment group had experienced increasing learning anxiety (Cohen's d0.203, p0.046) and self-efficacy (Cohen's d1.793, p0.000) over time. Moreover, we conducted a series of non-parametric tests to compare the effects of AI-enabled visual reports and teacher feedback, but found no significant differences except for an increased self-efficacy (Cohen's d0.046). Additionally, we had the students in the treatment group rate their favourable modules in the AI-enabled visual report and provide evaluative feedback. The study results provide important insights into the design and implementation of effective formative assessment supported by artificial AI and visualization techniques.