Search and Filter

Peoat: Personalization-Guided Evolutionary Question Assembly For One-Shot Adaptive Testing

Date

AI Overview

Why it matters

Iterative computerized adaptive tests can be impractical in resource-constrained or sensitive settings with high interaction costs. Selecting all personalized test questions at once without intermediate feedback remains a major challenge.

Study at a glance
  • Question: Can an evolutionary algorithm assemble personalized, one-shot adaptive tests that accurately estimate student abilities?
  • Setting and data: Two educational interaction datasets: JUNYI (54,564 learners, 565 exercises) and PTADisc (18,768 learners, 3,262 exercises).
  • Method: Algorithm benchmark evaluation (combinatorial evolutionary optimization compared against eight adaptive testing baselines).
  • AI tools: PEOAT
Key findings
  • PEOAT outperformed all eight baseline testing methods across both datasets, improving accuracy and diagnostic quality by over five to seven percentage points on average.
  • Performance advantages were especially pronounced at shorter test lengths, outperforming the next-best baseline by over 10 percentage points on JUNYI with five-item tests.
  • Removing personalization-aware initialization caused the largest drop in assessment accuracy, showing that incorporating student ability differences initially is vital.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

With the rapid advancement of intelligent education, Computerized Adaptive Testing (CAT) has attracted increasing attention by integrating educational psychology with deep learning technologies. Unlike traditional paper-and-pencil testing, CAT aims to efficiently and accurately assess examinee abilities by adaptively selecting the most suitable items during the assessment process. However, its real-time and sequential nature presents limitations in practical scenarios, particularly in large-scale assessments where interaction costs are high, or in sensitive domains such as psychological evaluations where minimizing noise and interference is essential. These challenges constrain the applicability of conventional CAT methods in time-sensitive or resourceconstrained environments. To this end, we first introduce a novel task called one-shot adaptive testing (OAT), which aims to select a fixed set of optimal items for each test-taker in a one-time selection. Meanwhile, we propose PEOAT, a Personalization-guided Evolutionary question assembly framework for One-shot Adaptive Testing from the perspective of combinatorial optimization. Specifically, we began by designing a personalization-aware initialization strategy that integrates differences between examinee ability and exercise difficulty, using multi-strategy sampling to construct a diverse and informative initial population. Building on this, we proposed a cognitive-enhanced evolutionary framework incorporating schema-preserving crossover and cognitively guided mutation to enable efficient exploration through informative signals. To maintain diversity without compromising fitness, we further introduced a diversity-aware environmental selection mechanism. The effectiveness of PEOAT is validated through extensive experiments on two datasets, complemented by case studies that uncovered valuable insights.