Date
Publisher
arXiv
Online learning has experienced rapid growth due to its flexibility and
accessibility. Personalization, adapted to the needs of individual learners, is
crucial for enhancing the learning experience, particularly in online settings.
A key aspect of personalization is providing learners with answers customized
to their specific questions. This paper therefore explores the potential of
Large Language Models (LLMs) to generate personalized answers to learners'
questions, thereby enhancing engagement and reducing the workload on educators.
To evaluate the effectiveness of LLMs in this context, we conducted a
comprehensive study using the StackExchange platform in two distinct areas:
language learning and programming. We developed a framework and a dataset for
validating automatically generated personalized answers. Subsequently, we
generated personalized answers using different strategies, including 0-shot,
1-shot, and few-shot scenarios. The generated answers were evaluated using
three methods: 1. BERTScore, 2. LLM evaluation, and 3. human evaluation. Our
findings indicated that providing LLMs with examples of desired answers (from
the learner or similar learners) can significantly enhance the LLMs' ability to
tailor responses to individual learners' needs.
Study at a glance
- Question: Can AI generate personalized answers to learners' questions in online learning?
- Setting and data: Questions and answers from the StackExchange platform covering language learning and programming.
- Method: Evaluates AI-generated answers across different prompting strategies using automated metrics, AI review, and human evaluation.
Why it matters
Providing learners with answers customized to their specific questions is crucial for enhancing online learning. Generating these personalized answers with Large Language Models could enhance engagement while reducing the workload on educators.
Key findings
- In online language learning and programming forums, providing language models with desired answer examples significantly improved their ability to meet individual learners' needs.
- Example answers taken from either the target learner or similar learners effectively helped the models personalize their responses.
- Human and automated evaluations confirmed that language models can generate customized answers to specific learner questions across technical and language subjects.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
