Date
Publisher
arXiv
One-on-one tutoring is an effective instructional method for enhancing learning, yet its efficacy hinges on tutor competencies. Novice math tutors often prioritize content-specific guidance, neglecting aspects such as social-emotional learning. Social-emotional learning promotes equity and inclusion and nurturing relationships with students, which is crucial for holistic student development. Assessing the competencies of tutors accurately and efficiently can drive the development of tailored tutor training programs. However, evaluating novice tutor ability during real-time tutoring remains challenging as it typically requires experts-in-the-loop. To address this challenge, this preliminary study aims to harness Generative Pre-trained Transformers (GPT), such as GPT-3.5 and GPT-4 models, to automatically assess tutors' ability of using social-emotional tutoring strategies. Moreover, this study also reports on the financial dimensions and considerations of employing these models in real-time and at scale for automated assessment. The current study examined four prompting strategies: two basic Zero-shot prompt strategies, Tree of Thought prompt, and Retrieval-Augmented Generator (RAG) based prompt. The results indicate that the RAG prompt demonstrated more accurate performance (assessed by the level of hallucination and correctness in the generated assessment texts) and lower financial costs than the other strategies evaluated. These findings inform the development of personalized tutor training interventions to enhance the the educational effectiveness of tutored learning.
Study at a glance
- Question: Can AI models effectively and affordably assess tutors' use of social-emotional learning strategies?
- Setting and data: Not stated in the abstract
- Method: Compares four AI prompting strategies across two language models to assess tutor practices and costs.
Why it matters
Novice math tutors often neglect social-emotional learning, which promotes equity and student development. However, evaluating tutor ability during real-time tutoring remains challenging because it typically requires experts.
Key findings
- In math tutoring, generative AI supplied with reference materials evaluated tutors' social-emotional practices more accurately and with fewer errors than other tested methods.
- Generative AI supported by reference data incurred lower financial costs than alternative prompting approaches for automated tutor assessment.
- Novice math tutors often prioritize subject-matter instruction while neglecting social-emotional strategies that nurture student relationships.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
