Search and Filter

Supporting Tutors In The Gig Economy With Automated Feedback: A Case Study On Ringle

Date

AI Overview

Why it matters

Online tutoring platforms rely on learner feedback to evaluate quality, but such ratings are often overly positive, lack actionable guidance, and make monitoring tutor quality difficult at scale.

Study at a glance
  • Question: How do online English tutors perceive and respond to AI-powered automated feedback alongside learner ratings?
  • Setting and data: Ringle online tutoring platform; 36 newly onboarded tutors, 327 tutoring lessons, and 10,000 learner feedback records from 6,256 learners.
  • Method: Mixed methods study using surveys, statistical score comparisons (Wilcoxon signed-rank tests), and inductive thematic analysis of open-ended responses.
Key findings
  • Tutors rated automated feedback significantly more negatively than learner feedback across understanding, accuracy, fairness, favorableness, evaluator qualification, and impact, though feedback uptake showed no difference.
  • Automated feedback provided a broader, more fine-grained distribution of scores across lessons, whereas learner feedback was heavily skewed toward positive ratings or frequently missing.
  • Tutors valued automated feedback for clarifying platform expectations and supporting self-monitoring, but score discrepancies with learner feedback caused confusion and distrust.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

The rise of online tutoring platforms in the gig economy has made education more scalable, flexible, and on-demand. These platforms rely on learner evaluations as the primary feedback for tutors and platforms. However, such feedback offers limited guidance for tutors' improvement and makes it difficult to monitor tutor quality at scale. To this end, we explored AI-powered automated feedback and how tutors perceive and respond to it. We deployed a research probe on Ringle, a popular online English tutoring platform, that analyzed tutors' lessons and provided automated feedback. We then surveyed 36 tutors about their experience. Our findings reveal that while tutors perceived automated feedback more negatively than learner feedback, they found it useful for self-monitoring and understanding platform expectations, though discrepancies between them often caused confusion. Based on these insights, we propose design considerations for feedback systems for online educational gig platforms.