Date
Publisher
arXiv
A large amount of valuable academic content is only available in its original
language, creating a significant access barrier for the global student
community. This is a challenge for translating in several subjects, such as
history, culture, and the arts, where current automated subtitle tools fail to
convey the appropriate pedagogical tone and specialized meaning. In addition,
reading traditional automated subtitles increases cognitive load and leads to a
disconnected learning experience. Through a mixed-methods study involving 36
participants, we found that GlobalizeEds dubbed formats significantly reduce
cognitive load and offer a more immersive learning experience compared to
traditional subtitles. Although learning effectiveness was comparable between
high-quality subtitles and dubbed formats, both groups valued GlobalizeEds
ability to preserve the speakers voice, which enhanced perceived authenticity.
Instructors rated translation accuracy and vocal naturalness, whereas students
reported that synchronized, identity-preserving outputs fostered engagement and
trust. This work contributes a novel human-centered AI framework for
cross-lingual education, demonstrating how multimodal translation systems can
balance linguistic fidelity, cultural adaptability, and user control to create
more inclusive global learning experiences.
Study at a glance
- Question: How does an identity-preserving, multimodal translation system affect teaching perceptions and student learning experiences compared to subtitles?
- Setting and data: 18 multilingual instructors (mostly university level) and 18 university students (Vietnamese and international).
- Method: Mixed methods system evaluation and a within-subjects comparative experiment with surveys and interviews.
Why it matters
Valuable academic lectures are often trapped in their original language. Automated subtitle tools struggle to convey pedagogical tone and specialized meaning, while reading subtitles increases cognitive load and causes a disconnected learning experience.
Key findings
- Most students (77.7%) preferred dubbed formats over subtitled ones, reporting higher concentration (mean 4.06 out of 5) and reduced cognitive effort.
- Instructors evaluated the system positively for teaching effectiveness and student understanding (both mean 4.28 out of 5), though vocal naturalness scored lower at 3.67.
- Learning effectiveness ratings were comparable between LLM-translated subtitles and dubbed formats (median 4.0 out of 5), while lip synchronization showed no statistically significant quality improvement.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
