Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Globalizeed: A Multimodal Translation System That Preserves Speaker Identity In Academic Lectures

Date
Publisher
arXiv
A large amount of valuable academic content is only available in its original language, creating a significant access barrier for the global student community. This is a challenge for translating in several subjects, such as history, culture, and the arts, where current automated subtitle tools fail to convey the appropriate pedagogical tone and specialized meaning. In addition, reading traditional automated subtitles increases cognitive load and leads to a disconnected learning experience. Through a mixed-methods study involving 36 participants, we found that GlobalizeEds dubbed formats significantly reduce cognitive load and offer a more immersive learning experience compared to traditional subtitles. Although learning effectiveness was comparable between high-quality subtitles and dubbed formats, both groups valued GlobalizeEds ability to preserve the speakers voice, which enhanced perceived authenticity. Instructors rated translation accuracy and vocal naturalness, whereas students reported that synchronized, identity-preserving outputs fostered engagement and trust. This work contributes a novel human-centered AI framework for cross-lingual education, demonstrating how multimodal translation systems can balance linguistic fidelity, cultural adaptability, and user control to create more inclusive global learning experiences.
Study at a glance
  • Question: How does an identity-preserving, multimodal translation system affect teaching perceptions and student learning experiences compared to subtitles?
  • Setting and data: 18 multilingual instructors (mostly university level) and 18 university students (Vietnamese and international).
  • Method: Mixed methods system evaluation and a within-subjects comparative experiment with surveys and interviews.
Why it matters

Valuable academic lectures are often trapped in their original language. Automated subtitle tools struggle to convey pedagogical tone and specialized meaning, while reading subtitles increases cognitive load and causes a disconnected learning experience.

Key findings
  • Most students (77.7%) preferred dubbed formats over subtitled ones, reporting higher concentration (mean 4.06 out of 5) and reduced cognitive effort.
  • Instructors evaluated the system positively for teaching effectiveness and student understanding (both mean 4.28 out of 5), though vocal naturalness scored lower at 3.67.
  • Learning effectiveness ratings were comparable between LLM-translated subtitles and dubbed formats (median 4.0 out of 5), while lip synchronization showed no statistically significant quality improvement.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?
Who age?