Date
Publisher
arXiv
Interprofessional education has long relied on case studies and the use of standardized patients to support teamwork, communication, and related collaborative competencies among healthcare professionals. However, traditional approaches are often limited by cost, scalability, and inability to mimic the dynamic complexity of real-world clinical scenarios. To address these challenges, we designed and developed AIMS (AI-enhanced Immersive Multidisciplinary Simulations), a virtual simulation that integrates a large language model (Gemini-2.5-Flash), a Unity-based virtual environment engine, and a character creation pipeline to support synchronized, multimodal interactions between the user and the virtual patient. AIMS was designed to enhance collaborative clinical reasoning and health promotion competencies among students from pharmacy, medicine, nursing, and social work. A formal usability testing session was conducted in which participants assumed professional roles on a healthcare team and engaged in a mix of scripted and unscripted conversations. Participants explored the patient's symptoms, social context, and care needs. Usability issues were identified (e.g., audio routing, response latency) and used to guide subsequent refinements. Findings suggest that AIMS supports realistic, profession-specific, and contextually appropriate conversations. We discuss technical innovations of AIMS and conclude with future directions.
Study at a glance
- Question: What usability issues arise when healthcare students interact with an AI-driven virtual patient simulation?
- Setting and data: Four healthcare students across medicine, pharmacy, nursing, and social work at a major southeastern U.S. research university.
- Method: System development and a usability testing session analyzed using Nielsen's Usability Severity Rating Scale.
Why it matters
Traditional healthcare training uses case studies that lack real-world complexity and actor simulations that carry high costs, creating an urgent need for realistic, scalable virtual alternatives.
Key findings
- Usability testing uncovered 22 total issues across both clinical scenarios, including 3 catastrophic, 9 major, 9 minor, and 1 cosmetic issue.
- Catastrophic failures involved speech recognition and audio input breakdowns, such as the system mistakenly capturing facilitator speech as user input.
- Major issues included audiovisual desynchronization between speech and avatar lip movements, microphone control challenges, virtual patient interruptions, and participant confusion about clinical objectives.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
