Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Build Ai Assistants Using Large Language Models And Agents To Enhance The Engineering Education Of Biomechanics

Date
Publisher
arXiv
While large language models (LLMs) have demonstrated remarkable versatility across a wide range of general tasks, their effectiveness often diminishes in domain-specific applications due to inherent knowledge gaps. Moreover, their performance typically declines when addressing complex problems that require multi-step reasoning and analysis. In response to these challenges, we propose leveraging both LLMs and AI agents to develop education assistants aimed at enhancing undergraduate learning in biomechanics courses that focus on analyzing the force and moment in the musculoskeletal system of the human body. To achieve our goal, we construct a dual-module framework to enhance LLM performance in biomechanics educational tasks: 1) we apply Retrieval-Augmented Generation (RAG) to improve the specificity and logical consistency of LLM's responses to the conceptual true/false questions; 2) we build a Multi-Agent System (MAS) to solve calculation-oriented problems involving multi-step reasoning and code execution. Specifically, we evaluate the performance of several LLMs, i.e., Qwen-1.0-32B, Qwen-2.5-32B, and Llama-70B, on a biomechanics dataset comprising 100 true/false conceptual questions and problems requiring equation derivation and calculation. Our results demonstrate that RAG significantly enhances the performance and stability of LLMs in answering conceptual questions, surpassing those of vanilla models. On the other hand, the MAS constructed using multiple LLMs demonstrates its ability to perform multi-step reasoning, derive equations, execute code, and generate explainable solutions for tasks that require calculation. These findings demonstrate the potential of applying RAG and MAS to enhance LLM performance for specialized courses in engineering curricula, providing a promising direction for developing intelligent tutoring in engineering education.
Study at a glance
  • Question: Can retrieval augmentation and multi-agent systems improve language models' ability to solve undergraduate biomechanics problems?
  • Setting and data: 100 conceptual true/false questions and 30 calculation problems sourced from undergraduate and graduate biomechanics instructional materials.
  • Method: Model or benchmark evaluation comparing baseline models, retrieval-augmented models, and multi-agent systems across repeated trials.
Why it matters

General-purpose language models often hallucinate and struggle with complex, multi-step engineering reasoning. Finding ways to ground models and automate multi-step calculations is needed to build reliable AI assistants for specialized courses.

Key findings
  • Retrieval-Augmented Generation noticeably improved accuracy across all tested models on conceptual true/false questions, with Qwen-2.5-32B reaching the highest answer stability at 97%.
  • On multi-step calculation problems, a hybrid multi-agent system combining Qwen2.5-VL and Mistral achieved 81.7% accuracy, nearly matching a fully multimodal GPT-4o setup at 82.0%.
  • Domain-specific prompt framing and lower temperature settings consistently improved both answer accuracy and output stability across evaluated models on conceptual biomechanics tasks.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Who is the user?
Who age?