Search and Filter

Large Language Models For Education And Research: An Empirical And User Survey-Based Analysis

Date

AI Overview

Why it matters

No comprehensive evaluation has systematically compared ChatGPT and DeepSeek across both technical performance and user perceptions within education and research settings.

Study at a glance
  • Question: How do ChatGPT and DeepSeek compare in technical performance and user perceptions for educational and research tasks?
  • Setting and data: Survey of undergraduate and graduate students, faculty, and researchers, primarily in computer science; plus Codeforces programming problems and subject-matter test sets.
  • Method: Model benchmarking across academic tasks combined with a user survey containing multiple-choice and open-ended questions.
  • AI tools: ChatGPT, DeepSeek, GPT-4, DeepSeek-V3, DeepSeek-R1-Zero
Key findings
  • DeepSeek achieved a 100% success rate across evaluated Codeforces programming tasks, while ChatGPT occasionally failed due to wrong answers, compile errors, or time-limit exceptions.
  • Both models attained 100% correctness on tested chemistry problem sets, successfully solved calculus-based derivations, and reached roughly 90% accuracy on gold-standard medical diagnostic tasks.
  • Surveyed users perceived ChatGPT as superior for response accuracy, educational relevance, and writing tasks, while viewing DeepSeek as stronger for technical coding support.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Pretrained Large Language Models (LLMs) have achieved remarkable success across diverse domains, with education and research emerging as particularly impactful areas. Among current state-of-the-art LLMs, ChatGPT and DeepSeek exhibit strong capabilities in mathematics, science, medicine, literature, and programming. In this study, we present a comprehensive evaluation of these two LLMs through background technology analysis, empirical experiments, and a real-world user survey. The evaluation explores trade-offs among model accuracy, computational efficiency, and user experience in educational and research affairs. We benchmarked these LLMs performance in text generation, programming, and specialized problem-solving. Experimental results show that ChatGPT excels in general language understanding and text generation, while DeepSeek demonstrates superior performance in programming tasks due to its efficiency-focused design. Moreover, both models deliver medically accurate diagnostic outputs and effectively solve complex mathematical problems. Complementing these quantitative findings, a survey of students, educators, and researchers highlights the practical benefits and limitations of these models, offering deeper insights into their role in advancing education and research.