Date
Publisher
arXiv
Large language models (LLMs) and prompt engineering hold significant
potential for advancing computer programming education through personalized
instruction. This paper explores this potential by investigating three critical
research questions: the systematic categorization of prompt engineering
strategies tailored to diverse educational needs, the empowerment of LLMs to
solve complex problems beyond their inherent capabilities, and the
establishment of a robust framework for evaluating and implementing these
strategies. Our methodology involves categorizing programming questions based
on educational requirements, applying various prompt engineering strategies,
and assessing the effectiveness of LLM-generated responses. Experiments with
GPT-4, GPT-4o, Llama3-8b, and Mixtral-8x7b models on datasets such as LeetCode
and USACO reveal that GPT-4o consistently outperforms others, particularly with
the "multi-step" prompt strategy. The results show that tailored prompt
strategies significantly enhance LLM performance, with specific strategies
recommended for foundational learning, competition preparation, and advanced
problem-solving. This study underscores the crucial role of prompt engineering
in maximizing the educational benefits of LLMs. By systematically categorizing
and testing these strategies, we provide a comprehensive framework for both
educators and students to optimize LLM-based learning experiences. Future
research should focus on refining these strategies and addressing current LLM
limitations to further enhance educational outcomes in computer programming
instruction.
What is the application?
Who age?
Study design
