Interactions with Prompt Problems: A New Way to Teach Programming with Large Language Models

Authors

James Prather,

Paul Denny,

Juho Leinonen,

David H. Smith IV,

Brent N. Reeves,

Stephen MacNeil,

Brett A. Becker,

Andrew Luxton-Reilly,

Thezyrie Amarouche,

Bailey Kimmel

Date

01/2024

Publisher

arXiv

Link

http://arxiv.org/pdf/2401.10759v1

Large Language Models (LLMs) have upended decades of pedagogy in computing education. Students previously learned to code through \textit{writing} many small problems with less emphasis on code reading and comprehension. Recent research has shown that free code generation tools powered by LLMs can solve introductory programming problems presented in natural language with ease. In this paper, we propose a new way to teach programming with Prompt Problems. Students receive a problem visually, indicating how input should be transformed to output, and must translate that to a prompt for an LLM to decipher. The problem is considered correct when the code that is generated by the student prompt can pass all test cases. In this paper we present the design of this tool, discuss student interactions with it as they learn, and provide insights into this new class of programming problems as well as the design tools that integrate LLMs.

Study at a glance

Question: How do students interact with an AI tool that teaches programming by having them write prompts?
Setting and data: Not stated in the abstract
Method: Tool design description and discussion of student interactions

Why it matters

Large Language Models have upended computing education pedagogy because free code generation tools can solve introductory programming problems with ease. Previously, students learned to code through writing small problems with less emphasis on reading and comprehension.

Key findings

In introductory computing education, free AI code generation tools can easily solve programming problems described in natural language.
Students engage with new coding exercises by translating visual input-output transformations into prompts for an AI tool.
Student prompts are considered correct when the code generated by the AI successfully passes all required test cases.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

What is the application?

Teaching – Assessment and Feedback,

Learning – Student Support

Who is the user?

Student

Who age?

Post-Secondary

Why use AI?

Outcomes – Other Academic,

Outcomes – Durable Skills,

Reimagined Schooling