Date
Publisher
arXiv
Recent advancements in artificial intelligence (AI) are fundamentally
reshaping computing, with large language models (LLMs) now effectively being
able to generate and interpret source code and natural language instructions.
These emergent capabilities have sparked urgent questions in the computing
education community around how educators should adapt their pedagogy to address
the challenges and to leverage the opportunities presented by this new
technology. In this working group report, we undertake a comprehensive
exploration of LLMs in the context of computing education and make five
significant contributions. First, we provide a detailed review of the
literature on LLMs in computing education and synthesise findings from 71
primary articles. Second, we report the findings of a survey of computing
students and instructors from across 20 countries, capturing prevailing
attitudes towards LLMs and their use in computing education contexts. Third, to
understand how pedagogy is already changing, we offer insights collected from
in-depth interviews with 22 computing educators from five continents who have
already adapted their curricula and assessments. Fourth, we use the ACM Code of
Ethics to frame a discussion of ethical issues raised by the use of large
language models in computing education, and we provide concrete advice for
policy makers, educators, and students. Finally, we benchmark the performance
of LLMs on various computing education datasets, and highlight the extent to
which the capabilities of current models are rapidly improving. Our aim is that
this report will serve as a focal point for both researchers and practitioners
who are exploring, adapting, using, and evaluating LLMs and LLM-based tools in
computing classrooms.
Study at a glance
- Question: How should computing educators adapt their teaching to navigate the challenges and opportunities of generative AI?
- Setting and data: Computing students and instructors across 20 countries, 22 educators across five continents, 71 research articles, and computing education datasets.
- Method: Literature review, survey, in-depth interviews, ethical framework analysis, and testing AI model performance on computing datasets.
Why it matters
As large language models fundamentally reshape computing by generating and interpreting code, computing educators face urgent questions about how to adapt their pedagogy to address the challenges and leverage the opportunities presented by this new technology.
Key findings
- In computing education across five continents, 22 educators have already adapted their curricula and assessments in response to generative artificial intelligence.
- Large language models can effectively generate and interpret both computer programming code and natural language instructions.
- Performance evaluations on computing education datasets show that artificial intelligence model capabilities are rapidly improving.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
