Date
Publisher
arXiv
We present a method for generating large numbers of isomorphic physics
problems using ChatGPT through prompt chaining and tool use. This approach
enables precise control over structural variations-such as numeric values and
spatial relations-while supporting diverse contextual variations in the problem
body. By utilizing the Python code interpreter, the method supports automatic
solution validation and simple diagram generation, addressing key limitations
in existing LLM-based methods. We generated two example isomorphic problem
banks and compared the outcome against simpler prompt-based approaches. Results
show that prompt-chaining produces significantly higher quality and more
consistent outputs than simpler, non-chaining prompts. This work demonstrates a
promising method for efficient problem creation accessible to the average
instructor, which opens new possibilities for personalized adaptive testing and
automated content development.
Study at a glance
- Question: Can connecting prompts and computer tools help ChatGPT reliably generate variations of physics problems?
- Setting and data: Two banks of generated physics problems
- Method: Compares problems created using linked prompts and tools against problems made with simpler prompts
Why it matters
Existing large language model methods have key limitations in reliably generating isomorphic physics problems, particularly in validating solutions and generating diagrams. Instructors also need precise control over structural and contextual variations.
Key findings
- In physics, chaining ChatGPT prompts produced significantly higher quality and more consistent problems than using simpler, single-step prompts.
- ChatGPT achieved precise control over numbers and spatial relationships while generating diverse contexts for structurally identical physics problems.
- Pairing ChatGPT with a coding tool enabled it to automatically verify solutions and generate simple diagrams.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
