Date
AI Overview
Why it matters
Large language models often suffer from an artificial hivemind effect that produces repetitive educational content. Maintaining task diversity is crucial to adequately cover learning objectives and different difficulty levels.
Study at a glance
- Question: How can large language models generate a more diverse set of high-utility practice tasks?
- Setting and data: Python programming tasks across 20 contexts, evaluated with automated metrics and three experienced programming education tutors.
- Method: Model or benchmark evaluation and user study with expert clustering, rating, and pairwise comparisons.
- AI tools: Qwen3-235B-A22B, Qwen3-Embedding-0.6B, Gemini 2.5 Flash-Lite
Key findings
- The two-stage framework produced about 1.6 times more distinct, high-utility programming tasks than baseline prompting methods at 50 tasks.
- Expert evaluators clustered high-utility tasks into roughly 16 distinct groups for the two-stage framework, compared to about 9 for chain-of-thought prompting.
- Experts preferred tasks from the two-stage framework in 96.67% of pairwise novelty comparisons and gave them higher interestingness ratings.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Large language models (LLMs) have significant potential for generating educational questions and problems, enabling educators to create large-scale learning materials. However, LLMs are fundamentally limited by the ``Artificial Hivemind'' effect, where they generate similar responses within the same model and produce homogeneous outputs across different models. As a consequence, students may be exposed to overly similar and repetitive LLM-generated problems, which harms diversity of thought. Drawing inspiration from Wallas's theory of creativity and Guilford's framework of divergent-convergent thinking, we propose CreativeDC, a two-phase prompting method that explicitly scaffolds the LLM's reasoning into distinct phases. By decoupling creative exploration from constraint satisfaction, our method enables LLMs to explore a broader space of ideas before committing to a final problem. We evaluate CreativeDC for creative problem generation using a comprehensive set of metrics that capture diversity, novelty, and utility. The results show that CreativeDC achieves significantly higher diversity and novelty compared to baselines while maintaining high utility. Moreover, scaling analysis shows that CreativeDC generates a larger effective number of distinct problems as more are sampled, increasing at a faster rate than baseline methods.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
