Date
Publisher
arXiv
As artificial intelligence (AI) continues to advance, it demonstrates capabilities comparable to human intelligence, with significant potential to transform education and workforce development. This study evaluates OpenAI o1-preview's ability to perform higher-order cognitive tasks across 14 dimensions, including critical thinking, systems thinking, computational thinking, design thinking, metacognition, data literacy, creative thinking, abstract reasoning, quantitative reasoning, logical reasoning, analogical reasoning, and scientific reasoning. We used validated instruments like the Ennis-Weir Critical Thinking Essay Test and the Biological Systems Thinking Test to compare the o1-preview's performance with human performance systematically. Our findings reveal that o1-preview outperforms humans in most categories, achieving 150% better results in systems thinking, computational thinking, data literacy, creative thinking, scientific reasoning, and abstract reasoning. However, compared to humans, it underperforms by around 25% in logical reasoning, critical thinking, and quantitative reasoning. In analogical reasoning, both o1-preview and humans achieved perfect scores. Despite these strengths, the o1-preview shows limitations in abstract reasoning, where human psychology students outperform it, highlighting the continued importance of human oversight in tasks requiring high-level abstraction. These results have significant educational implications, suggesting a shift toward developing human skills that complement AI, such as creativity, abstract reasoning, and critical thinking. This study emphasizes the transformative potential of AI in education and calls for a recalibration of educational goals, teaching methods, and curricula to align with an AI-driven world.
Study at a glance
- Question: How well does an advanced AI model perform on complex thinking tasks compared to humans?
- Setting and data: Validated cognitive test scores from OpenAI o1-preview and humans, including psychology students.
- Method: Comparison of AI test results with human performance across 14 higher-order thinking dimensions.
Why it matters
As artificial intelligence continues to advance, it demonstrates capabilities comparable to human intelligence, with significant potential to transform education and workforce development. This creates a need to systematically evaluate AI's higher-order cognitive capabilities against human performance.
Key findings
- OpenAI o1-preview outperformed humans by 150% across most higher-order thinking categories, including systems thinking, computational thinking, data literacy, and scientific reasoning.
- The AI tool underperformed humans by approximately 25% in critical thinking, logical reasoning, and quantitative reasoning.
- Both the AI and humans achieved perfect scores in analogical reasoning, but human psychology students outperformed the tool in abstract reasoning.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
