Search and Filter

What Makes An Expert? Comparing Problem-Solving Practices In Data Science Notebooks

Date

AI Overview

Why it matters

Data science requires procedural skills that are often tacit and difficult to teach directly. Generative AI tools risk reinforcing superficial, novice habits unless educators understand how expertise develops.

Study at a glance
  • Question: How do the problem-solving processes and coding patterns of novice and expert data scientists differ in computational notebooks?
  • Setting and data: 440 Kaggle Jupyter notebooks from the Quora Insincere Questions Classification competition, spanning 339 novices and 101 experts.
  • Method: Multi-level sequence analysis combining hierarchical clustering, process mining, Markov models, and sequential pattern mining.
Key findings
  • Overall workflow structure was significantly associated with expertise: novices were overrepresented in long, linear workflows, while experts were overrepresented in shorter, iterative workflows.
  • Experts and novices showed largely similar high-level transition sequences between technical data science phases, with no statistically significant differences in Markov model clusters.
  • Fine-grained coding patterns differed by expertise, with novices repeatedly engineering features or displaying tables, whereas experts more frequently chained model training, compiling, and evaluating actions.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

The development of data science expertise requires tacit, process-oriented skills that are difficult to teach directly. This study addresses the resulting challenge of empirically understanding how the problem-solving processes of experts and novices differ. We apply a multi-level sequence analysis to 440 Jupyter notebooks from a public dataset, mapping low-level coding actions to higher-level problem-solving practices. Our findings reveal that experts do not follow fundamentally different transitions between data science phases than novices (e.g., Data Import, EDA, Model Training, Visualization). Instead, expertise is distinguished by the overall workflow structure from a problem-solving perspective and cell-level, fine-grained action patterns. Novices tend to follow long, linear processes, whereas experts employ shorter, more iterative strategies enacted through efficient, context-specific action sequences. These results provide data science educators with empirical insights for curriculum design and assessment, shifting the focus from final products toward the development of the flexible, iterative thinking that defines expertise-a priority in a field increasingly shaped by AI tools.