Date
Publisher
arXiv
Supervised fine-tuning (SFT) of large language models (LLMs) for specialized tasks requires high-quality datasets, but manual curation is prohibitively expensive. Synthetic data generation offers scalability, but its effectiveness relies on complex, multi-stage workflows, integrating prompt engineering and model orchestration. Existing automated workflow methods face a cold start problem: they require labeled datasets for reward modeling, which is especially problematic for subjective, open-ended tasks with no objective ground truth. We introduce AutoSynth, a framework that automates workflow discovery and optimization without reference datasets by reframing the problem as a Monte Carlo Tree Search guided by a novel dataset-free hybrid reward. This reward enables meta-learning through two LLM-as-judge components: one evaluates sample quality using dynamically generated task-specific metrics, and another assesses workflow code and prompt quality. Experiments on subjective educational tasks show that while expert-designed workflows achieve higher human preference rates (96-99% win rates vs. AutoSynth's 40-51%), models trained on AutoSynth-generated data dramatically outperform baselines (40-51% vs. 2-5%) and match or surpass expert workflows on certain metrics, suggesting discovery of quality dimensions beyond human intuition. These results are achieved while reducing human effort from 5-7 hours to just 30 minutes (>90% reduction). AutoSynth tackles the cold start issue in data-centric AI, offering a scalable, cost-effective method for subjective LLM tasks. Code: https://github.com/bisz9918-maker/AutoSynth.
Study at a glance
- Question: Can an automated search framework generate high-quality synthetic training data for subjective educational tasks without reference datasets?
- Setting and data: Generated datasets of 1,000 examples and test sets of 10 queries per task across math concept explanation and interdisciplinary lesson planning.
- Method: System development and model evaluation using Monte Carlo Tree Search, supervised fine-tuning, and automated and human pairwise evaluations.
Why it matters
Fine-tuning language models requires high-quality training data, but manual curation is expensive. Existing automated workflow methods require pre-existing labeled datasets for reward modeling, which fails for subjective, open-ended tasks lacking reference data or ground truth.
Key findings
- Models trained on AutoSynth-generated data achieved 40% to 51% human evaluation win rates, substantially outperforming the base model's 2% to 5% win rates.
- AutoSynth-trained models earned higher automated metric scores than expert-designed workflows (4.25 versus 4.18 for math, 4.82 versus 4.75 for lesson planning), though experts achieved higher human preference rates.
- AutoSynth reduced human preparation effort by over 90%, requiring 30 minutes of human involvement compared to 5 to 7 hours for expert workflow engineering.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Study design
