Date
AI Overview
Why it matters
Aligning educational offerings with labor market demands requires matching course syllabi with job skills, yet research has largely focused on employers and candidates rather than educational institutions.
Study at a glance
- Question: How accurately can language models match university course descriptions to standardized occupational skills?
- Setting and data: Five Finnish universities, 2019–2025 graduate-level course descriptions, matched to European Skills, Competences, Qualifications and Occupations (ESCO) skills.
- Method: Dataset construction and model benchmark evaluation comparing bi-encoders, BERT, and domain-specific models with synthetic data augmentation.
- AI tools: GPT-4o, BERT, ESCOXLM-R, ESCOXLM-R_ENG, jobGBERT, me5-base-course-skill
Key findings
- A baseline BERT model combining course titles and individual sentences achieved an 87% F1-score and 89% recall for matching courses to skills.
- Training models with synthetic course sentences generated by GPT-4o increased recall for positive skill matches by 6% to 8%.
- Domain-specific models showed lower recall than the baseline BERT model, though ESCOXLM-R_ENG achieved the highest accuracy at 86%.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Skill extraction and recommendation systems have been studied from recruiter, applicant, and education perspectives. While AI applications in job advertisements have received broad attention, deficiencies in the instructed skills side remain a challenge. In this work, we address the scarcity of publicly available datasets by releasing both manually annotated and synthetic datasets of skills from the European Skills, Competences, Qualifications and Occupations (ESCO) taxonomy and university course pairs and publishing corresponding annotation guidelines. Specifically, we match graduate-level university courses with skills from the Systems Analysts and Management and Organization Analyst ESCO occupation groups at two granularities: course title with a skill, and course sentence with a skill. We train language models on this dataset to serve as a baseline for retrieval and recommendation systems for course-to-skill and skill-to-course matching. We evaluate the models on a portion of the annotated data. Our BERT model achieves 87% F1-score, showing that course and skill matching is a feasible task.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
