Skip to main content Skip to navigation

Vacancy Data

Vacancy Data.


Job portals have become a focal point for researchers and policymakers due to their cost-effectiveness and ability to offer comprehensive labour market data. At IER data group, we have been developing a vast vacancy database sourced from the main UK job portals since February 2019. Our project is funded by the UK Department for Education, enabling us to conduct groundbreaking research and analysis that drives change in the labour market.

Our process begins with the use of advanced web scraping techniques to automatically collect vast volumes of information from these portals. This raw data is then cleaned and standardised to ensure consistency and reliability. Each job vacancy is coded using various variables, including text patterns within the postings, geographical areas, required skills, and sector classifications such as the Standard Industry Classification (SIC). To further enhance our analysis, we use sophisticated tools like CASCOT to identify occupational groups based on the SOC2020 and ISCO 08 classification system. This enables us to categorise job titles accurately and efficiently.

Luke Bosworth (Senior Project Assistant), Rosie Day (Computing Officer), Dr Jeisson Cardenas-Rubio (Senior Research Fellow).