Piloting a new online job-scraping tool

As part of extending the data available in its labour market information project – LMI for All – the team at IER is currently piloting a new web scraping technique to collect UK vacancy data. LMI for All brings together different sources of labour market data to support careers advice. It is used by government, careers organisations and schools across the UK and is increasingly influential internationally. The team’s current task is to develop a method to apply the constant stream of updated vacancies to the UK occupational classification system. This data will enable the creation of trend data to analyse real-time labour demand in the UK. Web scraping consists of a computerised method to automatically collect information from across the Internet (e.g. job portals). For the LMI for All pilot, the vacancy information was scraped daily from two of the significant job portals in the UK: The Guardian and Reed, and from February 2019 to date. These two job portals are significant because they receive a high number of visits per day and include lots of information about the jobs being advertised.

Steep fall in vacancies for waiters/waitresses

The data shows some very bad news for some occupations. The obvious example is waitering jobs. Vacancies for these jobs have collapsed during the Covid-19 crisis, as we show below Graph 1 indicates the number of job vacancies posted online (y axis) from February 2019 to April 2020 (x axis).

Graph 1: Trends in job vacancies for waiters/waitresses in the UK

Source: authors’ own data

This stark decline in vacancies for waitering jobs is not a surprise. The hospitality industry has been mothballed during the Covid-19 crisis, with restaurants and hotels closed for business. It is likely that full reopening will be difficult for companies in this industry. Restaurant and hotels will likely operate below capacity. Unfortunately, furloughing just hides looming unemployment for many workers in this industry. Indicatively, the Restaurant Group, which owns Italian American style restaurant chain Frankie and Benny's, announced the closure of 125 restaurants in June with up to 300 job losses. More similar announcements are likely.

In the UK context, these jobs tend to have low job quality as indicated by pay (not untypically minimum wage) and formal skill levels. They can be good jobs, however, in terms of providing people with entry routes into work and for students needing to pay their way through study. They have a socially useful function therefore beyond helping feed customers.

Increased demand for nurses

There is some good news. Other jobs show a step rise in job vacancies. The obvious example is nursing. Not surprisingly given the health crisis, nurses are in big demand, with a steep rise recently in the number of job vacancies, as we show below. Graph 2 indicates the number of job vacancies posted online (y axis) from February 2019 to April 2020 (x axis).

Graph 2: Trends in job vacancies for nurses in the UK

Source: authors’ own data

Nursing jobs tend to be good jobs – at least as measured by skill level. Many also offer career progression even if there are inequities in pay. The problem is that even before the Covid-19 crisis, the UK’s health service was struggling to fill its nursing vacancies. This problem is global. Worldwide there is a shortage of six million nurses. A further one million nurses are needed in Europe alone. In the UK, the Government urgently needs to rethink its workforce planning and delivery for nurses if the country’s shortfall is to be addressed. Given the likely global labour market competition for new nurses, one option would be to focus on retaining older existing nurses. Currently, too many leave the NHS from the age of 50. Understanding why they leave and what would make them stay is now imperative.

Knowing where the jobs are and aren’t will be important

More broadly, this vacancy data shows that the shape of the UK workforce is likely to change, at least in the short-term. The impact of the Covid-19 crisis on jobs will not be even. The stock, or volume, of some jobs will decline, others will increase. To capture these changes, IER is creating a dataset that provides key insights into the UK labour market. In real time, we will be able to analyse demand for particular occupations and the skills needed within these occupations. As the UK economy contracts over the year, identifying where the job opportunities exist will be vital, not just for career advisers and those working in job centres but also for the unemployed as they retrain or new entrants as they come on to the labour market looking for jobs.