The country’s national statistics regulator has given the DfE achievement rate data a clean bill of health but called for greater transparency.
FE Week reported last month the 2017/18 national achievement rate tables included a list of more than 30 apprenticeship providers with data the DfE had admitted was unreliable.
The DfE said at the time the unreliable data was included to “provide a complete view of performance” and acknowledged that by doing so this year’s rate is 1.5 per cent lower than it would be if it was excluded.
Despite including the provider data it they admitted “the data we hold does not allow us to calculate a reliable estimate and therefore provides an unfair measure of performance.”
After FE Week shared the finding with the UK Statistics Authority, which had previously pressured the DfE into changing the way it presents its achievement data for 2015/16, the authority said it would look at the matter further.
The regulator has now sent and published a letter to Neil McIvor, the DfE’s chief data officer and chief statistician, saying it was “content” with the publication of the tables and with “the approach taken to include all data in the aggregate, headline position, particularly given that excluding it risks over-estimating the national position”.
However, it also said “it would be helpful to give more prominence to this in the headline publication and explain the context around the reasons for incomplete individual learner records”.
“In particular, the fact that these provider outcomes can be very unpredictable from year to year could be emphasised as one of the key limitations of the data,” wrote Ed Humpherson, director general for regulation at the authority.
In the same letter, Humpherson said the regulator investigated the “most recent concerns raised by FE Week” and noted that the 34 providers redacted in the formal performance tables have been redacted due to” quality issues”, which are the result of operational, rather than statistical, issues.
“We are aware that you work with the providers on an individual basis to try to resolve these issues,” he said.
We reported at the time that the government’s achievement rate data must comply with the UK Statistic Authority’s “code of practice”, which says official statistics must ensure they are “worthy of trust”.
“The code ensures that the statistics published by government serve the public,” according to the authority’s website.
“When producers of official statistics comply with the code, it gives users of statistics and citizens confidence that published government statistics are of public value, are high quality and are produced by people and organisations that are worthy of trust.”
FE Week analysis of the national achievement rate tables for 2017/18 showed there were 23,940 apprenticeships across 34 providers that have unreliable data, including at two large high-profile colleges.
The total combined cohort for all providers in 2017/18 was 412,190, which means data for 5.8 per cent of the apprenticeships in that year cannot be trusted.
FE Week took a look back at the achievement rate data for 2016/17 and found that the DfE also included the unreliable data in the headline rate in that year. “If we had excluded them for apprenticeships, then the national rate would be 68.6 per cent, which would be 0.9 per cent higher than the published figure,” it said at the time.
In 2016/17 there were 10,610 apprenticeships across 21 providers that were redacted for having unreliable data, which was 2.5 per cent of the 409,020 total cohort.
The largest provider to be excluded for having unreliable data in 2017/18 was the troubled West Nottinghamshire College, which had an apprenticeship cohort of 4,900.
The college run into financial difficulties last year that has led to substantial government bailouts, a high number of job losses, and, ultimately, the resignation of its longstanding, high-profile principal, Dame Asha Khemka.
The other high-profile provider to be excluded from the DfE’s official achievement rate data for 2017/18 was grade one Dudley College.
Its chief executive Lowell Williams last week apologised after FE Week reported how an audit exposed dodgy data with regards to late withdrawals that resulted in more than £500,000 being paid back to the government.
A DfE spokesperson said: “We are pleased the UK Statistics Authority is satisfied with our reporting of the 2017/18 national achievement rate tables and recognise the Data Dissemination project we have undertaken to improve these statistics. We will consider the wider feedback in the letter.”
Your thoughts