Providers are fuming at the Education and Skills Funding Agency after officials started conducting short-notice mystery audits in the wake of the 3aaa scandal, FE Week can reveal.
In what appears to be the second major investigation into achievement rate data at FE providers in the past ten years, the agency began carrying out a significant number of data reliability checks at providers in February.
They’re understood to have been prompted by ESFA’s concern at the unpublished results of the qualification achievement rates for 2017/18, as well as its investigation into disgraced apprenticeship firm Aspire Achieve Advance (3aaa). The company went bust in October after the government pulled its skills funding contracts following allegations of fraud.
They would not say what they were looking for and haven’t been in touch since
It is understood the agency will soon write to the whole FE sector expressing concern over data gaming, just as Geoff Russell, then-chief executive of the Learning and Skills Council, infamously did in 2010.
FE Week has spoken with numerous providers who have experienced the new audits, which have all followed the exact same format.
They were given two days’ notice to hand over up to 100 apprentice files – dating back to 2015/16. Staff from ESFA’S provider risk and assurance team and its counter-fraud team then turned up onsite to go through the files.
After two days of working through the files they left, without taking away any of the material. Providers were not told what they had been looking for, nor anything about what the officials found.
One provider, who was angered by one of these audit visits, and who did not wish to be named, told FE Week: “The ESFA staff refused to say why they needed to audit a huge number of apprentice files. This was also the experience of other providers we have spoken to.
“They would not say what they were looking for and haven’t been in touch since they spent two days reviewing the files. Gathering so much paperwork for them with just two days’ notice took a significant amount of resource, and to not tell us why really makes me angry.”
FE Week understands that since the demise of 3aaa the ESFA has found that significant sums of funding was paid for learners at the provider that should have been withdrawn.
This inflated not only the company’s funding, but also their achievement rates – something that almost resulted in a second ‘outstanding’ rating from Ofsted last May.
FE Week understands the ESFA is now urgently looking into whether other providers have inappropriately taken funding and boosted achievement rates for apprentices, misusing reporting of withdrawals as well as the way they recorded breaks in learning and transfers.
The agency has now closed a loophole relating to the coding of transfers.
When asked about the mystery audits, including their scope and purpose, an ESFA spokesperson would only say that the agency’s “priority” is to “protect learners and ensure public money is being used effectively by training providers and these checks are a key part of our assurance process”.
He added: “As recent cases have demonstrated, if we find evidence of any wrongdoing we will not hesitate to take appropriate action.”
The audits took place in February as this is the window between providers giving feedback on provisional 2017/18 qualification achievement rate data and the date on which it becomes final.
It is understood that providers with what is considered to be suspicions or unreliable data will be excluded from the official figures when they are released later this year.
Writing in his weekly update on Thursday, Association of Employment and Learning Providers boss Mark Dawe warned his members that the ESFA is “looking at the data far more closely, running various reports to check integrity and also coming to the provider for clear evidence”.
He added there will be “less tolerance for ‘mistakes’ and no tolerance for potential manipulation”.
This all comes at a time when Ofsted is turning its focus to providers’ “integrity” during inspections.
The shaky history of achievement rate data
Achievement rate data has been an increasing cause for concern among ESFA officials in recent years.
In early 2017 the agency admitted to a “loophole” relating to breaks in learning in its calculation for 2014/15, which, until it was closed “artificially”, boosted the rate for around 10 per cent of providers.
Later in 2017 the ESFA attempted to hide the botched data by publishing revised national achievement rate tables for 2015/16 for individual providers but without comparable figures for previous years, which are needed to give any kind of indication of providers’ progress.
Activities in 2010 were a response to circumstances at the time
But following a call for an investigation into the agency’s failure to be forthcoming with the necessary data, the ESFA U-turned and eventually released the comparable figures.
And in the 2015/16 achievement rate data some providers, notably the biggest college group in the country, NCG, were absent from the tables owing to what they claimed were “data glitches”.
After being omitted, Ofsted delayed its inspection into NCG as the inspectorate couldn’t analyse the group’s achievement rates. When Ofsted eventually went knocking in 2018 it downgraded NCG from ‘good’ to ‘requires improvement’.
And during the course of its enquiries into the now-defunct 3aaa, Ofsted admitted to FE Week that the inspectorate didn’t visit providers with big falls in achievement rates because they had “limited resources”.
The ESFA’s audits appear to be a re-run of what happened 10 years ago, which led to Ofsted doing their own data checks ahead of inspection, but which stopped only a few years later.
In 2009 the Learning and Skills Council – the predecessor organisation to the Skills Funding Agency – along with Ofsted, the Data Service and the Information Authority looked into “suggestions that data manipulation goes beyond routine data cleansing to improve the accuracy of the data”.
They concluded: “Some of the practices identified at the fact-finding visits have led to an artificial increase in success rates.”
Geoff Russell then wrote to the sector (see here) to “ensure this practice ceases with immediate effect”.
FE Week asked Ofsted why it had stopped carrying out the data checks that it had introduced following the Russell letter, to which a spokesperson replied: “Activities in 2010 were a response to circumstances at the time. ESFA are responsible for the quality and reliability of provider data and they have been taking forward this role ever since.”
Asked if Ofsted plans to change its approach, such as returning to checking data, under the new Education Inspection Framework set to be released this year, the spokesperson said: “Ofsted inspectors pursue relevant lines of enquiry during inspection, and will continue to do so under the new framework.”
He would not say whether the education watchdog currently has full confidence in the credibility of official achievement rate data.
Your thoughts