One of England’s largest apprenticeship providers was left out of this year’s qualification achievement rates (QARs) after a systems error caused the duplication of 700 learners, leaving its figures “unreliable”.
Several other big providers that were included in the government’s official performance tables told FE Week they experienced similar problems but did not spot the issues until after the deadline for challenges, adding that their true rates are higher than those published.
But officials have insisted the issue was not widespread and is understood to not have had a significant impact on national results.
ILR transitions create duplications
Apprenticeship achievement rates were published at national and provider-level on March 31 for the 2021/22 academic year.
The data showed that the achievement rate for standards stood at 51.4 per cent, down 0.4 percentage points on the year before and a long way off the 67 per cent achievement rate target that ministers have set for 2025.
Two providers were redacted from the formal QAR dataset – one of which was Paragon Education and Skills Ltd, a long-running apprenticeship firm that had 3,270 leavers in 2021/22.
The company’s commercial director Paul O’Hagan told FE Week the redaction was based on “an error in one field of our ILR (individualised learner record) upload” caused after Paragon migrated to a new systems provider.
“There was a duplication of 700 learners at R04,” he explained. “What that meant was our stated QAR was coming out 10 percentage points less than it actually was. The transition created the duplication, and this was not picked up by the ESFA data self-assessment toolkit or the software we use to check success rates from multiple year ILRs.”
O’Hagan said the ESFA was “very supportive in terms of redacting the data based on the premise that it wasn’t reflecting of actually where we are as a business from a QAR perspective”, adding that the firm’s systems provider was “very open and very supportive with taking all the actions, so it’s not an issue moving forward”.
When the data duplication was removed, Paragon’s QAR moved from 39 per cent to about 49.9 per cent.
Nine of the 20 largest apprenticeship providers in England are recorded as having overall achievement rates of below 50 per cent in the DfE’s official QAR data.
They include the likes of Lifetime Training (45 per cent), Babington Business College (48.1 per cent), and HIT Training (36.1 per cent).
Several other providers that spoke to FE Week said they experienced similar data issues to Paragon – there was a problem with learner identification during their migration to new systems, which counted some leavers twice.
Jill Whittaker, managing director of HIT Training, said this impacted her provider, leaving its QAR around five percentage points lower than what it truly is.
Some of those affected said they experienced the issue when moving to Bud Systems.
A spokesperson for Bud said: “Bud was made aware of a system error that affected a small number of our customers. Bud worked closely with our customers and the ESFA to diagnose the issue and the extent of the impact. The ESFA concluded that the error did not have a significant impact on national QAR production. This issue has now been rectified within the software.”
A DfE spokesperson added: “We have a clear and robust process for calculating QARs based on data submitted by providers, who are responsible for the quality of their own data.
“Every year, providers have had the opportunity to report any issues regarding QAR, and we have worked with them to address these concerns. Any concerns raised have been fed into the usual Quality Assurance processes supporting final publication.”
QARs are for a ‘different world’
Regardless of the system issues, there are sector-wide concerns that the way in which apprenticeship achievement rates are calculated is out of date and represents a regime prior to the introduction of an employer-led system.
Lifetime Training, for example, said its QAR reflects the Covid-19 period, when its core sectors of hospitality, care and retail were facing significant challenges which impacted learners and caused many withdrawals.
A spokesperson for the provider said the current QAR calculation is a “one-dimensional metric which does not consider sectoral differences around turnover and workforce dynamics”.
Whittaker echoed this. “When the ESFA measure QAR it is assumed that all leavers are the providers fault, when most leavers are down to the learner or employer. How can the provider be responsible for learners who change career or move to an employer who doesn’t want to continue the apprenticeship?” she told FE Week.
“QAR was developed to measure the success of apprenticeship frameworks. We are now in a different world, working under a different regime which is very much controlled by employers, not by providers. It is time to update the current outmoded measurement systems to reflect this.”