At this year’s AELP autumn conference, ESFA director for apprenticeships and bootcamps, Kate Ridley-Moy said that while she recognised challenges with the qualification achievement rate (QAR), it had to stay. It is the standard measure of success rates across all qualifications, she argued, adding that her team were looking at other measures to support understanding of the QAR. Let’s unpack this, and hopefully give Kate and her team some useful insights.
A level playing field?
The QAR for apprenticeships measures the number of people who fully complete an apprenticeship as a percentage of all leavers, with some added complications for cohort years. This is not the same methodology as for qualifications like A levels, where the calculation is the number of passes as a percentage of all people who took the exam. If we were to measure apprenticeships on the same basis as A levels, we would be celebrating very high success rates (97 per cent, in fact, according to Gillian Keegan).
Further, many of the reasons people don’t fully complete apprenticeships are entirely beyond a provider’s influence. Redundancy, illness and lack of time to complete the 20 per cent off-the-job training requirement are all regularly given as reasons for leaving, as well as more positive ones like promotion or role changes.
At HIT we track leave reasons very carefully to focus our efforts on reducing the number of leavers. For example, detailed tracking of one of our programmes revealed that some people were losing interest early. Our curriculum team surveyed the apprentices, made slight changes to the sequencing of delivery, and the number of leavers dropped.
Meaningful data
Next, QAR is averaged across all of the programmes a training provider delivers. This is not very meaningful to employers or prospective apprentices.
The success rates for our level 4 brewer programme are among our highest, at over 80 per cent. Most who take that apprenticeship are under thirty, have their sights set on a defined career and a good level of education, including English and maths.
Compare this with the fifty-year-old adult care worker who desperately wants to improve their skills but struggled at school, and as well as caring professionally has both younger and older generations of their family to care for.
Many such learners drop out mid-way because the pressures on them are just too much, but our tracking shows that they always (and I mean always) leave having developed useful knowledge and skills for their role.
Dysfunctional skills
Finally, people rightly should be able to continue studying English and maths in their adult lives, but why are apprenticeships the only programme of learning where a pass in these subjects is a condition of success?
An apprenticeship is about occupational competency proven over time. If the apprentice can become competent without having passed functional English and maths, then it is that competency that matters.
QAR squared
So to improve QAR, let’s have two versions alongside each other: one with all leavers, and another with just those that are within the influence of the training provider.
We should be held accountable for making adjustments if learners lose interest. It’s our responsibility to have effective initial assessments and honest conversations if a learner is unlikely to complete an element of the programme. And it’s simply unacceptable not to get close enough to learners to understand the risk factors that might cause them to leave.
But holding providers accountable for learners being made redundant or promoted does the whole sector a disservice. Discounting these leavers, HIT’s QAR success rate for 2022/23 would be 70 per cent, above the government 67 per cent target.
If QAR truly aims to improve the skills system for everyone, the only way is to develop a better understanding of the reasons people don’t finish their apprenticeships. The ESFA should therefore work with providers and employers to improve its inadequate list of leave reasons.
I’m sure our own extended list is similar to that used by other providers, but until the choices available on the ILR reflect this then the rather un-useful ‘other’ category will be what the ESFA see on our returns.
And that helps no one improve.
This article is riddled with errors which is a shame because I agree with the sentiment. The way in which the A Level QAR works IS the same as apprenticeships and based over 2 years for many (unless they’re doing AS). How can the education minister not get that? Furthermore, QAR isnt averaged over programmes, it’s totalled over programmes and therefore represents programmes proportionally.
This article appears to be comparing the achievement rate in relation to apprenticeships with pass rates for A levels which is misleading. Comparing apples with pears. I believe that “success rates” were replaced by “achievement rates” by Hancock some years ago ( just because he felt like it).
I don’t think this article is riddled with errors, I think it’s more a matter of interpretation.
A QAR for a provider is a headline figure across all the frameworks and standards (or other programmes) they deliver. Apprenticeship providers worth their salt will know about the outcome for every single apprentice and know that different frameworks / standards have different factors at play because of the context of the occupation and industry those apprentices are employed in.
The article is also correct in stating that seemingly low achievement rates are not all the ‘fault’ of the provider. If we want to get into a blame game then sometimes it will be the provider not being up to scratch, sometimes an apprentice will decide to take a different path, sometimes an employer will pull the plug (for numerous reasons). Or indeed, a blend of ‘blame’ across two or all three of the parties involved.
Just blaming the provider is misplaced. It’s easy to see why employers aren’t criticised as they pay the levy or make a financial contribution. It’s also easier to blame a provider than an apprentice. But it doesn’t make it fair!
In my view, the blame game is the lazy or divisive option and prevents any deep understanding or appreciation of what is actually happening.
Take A levels. When Covid hit, did A level students remain on programme beyond when they were supposed to finish? No, they finished on time, flexibilities were introduced to exams and how that learning was assessed and graded. Therefore, any fluctuation to achievement rates was directly visible and contained within the time period originally intended.
Now take apprenticeships. When Covid hit, many apprentices had learning delayed or were furloughed etc. So their eventual achievement (or not) got displaced into future QAR years. This not only delays the achievement, but it will have sucked achievement volumes away from the last few achievement rate years, exposing the withdrawals through weight of numbers, which pulled down achievement rates. The DfE doesn’t publish data on this displacement effect, but it’s obvious that lower recent achievement rates are not entirely because there were proportionately more withdrawals or non achievers, but because a heavy ‘weight’ of ‘potential’ achievement was displaced forward into future years.
The irony is that increasing achievement rates now are not necessarily solely because of improvements in quality, but because that ‘potential displaced achievement’ is now being realised as providers & EPAO’s continue to deal with the aftermath of Covid.
(see also the recent comments about achiever volumes being up 20% on last year…)
The real difficulty for providers is that all those apprentices where their potential achievement is ‘displaced’ into a future academic year, have an ongoing cost, but are out of funding.
This is about so much more than QARs & these few words barely scratch the surface.
The article is right to call current systems and measures uninformative and unproductive. I’d go further and say there is a concerning the lack of appetite for deeper insight and collaboration across the system to improve the situation.
(don’t even get me started on apprentices that transfer to other providers, but on the same standard and employer – they get counted as leavers twice, but can only ever be one achiever. At best, in QAR terms, they can only ever be half an achievement!)