SFA close achievement rate “loophole” that boosted some provider rates by more than 20 percent

The Department for Education has admitted to a “loophole” in the Skills Funding Agency qualification achievement rate calculation, which until now “artificially” boosted the rate for around 10 per cent of providers.

In some cases, providers benefited by more than 20 QAR percentage points.

Figures published on February 16 in the updated January Statistical First Release (SFR) show the 2014/15 achievement rate as 71.1 per cent before closing the loop-holes, which falls 4.7 percentage points to 67 percent once recalculated. The fall was most prominent for 25+ Level three apprentices, resulting in a 7.6 percentage point fall.

The QAR is an important performance measure, as it is used by Ofsted and the SFA can stop providers delivering courses that fall below thresholds, called ‘Minimum Standards’.

On page 17 in the SFR document, the DfE describes one of three loopholes it has now closed as “inappropriate use of the planned break exclusion rule”, and admitted “some providers reported nearly all withdrawals as planned breaks and therefore received a significantly higher QAR”.

The loopholes relate to how the SFA compares learner records in several annual data returns from providers. Where learners disappear or do not have a completion status, they will now be included in the calculation and counted as a ‘fail’.

According to the DfE document, the SFA analysis “identified that approximately 10 per cent of apprenticeship providers were receiving an artificially high QAR rate for apprenticeships because of how they were using three loopholes in the methodology”.

Some gained “a significant advantage of more than 20 per cent in their overall QAR”, while other providers “were able to avoid falling below the minimum standard threshold which was 55 per cent at the time”.

The minimum standard threshold for apprenticeships rose for performance in 2015/16 to 62 per cent, which is now above higher level apprenticeship average of 58.6 per cent.

Steve Hewitt, MIS and funding consultant for FE Associates, said: “That’s what happens when you change methodologies. But I think in general it’s a good thing because, if there were organisations that were playing the game to exclude these learners, if there were organisations who were trying to fiddle their data, this is a loophole that has now been closed and that’s good because we all want data to be as accurate as it can be.”

The overall achievement rate for apprenticeships in 2015/16 now stands at 67 per cent, meaning just two out of three starts counting towards the governments 3m target are successfully completing the course.

The DfE document said the “impact is less pronounced” on non-apprenticeship provision. 

Recalculated rates below

Your thoughts

Leave a Reply to Liam Ryan Cancel reply

Your email address will not be published. Required fields are marked *

6 Comments

  1. Is it a good idea to apply this retrospectively though? It invalidates all historical data undermining the good work that providers may have done for example in addressing achievement gaps. It will expose providers to criticism from ofsted if previously ‘ok data’ now looks like groups experienced disadvantage. How can a provider improve their 2014-15 data in 2017? They can’t. I can understand applying this methodology in 2015-16 but I’m at a loss as to the justification for back-dating it.

    • Simon George

      Completely agree with Matt. The term loophole implies training organisations were at fault, when in reality the fault is elsewhere. While it is fine to move the goalposts and change this system moving forward, it should not be applied retrospectively.

  2. Randy Jackson

    Back dating it exposes the providers that have benefited from this, but it serves little other purpose as Ofsted would normally only be interested in the most recent data set.

  3. Rachel Wilks

    I think this is a way of making the providers look like they have done something wrong to shift blame away from SFA officials who are less than competent at the best of times.

  4. Surely if providers have been misusing the “Break in Learning” rules to cover withdrawals then there will be evidence (or rather no good evidence to back up the BIL)? They should punish the guilty parties not disincentivise the legitimate use of BILs for things such as maternity leave. It’s probably just the usual story of the guilty parties being part of the “too big to fail” group that get away with blue murder!

  5. Swivvel-eyed-looney

    I can only speak for my own organisation, but the issue in my view is that there is an artificial cut off point (i.e. final ILR return date) where you can only set a status based on your most up to date understanding of an apprentice’s status at that time. It is no surprise that this has hit providers who have a greater volume of adults, and particularly a greater volume of women. I am irritated by those who have been quick to assume this is a scam and a loophole that has been deliberately exploited by all. What are we meant to do with an apprentice that has been diagnosed with cancer and is living in the hope that they will make a full recovery and eventually get their life back on track. What if, despite their best intentions, they were not able to do this by some artificial cut off point. What if, as is sometimes the case, the diagnosis becomes terminal and, heaven forbid, after the ILR cut off point. For those of you who have been quick to jump to conclusions, go and spend a week working with a large apprenticeship provider and you might be a bit more balanced in your judgements. As I said, I can only speak for my own organisation. We have been affected by the changes (we are still well above the national rate) and I can say we have acted 100% with integrity. I am irritated by the usual ‘quick to condemn’ keyboard warriors.