SFA close achievement rate "loophole" that boosted some provider rates by more than 20 percent

The Department for Education has admitted to a “loophole” in the Skills Funding Agency qualification achievement rate calculation, which until now “artificially” boosted the rate for around 10 per cent of providers.

In some cases, providers benefited by more than 20 QAR percentage points.

Figures published on February 16 in the updated January Statistical First Release (SFR) show the 2014/15 achievement rate as 71.1 per cent before closing the loop-holes, which falls 4.7 percentage points to 67 percent once recalculated. The fall was most prominent for 25+ Level three apprentices, resulting in a 7.6 percentage point fall.

The QAR is an important performance measure, as it is used by Ofsted and the SFA can stop providers delivering courses that fall below thresholds, called ‘Minimum Standards’.

On page 17 in the SFR document, the DfE describes one of three loopholes it has now closed as “inappropriate use of the planned break exclusion rule”, and admitted “some providers reported nearly all withdrawals as planned breaks and therefore received a significantly higher QAR”.

The loopholes relate to how the SFA compares learner records in several annual data returns from providers. Where learners disappear or do not have a completion status, they will now be included in the calculation and counted as a ‘fail’.

According to the DfE document, the SFA analysis “identified that approximately 10 per cent of apprenticeship providers were receiving an artificially high QAR rate for apprenticeships because of how they were using three loopholes in the methodology”.

Some gained “a significant advantage of more than 20 per cent in their overall QAR”, while other providers “were able to avoid falling below the minimum standard threshold which was 55 per cent at the time”.

The minimum standard threshold for apprenticeships rose for performance in 2015/16 to 62 per cent, which is now above higher level apprenticeship average of 58.6 per cent.

Steve Hewitt, MIS and funding consultant for FE Associates, said: “That’s what happens when you change methodologies. But I think in general it’s a good thing because, if there were organisations that were playing the game to exclude these learners, if there were organisations who were trying to fiddle their data, this is a loophole that has now been closed and that’s good because we all want data to be as accurate as it can be.”

The overall achievement rate for apprenticeships in 2015/16 now stands at 67 per cent, meaning just two out of three starts counting towards the governments 3m target are successfully completing the course.

The DfE document said the “impact is less pronounced” on non-apprenticeship provision. 

Recalculated rates below