Following the fallout from the first-year health and science T Level results last summer, awarding body NCFE has been on a drive to ensure problems aren’t repeated in 2023. Jason Noble spoke to chief David Gallagher to find out what happened behind the scenes and whether the problems were avoidable.
“There was a lot of pressure,” says David Gallagher, chief executive of awarding body NCFE.
“As a parent, if my boys had been in this cohort and got poor grades, whatever the reasons, I would have felt the system had let me down. I would have felt that, so I’m not going to shy away from that at all.”
But he stressed that T Levels – designed by the DfE to be the flagship technical equivalents of A-levels – are “broader, deeper, more rigorous, more challenging, more aligned to employers’ needs,” than qualifications that have gone before and are “significantly harder”.
While eyes were on the first full results for T Levels in the first three courses – construction, digital, and education and childcare – first year results for the second wave of T Levels were also released to students.
Those first-year results for the health and science T Levels soon came under the spotlight as scores of disheartened students received lower than predicted grades, and social media exploded with angry learners trying to piece together what had happened.
It appears that the problems could have been avoided.
The primary issue was the amount of science content in the core papers, although other concerns such as the availability of the course textbook and uploading elements of the employer-set project also caused concern.
The issues meant NCFE had to regrade students after an Ofqual review determined that the 2022 core assessment papers “do not secure a sufficiently valid or reliable measure of student performance”. The Ofqual review found “question errors, inadequate mark schemes and questions covering areas not explicitly in the specification”.
According to Gallagher, NCFE itself had questions about the outline common core science content, which is designed by the Institute for Apprenticeships and Technical Education’s (IfATE) route panel, from as early as January 2020 and querying the breadth and depth of the technical annex.
“We raised that question on a number of instances in the design and development process,” he said.
“In the lead into ultimately designing the assessments, the exams, the questions within, there were questions and concerns raised about the core – centres, those involved in design and development, and NCFE raised those questions and those concerns.”
Gallagher said the concerns were fed back through the “usual contract management processes” and added: “We felt that they were highlighted in a way that gave the institute enough information to assess whether there was a risk, whether there was an issue from their perspective. But we absolutely have to take responsibility that we took forward the development of the papers based on the spec that had been agreed.”
However, Gallagher says it was only when looking at the grade boundaries and profiles after the assessments had been marked that it became clear there may be an issue with results.
According to NCFE, the distribution curve of grades was what they would expect to see, but the curve itself was lower. The lines were further blurred by the fact that the courses were designed to be more rigorous than the BTEC and level 3 equivalents they were replacing.
NCFE was initially unsure whether exam preparedness was the issue, with the Covid-19 pandemic meaning students had not sat exams for a couple of years.
There were 1,115 results issued across the three pathways – health, science and healthcare science – and while exact numbers affected were never revealed (Gallagher says the nature of the contracts meant some details couldn’t be revealed) an FE Week investigation in the autumn found a third had either switched to alternative courses or dropped out of college entirely as a result.
Since then, the IfATE has confirmed it is working with NCFE to reduce pure science content in the health core papers, while a separate investigation by Ofqual resulted in an enforced undertaking being signed by NCFE to deliver an action plan of improvements, offering assurances that those issues will not recur in 2023.
The organisation was understandably in the spotlight, but a web of input during development of the courses – including the DfE and IfATE, with oversight by Ofqual – has created a murky landscape which makes it difficult to pinpoint responsibility.
The journey began with IfATE’s route panel creating the outline content, which was used as the basis for development.
NCFE added detail, which was built into a qualification specification with work alongside industry experts, providers and employers. That work was subsequently validated by panels of experts and IfATE’s route panel prior to approval to ensure it met the original outline content.
Gallagher is clear that “the nature of the design and development of T Levels means that the institute and Ofqual are very intimately involved at every stage of design, development, scrutiny, approval”.
“When things are going well, in the design, development and delivery side of things, particularly pre-this issue, where things are going well on paper it is clear and it works,” he said.
“When there is a particular issue, an issue that needs addressing urgently such as this did, it put a significant amount of stress into that sort of structure between the respective stakeholders that I’m not sure is optimal.”
Gallagher said it could involve “hundreds of individuals” as respective organisations have to take account of implications for students, colleges, and the T Level brand, as well as contractual and regulatory considerations.
He questioned the unique model of a single awarding body per T Level. “In the event that something hasn’t gone right – and this could be in design and development, it could be delivery – with one awarding organisation, all of the eggs are in that basket. Is that sort of licencing model right? That for me, there’s a question mark there.
“That’s not me saying it’s not right, that’s me saying, is that right? Is that going to give us what we need?
“The capacity that is required in the system to deliver this reform, right from assessment design, assessment development, qualification development, provider development where we set centres up for success, quality assurance that wraps around that, there is insufficient capacity in the system to do everything that the policy is driving towards.”
It leads neatly to a looming question for the future. Awarding organisations will have to decide on whether to re-procure to continue delivering the first T Level courses, a process expected to launch in the next couple of months.
Gallagher said NCFE is “really focusing on what we’ve got now” and has not yet made a decision on whether to re-procure, but suggested that “investing really heavily” in resources and industry partnerships was “a sign of our intent”.
NCFE has bagged more T Level contracts than any other awarding body. Alongside the health and science courses, NCFE also has contracts for the T Levels in education and childcare, digital business services, and digital support and services. It has also won the tender for three of the five launching this September: hair and beauty, craft and design, and media and broadcast.
Contract values for all of NCFE’s T Levels, including the new ones for this year, total nearly £35 million, with the health, science and healthcare science T Levels alone accounting for more than £10 million.
But can NCFE reassure students that there won’t be a repeat of the issues this summer?
Gallagher concludes: “I am very confident that we’ve done everything that we possibly can to prepare centres, to prepare educators and to make sure that we’ve got robust, reliable and valid assessments to support all of these T Level learners.”