Senior staff from the exams regulator were hauled infront of the education select committee today where they were grilled on this year’s results fiasco.

Ofqual has been criticised for keeping quiet while chaos reigned on their now infamous grades-awarded-by-mutant-algorithm system, while ministers pushed blame their way. But the Ofqual team’s accounts seem to have shifted that blame squarely back onto ministers.

There was a hell of a lot to cover, but here are some of the key points:

 

1. Williamson’s claims he was unaware of problems look like bunkum

One of the more interesting developments post the exams fallout was the continued insistence from ministers, mostly education secretary Gavin Williamson, that he first knew there were issues with calculated grades a couple of days AFTER pupils got results.

MPs were told today that Ofqual was meeting with ministers on a weekly basis throughout the period. The exams regulator’s first advice to ministers was also it would be “challenging if not impossible to attempt to moderate estimates in a way that is fair for all students”.

They also briefed Number 10 on August 7^th in a paper that was “very alert to the risks” the results posed to disadvantaged, outlier students, centre expecting improved grades and those with lower entry cohorts (who were actually advantaged by the algorithm).

 

2. Ofqual advised against calculated grades back in March

Interestingly, Taylor also revealed Ofqual’s initial recommendations to Williamson at the outset in March was to, first of all, try to hold exams in a socially distanced manner.

The second option was  to delay exams, and the third option was to look at some form of calculated grade, but Taylor suggested this should be some form of teacher certificate, rather than attempting to replicate exam grades.

However, Taylor said the decision soon came, from Williamson, to scrap exams and award calculated grades – without any further consultation with Ofqual.

 

3. How the 11^th hour ‘mock appeal’ announcement started the downfall

Following the outcry in Scotland over their calculated results, Williamson announced his ill-fated “triple lock” appeals route – allowing pupils to appeal a grade if they had a mock that was higher.

It appears this was a key first link in a chain of chaos that led to pupils being awarded CAGs.

Up until this stage, it seemed Ofqual – while knowing its algorithm had flaws – was confident it could deal with the fallout. Its approach was thus: a system to contact schools and colleges who had pupils with “outlier” grade changes to make sure they knew they had a “good” case to appeal, and also the autumn resits (with universities holding open students’ places).

Taylor said he told Williamson he was “not confident” the new mocks appeals route “could be delivered … to ensure valid and trustworthy grades were issued”.

Nonetheless, Ofqual decided it should find a way to implement this, and after getting agreement from the DfE and Williamson’s office, published the guidance on Saturday afternoon.

But Ofqual was contacted by Williamson on Saturday evening at around 8pm saying this guidance wasn’t in line with government policy (and the guidance was swiftly removed).

A hastily-convened Ofqual board, meeting at 10pm that evening, then realised the situation was “rapidly getting out of control”, Taylor said, with policies “we felt would not be consistent with our legal duties”. (He said the mock appeals route would have led to around 85 per cent of pupils receiving their CAG, and a small number not receiving them – but this would have also taken an age to happen).

By Sunday evening, the opinion from Ofqual was there was no way to carry on “in an orderly manner” and the “only way out” was to move towards awarding CAGs.

 

4. Lessons learnt? Don’t try to replicate grades

As we’ve covered here, Ofqual has now admitted it was a “fundamental mistake” to believe that calculated grades would ever be accepted by the public – and said tweaks to the algorithm wouldn’t have made it any better.

Elaborating on this, Taylor said one of the lessons learned was “if you can’t replicate normal grades, don’t pretend you can” – pointing out that the objective was actually to enable progress for students onto their next step (rather than award GCSE grades) and that’s what seemed to have got lost in all this.

Note: this is important for decisions made for next year’s exam series – and it looks like any form of CAGs is totally off the table, with the focus instead on finding ways to deliver tests come what may.

 

5. College ‘concerns’ over ‘ill-equipped’ students

Julie Swan, Ofqual’s executive director, told MPs that the regulator is now hearing “some concern” from general FE colleges and sixth form colleges over students which received their teacher-assessed grades from schools that were over “optimistic” about what they would have realistically achieved in an exam.

The centre assessed grades now “indicate a level of ability that isn’t actually very accurate”.

“There are concerns that they [students] might progress onto a course for which they are now ill-equipped for,” she added.

 

6. Delaying exams next year has ‘enormous benefits’

Currently, government is proposing exams go ahead with some tweaks to assessments in some subjects.

But what about local lockdowns throwing this off-course? Swan said having an extra set of papers for pupils that may be ill, for instance, is something being considered.

Taylor also said another plan b could be using online tests.

The government has been consulting on plans to delay exams since June. While Taylor said there would be “enormous benefit” to delays, Ofqual still won’t commit to a date for when schools, colleges and pupils will have clarity. All they would say was a decision would be “weeks, rather than a period of months” away.

 

7. A whole new (transparent) world?

One of the big criticisms of Ofqual has been its secrecy over how the algorithm will work. This was touched upon a few times during the hearing.

Taylor said the “primary constraints” on transparency was not to publish information on how the system would work that could influence how teachers made their grades.

They also held onto publishing full details of the algorithm because that could lead to pupils identifying what grades they got before results day.

Michelle Meadows, executive director for strategy, added: “Our aim throughout has been to be transparent about the limitations of the model, and that’s why we did publish those metrics on results day so people could understand the limitations of what’s possible with statistical moderation like this.”

Meanwhile, Ian Mearns MP called for “deep, forensic” analysis on the CAGs submitted by schools and colleges to work out, for instance, how many schools and colleges “were gaming it”.

Taylor said he wouldn’t publish the full data – as it involves confidential informaiton – but said it’s “absolutely essential that independent researchers have access to this, in a secure way, so lessons can be learnt”.

And Taylor also committed to publishing Ofqual board minutes (the last of which was published in September last year).

 

8. But will Ofqual survive?

Committee chair Robert Halfon ended the hearing by asking whether Ofqual was still fit for purpose amid rumours the body could have a similar ending to Public Health England.

One of Halfon’s big issues was over a lack of communication from the regulator during the chaos (or “hiding away in the Ofqual attic”, as he put it). He highlighted Ofqual’s communications team included 11 people, with a job advert out for another comms person on £80,000.

Dame Glenys Stacey, who has stepped in as acting chief regulator after Sally Collier stepped down, said she has a “strong interest” in communications capacity, and that structure was “under active review”.

But she claimed the whole debacle had “shown the importance of an independent regulator”.

“We do have very important, very rare skills and expertise .. yes we’ve had an enormously difficult year… but we do have an important job to do,” she added.

Taylor said there had been a “major blow to confidence in Ofqual”, but claimed the regulator had a good track record over the previous decade on, for instance, delivering new GCSEs.

When asked about reform, Stacey added: “Yes I will look at the way Ofqual works, of course I will, and if changes are needed I’ll put those in play.”

Ofsted has today published further details about how their “interim visits” of FE providers will work this autumn.

The watchdog stressed these will not be routine inspections and will not result in a judgement grade.

The guidance follows an announcement in July that Ofsted would use a phased return to routine inspections, following a pause to this activity as result of Covid-19. The full inspection regime is planned to resume in January 2021.

Some monitoring visit activity of new providers will also resume from this month.

Here is what providers need to know…

 

When will the interim visits begin?

September 28 to the end of December 2020.

 

The purpose of interim visits which could trigger a full inspection

Ofsted states that the interim visits are to help learners, parents, employers and government “understand how providers are meeting the needs of students and apprentices in this period, including learners with high needs and those with special educational needs and disabilities”.

While existing Ofsted grades will not change as a result of the visit, the watchdog will publish the outcomes of the visits in a “brief report to share what steps are being taken to enable learners to resume a full programme of education or training”.

However, “significant concerns” identified on an interim visit could trigger an early full inspection or a monitoring visit.

 

How long will the visits last and what will Ofsted explore?

Each interim visit will last two working and days and inspectors will explore three themes:

1. What actions are leaders taking to ensure that they provide an appropriate curriculum that meets the reasonable needs of learners and stakeholders and adapts to changed circumstances?
2. What steps are leaders, managers and staff taking to ensure that the approaches used for building knowledge and skills are appropriate to meet the reasonable needs of learners?
3. How are leaders ensuring that learners are safe and well informed about potential risks, including from online sources?

Leaders “do not need to prepare anything for the visit, beyond what is part of normal business for the provider”.

Inspectors will try to work on site at the provider’s premises “wherever possible”, but recognise it may be necessary to carry out some meetings, discussions or aspects of the visit remotely.

Conversations with senior leaders will be the “main source of evidence”. Inspectors will want to “discuss the impact of Covid-19 and identify any barriers that the provider is facing in managing the return to full education and training for all learners”.

Inspectors will talk to staff and learners “if it is safe and appropriate to do so” and they may also speak to subcontractors, employers and other stakeholders.

Ofsted made clear they will not carry out “deep dives” or directly observe teaching or training. However, they may look at the provider’s records, especially on safeguarding.

 

Providers will be given at least 2 days’ notice, but can request a deferral

The watchdog said they will normally notify a provider of an interim visit up to two working days before the on-site stage of the visit, but this could be extended to four working days if the provider or college is particularly large.

A provider may request a deferral of an interim visit but it will be for Ofsted to decide whether a deferral should be granted “in accordance with our deferral policy”.

 

Which providers will be chosen for autumn visits?

Ofsted will prioritise providers with an ‘inadequate’ or ‘requires improvement’ inspection grade, providers that the watchdog has identified “risks and concerns” about, as well as a sample of ‘good’ and ‘outstanding’ providers and newly merged colleges.

 

Letters will be published within 38 working days of a visit

Ofsted says it will write to the provider within 18 working days of the end of the visit, setting out in a draft report what inspectors found.

The provider will then have five working days to comment on the draft report, inspection process and findings. Ofsted will “consider all comments and we will respond to the comments when we share the final report with the provider” within 30 working days after the visit.

If the provider wishes to submit a formal complaint, it will have until the end of the fifth working day after receiving the final report to do so.

Ofsted will normally publish the report on its website within 38 working days of the end of the visit, but this could be delayed if a complaint is being investigated.

The watchdog said it may share the provisional findings, before a report is published, with the FE Commissioner, ESFA, DfE, Ofqual, Office for Students or devolved authorities.

 

New provider monitoring visits to continue

From September 28, Ofsted will resume monitoring visits to new providers if they were judged to be making ‘insufficient progress’ against one or more themes at their previous early monitoring visit and would have been due their full inspection up to or during this interim phase but have not received it because of the suspension of routine inspections

Both of the above factors must apply.

The “fundamental mistake” made by Ofqual throughout this year’s exams debacle was to believe that its calculated grades “would ever be acceptable to the public”, the regulator’s chair has said.

Roger Taylor appeared in front of the education select committee this morning to answer for the errors made in deciding GCSE and A-level grades this summer, namely the “mutant algorithm” as described by prime minister Boris Johnson.

He lifted on the lid on the internal battle between Ofqual and the education secretary Gavin Williamson – stating that policies such as the last minute decision to use mock grades were getting “out of control”.

He also revealed that the regulator’s “first choice” at the beginning of lockdown was to hold socially distanced exams this summer – but Williamson opted to cancel them and use standardised grades without consultation.

In a statement provided to the committee by Taylor, published during the hearing, he said: “The blame lies with us collectively – all of us who failed to design a mechanism for awarding grades that was acceptable to the public and met the secretary of state’s policy intent of ensuring grades were awarded in a way consistent with the previous year.”

Delivering calculated grades was ‘impossible task’

But he said it was an “impossible task”, adding a “better” algorithm would not have made outcomes more acceptable, nor that “more effective communications effort would have overcome this”.

He added that with “hindsight it appears unlikely that we could ever have delivered this policy successfully” and during the hearing stated that “the fundamental mistake was to believe that this would ever be acceptable to the public”.

Taylor told the committee of MPs that Ofqual’s “initial advice” to Williamson was that the “best way to handle this [lockdown] was to try and hold exams in a socially distanced manner, that our second option was to delay exams but the third option if neither of these were acceptable would be to have to try and look at some form of calculated grade”.

But it was Williamson who then “subsequently took the decision and announced without further consultation with Ofqual that exams were to be cancelled and a system of calculated grades were to be implemented”.

Ofqual could have rejected the government’s demand for such a system – but said it didn’t because it was “decided that this was in the best interests of students, so that they could progress to their next stage of education, training or work”.

DfE was ‘fully informed’ of the risks

Taylor also said the DfE was “fully informed about the work we were doing and the approach we intended to take to qualifications, the risks and impact on results as they emerged. However, we are ultimately responsible for the decisions that fall to us as the regulator.”

This explicitly contradicts claims by Williamson that he was unaware of problems with the results until after they were delivered to pupils.

On August 11 the Department for Education announced eleventh hour plans to give students the choice to use mock grades instead of standardised grades before telling Ofqual, according to Taylor.

The chair said the regulator’s advice to the education secretary at this point was that “we could not be confident that this could be delivered within the statutory duties of Ofqual to ensure that valid and trustworthy grades were being issued”.

But the “secretary of state as he is entitled to do nonetheless announced that that was the policy of the government”.

Taylor claimed Ofqual was “very concerned that this idea of a valid mock exam had no real credible meaning but we consulted very rapidly and developed an approach that we felt would be consistent with awarding valid qualifications”.

They then agreed that “with the DfE and with our understanding with the secretary of state’s office” and published this on the August 15.

Taylor said Ofqual was then contacted by Williamson later that evening and was “informed that this was in fact not to his mind in line with government policy”.

The Ofqual board was pulled together late that evening and they “realised we were in a situation that was rapidly getting out of control, that there were policies being recommended and strongly advocated by the secretary of state that we felt would not be consistent with our legal duties and that there was a growing risk around delivering any form of mock appeals result in a way that would be acceptable as a reasonable way to award grades”.

New routine funding audits will restart on a remote basis for independent training providers from this month – but colleges will continue to have theirs put on hold until November, the government has announced.

The Education and Skills Funding Agency will also carry out a “risk assessment on restarting site visits” for audits and investigations, so that officials have the “ability to do so, should we need to”.

The agency had paused all audits, including those that had already started, in March owing to the Covid-19 pandemic.

Coronavirus guidance for training providers and colleges was updated yesterday and said that where funding audits and investigations were already in progress prior to the lockdown, the ESFA has now “sought to complete the work, taking into account training providers’ capacity to resolve any issues”.

The agency’s guidance did not provide a reason for the discrepancy between college and private provider restart dates but FE Week understands it is because the agency does not want to overburden colleges as they begin their full reopening this month.

The ESFA stated in its guidance that they “recognise the challenges” training providers face as a result of coronavirus and will be “sensitive to these in agreeing arrangements for both remote funding audits and site visits with providers” going forward.

“It may be necessary for the ESFA to contact training providers during the coronavirus outbreak in order to continue to maintain effective oversight and protection of public funds,” the guidance continued.

“Where such contact is necessary, we will continue to be sensitive to the challenges training providers face as a result of coronavirus.”

Higher education institutions funded by the ESFA are currently not in scope for audit restarts, but these are due to resume next year.

Other parts of the post-16 regulatory system, such as initial Ofsted visits, are also set to restart this month following a pause due to Covid-19.

The Ofsted visits will not be full inspections, but will “look at how effectively leaders are enabling provision to resume fully following an extended break in formal education, including considering remote education and safeguarding”. They will result in a published letter but no grade.

Full education inspections are due to restart in January 2021.

FE Commissioner activity in colleges resumed in July.