The Department for Education has admitted that a “completion” of a course on its Skills Toolkit may simply represent that a user has accessed learning materials.
Officials have also publicly confessed for the first time that they have no idea whether users of the free online content, which has cost almost £1 million to put on a new platform, live in the UK.
The admissions follow multiple investigations by FE Week which revealed how the flimsy data was being celebrated by ministers without a high level of transparency.
It also comes shortly after a public telling off by the Office for Statistics Regulation (OSR), which wrote to the DfE’s chief statistician earlier this month to raise issues with the figures.
A series of extra caveats have been added to the DfE’s publication of the Skills Toolkit “experimental” data in its monthly apprenticeship statistics release for the first time today.
As previously revealed by FE Week, course “registration” data has already been overcounted by the DfE and has led to revised estimates as they can simply include web hits.
The DfE now explains in their data release that course providers outline their method of reporting to the department “after being given a clear mandate to ensure that they are accurate to the best of their knowledge and understanding and are asked to report a change in their methods of reporting or any errors/issues they encounter as quickly as possible”.
Provider reports are then “checked for unusual patterns”.
‘We expect this to be the UK in most cases’
This publication also revealed in January that the course registrations could be coming from anywhere in the world as many of the providers do not filter users by geographical location.
The DfE’s data now includes a line which states: “DfE do not receive data from course providers on the country the learner accessed the course from, but we expect this to be the UK in most cases.”
Additionally, FE Week reported earlier this month on how course “completions” were being counted when users spent three minutes looking at one of the online resources.
The DfE’s publication of the data now states: “How providers record completions and define what a completion is can vary, and again depending upon the course format (for e.g., interactive videos, modules etc), the completion stage could be different for various course types and a ‘completion’ may simply represent that a user has accessed learning material.”
In the OSR’s letter, deputy director for regulation Mary Gregory raised concern that the Skills Toolkit data was published under the “additional analysis section” of the publication which “may not be clear to users looking for these data in the release”.
The DfE has now included commentary at the top of the publication to flag where users can find the relevant data.
The course content on the Skills Toolkit has not been developed by the government, but more than £1 million has been spent to develop and promote its “platform”. The platform launched in April 2020 and consists of a web page on the National Careers Service with short course descriptions and links to the external websites.
The government says the free educational content being promoted aims to help people who are out of work to boost their digital and numeracy skills during the pandemic.
Education secretary Gavin Williamson described the free online courses as having a “transformational impact on so many people taking furlough” during a speech in October.
Your thoughts