More than 2,000 secret changes to FE Choices

An investigation into changes on FE Choices has found more than 2,000 data records were adjusted without public knowledge.

The Data Service, which is part of the Skills Funding Agency, today revealed that 2,639 changes were made to “unique values” in the Learner Satisfaction indicator of the performance comparison site, which went live on January 26.

As revealed by FE Week last month, FE Choices, formerly known as the Framework for Excellence, is under scrutiny for unannounced post-publication alterations.

Despite the changes to the comparable data fields, made on January 30, neither the Agency nor the Service made a public announcement, which could leave them in breach of the Code of Practice for Official Statistics by the UK Statistics Authority.

The Code requires that prompt public announcements are made on errors which are discovered in statistical reports.

Although the FE Choices website says the changes were made on January 30, the Data Service said the data was revised on February 6.

A statement, posted on the Service’s website, read: “The revisions related to the data from the survey of learners.

“The Learner Satisfaction national, provider and organisation type aggregate scores were not- affected, but supporting scores on the sub-scales were affected for some organisation types and providers.”

The data changes, according to the Service, were:

  1. The comparative data at organisational type and national level was changed to exclude zeros from the calculations. This caused a change in the minimum value and medians. This change was to background data and no headline score was affected.
  2.  Changes were made to correct errors whereby the code for question 7 was wrongly referring to question 6 data (for the 19+, all levels category) and the code for comparative data for question 2 was referring to data from question 1.

The changes, according to the Service, represent 0.7 per cent of the 371,987 total unique values.

However, a closer look shows that 205 unique values for “Organisation Type detailed scores” were changed – some 12.7 per cent of the 1,620 total values.

Meanwhile, 30 of the 405 unique values for “National detailed scores” were changed, which equates to 7.4 per cent.

The Skills Funding Agency said they have “duty to let users know as soon as possible” about what changes were made.

A statement released by the Agency today said: “The Skills Funding Agency has published information on what changes were made to the data on FE Choices, in compliance with statistical protocols.

“The investigation process is ongoing and a full report will be published in due course.”

The Agency also said it is identifying lessons to learn from the episode.

“The Skills Funding Agency and the Data Service continue to abide by the Code of Practice for Official Statistics and have taken swift action to conduct a thorough investigation.

“The Agency and the Service are identifying lessons to learn and steps to take to ensure that there is no recurrence by reinforcing practice, quality management and testing procedures,” the statement said.

As well as the investigation by the Agency, the Department for Business, Innovation and Skills, under a request by the FE and skills minister, John Hayes MP, and the UK Statistics Authority have launched their own inquiries.