Colleges ‘fixing’ the success rate figures. Where will it end?



An overwhelming majority of staff working in FE colleges have concerns with the credibility of success rate data.

A survey conducted by Lsect, publisher of FE Week, revealed that after excluding those who responded ‘don’t know’, 90 per cent said “yes” to the question: “Across colleges, do you think there is a problem with success rate data credibility?”

Becky Dowst, CIS manager at Totton College, said: “The artificial inflation of college success rates causes the national rates to rise, putting colleges under greater pressure to improve their own success rates still further…where will it end?”

Angie Tithecott, head of funding and performance review at Canterbury College, added: “Over the years I have had other staff within my college come back and tell me how other colleges improve their rates; this has been using methods I would have said were not acceptable, like changing end dates and removing fails.”

The survey, which had 106 responses from 94 colleges, followed a confidential report, seen by FE Week, from Tenon Education Training and Skills Limited. The document, which FE Week reported in its last edition, was written for a secret group of FE colleges and suggested headline success rates could be improved by up to 10 per cent by adopting unfair practices.

Jon Brown, MIS manager for Loughborough College, said: “There is certainly pressure within colleges to find ways to maximise success rates (other than by making sure the data is correct).

“This is unhelpful to say the least and in discussions with colleagues in the sector, there is definitely a spread of views as to whether this is acceptable.

“Given this, it is highly likely that a significant number of colleges ‘fix’ the success rates.

“The problem is that most of the information is anecdotal, but there is a growing unease that some colleges have knowingly decided to take actions that disguise the true picture.”

If we receive reliable intelligence about any provider manipulating their data the Agency will investigate fully.”

The Tenon report claimed: “the use of practices to improve success rates is widespread within FE colleges.”

Seventy-per cent of respondents to the Lsect survey agreed when asked if they thought this was true. Dean Carey, head of MIS at South Nottingham College, said: “I’ve worked in four colleges in an MIS role and there has always been pressure applied to the MIS/data teams to ‘sort’ the data out either with individual department heads or cross-college.

“It requires these teams to maintain a strong stance and make managers aware of the risks and ultimately providers can be setting themselves up to fail by artificially inflating success rates.”

He added: “We’ve all seen the letters from the funding agencies warning us not to adjust data and there will be tighter checks done, etc.”

Mr Carey said the reality was that resources were being squeezed at the Skills Funding Agency (SFA).

However, 24 survey respondents disagreed that the practices were “widespread”.

Neil Reeves, MIS team leader at City College Brighton & Hove, said: “There may be some instances where the practices outlined are occurring, but I feel they may be looking for a scandal that isn’t there.”

A spokesperson for the SFA told FE Week: “The Agency does not use overall success rates as a means of managing and regulating funds, or as a trigger for formal intervention.

“If we receive reliable intelligence about any provider manipulating their data the Agency will investigate fully.”

The Tenon report also said colleges were using specific practices “to support in the manipulation of inspection grades.”

However, a spokesperson for Ofsted told FE Week:  “Ofsted is confident of the reliability of the national success rates data for this purpose.

“Ofsted also uses a range of other sources of evidence when arriving at inspection judgments including the provider’s own in-year performance data, the provider’s self-assessment report, previous inspection findings, observations of teaching, training and assessment and the views of staff, learners and employers.”



Your thoughts

Leave a Reply

Your email address will not be published.

4 Comments

  1. Mortimer Lake

    Hmmm smacks of the Freakonomics principle (http://en.wikipedia.org/wiki/Freakonomics) where high school teachers and sumo wrestlers were compared in an analysis of cultures that encourage cheating.

    Wherever there is a system that demands results and continually better performance, then you will have a culture that encourages inventive/imaginative ways of improving the results, some maybe less honest.

  2. Malcolm Today

    Errr. Elvis is dead and a man has even set foot on the moon.
    Colleges have clearly needed to manipulate their data for years. They have been dealing with the same standard of poorly educated, innumerate and illiterate post-16 year-olds that have been let down by the poor schooling system but . . . loe and behold, their success rates have continued to go up and up. And this is with all the cuts in funding and cuts in support that has gone on since this latest shower got into government.
    The Agency won’t do a thing though. How can they? Colleges are too big to fail. Grade 3 colleges take over running Grade 4 colleges. Grade 4s object to Ofsted and so on and keep on getting funded. Same story, different year.
    But here’s a good idea, why doesn’t “the agency” give experienced and caring private training providers a bit of a kicking. They’re an easy target. Then they can always give the funding to colleges that manipulate their success rates and then sub-contract their funding (top slicing 30%) to brand new (here today, gone tomorrow) dodgy providers that employ the learners (it’s still happening you know) themselves and deliver 12 week apprenticeships.
    Hang on a minute. They tried that didn’t they? But don’t be surprised, when the Level 2 numbers plummet and the post-19 Employers scarper (because they don’t need 19+ minimum-12-month Admin apprenticeships) and “the agency” has a big underspend on apprenticeships, that “the agency” doesn’t do it all again! Then when FE Week does a big “exposé” “the agency” can give good private training providers another kicking and then go back to sleep for a couple of years and wait to be offered their next redundancy/retirement packages when the next round of government departmental restructuring occurs.

  3. ernest younger

    Interesting stuff. I would also note: teachers coaching students through assignments; shifting to in-house teacher assessed accreditation schemes in preference to exam based schemes; teachers training students to get through a specific exam; giving students site of the exam questions prior to the exam.

  4. Jon Carr

    I think this article (on FE Week’s own pages) presents a more accurate assessment of the current situation:

    http://feweek.co.uk/2012/06/08/data-management-abuse-or-inadequacy/

    Maybe I’m being naive, but my experience is that MIS Managers are trying to do the right thing and have made significant changes to practices (eliminating data manipulation) since the infamous Geoff Russell letter at the end of 2009.

    In the words of Mark Smith in the article linked to above – ‘Do a small number of providers occasionally do stupid things which tarnish the reputation of the many – yes. Do we see any widespread evidence of systematic abuse? – very little’. A very different conclusion from that of Tenon’s ‘secret audit’.