Assessment organisations put to the customer service test in our mystery shop

Assessment organisations put to the customer service test in our mystery shop

End-point assessment has been a huge source of anxiety for providers across the sector this year, amid a growing scandal over the glaring lack of apprentice assessment organisations approved to deliver the new tests.

Every apprenticeship standard approved for delivery must now include an EPA carried out by an independent organisation listed on the government’s register of AAOs.

But there aren’t enough AAOs – a state of affairs that’s increasingly worrying to FE Week readers – so we decided to carry out a mystery shopper exercise to see what the customer experience was like.

We first reported on delays to government approval of AAOs in April last year, when only 13 had been cleared to deliver EPA for 23 of the 88 apprenticeship standards ready for delivery at the time.

Then Dr Susan Pember, a former senior civil servant at the Department for Business, Innovation and Skills, stepped up in June 2016 to say it was “morally wrong” to start an apprentice on a course without having the end-point assessment in place.

Her comments made waves, and in October – when it was revealed that almost 60 per cent of apprenticeship standards were still without – others from the sector expressed their worries about the EPA process.

Mark Dawe, the chief executive of the Association of Employment and Learning Providers, described EPA as “a nightmare” at the time.

Apprenticeships and skills minister Robert Halfon provided cold comfort to a now-panicking FE sector when in November 2016 he said he was “pretty sure” no apprentices would reach the end of their course without an AAO in place.

And in December, FE Week analysis showed that 78 standards were still without an AAO – just over half of the total approved for delivery. Currently 105 out of 163 have an AAO in place.

For our investigation, FE Week chose one of the apprenticeship standards with most starts – the level two ‘customer service practitioner’ – and contacted each of the AAOs approved to deliver EPA for this course.

We called up pretending to be a college with apprentices on roll who were ready to do their EPA, and asked each organisation to explain what service they could offer.

All of the companies contacted were regulated awarding organisations as well as approved AAOs, though they had been approved for delivery of EPA for different lengths of time.

Based on our customer service experience and the details provided by each AAO, we have rated them out of five stars – check out the results below.

Summary of the standard

The customer service practitioner apprenticeship standard is a level two apprenticeship, which takes a minimum of 12 months to complete.

In the role, apprentices must “deliver high-quality products and services to the customers of their organisation”.

According to the standard, this can be “delivered from the workplace, digitally, or through going out into the customer’s own locality”.

Apprentices are also expected to “demonstrate excellent customer service skills and behaviours as well as product and/or service knowledge when delivering to your customers”.

The customer service practitioner apprenticeship standard is the third most popular standard this year, while the equivalent framework is the seventh.

The standard was developed by businesses including Boots, British Gas Services, BT and Superdrug.

It falls under funding band six, which means a maximum of £4,000.Knowledge requirements include “knowing your customers” and “understanding the organisation”, while examples of skills needed are “communication” and “personal organisation”.

Apprentices on this standard will be required to have or achieve level one in English and maths, and to have taken level two in English and maths tests before completing their apprenticeship.

The apprenticeship should be reviewed after a maximum of three years.The government is phasing the withdrawal of old apprenticeship frameworks to be replaced by the new standards, and is currently consulting on the removal of a fourth batch.

Customer Standard Practitioner in numbers

End-point assessment details

The EPA for this standard comes in three parts: an apprentice showcase, a practical observation, and a professional discussion.

These elements have weightings of 65 per cent, 20 per cent, and 15 per cent respectively, and the observation and discussion should take one hour for each.

The apprentice showcase allows candidates to present examples of their development over the training period, and can be assessed face to face or remotely.

The practical observation must include customer interaction, and should enable the apprentice to demonstrate their skills, knowledge and behaviour in situations such as “dealing with a customer complaint” or “handling a general enquiry”. As a minimum, the observation should cover “presentation,equality, interpersonal skills, communication and personal organisation”.

The professional discussion should take place between the apprentice and the independent assessor after the observation.

It should be an opportunity for the assessor to ask any questions they may have and explore aspects of the work in more detail – especially with regard to scenarios that didn’t come up during the observation.

The apprenticeship includes pass and distinction grades, which are determined by the EPA.

To gain a distinction, the apprentice must meet the pass criteria and also consistently perform above the required level for the role.

There should be an opportunity for apprentices to resubmit work or to be reassessed, if any part of the EPA is not adequate when first completed.

Further development should be provided before any retakes and practice runs for the EPA are recommended.

External Quality Assurance for the EPA for this apprenticeship standard is delivered by Ofqual.

 Innovative EPA plans

The assessment plan available for this standard, which was published last August 1, recommends the use of technology in the EPA process.

The aim is to make the process more efficient and more convenient for employers.

The plan says: “Employers within the sector have strongly expressed that EPA methods should be flexible, and where possible delivered virtually due to the impact of demand fluctuation on the quality of customer service within the sector.”

The apprentice showcase, for example, can be “assessed face to face or remotely”.

The apprentice may choose to deliver a presentation in person, or could submit a digital report, storyboard or journal to the assessment organisation instead.

FE Week’s research revealed that some assessment organisations are looking at innovative ways to deliver the minimum one-hour practical observation by using technology to carry out this part of the assessment remotely as well.

iCQ in particular showed a range of different uses of technology to streamline the process. It provides a flexible, browser-based system called ‘iLearner’, which includes e-portfolio and funding management tools to track the progress of the learner.This system will also include option to book into an online diary when the apprentice is ready for EPA.

iCQ recently developed a web video tool, to allow reviews or gateway meetings between the employer, provider and apprentice to take place remotely.These web videos can be recorded and time and date stamped.

The intention is to use the iLearner web video tool to conduct remote professional discussions, and iCQ also told FE Week it “anticipates using this tool for practical observations”. This is “subject to external quality assurance organisation agreement”.

Active IQ also said “assessment can be carried out remotely via video or in person”, but was unable to give further detail on how this would work in practice before the time of going to press.