How to protect the validity of apprentice observation

Linda Martin sets out how independent assessors can uphold the validity of observation and ensure apprentices get the most out of the assessment method

Linda Martin sets out how independent assessors can uphold the validity of observation and ensure apprentices get the most out of the assessment method

22 Jan 2023, 5:00

In the context of apprenticeship standards, observation offers an opportunity to study the unstructured and natural performance of an apprentice. It has a great deal of face validity. An independent assessor can witness first-hand the ability of an apprentice to conduct tasks in the real world in direct relation to ‘Knowledge, Skills, and Behaviours’ assessment criteria.

While it can present a great chance for the apprentice to display their competence, as assessors and educators we must uphold the validity of assessments involving observation. There are several ways we can achieve this.

Encourage the best environment for success

An element of chance always remains in observations. However, we should consider, in the sphere of what we can control, how to provide the best opportunity to meet the required assessment criteria.

For example, if the apprentice is working front of house in the hospitality sector and they need to demonstrate particular abilities in dealing with the public, then the observation should be scheduled for a busy shift. This will give the apprentice the best shot at practising the necessary customer interaction.

Remain mindful of the assessor’s presence

We need to be aware that the independent assessor is not completely removed from the scene, and this may affect performance in several ways.

Firstly, there’s the factor termed reactivity, where people may try to give a good impression rather than performing as they naturally would if unobserved. So, assessors must be aware of their presence and stick to observational assessment best practice. When new independent assessors join a team there should be careful training programmes put in place to minimise their personal impact on observations.

Secondly, an independent assessor hovering in the background making notes may not also always be welcome. Say the apprentice is handling a verbal complaint from an upset customer, the complainer and employer would not appreciate the presence of a lurking assessor. In these scenarios, it’s important that assessors tactfully retreat out of view, and even out of earshot, but observe and follow up afterwards by asking pertinent questions.

Be self-aware of unintentional biases

Unintentional biases, such as confirmation bias or affinity bias, could mean an assessor is influenced by previous evidence. For example, they may have a more favourable opinion of a particular apprentice because they appear to have similar qualities or characteristics that resonate with the assessor. Therefore, when starting an assessment, they could unconsciously decide that the apprentice will be strong or weak based on personal preconceptions.

Additionally, theories such as growth mindset show that our brains are pre-disposed to note flaws and imperfections before appreciating what is accurate and correct

The risk of bias could lead the independent assessor to find evidence which supports that bias and fail to see evidence that does not. Valid, fair, credible, and reliable assessment practices rely on the independent assessor remaining true to the relevant standards and assessment specification as set by the EPAO.

Training independent assessors to be self-aware of their potential biases is key to our ethos at Professional Assessment Ltd. By encouraging our staff to face up to their biases, they can make assessment judgments based on the presented evidence that are both as fair and as accurate as possible.

Relax

Independent assessors need to determine competence against an agreed set of standards. However, assessors ultimately want apprentices to perform well and see what the apprentice is really capable of. High levels of anxiety rarely enhance performance, so an independent assessor should do their best to put an apprentice at ease during an observed assessment.

Overall, the advantages of observation can be maximised by careful planning of activities with apprentices and proper training with independent assessors from the outset, so all stakeholders can benefit from higher levels of face validity and from this naturalistic assessment.

More from this theme

Assessment, Long read

AI’s threat to the essence of learning is far bigger than we realise

FE Week has uncovered significant evidence of the harm that generative AI is doing to the integrity of assessment,...

Jessica Hill
Assessment

Ofqual and DfE studying ‘feasibility’ of ‘fully digital’ exams

Study includes looking at whether national infrastructure up to scratch for on-screen exams

Samantha Booth
Apprenticeships, Assessment

Apprentices to take ‘integrated’ EPA and mandatory qual assessment

New guidance on mandatory qualifications could see fewer apprentices drop out early

Shane Chowen
Assessment, Exams

Ministers confirm exams and grading plan for 2023

GCSE and A-level grades will return back to pre-pandemic levels

Samantha Booth
Assessment

Saxton: Adaptive testing could end exam tier ‘ceilings’

'What I found very difficult with tiering was having to have conversations with students and parents about ‘we just...

Will Nott

Your thoughts

Leave a Reply

Your email address will not be published. Required fields are marked *