In the context of apprenticeship standards, observation offers an opportunity to study the unstructured and natural performance of an apprentice. It has a great deal of face validity. An independent assessor can witness first-hand the ability of an apprentice to conduct tasks in the real world in direct relation to ‘Knowledge, Skills, and Behaviours’ assessment criteria.
While it can present a great chance for the apprentice to display their competence, as assessors and educators we must uphold the validity of assessments involving observation. There are several ways we can achieve this.
Encourage the best environment for success
An element of chance always remains in observations. However, we should consider, in the sphere of what we can control, how to provide the best opportunity to meet the required assessment criteria.
For example, if the apprentice is working front of house in the hospitality sector and they need to demonstrate particular abilities in dealing with the public, then the observation should be scheduled for a busy shift. This will give the apprentice the best shot at practising the necessary customer interaction.
Remain mindful of the assessor’s presence
We need to be aware that the independent assessor is not completely removed from the scene, and this may affect performance in several ways.
Firstly, there’s the factor termed reactivity, where people may try to give a good impression rather than performing as they naturally would if unobserved. So, assessors must be aware of their presence and stick to observational assessment best practice. When new independent assessors join a team there should be careful training programmes put in place to minimise their personal impact on observations.
Secondly, an independent assessor hovering in the background making notes may not also always be welcome. Say the apprentice is handling a verbal complaint from an upset customer, the complainer and employer would not appreciate the presence of a lurking assessor. In these scenarios, it’s important that assessors tactfully retreat out of view, and even out of earshot, but observe and follow up afterwards by asking pertinent questions.
Be self-aware of unintentional biases
Unintentional biases, such as confirmation bias or affinity bias, could mean an assessor is influenced by previous evidence. For example, they may have a more favourable opinion of a particular apprentice because they appear to have similar qualities or characteristics that resonate with the assessor. Therefore, when starting an assessment, they could unconsciously decide that the apprentice will be strong or weak based on personal preconceptions.
Additionally, theories such as growth mindset show that our brains are pre-disposed to note flaws and imperfections before appreciating what is accurate and correct
The risk of bias could lead the independent assessor to find evidence which supports that bias and fail to see evidence that does not. Valid, fair, credible, and reliable assessment practices rely on the independent assessor remaining true to the relevant standards and assessment specification as set by the EPAO.
Training independent assessors to be self-aware of their potential biases is key to our ethos at Professional Assessment Ltd. By encouraging our staff to face up to their biases, they can make assessment judgments based on the presented evidence that are both as fair and as accurate as possible.
Relax
Independent assessors need to determine competence against an agreed set of standards. However, assessors ultimately want apprentices to perform well and see what the apprentice is really capable of. High levels of anxiety rarely enhance performance, so an independent assessor should do their best to put an apprentice at ease during an observed assessment.
Overall, the advantages of observation can be maximised by careful planning of activities with apprentices and proper training with independent assessors from the outset, so all stakeholders can benefit from higher levels of face validity and from this naturalistic assessment.
Your thoughts