Ever since the early days of teaching machines in the 1920s, the PLATO system of the 1960s and the Intelligent Essay Assessor of the 1980s, there’s been an ever-changing landscape in the use of technologies to support formative assessment practices within the education sector.
In recent years, artificial intelligence (AI) has emerged as a powerful tool for enhancing assessment practices, and the FirstPass platform, developed by Bolton College in collaboration with NCFE, is leading the way in leveraging the use of natural language processing to support students and teachers in this exciting space.
FirstPass is an online platform that utilises AI to provide real-time feedback to students as they compose their responses to open-ended questions. The basic premise behind the platform is a simple one: that real-time effective feedback enables learners to develop positive meta-cognition skills as they identify areas for improvement and adjust their work before submitting it to their teachers for commentary and feedback.
The Assessment Innovation Fund, a £1 million initiative launched by NCFE, provided Bolton College with the necessary funding to pilot FirstPass and assess its effectiveness in enhancing formative assessment practices. The pilot involved six further education (FE) colleges across England and Wales, spanning a diverse range of vocational subjects. The results of the pilot were very encouraging, demonstrating that real-time feedback can positively aid students as they author responses to open-ended questions.
Key findings from the pilot
Learners reported feeling more engaged and motivated due to the immediate feedback provided by FirstPass. Real-time feedback gave learners the opportunity to assess their first drafts, enabling them to make changes and edits before submitting. Results showed that nine in 10 learners (92.3 per cent) found that the feedback they received from FirstPass helped them to compose better answers to open-ended questions.
Teachers reported that the FirstPass platform led to improved submissions from learners. Consequently, they spent less time providing remedial feedback and 82 per cent stated that FirstPass would be an effective AI service for supporting them with their formative assessment practice.
One of the unique traits of the FirstPass platform is that it leverages the power of crowdsourcing to support the training of subject topic classifiers that underpin the delivery and assessment of open-ended questions over the platform.
The pilot demonstrated that teachers and other subject specialists can train AI models through a platform like FirstPass. The participatory model lends itself particularly well to the education sector where the larger group are motivated towards shared goals.
Implications for formative assessment
The success of the FirstPass pilot and the advances that we have all witnessed in the past 12 months demonstrate the potential of using AI to support formative assessment practices within the education sector. In some instances, the things we have taken for granted will be displaced. In other cases, AI will open a new vista of opportunities within the formative assessment landscape.
We have come a long way since the advent of the first analogue teaching machines in the 1920s. In 2023, the FirstPass pilot has demonstrated that AI can play a transformative role in enhancing formative assessment practices within the further education sector; especially regarding its ability to deliver immediate real-time feedback, reducing teacher workload around remedial feedback, and improving learning outcomes.
As AI continues to evolve and advance, FirstPass and other emerging services have the potential to transform the way formative assessment is undertaken. Thanks to continued support from NCFE and the Assessment Innovation Fund, we’re able to progress our research into a second phase where we will look to further develop and test the platform.
This research will focus on formative assessment in two specific courses: the Level 2 Diploma in Early Year Practitioner and the Technical Award in Child Development and Care at Levels 1 and 2.
Additionally, we are being funded by the Education Endowment Foundation (EEF) to undertake a small-scale pilot with 8 to 12 further education colleges. The pilot will assess how FirstPass can support formative assessment practices to aid learners within GCSE English Language resits. If it is successful, the EEF will increase its funding to support a larger scale project during 2024-25.
Re AI article, Enhancing Assessment at Bolton College
How many meta-cognition skills are there, and how do
they differ from ‘ordinary skills’ like editing what you have
written because you’re interested in your chosen subject?