The ethics of AI assessment: five big issues
What do teachers, students and parents think about AI marking essays?
Over the past couple of weeks, my colleague Chris and I have been out and about talking to teachers, students and parents about what they do - and do not - want from an AI assessment system.
Here are five big issues that keep recurring.
Speed & efficiency matter
Students care about getting feedback quickly, teachers care about excessive workload and senior teams care about students getting enough exam practice.
There is nothing wrong with any of this. As I’ve written about in a completely different context, speed matters. Speed is not opposed to quality; it is an aspect of quality. If you get feedback on your essays within a couple of hours of completing the essay, you will be much more likely to understand it and act on it.
And yet it is obviously wildly unrealistic to expect human teachers to routinely provide feedback on essays within a couple of hours. In fact, one of my biggest bugbears as a teacher was when a student would hand an essay in a week late, and then turn up at the staff room door at the end of the day asking if I’d marked it!
If AI can deliver quicker feedback, that’s definitely a good thing.
Accuracy
Everyone worries about AI errors, and about what the process is for dealing with them. Again, this is perfectly legitimate: one of the things that we’ve written a lot about is that traditional exam systems have well-established processes for dealing with human errors which don’t work with AI (we are building the processes for AI- see here).
But the flip side is that everyone understands that humans make errors too. I have spoken to groups of students and teachers who were genuinely shocked to learn that currently, with human marking, in GCSE English Literature you only have a 52% chance of getting your true grade.
Human contact
Students care what their teachers think about them and they want their teachers to read what they write.
However, there were some ways in which they were interested in the concept of AI feedback for its own sake - not just because it would be quicker.
For example, one student told us they liked the idea of AI feedback because it might offer a different perspective from their teacher, and pick up things their teacher had not thought of.
De-skilling
Another concern we hear - particularly from senior teams - is about de-skilling: what if teachers lose the capacity to mark essays and give feedback on student writing?
In some areas, I don’t care about de-skilling. For example, I have seen so many examples of teachers staying late formatting PowerPoint slides and trying to find just the right image for their worksheet, and I have never been convinced it’s a good use of their time. Andrew Old has written about this recently and I largely agree with him. If a teacher never designed another resource again I would not be that bothered.
However, when it comes to assessment, I am much more concerned about the possibility of teachers losing important skills. If a teacher never read another student essay again I would be very concerned.
We have to design systems that reduce workload and speed things up, but that preserve teachers engaging with student writing.
The environment
Recently, we have been hearing more concerns about the environment, particularly about how much water AI uses, and therefore how much water it would take to mark an essay.
We are not unconditional AI boosters, and we are always willing to consider the downsides of the technology. But on water use specifically, I think the concerns have been overblown. Andy Masley has done some excellent analyses of this, including showing that one of the most famous analyses of AI water use confused cubic metres and litres and was out by a factor of 4500!
Find out more
As ever, if you want to find out more about what we do, you can join one of our intro webinars. The next one is on Monday April 27. These webinars are very popular - we show you how the system works and at the end we give 30 free credits to all attendees, so you can try it yourself on a class set of essays in any subject.
If you work in a school, you can also book a 30-minute call with me here where I can get you set up on our system with 30 free credits.
I think the deskilling of teachers is an interesting concept. I start teaching before there was an accepted view of how planning should look. The arrival of Ofsted did change that I think but I'd suggest that laptops and printers changed it more . When I was a head I certainly never wanted to read everyone's planning. I had better things to spend my time on like being in classrooms, talking to kids and looking at outcomes. Student teachers and recently qualified teachers seemed to really go to town. Colour coding, highlighting etc. You'd never do all that if you had to write it out by hand. When I was a local authority adviser I did watch an awful lot of lessons. I rarely examined the lesson plan in detail unless something wasn't going well. I saw a lot of overplanning and that's definitely a waste of teacher time. In the film Happy Go Lucky, there's a scene where two of the characters who are primary teachers are seen spending Saturday morning making masks from paper bags and loads of scrap material. Later in the film you see the kids doing the activity in class. And having fun doing it. It's a great example of the difference between planning and preparation. The best teachers are engaged in preparation 24/7. I think so anyway. They are sparked of by things they see, read, hear. They get an idea and squirrel it away until it's useful. I don't believe AI can do that for you.
Thurday afternoon. In five minutes I ran 28 handwritten scripts - pencil text in four boxes on one page - through the scanner creating a single Pdf. Then I spent two (interesting) hours discussing the assessment criteria with Claude and how to give feedback to the students. Traditionally, it would have taken two-three boring hours just to assess and give perfunctory feedback to 28 students, good 5/6 etc. In contrast, I was able to upload a Pdf copy of their work and personal, targetted feedback on their OneNotes, within hours of them completing the work. Interestingly, I have been working with AI and this class for 18 months now. When I was calibrating the levels with Claude (he/she/it) pointed out that one of the students had improved significantly over the last two assignments. I hadn't noticed. I hadn't asked Claude for the info, but Claude knew that I would welcome the news.