If students are banned from using AI, is it fair for teachers to use it?

The ethics of new assessment technology

As regular readers of this Substack will know, we’ve been working on ways to use AI to assist teachers in marking writing.

We’ve also written about how important it is that students don’t use AI to write their assessments, and that we probably need to go back to more closely supervised in-person assessments to avoid AI plagiarism.

A few people have argued this is hypocritical. If we are banning students from using AI, shouldn’t we ban teachers from using it to mark their work?

Let’s explore this argument in more detail.

With students, what matters is not the product but the process

The value of the work students produce in an assessment is not in the work itself but in understanding it represents and the thinking that went into creating it. Imagine two students write an essay. One struggles hard, reads a lot, writes and redrafts it, and produces something that is OK but not great. Another produces something perfect by pasting the prompt into ChatGPT and copying the output into a Word doc. Who has done better?

If all we cared about was the product, then it would be the second student. But we don’t. We care about the process. The first student’s response has led to them learning more and their essay represents greater understanding of the topic than the second student’s essay. Fundamentally, it does not matter that a computer can answer this question better than the student. What matters is what the student has learnt from answering it, and what it tells us about their understanding.

The basic assessment principle here is the difference between the sample and the domain. The sample is the test itself. The domain is the student’s wider understanding. The sample only matters if it tells us something valuable about the domain — otherwise it is worthless. This is one of the most fundamental principles of assessment – and one of the most poorly understood. We have explored this argument in greater detail here.

With professionals, what matters is not the process but the product

The above dynamic is completely reversed in the workplace. If my accountant comes to me and says, I have found an AI application that will allow me to process your tax return 10x as quickly and as a result I can cut my fees, then I am fine with that.

If my doctor tells me she can process my test results quicker and come up with a more personalised and accurate diagnosis and treatment plan by using an AI tool, then I am fine with that too.

What I care about with these professionals is the final product.

Of course, there is a caveat to this. If the AI tool produces a mistake, I expect the professional to catch the mistake, and if they don’t, I am holding them responsible, not the AI tool. I suspect that this might become a growing aspect of a professional’s job. They will increasingly use more and more technology & AI to help them do their job. The value they add will be using their expertise to choose the right tool, sense check its outputs, and be ultimately responsible for the output that goes to the customer.

(As an aside, the high level of expertise these humans have is something they have acquired as part of a lengthy learning process that begins with fundamentals and works up to more advanced skill. The existence of advanced AI tools does not let humans short circuit this learning process and jump straight to the advanced level. Again, we explore this in more detail here.)

With teachers, do we care about the product or the process?

Broadly speaking, I think teachers are like doctors & accountants in that we care about the product, not the process. If a teacher says they can use an AI tool to speed up the process of marking, return marked work to students quicker, provide more personalised feedback and potentially allow for more work to be set and marked, then that is all positive.

The same caveat that applies to doctors & accountants applies to teachers in that they have to guarantee the outputs of the AI tool they are using.

But there is perhaps also a further caveat that only applies to teachers – which is that students often really care about what their teachers think about their work and want their teacher to read what they have written and have an opinion about it. I don’t really care how much human attention my tax return gets – I just want it to be accurate. A ten year old does really care about whether their teacher has read their work or not.

This is not an argument against teachers using AI to help with marking – it’s just an argument for designing it carefully to retain human input.

So should teachers use AI to help them assess?

Our current model of AI marking & feedback works as follows.

• Teachers still read their students’ work and make judgements about it. AI judging supplements the human judgements, but every piece of writing is still read by 2 human teachers.

• The teachers leave audio comments on the work. The AI transcribes and combines all the audio feedback and produces a paragraph of written feedback for the student and a summary report for the teacher.

This model is much quicker & more accurate than the current model, and involves each piece of writing being read by two teachers, not one - which is twice as many as most traditional non-AI approaches to marking.

We are running a free trial for secondary schools in England who would like to try out this approach. You can read more about the project here and sign up for a short intro webinar here on Monday 10 February at 4pm.