Using AI to judge the best Christmas film quote

Have yourself a very merry Christmas Judgement Day

Every year at No More Marking we run a fun and festive Comparative Judgement Christmas task where we get people to judge their favourite Christmas song, chocolate, film, etc.

This year we are adding AI into the mix, so you can judge your favourite Christmas film quote and see if our new robot overlords agree with you!

It’s also a nice way of seeing how our new AI features work.

AI bless us, everyone!

Here is how it works.

• Click on this link to register as a judge.

• You’ll be presented with a pair of quotes from a famous Christmas film (you’ll also get the name of the film.) The interface will look like this.

• Read both quotes and have a think about what the best one is!

• You can click on the button with the snail in the bottom right hand corner to see which quote our AI picks. You will also get to see a sentence where the AI explains its decision!

• You can then make your own decision by clicking on the button that says “left” or “right". You’ll then be taken to a new decision.

We will share the overall results in a week or so’s time on our social media feeds.

Now we have AI judges. Ho ho ho!

In the big writing projects that we run, we typically get about 80-85% agreement between humans and AI.

In these big writing projects, we don’t include the button on the bottom right that gives you real-time information from the AI. Instead, we get teachers to complete their judgements independently, with no influence from the AI. You can then download the results of the AI judging and the AI feedback separately - and see the percentage agreement between your teachers and the AI.

Faith is believing in things when AI tells you not to

We don’t necessarily expect human & AI choices on Christmas film quotes to align! In fact, we don’t necessarily expect human choices on Christmas film quotes to align!

For our serious assessment projects we spend a great deal of time validating and fine-tuning to ensure that the AI decisions align with the decisions of teachers.

If you’d like to try out a writing assessment, register for one of our introductory webinars here.

Merry Christmas!