13 Comments
User's avatar
James Cantonwine's avatar

I'm really excited about the potential here. It seems like a big shift to get teachers (at least in the US) bought into comparative judgement, but I think the opportunity here to add AI as a long-term time saver might be the hook needed to get folks on board.

Expand full comment
Brian Huskie's avatar

Very interesting! I have shared this with my grade 9 teacher colleagues (Albany, NY). I think on balance this is a good thing - and probably inevitable, whether I think it's good or not - but I have two concerns. One you addressed: over time there will be a temptation, both in terms of "work smarter not harder" and in terms of saving money, to lean all the way into AI grading. Over time, I imagine teacher knowledge and skill will atrophy. Related, this might be particular to NY public high schools, but twice a year the entire English department scores high stakes standardized tests together. As annoying as it can be, I think there is some (difficult to measure) value in the process that we would lose. The other concern is adding diesel fuel to a trade-off. That is, "rubric writing" (or, in this case, comparative writing, which I believe amounts to the same thing) is how you develop as a writer, but it's also how writers get put into boxes. AI, I imagine, would accelerate the amount of efficiency and value placed on standardized writing. Which, on one hand, is foundational, but on the other hand, can be stifling.

Expand full comment
Sarah Findlater's avatar

So much potential for this if used effectively for sure!

Expand full comment
Terry Mackie's avatar

An old Welsh educator is speechless (for once) at this frankly beautiful and invaluable production of human ingenuity. Schools, children and parents will be eternally grateful. To all the NMM team, 👏👏👏

Expand full comment
AnonymousEd's avatar

Keen to try this in Sydney, Australia.

Expand full comment
Tom Millinchip's avatar

This is awesome and hugely exciting. When you fancy chucking out some trials further afield, our little school in The Bahamas would be up for it!

Expand full comment
Edrith's avatar

Really fascinating.

Expand full comment
Ian Thomas's avatar

Sounds amazing, with lots of potential. The AI marking might also be useful in capturing human biases. This appeared to be achieved in the example by humans reviewing the biggest discrepancies between humans and AI. Helping more students get the feedback that is right for them.

Expand full comment
Ranga's Take by Daren White's avatar

Really promising findings. I've also had greater success in smaller scale tests when asking for a comparative judgment in tools rather than asking for a specific grade from a rubric.

Expand full comment
Mark Aveyard's avatar

Can I ask how you do this exactly? Which platform, what kinds of prompts?

Expand full comment
Ranga's Take by Daren White's avatar

The original post is regarding No More Marking, but I've had interesting results using NotebookLM for comparative judgements

Expand full comment
Kelley Bulmer's avatar

As an English teacher in Massachusetts, USA, I have some students who have been caught using AI to write essays. It's pretty evident when they use it because their essay is written too well! I found this article interesting as a way to save me time on some simple written essays that are graded for effort and not as a summative assessment. I find there is potential to use this as a grading tool to save me some time in grading simple written responses, but I wish I could be more positive about its ability to grade summative assessments. Maybe with more development, you can raise my confidence bar soon! In any event, this was interesting given that grading takes up a major portion of my preparation time!

Expand full comment
Jan's avatar

I find all research of interest though I'm not convinced on this one. I keep returning to the thought of whatever type of assessment you use are you truly evaluating its effect? More to the point how are you evaluating its effect. System X might be more cost effective than system Y but does it improve learning. No formal assessment is worthwhile if it doesn't result in improved learning. In the same way that getting A stars at A level doesn't really mean much unless you use what you learned after the exams.

Expand full comment