I'm really excited about the potential here. It seems like a big shift to get teachers (at least in the US) bought into comparative judgement, but I think the opportunity here to add AI as a long-term time saver might be the hook needed to get folks on board.
Very interesting! I have shared this with my grade 9 teacher colleagues (Albany, NY). I think on balance this is a good thing - and probably inevitable, whether I think it's good or not - but I have two concerns. One you addressed: over time there will be a temptation, both in terms of "work smarter not harder" and in terms of saving money, to lean all the way into AI grading. Over time, I imagine teacher knowledge and skill will atrophy. Related, this might be particular to NY public high schools, but twice a year the entire English department scores high stakes standardized tests together. As annoying as it can be, I think there is some (difficult to measure) value in the process that we would lose. The other concern is adding diesel fuel to a trade-off. That is, "rubric writing" (or, in this case, comparative writing, which I believe amounts to the same thing) is how you develop as a writer, but it's also how writers get put into boxes. AI, I imagine, would accelerate the amount of efficiency and value placed on standardized writing. Which, on one hand, is foundational, but on the other hand, can be stifling.
An old Welsh educator is speechless (for once) at this frankly beautiful and invaluable production of human ingenuity. Schools, children and parents will be eternally grateful. To all the NMM team, 👏👏👏
Sounds amazing, with lots of potential. The AI marking might also be useful in capturing human biases. This appeared to be achieved in the example by humans reviewing the biggest discrepancies between humans and AI. Helping more students get the feedback that is right for them.
Really promising findings. I've also had greater success in smaller scale tests when asking for a comparative judgment in tools rather than asking for a specific grade from a rubric.
As an English teacher in Massachusetts, USA, I have some students who have been caught using AI to write essays. It's pretty evident when they use it because their essay is written too well! I found this article interesting as a way to save me time on some simple written essays that are graded for effort and not as a summative assessment. I find there is potential to use this as a grading tool to save me some time in grading simple written responses, but I wish I could be more positive about its ability to grade summative assessments. Maybe with more development, you can raise my confidence bar soon! In any event, this was interesting given that grading takes up a major portion of my preparation time!
I find all research of interest though I'm not convinced on this one. I keep returning to the thought of whatever type of assessment you use are you truly evaluating its effect? More to the point how are you evaluating its effect. System X might be more cost effective than system Y but does it improve learning. No formal assessment is worthwhile if it doesn't result in improved learning. In the same way that getting A stars at A level doesn't really mean much unless you use what you learned after the exams.
I'm really excited about the potential here. It seems like a big shift to get teachers (at least in the US) bought into comparative judgement, but I think the opportunity here to add AI as a long-term time saver might be the hook needed to get folks on board.
Very interesting! I have shared this with my grade 9 teacher colleagues (Albany, NY). I think on balance this is a good thing - and probably inevitable, whether I think it's good or not - but I have two concerns. One you addressed: over time there will be a temptation, both in terms of "work smarter not harder" and in terms of saving money, to lean all the way into AI grading. Over time, I imagine teacher knowledge and skill will atrophy. Related, this might be particular to NY public high schools, but twice a year the entire English department scores high stakes standardized tests together. As annoying as it can be, I think there is some (difficult to measure) value in the process that we would lose. The other concern is adding diesel fuel to a trade-off. That is, "rubric writing" (or, in this case, comparative writing, which I believe amounts to the same thing) is how you develop as a writer, but it's also how writers get put into boxes. AI, I imagine, would accelerate the amount of efficiency and value placed on standardized writing. Which, on one hand, is foundational, but on the other hand, can be stifling.
So much potential for this if used effectively for sure!
An old Welsh educator is speechless (for once) at this frankly beautiful and invaluable production of human ingenuity. Schools, children and parents will be eternally grateful. To all the NMM team, 👏👏👏
Keen to try this in Sydney, Australia.
This is awesome and hugely exciting. When you fancy chucking out some trials further afield, our little school in The Bahamas would be up for it!
Really fascinating.
Sounds amazing, with lots of potential. The AI marking might also be useful in capturing human biases. This appeared to be achieved in the example by humans reviewing the biggest discrepancies between humans and AI. Helping more students get the feedback that is right for them.
Really promising findings. I've also had greater success in smaller scale tests when asking for a comparative judgment in tools rather than asking for a specific grade from a rubric.
Can I ask how you do this exactly? Which platform, what kinds of prompts?
The original post is regarding No More Marking, but I've had interesting results using NotebookLM for comparative judgements
As an English teacher in Massachusetts, USA, I have some students who have been caught using AI to write essays. It's pretty evident when they use it because their essay is written too well! I found this article interesting as a way to save me time on some simple written essays that are graded for effort and not as a summative assessment. I find there is potential to use this as a grading tool to save me some time in grading simple written responses, but I wish I could be more positive about its ability to grade summative assessments. Maybe with more development, you can raise my confidence bar soon! In any event, this was interesting given that grading takes up a major portion of my preparation time!
I find all research of interest though I'm not convinced on this one. I keep returning to the thought of whatever type of assessment you use are you truly evaluating its effect? More to the point how are you evaluating its effect. System X might be more cost effective than system Y but does it improve learning. No formal assessment is worthwhile if it doesn't result in improved learning. In the same way that getting A stars at A level doesn't really mean much unless you use what you learned after the exams.