This week we will release our AI-enhanced Comparative Judgement site for general use to our customers, a landmark that represents the end of a long journey.
I have really valued NMM’s skeptical stance and insights on Generative AI - but I am having trouble understanding this post! It - and much of your recent work - calls ChatGPT into question, and yet you are launching a product using the very same technology? Is this a change of heart or a new (but skeptical) test of its functionality? Please do keep posting your ongoing thought processes and testing as most of us don’t have the intellectual commitment to spend months in a shed doing this sort of hard thinking — and we very much appreciate its fruits!
Just to add to what Chris said - the marks are not good enough, and we can tell that by comparing AI marks to human marks. But the written feedback is not as clear cut, and we are not best-placed to decide if it is good enough - our teachers are. So that's why we are making it available for low stakes use in June & July. It is good at rapidly picking out the spelling and grammatical errors in a text and there is clearly some value to that.
I think the broader issue is that generative AI has been so hyped that it is then too easy to react against that and say it's useless. I don't think it is going to be as transformative as many boosters say. But it may well be able to save teachers 20-25% of their time when writing feedback - that's not game changing but it is still pretty useful.
The other possibility is that whilst it is flawed now it will only get better. That's certainly a possibility, although again one we are a bit sceptical about!
Thank you for your thoughtful comment. We thought we owed it to our subscribers to be able to try out GPT generated feedback for themselves, rather than simply take our word for it. Perhaps there is some use case we haven’t considered? It can pick out spelling and grammatical mistakes at times - is that enough to justify its use? We don’t know, which is why we have put this release out there. I certainly wouldn’t trust any marks or grades it gives!
I have really valued NMM’s skeptical stance and insights on Generative AI - but I am having trouble understanding this post! It - and much of your recent work - calls ChatGPT into question, and yet you are launching a product using the very same technology? Is this a change of heart or a new (but skeptical) test of its functionality? Please do keep posting your ongoing thought processes and testing as most of us don’t have the intellectual commitment to spend months in a shed doing this sort of hard thinking — and we very much appreciate its fruits!
Just to add to what Chris said - the marks are not good enough, and we can tell that by comparing AI marks to human marks. But the written feedback is not as clear cut, and we are not best-placed to decide if it is good enough - our teachers are. So that's why we are making it available for low stakes use in June & July. It is good at rapidly picking out the spelling and grammatical errors in a text and there is clearly some value to that.
I think the broader issue is that generative AI has been so hyped that it is then too easy to react against that and say it's useless. I don't think it is going to be as transformative as many boosters say. But it may well be able to save teachers 20-25% of their time when writing feedback - that's not game changing but it is still pretty useful.
The other possibility is that whilst it is flawed now it will only get better. That's certainly a possibility, although again one we are a bit sceptical about!
Thank you for your thoughtful comment. We thought we owed it to our subscribers to be able to try out GPT generated feedback for themselves, rather than simply take our word for it. Perhaps there is some use case we haven’t considered? It can pick out spelling and grammatical mistakes at times - is that enough to justify its use? We don’t know, which is why we have put this release out there. I certainly wouldn’t trust any marks or grades it gives!
Has there been an update to the privacy policy to accommodate these changes?