Dear D-AI-sy, this blog is indeed very thoughtful. LLMs are not appropriate for use in a scale setting without a significant amount of model enhancements (and I don’t just mean prompt engineering). They have many valuable use cases as you know. One of them is content ideation and generation. I too am concerned with the amount of hype. In particular a lot of time seems to be wasted by people creating classroom lesson plans that have no/weak curricular alignment or even grounding. Everyone has been given the ability to print a personalized textbook but without the pedagogy and performance. This is a recipe for inequality. It’s also not really saving anyone very much time given all the crafting that goes into each LLM encounter. But it certainly is fun!
I do think there are many good short term use cases. Marking essays is not one of them. Editing helper is one. In fact the best LLM cases are on a 1-2-1 basis where the user has some degree of English reading and writing proficiency and background knowledge on the subject, as well as intrinsic motivation. This alone whittles down the addressable market of beneficiaries to the top 20% of learners (and not necessarily the ones we need to help most).
For teachers, for LLMs to be effective and really save time, they need to be adapted or enhanced for classroom use. LLMs have a material alignment problem. They don’t teach to a developmentally appropriate level. But they do serve as a powerful research assistant (for content that then needs to be carefully checked and aligned) and a writing assistant (for people who already have strong foundational skills and are using it to save time).
They will get better but not all in the same way. Some will be better at x and some at y. We are already seeing this. Specialised LLMs and associated technologies have tremendous potential value.
Dear D-AI-sy, this blog is indeed very thoughtful. LLMs are not appropriate for use in a scale setting without a significant amount of model enhancements (and I don’t just mean prompt engineering). They have many valuable use cases as you know. One of them is content ideation and generation. I too am concerned with the amount of hype. In particular a lot of time seems to be wasted by people creating classroom lesson plans that have no/weak curricular alignment or even grounding. Everyone has been given the ability to print a personalized textbook but without the pedagogy and performance. This is a recipe for inequality. It’s also not really saving anyone very much time given all the crafting that goes into each LLM encounter. But it certainly is fun!
I do think there are many good short term use cases. Marking essays is not one of them. Editing helper is one. In fact the best LLM cases are on a 1-2-1 basis where the user has some degree of English reading and writing proficiency and background knowledge on the subject, as well as intrinsic motivation. This alone whittles down the addressable market of beneficiaries to the top 20% of learners (and not necessarily the ones we need to help most).
For teachers, for LLMs to be effective and really save time, they need to be adapted or enhanced for classroom use. LLMs have a material alignment problem. They don’t teach to a developmentally appropriate level. But they do serve as a powerful research assistant (for content that then needs to be carefully checked and aligned) and a writing assistant (for people who already have strong foundational skills and are using it to save time).
They will get better but not all in the same way. Some will be better at x and some at y. We are already seeing this. Specialised LLMs and associated technologies have tremendous potential value.
Thanks for your tireless work,
Sof