Thanks for this. Very interesting as always. How effective can essays be in testing factual knowledge if they are positively marked for knowledge? So, essay A doesn’t need to be penalised for the errors but can be rewarded for, eg, knowing that William was from Normandy and that he defeated Harold Godwinson.
To extend your football metaphor. We don’t have to penalise the shots off target, we can just ignore them and count the ones that hit the target. This rewards more the pupils who take the most shots, and rewards even more those who score the most goals.
That feels like the approach we currently take in Geography (GCSE and A Level) already anyway.
The rubrics for the current GCSE & A levels are not terrible but they're not brilliant either. My dream assessment would be a long series of short answer / MCQ / closed questions, and then a couple of completely open ended essays marked with CJ. GCSEs / A levels have a lot of of 2-4 markers which are the worst of both worlds - hard and time consuming to mark reliably but don't promote open ended thinking either.
This is interesting. When you say a long series of short answer or MCQ style questions, what does that look like in practice?
Roughly how many questions are we talking about, and how many marks per question in an ideal assessment? And what overall weighting would you give to those versus the open-ended essays?
You’re clear that 2 to 4 markers are the worst of both worlds. What does best case look like in terms of structure, number of questions and mark allocation?
As a teacher of A level chemistry Iwelcomed, with open arms, the Shift to "Highly Structured" Questions.
Before 1990, Chemistry A-levels often included "Section C" or "Paper 3" style questions that required continuous prose.
This was done to increase "construct validity", ensuring students were being tested on their chemistry knowledge rather than their ability to write a structured English essay.
By 1995, most questions had been broken down into sub-parts (a, b, c, d) that guided the student through the logic.
The downside was that students became less able to link ideas and think critically.
I assumed the death of the "chemistry essay" was the availability of skilled exam markers.
The error over the month of the Battle of Hastings doesn’t impact on the overall strength of the argument, but at what point does a factual error become so important as to undermine the whole premise of the essay? Eg if a candidate thought that Harold won the battle, or that it was fought between Harold and Tostig?
I can't remember the details but when I was preparing for english language GCSE back in 2008 I used to put two compound sentences in the first paragraph of any essay one with a conjunction, and one with a semicolon then write the entire remainder in single clauses because I couldn't think and punctuate at the same time under exam conditions. I'm pretty sure it came across as deranged but it seemed to "defeat the rubric".
This analysis can be extended to any verbal communication. The way a commercial or political speech is evaluated by the general public may not correlate with the accuracy of the claims in the communication.
Thanks for this. Very interesting as always. How effective can essays be in testing factual knowledge if they are positively marked for knowledge? So, essay A doesn’t need to be penalised for the errors but can be rewarded for, eg, knowing that William was from Normandy and that he defeated Harold Godwinson.
To extend your football metaphor. We don’t have to penalise the shots off target, we can just ignore them and count the ones that hit the target. This rewards more the pupils who take the most shots, and rewards even more those who score the most goals.
That feels like the approach we currently take in Geography (GCSE and A Level) already anyway.
The rubrics for the current GCSE & A levels are not terrible but they're not brilliant either. My dream assessment would be a long series of short answer / MCQ / closed questions, and then a couple of completely open ended essays marked with CJ. GCSEs / A levels have a lot of of 2-4 markers which are the worst of both worlds - hard and time consuming to mark reliably but don't promote open ended thinking either.
This is interesting. When you say a long series of short answer or MCQ style questions, what does that look like in practice?
Roughly how many questions are we talking about, and how many marks per question in an ideal assessment? And what overall weighting would you give to those versus the open-ended essays?
You’re clear that 2 to 4 markers are the worst of both worlds. What does best case look like in terms of structure, number of questions and mark allocation?
As a teacher of A level chemistry Iwelcomed, with open arms, the Shift to "Highly Structured" Questions.
Before 1990, Chemistry A-levels often included "Section C" or "Paper 3" style questions that required continuous prose.
This was done to increase "construct validity", ensuring students were being tested on their chemistry knowledge rather than their ability to write a structured English essay.
By 1995, most questions had been broken down into sub-parts (a, b, c, d) that guided the student through the logic.
The downside was that students became less able to link ideas and think critically.
I assumed the death of the "chemistry essay" was the availability of skilled exam markers.
The error over the month of the Battle of Hastings doesn’t impact on the overall strength of the argument, but at what point does a factual error become so important as to undermine the whole premise of the essay? Eg if a candidate thought that Harold won the battle, or that it was fought between Harold and Tostig?
Yes, and that is the power of Comparative Judgement - it gives the judge the discretion to make the distinction between a minor and a major error.
How can that be accommodated within current prescriptive mark schemes?
There aren't mark schemes at KS3 & most of primary! And actually the results from CJ do tend to agree with GCSE / KS2 TAF most of the time.
https://observablehq.com/@nomoremarking/the-reliability-of-grading-writing-using-the-teacher-asses
(Am assuming you are in UK)
Yes but very secondary-focused
I can't remember the details but when I was preparing for english language GCSE back in 2008 I used to put two compound sentences in the first paragraph of any essay one with a conjunction, and one with a semicolon then write the entire remainder in single clauses because I couldn't think and punctuate at the same time under exam conditions. I'm pretty sure it came across as deranged but it seemed to "defeat the rubric".
This analysis can be extended to any verbal communication. The way a commercial or political speech is evaluated by the general public may not correlate with the accuracy of the claims in the communication.
So interesting!