8 years of maintaining standards in Primary Writing in England & Wales

From pilot to realisation

In the summer of 2016 we ran our first writing pilot with 5 schools and 256 pupils aged 10 to 11. Our idea was to use Comparative Judgement to help schools measure their pupils’ progress in their writing over time and to be able to compare the progress of their pupils to pupils in other schools. Since then we have assessed over 2 million pieces of writing from pupils aged 4 to 16 globally.

We started the project because schools at the time were disenchanted with the assessment of writing that was required at that time by the Department for Education. The disenchantment was at two levels. Firstly, schools felt that the assessment was unfair as different standards were being applied in different areas of the country, and that they may be disadvantaged by local authorities applying harsh or generous standards in the absence of any national system of moderation. Secondly, they felt that the approach constrained the teaching of writing and rewarded mechanical features over freedom of expression.

So why did we think that we could improve on the current system? We had been using Comparative Judgement for a number of years for various research projects and had begun to understand its power. Typically the advantages of Comparative Judgement are considered to be the following:

• The increased reliability from using relative judgement (which piece is better?) over absolute judgement (what is this piece worth?)

• The potential to capture the tacit knowledge and common sense of experts, without the distortions of hyper-specific rubrics

• The advantage of aggregating judgements from many judges on every piece of writing compared to using the judgement of a single expert

• The use of a powerful statistical model to model the quality of work over the multiple aggregated judgements

Beyond these qualities we had the grain of an idea that we could build a massive matrix of decisions across the country that would ensure fair comparisons, not just between schools but over time.

In the following series we will consider what we have learned since 2016. In particular we will look at:

1. What we have learned about the the national system of writing we were seeking to complement. How reliable is national writing moderation? Can it measure how standards in writing are rising or falling? Can schools trust national writing moderation to give them a fair measure of their teaching of writing?

2. What we have learned about Comparative Judgement. Can it measure how standards in writing are rising or falling? Can schools trust our matrix of national decisions to give them a fair measure of their teaching of writing?

3. What we have learned about standards in writing over time. Have standards risen or fallen? How certain are we about our findings?

4. What we have learned about teaching writing, what makes effective writing, and what is holding children back.

5. Is it possible to develop a “science of writing” and if so, what part could Comparative Judgement play?

6. What role does Comparative Judgment have in the age of AI?

To stay updated on future posts, click on the button below to subscribe to our Substack.