Taking moderation to extremes

or

What we have learned about KS2 writing and assessment in 2016

Like a portakabin in the playground, the ‘interim’ assessment criteria for KS1 and 2 writing look likely to be in use indefinitely. Although the STA website emphasises that the 2017 frameworks are for one year only, they said that was the case for 2016, yet here they are again, unchanged and unaffected by the controversy that surrounded their birth.  Whatever one thinks of the descriptors, this probably counts as a blessing, as, for 2016 -17, we begin the year with an idea of how we have to assess our pupils’ writing; something we lacked for most of 2015-16. We also have the added benefit, dubious though it may be, of experience, having applied the criteria during the summer assessment season.  As someone with responsibility for overseeing KS2 writing moderation in 2016 and going into several schools to monitor, challenge and support the process, I thought it might be useful to share a few thoughts, informed by my own observations and by the feedback received from our esteemed moderators at both KS2 and KS1.

Context and controversy

The interim frameworks were rolled out in September 2015 and were supplemented by exemplification materials which finally appeared around Christmas.  There was resistance to the exemplification materials, the criteria and the timescale for assessment (May 22^nd given as deadline for writing assessment at KS2 and 13^th June for KS1) which led to grudging changes to assessment and reporting arrangements (“for this year only”) in March 2016. The increased demand and ‘perfect fit’ approach were defended as part of a commitment to higher standards, with ‘robustness and fairness’ advanced as reasons for the early deadlines. The exemplification materials were also defended but the compulsion to use them, along with the accompanying tick-sheets that offered assistance in applying the criteria, was removed. Local Authorities and moderators were left in the middle somewhat. They had an obligation to ensure that assessment in the schools being sampled was robust and that criteria were applied consistently. Original plans were binned. Professional dialogue was reinstated, having been discarded in favour of a more authoritarian ‘scrutiny’, and moderators retrained to make them aware that schools couldn’t be required to submit assessments in advance of a visit (no matter that this actually accelerated and improved the process) and didn’t have to use the tick-sheets and exemplification, though moderators would. Fortunately for us, we had kept schools abreast of all the changes and had involved them throughout. Our moderators are practising teachers drawn from several schools and collectively, and often individually, possessed of vast experience and expertise. As a result, the majority of schools were prepared to submit their assessments in advance and were welcoming and hospitable to moderators who, as had always been the case in the past, were there as professional colleagues, helping them to ensure their assessments were accurate, no matter what they thought of the strictures and criteria imposed upon them.

The Frameworks and Exemplification

The writing framework at KS1 and KS2 is really the GPS test in another form. The descriptors, whether bulleted or dropped into a table for ticking, mention purpose and audience but are simply a list of grammatical features that have to be included to meet the expected standard, whether purpose, audience and context require them or not. The attempt, of course, is to replace the old ‘best fit’ approach where assessors make a judgement about the qualities of a collection of writing against a prose descriptor (or, with the now defunct APP grids, several prose descriptors) and arrive at a grade level or standard.  The ‘best fit’ approach is mistrusted because its descriptors are open to interpretation and, even with levels and sub-levels, it provides only a broad idea or approximation of how capable a writer the child is. It does not guarantee ‘mastery’ of each and every attribute. A tick-list (what some commentators like to refer to as the ‘perfect fit’ approach, with its suggestion of rigour and precision), ostensibly details what a child can or has to do, but does not evaluate effectiveness or quality. The approach is an abandonment of much past practice including that of former Tory governments, as is illustrated by this from the Cox Report of 1989:

The main principle is that the secretarial aspect should not be allowed to predominate in the assessment while the more complex aspects of composition are ignored. It is evident that a child may be a poor speller, but write well-structured and interesting stories, or be a good speller, but write badly structured and boring stories. (1)

            The dismay that many teachers felt on seeing the criteria and the exemplification was expressed very clearly across social media with Mike Rosen, amongst others, railing against this highly reductive approach and the view that “good writing can be defined or measured by using a criterion like ‘does this passage include an embedded relative clause’”.(2) Arguably, the criteria simply ask for the application of grammatical knowledge and ‘secretarial skills’ (or ‘technical skills’, depending upon one’s ideological frame perhaps) but, as Rosen and others ask, is this a viable or useful definition of writing for purpose and audience when there is nothing to suggest that successful address to these principles is of any value? The creators of the exemplification materials (the adults, not the children whose work has been commandeered) seem to be aware of the conflict and difficulty. Their commentaries included sections on ‘composition and effect’, which were the areas which, in the days of writing tests and level descriptors, gained most marks. Their comments are pointless in assessment terms though, and do not address anything in the framework. They deal with ‘complex aspects of composition’ and elements which actually are mentioned in the current national curriculum, such as ‘fluency’, enjoyment and understanding of language’, ‘effectiveness as well as competence’ and ‘conscious control’ but these aspects do not figure in the descriptors.

There were a few prosaic and pragmatic problems encountered with the tick boxes and bullet points of the framework. The ‘perfect fit’ descriptors often conflated more than one skill or attribute. Across a range of work teachers and moderators had to find children “using present and past tense mostly correctly and consistently” (KS1) or “using passive and modal verbs mostly appropriately” (KS2). Often of course, this did not occur simultaneously and a single tick or highlighter stroke could not convey whether the statement had been met or not. Trying to judge whether a child had met the standard for the single bullet point, “using inverted commas, commas for clarity, and punctuation for parenthesis mostly correctly, and making some correct use of semi-colons, dashes, colons and hyphens” (KS2) caused mild apoplexy. A single tick, or the alternative approach of annotating the work, disingenuously proposed by the then Education Secretary, could not usefully cover the entirety of this descriptor and it had to be disaggregated. Ironically, we had to construct lengthier tick lists than those provided by STA so that we could manage the information and confirm what a child had achieved. This difficulty was compounded by the fact that, for a child to have met the standard, every bullet point or statement had to be demonstrated.

As the STA training for Local Authority leads demonstrated, reading several pieces of work and confirming that a particular standard had been met mostly, consistently, sometimes, occasionally, frequently or whatever takes a long time. Moderation visits were much lengthier than in previous years, especially in three form entry schools, and year 2 and year 6 teachers, who generally do not swim leisurely in oceans of free time, often found assessing every child against each criterion onerous, exhausting and wasteful. Even during the moderation visit, there was insufficient time to follow the exemplification model and ensure that 5 out of 6 pieces ticked enough boxes for a pass. Judgements were required and dialogue was essential. If there is a return to solitary scrutiny by moderators in 2017, they will need sleeping bags and camping equipment for every visit.  

In addition to time pressures, and despite our best efforts as an LA, teachers, who had received the exemplification material late in the year, did not always feel confident or fully prepared for the assessment process. At KS2 they were also preparing children to take tests in grammar, maths and reading which were based on a six-year curriculum of which their children had had eighteen months. At KS1, they were preparing to administer and mark new tests while gathering evidence to make assessments against new criteria in maths, reading and writing. They also had to decide whether to use the grammar test that the government had mistakenly published online a few months earlier and which was originally designed to inform their judgement of children’s writing. Nonetheless, many teachers gave up their time, conscientiously applied the criteria, read the exemplification materials and prepared for moderation with a thoroughness that was admirable but morale-sapping and debilitating. Previously, moderators had travelled in pairs but with the complexity of KS1 assessment and the need to check that the tests had been marked accurately by teachers, this was increased to three. At KS2 we stuck with two moderators per school but, while also moderating the moderators, I often acted as an additional body.

Moderation in all things

When in schools, moderators engaged teachers in a dialogue about the evidence. Receiving provisional assessments in advance meant they were able to select pupils whose work they wanted to see and get a good start to the long process of verifying or challenging judgements. Most teachers did want to talk about the particular qualities of the work being scrutinised and, as in previous years, were keen to point out where a child was using imaginative vocabulary, evocative similes, effectively deployed dialogue or clear expressions of opinion and evidence. They were rightly proud of the occasions in the writing where a particular group of words captured an idea or emotion, i.e. where children discovered the objective correlative! However, moderators had to be party-poopers and bring discussion back to tick sheets and grammatical features. Time was spent hunting down semi-colons and passive constructions at KS2 and at KS1 the exclamation sentence proved elusive and there were debates about horizontal strokes and spaces between letters. Where teachers had undertaken periodic assessment using the tick lists and had deployed them with children during the drafting and proof-reading process, the ticks were forthcoming. Where they hadn’t, aided by well-meaning organisations like NAHT and the unions who had put out guidance emphasising what could and couldn’t be done and had encouraged a retrogressive level-based judgement, things were less straightforward.  A collection of work might well have been a 4B or even a 5 last year but, if the passive voice was lacking or a semi-colon, dash or hyphen could not be found, it was not at the expected standard. The deferred deadline of June 30^th allowed moderators to give schools time to produce extra work which did allow pupils to tick the boxes. This may not be the case next year and in 2016 it added the pressure of an additional visit and extra assurances about independence.

Because teachers were talking to teachers, because there was space to review and because the majority of teachers at KS1 and KS2 had attended briefing sessions and network meetings or knew moderators and advisers they could ask, things went relatively smoothly. Because teachers knew their children extremely well and moderators were keen to help rather than condemn, the dialogue worked. Changes to STA instructions on cursive handwriting did have to be explained and there were many debates about what a ‘wide range of clause structures’ might be and which structures and vocabulary correctly indicated the required levels of formality. As formal writing was also a feature of the ‘greater depth’ standard at KS2, moderators often advised schools to include more formal writing in their plans for next year, suggesting discursive pieces and more stringent approaches to writing  investigations in science for example, where the passive could also be required.  There were also discussions about what constituted a short story (a requirement at KS2) or a narrative (a requirement at KS1), though, in the latter it mattered little as none of the criteria dealt with a text level judgement, unlike the now defunct Foundation Stage descriptors which at least referenced features of a narrative. At KS2 confirming that an atmosphere had been created and dialogue integrated so that character could be conveyed and action advanced did lead to some discussion and schools fared better when they sought to have children produce longer, more sustained narratives and free them from the tyranny of the old writing test or the Big Write model.  This criterion, like the first two KS2 ‘greater depth’ criteria, also seemed more in keeping with old judgements about composition and effect, and less about parading one’s SPAG knowledge.

Conversations with moderated schools unsurprisingly revealed that moderators and teachers shared a frustration with the new assessment system, the political whims that dictated the frameworks and the cack-handed implementation. The arbitrary decision that those working below ‘working below’, i.e. at pre-key stage standards were not to be moderated was concerning for some schools where there were pupils operating at this level. Moderators assisted in this at the school’s request and helped them to distinguish between the three sub-levels that describe pre-key stage 2 performance. How schools who weren’t moderated or who were but did not offer up their ‘pre-key stage’ pupils, got on may be anyone’s guess but with 75% of schools not getting a moderating visit some uncertainty must remain.

An uncertain future

We don’t yet know the details of the 2017 moderation process and whether it will be the same as this and previous years. Government bowed to pressure over the timing of the assessment but said it was only for this this year – thus it may well be that the writing deadline is May 22^nd or thereabouts and KS1 has to be wrapped up by June 13^th ish. KS2 grades/levels/standards or whatever we now call them may have to be submitted to LAs and STA by the end of the very testing month of May with moderators then checking the accuracy of assessments in selected schools. KS1 teachers would just be exhausted shadows of their former selves. Holiday patterns, with Easter vacations and the summer half-term break occurring at different times and for different lengths of time for different schools are likely to cause even greater confusion and complication next year than they did this. The ‘moderation’, as originally envisaged for 2016, may also become a ‘scrutiny’ and not the helpful, collaborative process which moderation defaulted to this year. On the basis of the original instructions to LAs for 2016, scrutineers would march into schools carrying the threat of a maladministration charge with them. They wouldn’t engage in professional dialogue but would sequester themselves in the building, issuing demands for more evidence if required and then present their verdict to the head, LA and STA.  Opportunities to have kids produce last minute pieces of writing so that they did not fall short of the expected standard for want of a colon, dash or embedded clause would be unavailable.

At our post-moderation debrief, moderators shared what they had learned and what they were going to be doing in their own schools next year. At KS1 there was a feeling that there has to be a continuing focus on subject knowledge and on ensuring that practice, e.g. with regard to cursive writing, allows pupils to hit the criteria. Teachers noted that they must make sure that pupils have the chance to use different sorts of sentence, including the mimicking of Swallows and Amazons’ exclamations that begin with a ‘what’ or a ‘how’ and contain a verb (What jolly wonderful lessons will emerge!). They also saw the inevitability of building in periodic internal moderation to identify what needed to be taught directly but were also determined to ensure immersion in a range of texts, regular reading and opportunities to write for pleasure and enjoy language.

Similar thoughts were in the minds of KS2 moderators, who were adamant that they wouldn’t teach to the interim frameworks because tick lists are not helpful in improving writing, whereas an attention to purpose and audience is. Using models of good writing produced by other children and by professional writers was strongly advocated, with attention being drawn to how they use punctuation and particular grammatical structures to achieve a particular effect. They felt it important to render unto Caesar what is Caesar’s however and thought the extended tick sheets were a necessity, when undertaking periodic assessment themselves or when engaging pupils in peer and self-assessment and as part of the all-important drafting and proof-reading process. Ensuring that there were opportunities for formal writing was agreed as vital, particularly where pupils aspire to be working at greater depth. This did mean further developing their understanding of what constitutes ‘formality’ and ensuring that the passive voice was encouraged in assignments such as the science report, news report, discursive essay and occasionally in fiction. There was also a desire to increase the knowledge of some of the more abstruse elements of the framework, such as the difference between separating independent clauses with a colon or a semi-colon.  Extended short stories will figure more, it was believed, with teachers concentrating on constructing extended narratives and teaching how to create character, develop action and dialogue and climb and descend the story mountain. All of this being done, of course, with a strong regard for pupil independence.  

Discussion about the 2016 moderation cycle inevitably turned to the tests with reading being the major concern at both KS1 and KS2. How writing, after years of lagging behind performance in reading, suddenly became the stronger discipline, was certainly an area for discussion. The problems with the reading tests at both KS1 and KS2, however, is a topic for another day.

1.    The Cox Report: English for ages 5 to 16. London: Her Majesty's Stationery Office 1989

2.     See michaelrosenblog.blogspot.co.uk . Michael Rosen also produced a poem on the subject called Writing at the Expected Level (20^th June 2016) and has written several pieces on the grammar test and writing assessment.