Assessing without levels: the leaked report

‘Same as it ever was?’

The long awaited report on primary assessment is due in September but a leaked copy has appeared in the Guardian. It is clearly unfair to comment on the leaked draft but what the hell?

Context
The report was commissioned  after a consultation on new descriptors  for teacher assessment highlighted the confusion and anxiety that had been  generated by the  wanton abandonment of an imperfect but functioning  system. It also came after the initial  publication of test frameworks which  described threshold performance (March 2014 then revised June 2015 DfE on  www.gov.uk). The end of key stage descriptors (October 2014 DfE on www.gov.uk)  offered up a confusing hierarchy which looked remarkably like a set of levels (a child at KS1 and 2 might be working below, towards, at or above age related expectations or attain ‘mastery’ – presumably grand mastery could be a prospective next step when government decide to up the ante!). The descriptors varied in number according to the subject and some were ludicrous – requiring teachers to distinguish degrees of pleasure in reading for instance.  They were very much in keeping with the plethora of other attempts to fill the levelless void with tick lists and sure-fire methods for measuring attainment and progress  in the new curriculum. Forbidden to mention ‘level’, we got  a series of synonyms instead - ladders, stages, steps- with tick lists varying between systems dependent upon what the creators thought were key objectives in each subject .  Experienced and immersed in a levelling system, while operating two different curricula at the same time, schools often experimented with  new assessment in years 1, 3,4 and 5 though  most stuck to levels, to APP and to other systems already in use. They largely followed  NAHT advice and awaited clarification.  Some foresaw and experienced struggles with  ill-informed Ofsted inspectors demanding accurate up to the minute data on the progress of individuals and groups in every class and struggled, in the face of test changes, curriculum revamps and an alien, or non-existent, nomenclature to communicate their understanding of pupils’, classes’ and the school’s  future and current attainment.  There was also anxiety about how they would communicate to parents, who basically understood levels, whether their child was doing well, all right or not so well.  Hopefully apocryphal, tales emerged of  numbers being substituted with flowers or fruit so that a daisy child would have tuliphood as a target and a mediocre apple be encouraged to aspire to peachiness or, in more affluent neighbourhoods, to kumquat status.

I had hoped that  the report would clarify things a bit and address the concerns raised in discussion with the primary schools with which I work.  These concerns included:

• How schools could efficiently and accurately  communicate a child’s current and past  performance on transfer to a new school or to a new class, especially when this did not occur at the end of a key stage where test results might at least give some idea of how best to cater for the newly arrived child.
• How schools could judge whether they, their teachers and their pupils were actually doing well and how they could be assured that their judgements were in line with those a mile or a hundred miles away.
• How hostile Ofsted inspectors could be convinced that school was doing the best by their children, given their  obsession with easily interpretable statistics, their conviction that progress was linear and easily observable, and their fetishistic faith that measuring ‘impact’ was effortlessly accomplishable, as well as being a useful word to throw around  when schools tried to explain what they were doing.
• Whether the transformation of programmes of study into lengthy tick lists was an appropriate substitute for the various level descriptors in operation and what a tick might actually mean about a child's understanding, knowledge and capacities. Whether a tick was a lifelong endorsement that a skill or fact had been acquired or a temporary indication was also a running concern.

The Report
The report is anodyne and its 12 sections, multiple sub-divisions  and 45 pages do not provide many satisfactory answers.  It may be of value as a discussion document and the 5 page section on guidance for assessment policies is transformable into a tick list of questions to ask oneself and thus may have some use. However, the whole, in its draft form, seems to do little more than catch up with the thinking that many primaries have already done. It appends lengthy quotations from the new  Ofsted framework  and the SEND Code of Practice, with which most schools should already be familiar and offers sound but very general advice on how to proceed. The 2009 Expert Group Report on Assessment seems to me to offer more though, again, it  will be familiar to most heads.  This report isn’t without value and will be a basis for discussion with schools, but it is a disappointment for those wanting a bit more clarity as we approach a busy year culminating in a new battery of tests,  and involving uncertainty about how test performance will be reported and anxiety about how to ensure 85% of pupils get to the new 'standard' of secondary readiness. On the basis of pronouncements from government, unemployment also beckons for heads who do not leap that newly raised bar.

The principle offered in the foreword is a sound one: the new curriculum requires “ radical, cultural and pedagogical change, from one which has been too dominated by the requirements of the national assessment framework and testing regime to one where the focus needs to be on high-quality, in-depth teaching supported by in-class formative assessment”. I think this would be supported by most teachers and most heads. Whether  or not it  is more honoured in the breach than the observance, Assessment for Learning is generally agreed to be the key to good learning (notwithstanding some recent TV shows which seem to argue that rote learning en -masse is the most effective means of developing mathematical knowhow).  I don’t think there would be much argument with ensuring that pupils  “know,  apply and understand the matters (sic), skills and processes specified in the ...programmes of study” ( again notwithstanding the fact that the national curriculum only applies to LA maintained schools).  However, reiterating this does not seem to me to address the concerns mentioned earlier which are more about accountability and quality control than about pedagogy directly.  Is AfL as applied in school X working and how do we know? Do we have to wait for the entirety of KS2 (or the five years of secondary schooling for that matter) to find out whether we are doing a good job, as measured by the end of key stage tests and assessments?  I don’t feel that the report actually gets to these questions.  It simply shrugs its shoulders and gives general advice on how schools need to do it for themselves ...but without levels.

The reiteration of Tim Oates’s criticism of levels and the rationale for removing them  is valid enough but seems to me to be an ideological sleight of hand, shying away from the implications of the argument and what should be a more radical attack on the whole assessment system. The accusation that levels ‘required aggregating a wide variety of data into a single number which did not represent pupil performance accurately or meant that levels were being assigned to individual pieces of work regardless of how much of the programmes of study they covered”  is accurate but is this not true at GCSE and A level where single numbers or  letters define performance even if qualified by an informative asterisk? Is it not going to be true of whatever is used to represent KS1 and 2 test performance be it a raw mark, standardised score or complex formula? That the DfE recently announced that the expected  standards will be equivalent to a 2B at KS1 and a 4B at KS2 also suggests we are exactly where we started, or at least that recourse has to be made to a commonly understood terminology. I would also suggest that the assertion made in the report that “secondary schools often found the information received from primary schools unreliable and unhelpful” is an unjust slur on primaries.  Secondaries all too often fail to do anything with the information  they receive and year 7 and 8 pupils continue to complain that they are forced to endure repetition of previous learning  because secondary teachers have not pursued an AfL approach and uncovered prior knowledge. Secondaries  also  too frequently substitute an alternative test (CATs for example)which simply offers up a number rather than a description of what a child knows, understands and can do.

The argument that levels lead to threshold hugging rather than pursuit of mastery is also true I think but seems equally applicable elsewhere – the all important C at GCSE or the AAB ticket to a Russell Group education and a 2.1 or better. The argument that progress had come to mean a shuffle, walk or stride through levels and sub-levels rather than a broadening or deepening of understanding is also a valid criticism but does the 'no-one left behind' approach of the new curriculum and “no longer grouping pupils according to levels” really mean a more personalised, mixed ability approach in this undifferentiated brave new world ? Maybe.  And maybe levels disappearing will mean pupils gain an “improved mind set about their own ability” but is this feasible with the continued existence of end of key stage tests, grammar schools, social and economic selection and pressure to drive 85% of pupils to the expected standard in years 2 and 6? I feel there is a bit of blind optimism here.

 When addressing the curriculum, the report emphasises the “specific knowledge pupils should acquire by the end of each key stage” suggesting that, without levels, schools will “develop approaches to in-school assessment which are better tied to curriculum content”. This seems to me to be dodging questions about how ‘mastery’ of  skills, processes and application is demonstrated. It emphasises the 'stuff ' or ‘matters’ that are easiest to assess and easiest to forget once the test is done.  Admittedly, ‘mastery’ is  helpfully addressed in a short section which explores its origins in Bloom’s work,  but the report's endorsement of an approach where knowledge(sic)  is broken into discrete units, presented  in a logical order and formatively assessed following “high-quality instruction” seems more applicable to content than process and skills and there is no real life example on offer. Indeed the examples of good practice cited throughout  the report seem to be vague  assertions, lacking specificity of subject, topic or outcome. A sense of hopeful generality and imprecision dominates.

A section on the purpose and principles of assessment is helpful, if not news to anyone likely to read the report, and there is a welcome emphasis throughout on dealing with children with SEN. Inclusion is writ large in the report and justifiably so. There is also a section, of great interest to heads I suspect, on data collection and reporting. This argues that there is no need for lots of recording of formative assessment  and it  is strong on defending teachers against excessive and pointless work.  I doubt that many teachers will argue against the judgement that “real learning is a slow process and can- at best- be observed only through indirect and approximate, measures. For most learners, even the best assessments are not accurate enough to reliably detect real changes in individuals’ deep learning over a period of less than a year”. In fact many have made this sort of argument over the years only to be ignored by government and by Ofsted inspectors claiming that such change is discernible in a twenty minute observation and perfectly captured by a matrix, graph or column of figures.  Will heads be able to quote this statement when challenged by an aggressive inspector demanding to know how well year 4 pupils are doing and why the school cannot pinpoint the precise degree of progress made? Will this be a valid reason for explaining KS2 results which drop below the 85% pass mark for schools?

Ofsted were allies in the report's construction, or at least were involved with the Commission – hence the repeated references to its new handbook. The section on accountability and inspection deals directly with the Ofsted view  (though not necessarily its actual practice of course). Citing the handbook at length the section is full of references to progress and meeting, or preferably exceeding, challenging targets set on the basis of prior attainment (more neutrally called ‘starting points’ and possibly alluding to new baselines as well as KS1 scores). A small section on ‘myths’ fails to answer any concerns about  how Ofsted continues to endorse the sorts of things this reports seems at times to argue against. Pleasingly, there is a strong recommendation for the “inspectorate and school leaders to develop a rigorous  and shared understanding of all aspects of summative and formative educational assessment” and a  euphemistic recommendation that there is some training for inspectors who may need support in developing their understanding of assessment without levels.

There are other recommendations too. Dissemination of the report and extensive and intensive CPD are obviously part of that. A national question bank is  also mooted with questions and model answers being available to schools. Though quality control is also recommended and teachers are invited to make deposits in the bank, the idea seems to me to be rather dangerous and analogous to those stockpiles of comments that schools sometimes used on reports and which failed to address the idiosyncrasies of individuals and groups. The recommended increase in training in assessment in ITT and throughout a teaching career seems a more productive avenue to me, as does the use of SLEs in assessment or  the development of chartered assessors.

Success criteria  are offered for measuring how assessment without levels has fared. I am not entirely convinced of their use as descriptors however.  Much is open to interpretation (one of the criticisms made of levels and perhaps inevitable when language is the medium used) and there is  little that is easily measurable or reportable:  pupils taking responsibility for their own learning, teacher confidence, greater responsiveness in children, stronger links between curriculum, pedagogy and assessment, meaningful conversations with parents and greater professional expertise. An aversion to sarcasm prevents me saying that I am  surprised they couldn't come up with a standardised score to measure each discrete step on the way.

The final published report may make my concerns look like the grumpy, ramblings of an ancient pedagogue.  I hope so. I also hope that any new descriptors for teachers to use at the end of key stages 1 and 2 and the reporting system for the KS2 tests help lead to a system which truly endorses assessment for learning and personalised learning. Everything will then be for the best  in this, the best of all possible assessment worlds.