If teachers decided to take a “staycation” this year they may have regretted their decision. Being exposed to English newspapers in the exams “silly season” is not an edifying experience.
This year may well have been worse than most. First, there was gnashing of teeth over the rise in the number of first-class degrees awarded by universities, followed by panic about a one percentage point drop in the number of 11-year-olds attaining level 4 in their English tests.
Perhaps we should be less worried about some changes in students’ performance in tests than about the difficulty many adults have in understanding assessment and what it can and cannot do.
Three misunderstandings are particularly worrying. First is the assumption that scores on tests can be completely accurate. A second is that a short test can validly capture achievement of something as multi-faceted as, say, writing. And a third is that tests can themselves drive up standards in learning - confusing the indicator that learning has occurred with learning itself.
This week the Economic and Social Research Council’s Teaching and Learning Research Programme (TLRP) published a commentary on assessment policy calling for more effort to be made to enhance public understanding based on good research evidence. Assessment in Schools. Fit for Purpose? is written by the journalist Warwick Mansell and the Assessment Reform Group, a group of 10 academics with decades of experience in this field, of which I am a member. We argue that all those who have an interest in school assessment systems - parents, teachers, policy-makers and journalists - need to consider the intended and unintended consequences of the decisions taken in this area of policy.
Assessment has been asked to perform an increasing number of functions in recent years. Although it seems attractive, simple and cost-effective to use data from single assessments for multiple purposes - “a no-brainer” to policy wonks - research has questioned whether it is possible to fulfil so many goals simultaneously.
Assessments that were originally designed to indicate what a student knows and understands of a subject have now become proxy measures of the quality of teachers, heads, schools, support services, local authorities, the Government and the nation.
With stakes this high, no wonder results are represented in dubious ways and the means of achieving them are subject to games-playing. If a school takes actions designed to improve its performance, such as drilling to earn marks at the expense of teaching for deeper understanding, the consequences for students are often against their long-term educational interests.
We argue that the fitness for purpose of the assessment system needs to be re-evaluated and the education of young people made the priority once more.
In recent years, the Assessment Reform Group has promoted formative assessment, also known as assessment for learning. This is not to say we are against tests and exams - much of our work has been concerned with enhancing their validity, reliability and usefulness. All forms of assessment can be beneficial or harmful, depending on their “quality”.
Fitness for purpose rests on the extent to which formative assessment, summative assessment by teachers, tests and exams and accountability systems have the quality needed to engender confidence and support good teaching. Research in all these areas provides guidance on how this can be achieved. But is anyone in the policy environment listening?
In our report, we pose four pressing challenges for policy-makers.
First, more must be done to promote effective in-class assessment. My own recent TLRP research on “learning how to learn”, undertaken with colleagues in four universities, showed that significant gaps still exist between teachers’ valuing of formative assessment and the majority’s capacity to implement it, except in mechanistic ways. Effective practice requires teachers to move from trying to ratchet up scores, by teaching to the tests, towards integrating observation, interpretation and redirection of students’ learning in the flow of everyday lessons.
Second, confidence in tests and exams needs to be enhanced. These assessments have far-reaching consequences, some unintended, so much more transparency is needed, possibly with “health warnings” about what inferences can validly be drawn from the data. Results should be published with information about the likely scale of measurement error, thus avoiding the nonsense of claiming that a one percentage point drop is in any way meaningful. Meanwhile, there is need for more work on the validity of measures so that, for instance, use of commas does not come to define whether an 11-year-old can write.
Third, the cost of assessment needs to be analysed and justified. We calculate that the annual direct and indirect cost of the assessment system is pound;750 million. From early years to A-level, young people in England experience a huge load of formal assessment. The public needs to ask if the sums devoted to this are well spent.
Fourth, political micro-management of the assessment process needs to be avoided. In England, the Qualifications and Curriculum Development Agency and Ofqual have official control over the details of how students are assessed but, in practice, ministers have exerted extensive powers. Nobody denies their right to determine overall assessment policy, but their recent record in specifying technical details - the pilot work on single- level tests being a case in point - raises questions about whether they are qualified to do so.
The next (election) year is likely to be crucial for deciding future directions of assessment in schools. The public and the profession need to ask some hard questions about assessment policy so that the education of our young people is enhanced rather than undermined.
Download: `Assessment in schools. Fit for Purpose?’Professor Mary James, Associate Director of Research, Faculty of Education, Cambridge University.
