Out of his British box

15th October 2004 at 01:00
Dylan Wiliam, assessment guru, is finding work rewarding and exciting on the other side of the Atlantic. He talks to Stephen Phillips

So why did Dylan Wiliam, doyen of the assessment world on this side of the Atlantic, decide to up sticks to join Educational Testing Service in Princeton, New Jersey? His answer is reminiscent of US bank robber Willie Sutton's celebrated deadpan logic for holding up banks: "Because that's where the money is."

Wiliam explains that ETS, creators of the SAT, the US university admissions test, is where the assessment experts are. "There are more people working on assessment here than anywhere else on the planet. It's a chance to have corridor conversations with all the people I've read and learnt from."

Wiliam is no slouch himself. He was joint architect of the national curriculum tests for 14-year-olds. Inside the Black Box, the 1998 pamphlet extolling the benefits of formative assessment he co-wrote with fellow Kings academic Paul Black, sold 40,000 copies and helped persuade ministers to embrace assessment for learning, informal classroom techniques to diagnose individual pupil understanding.

Wiliam and Black recommended practices such as comment-only marking, spurning grades, which tend to have less impact than comments (provided these contain specific, actionable feedback rather than vague, abstract observations). They advocated self or peer assessment - noting that students tend to be franker about each other than teachers - and open-ended questioning, which reveals a grasp of concepts rather than regurgitation of facts.

Their 2002 sequel, Working Inside the Black Box, another bestseller, documented efforts to translate such suggestions into a workable pedagogy in six Oxfordshire and Medway schools, and was amplified into last year's book, Assessment for Learning: Putting it into practice. The Government's promotion of formative assessment probably owes something to Wiliam and Black showing half-grade GCSE gains per student, per subject in the school trials.

In Princeton, Wiliam hopes to parlay ETS's clout as a non-profit organisation in traditional, external examinations (dubbed summative assessment in education jargon) into promoting formative (personalised) assessment in US schools.

"ETS is very good at tests, but there's been less emphasis on teachers'

needs and tools to give (them) an idea of where kids' thinking is at," he explains. Wiliam and his colleagues are developing training models for staff and field-testing assessment for learning in New Jersey and Pennsylvania to tweak the approach for US classrooms.

It shouldn't be too foreign, he points out. "These techniques are not new to many teachers. They have all been tried."

But there are marked differences between American and British testing philosophies, Wiliam observes. Though both governments share a common zeal for summative testing, US schools overwhelmingly go in for multiple-choice exams, largely shunning the constructive response approach of GCSEs and A-levels.

Multiple-choice tests "cover a large amount of content quickly and are cheap to score," he says. But many US staff complain that the drive to administer such tests, plus the high stakes attached, are forcing them to "teach to the test", dumbing down instruction to rote drilling.

Wiliam counters that the formative assessment he recommends has also produced gains on multiple-choice tests in studies, but laments that "teachers feel under pressure and it's just too tempting to go for lower-order instruction". By comparison, GCSEs and A-levels are "tests worth teaching to", he says. "Kids have to able to do actual maths and science and know how to write a sentence."

He contrasts US concern with paring down marking costs with Britain's more laborious approach, starting with the comparatively quaint UK "craft tradition of test construction, giving an examiner $1000 who then sits down at the kitchen table to write it. The marvellous thing is that (British tests) are as good as they are."

Wiliam predicts "convergence" between the two approaches, particularly with the advent of computer programs to automate marking of written test answers. It might be disquieting to some, contemplating grammatically correct, impeccably spelled nonsense earning top marks, but Wiliam says the technology is "nearly there in terms of (essay questions) in the US. In five or ten years we'll be able to assess a richer range of things as cheaply as multiple-choice answers."

Closer to hand, Wiliam supports a SAT-style aptitude test (being mulled by Whitehall officials to determine university admissions) as "a way to help British universities achieve greater diversity." A-levels, "explicitly test how well you've been taught", he says, so they favour pupils at well-heeled schools.

Still, African-American pupils have stubbornly lagged behind their white peers in SAT scores, notes Wiliam, something US researchers have ascribed to the "cultural capital" wealthier students enjoy compared with minorities who mainly come from the wrong side of the tracks.

He suspects the situation may also reflect teaching quality, even though the SAT attempts to measure native intelligence. Schools in many US cities are racially polarised: the inner-cities have less cash and fewer experienced staff while the white suburbs are better resourced. With funding largely based on local property tax receipts, the gaping inequities of America's school system have been an eye-opener, says Wiliam. "One finds schools funded at $19,000 per pupil and others at less than a third of this."

The No Child Left Behind (NCLB) Act represents an attempt to hold schools accountable for the test performance of all pupils, invoking punitive sanctions against schools if they miss so much as one target across myriad student categories. But the legislation has proved highly contentious amid complaints it sets unrealistic goals without coughing up enough funding and stigmatises schools for societal problems.

However while Wiliam acknowledges "flaws", he hails the Act as "extraordinarily subtle. The idea is that you can't allow the success of one kid to compensate for the failure of another. If it's minority kids failing, the community might be happy to accept this, but NCLB says you've got a problem - it focuses on the trailing edge rather than the leading edge."

Wiliam says he relishes American "can-do," versus the British "mustn't grumble" attitude. "People really believe things can change," he enthuses.

Effecting change is another matter amid a patchwork of different testing regimes in different states. He yearns for the simplicity of Britain's comparatively unitary education system. "It might be convenient to think of America as one country, but it's more useful to think of it like Europe with 51 education systems."

Log-in as an existing print or digital subscriber

Forgotten your subscriber ID?


To access this content and the full TES archive, subscribe now.

View subscriber offers


Get TES online and delivered to your door – for less than the price of a coffee

Save 33% off the cover price with this great subscription offer. Every copy delivered to your door by first-class post, plus full access to TES online and the TES app for just £1.90 per week.
Subscribers also enjoy a range of fantastic offers and benefits worth over £270:

  • Discounts off TES Institute courses
  • Access over 200,000 articles in the TES online archive
  • Free Tastecard membership worth £79.99
  • Discounts with Zipcar, Buyagift.com, Virgin Wines and other partners
Order your low-cost subscription today