Skip to main content
article icon

Named, ranked and blamed

news | Published in TES magazine on 8 March, 2013 | By: Stephen Exley

League tables that measure teachers individually are gaining popularity in the US, but their impact can be catastrophic. One such project resulted in a practitioner taking his own life after a poor rating

One autumn morning, Rigoberto Ruelas didn’t turn up for work. His colleagues at Miramonte Elementary School in Los Angeles were worried; in the 14 years he had taught at the school, he had seldom taken a day off.

Several days later, a police search-and-rescue team taking part in a training exercise in the Angeles National Forest spotted the 39-year-old’s abandoned vehicle. In a nearby ravine, they found his body lying 100 feet below a bridge. The Los Angeles county coroner later ruled that he had taken his own life.

Despite working in a tough, gang-ridden part of LA, Ruelas adored his job. He would tutor his pupils at weekends, and pushed them to aim high and go to college. Speaking after Rigoberto’s funeral, his brother Jose told journalists: “I want him to be remembered as a person that loved his career, he had a passion for his career. He loved the children and that’s why he taught.”

But according to the Los Angeles Times, he was one of the “least effective” maths teachers in the city. Days before Ruelas’ death in September 2010, the newspaper had entered uncharted territory. In the UK, school league tables have - for better or worse - become accepted as part of our education system. But the Los Angeles Times took things to a new level: it published individual ratings for each of the city’s 6,000 elementary school teachers. It used the value-added measure, comparing individual pupils’ progress in test scores to evaluate what effect their teachers had on their learning.

The project’s impact was seismic. In Ruelas’ case, his family said that he had become deeply depressed by his poor rating. After news of his suicide emerged, thousands of teachers turned up at the newspaper’s office in downtown LA, calling for readers to boycott the paper and demanding that the ratings be removed from its website. The banners on display were emblazoned with angry messages including, “We are more than a test score”, “Demoralising teachers hurts students” and “LA Times, how do you help our kids?”

But in spite of the outrage among the teaching community, value-added teacher ratings have not gone away. The scores can still be viewed on the Los Angeles Times website; Ruelas’ poor rating in maths - and his “average” effectiveness rating for teaching English - can still be viewed, just like those of the thousands of other teachers in the city.

And it’s not just the press that has shown an interest in the data. Los Angeles Unified School District (LAUSD) now calculates its own teacher scores to evaluate the performance of individuals, and similar approaches are in use in many other school districts, such as Chicago and Columbia.

US secretary of education Arne Duncan has also come out in support of the scores, arguing: “The truth can be hard to swallow, but it can only make us better and stronger and smarter.”

Teacher scores have even made their way to the eastern seaboard. In February 2012, The New York Times published performance data for 18,000 elementary school teachers in the city.

But is rating individual teachers a genuine means of improving education through accountability? Should an employee’s appraisal be kept private and used purely for professional development, or can putting evaluations in the public domain be a real force for educational improvement?

If LA teachers were unhappy about being publicly named and shamed by the Los Angeles Times, they made sure that the two journalists behind the project - Jason Felch and Jason Song - knew what it felt like. “They were burning me and Jason in effigies,” Felch explains. “There were personal attacks on us. Jason Song got more of it because his name is easier to rhyme than Felch.”

The project began in 2009, when stories of underperformance in LA schools prompted the reporters to start looking into how teachers were being held to account by the school district.

“What we realised,” Felch says, “was that there was absolutely no measure of performance. For decades in LAUSD, teachers have essentially been given drive-by evaluations - very quick visits from a principal sitting in the classroom, checking off. Nationally it was the same picture. Teachers all around the country were receiving no feedback on their performance.”

Progress v achievement

While value-added (and contextual value-added) scores may have fallen out of favour in the UK, they have started to become more popular in the US over the past three years, and the approach piqued Felch’s interest.

“Schools were being called failing schools only because they had poor children,” he says. “Value-added was an effort to correct that by bringing in socio-economics, and bringing in growth rather than achievement level.”

By looking at how much progress a pupil makes over a set period of time rather than raw attainment, the theory goes, schools with low socio- economic catchment areas can be judged fairly alongside their neighbours in more affluent areas. The argument, staff at the Los Angeles Times soon realised, could be extended to teachers: they could mine the data to extract the impact of individual teachers in terms of how much value they added to pupils’ education.

After six months of haggling with the school district, the newspaper finally got hold of the figures it wanted by using freedom of information legislation. “No one had ever asked for the data before,” Felch says. “No one had even thought to ask. Even internally here, people were telling us, ‘You’ll never get that. They’ll never give you the data. Even if they do, you’ll never be able to analyse them.’”

But analyse the data they did. The Los Angeles Times hired Richard Buddin, an education policy expert at RAND Corporation, to do the number crunching, before checking his work with several other academics and its own in-house data experts.

But while critics of the project were quick to damn Felch and his colleagues as journalists out to take a cheap shot at the teaching profession, he insists that their motives were genuine.

“In the United States, our whole education system is a self-fulfilling social prophecy,” he explains. “Because of our accountability structure with testing, poor kids do poorly, rich kids do great on tests. That makes us think that the schools that these rich kids go to are great schools.

“[The system] is built on this ridiculous fallacy. Yet parents, teachers, the state, resources, all of it is geared towards this fallacy. We saw value-added (scores) as a way to cut through the socio-economics that are skewing the whole picture, and really shine a light among students, teachers and schools that’s not just a reflection of socio-economics.

“[This is done] by comparing students with their own prior behaviour. So if a student comes from an inner-city family - dad’s not in the picture, mom’s on drugs - the assumption of value-added is that that (scenario) is a relative constant in this kid’s life.”

Felch argues that by looking purely at the relative progress made by pupils in successive exams, it is possible to strip out extraneous social factors, meaning that pupils - and teachers - can be compared on a like- with-like basis.

Using this measure of pupils’ relative progress, the Los Angeles Times rated the city’s elementary teachers on their effectiveness in teaching English and maths in terms of how much value they added - ie, whether pupils progressed more rapidly than would have been expected, based on their prior performance. Each teacher was classed as “least effective”, “less effective”, “average”, “more effective” or “most effective”.

As well as posting details about the calculations on its website, the Los Angeles Times also gave teachers the chance to raise their concerns.

“A lot of teachers felt maligned by the data,” Felch says. “One of the things I’m proud of is that we took those complaints seriously and allowed teachers to point out mistakes in the data and things that were unfair.”

He estimates that about 80 per cent of complaints were from teachers who simply thought they deserved a better rating; the remaining 20 per cent were legitimate grievances about errors in the data. As a result, some teachers’ scores were removed.

Teachers hit back

The comments that teachers posted next to their ratings offer an insight into the massive impact the project had on teachers’ lives. Some teachers offer reasons for their low scores, such as retirement, maternity leave or the fact that they didn’t actually teach the classes concerned. Others take the opportunity to express their pain and anger.

Angelica Barraza, a third-grade teacher at Hooper Avenue Elementary, writes: “I’ve seen the disheartening effect of your scoring system on excellent teachers that I have had the privilege of working alongside… One teacher in particular comes to mind. He’s the type of teacher who is first in and works through recess and lunch. A good teacher who was made to feel that his efforts as an educator were meaningless based only on test scores. ‘What more can I do?’ he asked as he reviewed the ratings himself, trying to figure out what led to his poor showing.”

Winnetka Avenue Elementary teacher Lilia Alzate - classed as “least effective” in English and maths - admits that she has been seriously affected. “Your publishing (of) these test scores (has) kept teachers awake at night, including myself. Could it also be that some who have suffered a degree of emotional instability may not have survived your ratings?”

Although Stephanie Logan, who teaches at Seventy-Fifth Street Elementary School, is classed as a “more effective” maths teacher, she is described as one of LA’s “least effective” English teachers. “I feel like I’m being punished for being responsible and not saying ‘no’ when I was asked to take (difficult) students,” she writes. “I feel hurt and humiliated to be rated like this. Should I have refused to take those students in?”

Her colleague James Melin, classed as a “less effective” maths teacher, puts his point across more forcefully. “Listen,” he writes. “I teach in an area of south Los Angeles that most of your readers wouldn’t want to drive through. I work at a job that most of your readers wouldn’t dare undertake because I am so underpaid for what I do. I work for a district that has seen it fit to lay me off the past three years, only to rehire me at the very last second.

“Nobody who matters give a hoot about your rating…I will be receiving ‘thank you’ notes from many of my students when they are in college or are productive adults in society. At that time, my effectiveness as a teacher can be measured.”

Felch acknowledges that the ratings aren’t 100 per cent accurate. “Our confidence in these figures varies,” he says, “and these are not exact figures, they are estimates. That’s the best you can do with this. The data’s strongest at the two extremes. It’s a big bell curve, in the middle it gets squishy.”

When I ask Felch about the impact on teachers, he is unrepentant. “The kneejerk reaction is that this is evil and wrong, and is going to perpetuate all the inequalities,” he explains. “When you understand the goals of value-added, I would think teachers would be excited. For the first time in their careers, they have an opportunity to succeed even if they teach poor kids. Here’s a system that will level the playing field, and try to take out of the equation all the socio-economic stuff they feel that they are blamed for by society.”

But it is not just teachers who have expressed reservations about the scores. Two years ago, Derek Briggs and Ben Domingue of the National Education Policy Center analysed the Los Angeles Times’ ratings. They concluded that the newspaper’s research “was demonstrably inadequate to support the published rankings”. In its next set of teacher scores, the newspaper altered its methodology.

Speaking at the Education International (the global federation of teacher unions) conference in London in January, Lily Eskelsen, vice-president of the National Education Association, added her voice to the debate.

“I wouldn’t mind the ranking so much if it was just used on the sports page where it belongs,” she quipped, adding: “We’re making decisions around bad data… (The Los Angeles Times reporters) put a small disclaimer on their work saying that yes indeed, they know that what they’re about to tell you is not accurate, and then they use that disclaimer as permission to proceed with giving you bad information.”

Schools join in

But it is not just the newspaper that is now making use of the data. Although LAUSD was initially reluctant to put the data in the public domain, it has - perhaps surprisingly - now decided to put together its own value-added scores.

After first publishing value-added data - dubbed Academic Growth over Time (AGT) - at a school level, it started a pilot scheme on individual teachers. Crucially, the teacher-level ratings are not made public, but many - no doubt scarred by their experiences with the Los Angeles Times - are still less than impressed.

According to the AGT system, Brent Smiley, who teaches social sciences at Lawrence Middle School in Chatsworth, near LA, freely admits that he is “one of the ‘least effective’ teachers in the district”.

When I ask him why, he pauses for dramatic effect. “No matter what I do,” he finally answers, “I can’t get 103 per cent of my kids over the bar.” He bursts out laughing.

Smiley’s problem, he explains, is that the pupils at his school are too good. “The kids I teach are gifted and highly gifted, the school’s a magnet for them. And so last year I had 97.7 per cent of my students reach advanced or proficient. I was only able to go up about 1.5 per cent (from the year before).”

Compared with the set goal of a 6 percentage point increase, Smiley had - through no fault of his own - fallen short.

The relative nature of the accountability system has created perverse incentives for teachers, Smiley explains. “I would be best served personally to have my students tank the testing every other year. That would mean that one year I’d be the ‘most effective’, the next year I’d be the ‘least effective’, and I’d get ping-pong balled. That is not healthy for anyone, that’s not what I’m doing. I don’t care a damn about my test scores (but) I owe it to the kids to get them to be as proficient as possible.”

So how does Smiley play the system to achieve such good scores for his pupils? “I’ve figured out how to beat the test,” he admits. “It’s just a vocab test they take through social studies, that’s all it is. It’s a piece of cake. We spent five minutes a week on it, and we hit 97.7 per cent.”

But the irony for Smiley is that, having learned how to game the system, he can teach the way he wants to. “By figuring out how to beat their test, it freed me up to go about teaching the right way. But not everyone has the luxury of kids who are at the upper echelon.”

By trying to create a new accountability system that aims to ensure that teachers are teaching well, some teachers are paradoxically having to focus on getting a good score, rather than on providing a good, rounded education for their pupils.

“What value-added models are doing,” Smiley argues, “is trying to give a very simple answer to one of the most complex questions that there is. What they are really trying to do is define teaching as a science. It’s not, it’s an art.”

Data limitations

Attempting to offer a genuinely objective assessment is no mean feat. Professor Alan Smithers, director of the Centre for Education and Employment Research at the University of Buckingham in the UK, says that he is “uneasy” about teachers being evaluated publicly, not least because of the limitations of the data.

“If you are looking at pupils’ test results,” he says, “they depend on the pupils’ abilities, motivations and aspirations to study. Whether or not a child learns is ultimately down to them. (The pupils’ attainment) reflects a whole range of teachers they have had before, not just who they had at a particular age.

“I don’t think it will lead to good teaching. This approach will encourage teachers to develop a box-ticking mentality - teachers will play it safe. This approach would be absolutely terrifying for them, even if it were totally accurate. If you use the data in that way, it will have a massive impact on staff.”

On this issue, the LAUSD agrees. Despite further requests from the Los Angeles Times for the data that would allow it to update its teacher ratings once more, the district has steadfastly refused to release any details that would allow teachers to be identified by name. In December, the Los Angeles Times submitted a lawsuit to try to force the district to comply. The case has not yet been decided.

What has been decided, however, is a new approach to teacher evaluation in LA. Controversial moves to use teachers’ individual AGT scores in their formal evaluations have been watered down. In January, the LA teachers’ union, United Teachers Los Angeles, voted to go along with an agreement to base teacher evaluations on three factors: combination of raw test data, school performance and “robust classroom observation”. Although AGT scores won’t be directly used in evaluations, they can be referred to to provide “context” to a teacher’s performance.

This serves to illustrate the limitations of relying solely on test data. As the Los Angeles Times has already admitted, its scores “do not capture everything about a (teacher’s) performance”.

But the most poignant reminder of the limitations of teacher ratings comes from their most well-known victim, Rigoberto Ruelas. On the day of his funeral, LAUSD revealed that, in his final evaluation, he had scored the highest grade possible.

But more than two years after his tragic death, Ruelas’ name still appears on the Los Angeles Times’ website. Irrespective of the views of his colleagues and pupils, he remains one of Los Angeles’ “least effective” maths teachers.


Photo credit: AP

Subscribe to the magazine

4 average rating

Comment (24)

  • Unsuitable or offensive? Report this comment

    9 March, 2013


  • how sad, I feel so sorry for his family

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • Teachers now understand the pressure that we all put our children through.
    This does not make it right to measure people against each other.

    The surround factors that influence all our day to day are not shown and these would lead to a better more fairer system. But I guess that would cost far too much money to sort out.

    I feel really sorry for those great teachers who are marked as Least effective, though no fault of there own.
    How horrible it must be to be labeled in such a way.

    We pile on the pressure for both pupils & teachers alike, is this really the best way to get great results?

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • Unsuitable or offensive? Report this comment

    Rating: 4 out of 5 stars
    9 March, 2013


  • Journalists should be rated. Let people who know very little if anything about journalism construct a table using categories like honesty, objectivity, and accuracy.

    My prediction is that the bell curve would be very skewed, with the vast majority being least effective, and possibly one or two most effective.

    Of course, the results should be published on the front page.

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • As mentioned in the article, it is a matter od concern that by playing strategically, you can manipulate scores. This invalidates them. The people who are thinking tactically get promoted to positions of leadership, but those concentrating on the job of teaching feel undervalued. As I have witnessed, good teachers can then end up jaded and moving out of teaching.
    We experienced exactly this in our school:
    The children are tested each year. One excellent teacher achieved fantastic results with a particularly low achieving class. She worked incredibly hard for the whole year and managed to bring most of the group up by 4 or more sub-levels in maths. The scores were moderated down by 3 levels by the leadership, one of whom was going to be the next teacher for the class. When the children progressed to this next teacher, they miraculously achieved 4 sub-levels in one term.

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • In my experience, it's easy for a teacher to play the system. They learn the right things to do in an assessment, jump through the right hoops... and that's all they do. If they can't teach, haven't the subject knowledge, are not interested in honouring their students (nor themselves for that matter), then they can by lauded if they feed the right lines to management and the assessors whilst at the same time be criticised by students and other staff.

    This sort of thing is also great to hide all the sheer amount of corruption in education behind, which is rife.

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • “Value-added was an effort to correct that by bringing in socio-economics, and bringing in growth rather than achievement level. By looking at how much progress a pupil makes over a set period of time rather than raw attainment, the theory goes, schools with low socio-economic catchment areas can be judged fairly alongside their neighbours in more affluent areas."

    Because of course, socio-economic factors have no affect whatsoever on the amount of progress a pupil is able to make in class...

    “a pupils’ test results...depend on the pupils’ abilities, motivations and aspirations to study. Whether or not a child learns is ultimately down to them. (The pupils’ attainment) reflects a whole range of teachers they have had before, not just who they had at a particular age."

    Absolutely. Couldn't have put it better myself, which is precisely why this is such a ridiculous, painfully reductive and harmful way of attempting to quantify the quality and value of a teacher's teaching, which is of course about so much more than test scores. What an insult to all the hard-working teachers out there who are having all their efforts thrown back in their faces.

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • Why is one teacher getting the blame (or credit) for over 11 years worth of the child's education? They may only teach them for 1 year - they have no control over what they have experienced previously throughout school. That's like congratulating or blaming only year 6 teachers after SATS results - we are all in this together.

    Unsuitable or offensive? Report this comment

    9 March, 2013

    Jam Drop

  • The teaching profession is ruled by narcissists who are able to exploit the empathic vulnerability of their colleagues. The violence expressed in those of who turn to suicide and other forms of self destruct is nothing more than a counter transference to the management's need for approval and 'success'.
    I left teaching to become a psychotherapist. Need I say more?

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • Bean counters and politicians - try teaching....see if you last more than 10 minutes ...... :)

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • Time all teachers gave up - what is the point of trying if they are constantly attacked like this .

    Short sighted , smallm minded bean counters and politicians now rule education - they would not last 10 seconds in some of the schools that I have taught in!

    Unsuitable or offensive? Report this comment

    9 March, 2013


  • It's easier to look at the surface features - teachers' performance, children's progress etc. in terms of raw data. - but more difficult to analyse the underlying contexts affecting both students and teachers . These contexts are primarily socio- economic in nature and they affect students' outlook and perception of education; poverty, leading to low self-esteem; media glamour and the materialistic outlook on life in stark contrast to squalid living conditions of some; the effect this has on the idea of working assiduously to achieve some long-term goal rather than seek instant gratification.
    The erosion of basic family values result in students' difficulties in benefiting from a formal education in the way they should. The society's and media undue attention on stars and superstars and the get rich quick mentality - and some indeed do. Parents themselves lacking basic parenting skills and unprepared to raise children in a nurturing environment. Some still act like youngsters themselves and are less than role- models for their children. An 'effective teacher' might not necessarily produce the desired results for a whole host of reasons - some of which have been identified.
    The lack of respect for teachers is what motivates policies to dehumanize and belittle professionals. It has nothing to do with children's progress. Teachers under constant pressure will not perform better but will find ways of cheating as they do here in England. There is no-one more despicable than people who do not appreciate the hard work of those who go beyond the call of duty to make a difference in the lives of young people but would rather blame them covertly for the ills of society. We live in a sick world which appears to be spiralling downwards at an alarming rate. Allow schools to use their procedures for dealing with teacher performance; engage in consultation instead of denigration, and assist schools and teachers when necessary in improving teachers' effectiveness.

    Unsuitable or offensive? Report this comment

    10 March, 2013


  • "through no fault of 'there' own"?! What chance have students got of improving 'their' literacy if teachers themselves don't know the difference between basic homophones?

    Unsuitable or offensive? Report this comment

    10 March, 2013


  • One if the most difficult factors to cope with in analysing 'value added' is that children with low initial attainment will generally not progress as fast as children with moderate to high attainment. The evidence is already there in the prior attainment. Think about it: if a child's moved 9 sublevels in 6 years when you get them, you should expect about 1.5 sublevels if you do a sound job. If they moved 6 sublevels in the same period, why would you expect more than 1 sublevel?

    This is, to an extent, included in the UK value added figures if you look at them byindividual child. But as soon as you average them you run into trouble.

    It's quite possible that this is what affected the appalling case described in this story; maybe noone thought to compare that average added value with a realistic expectation for those particular students.

    But one thing you cannot fix: however you do it, someone's going to end up at the bottom of the league table _even if all of them are doing brilliant jobs_.

    Unsuitable or offensive? Report this comment

    10 March, 2013


  • The teacher is the most important feature of any classroom. We make a difference is more ways than can be measured by test results. I think that the best way to improve teaching and learning is to scrap regular testing (and league tables) and to allow our professional teachers to get on with their job in the best way for them and their students. Reporting on individual teachers is dangerous, unnecessary and highly unreliable.

    Unsuitable or offensive? Report this comment

    10 March, 2013


  • For someone clinging desperately to the dream of retirement in a few years, the whole scenario currently is a nightmare. I believe that my duty is to try and dissuade as many people as possible from entering or remaining in the profession, before it's too late to get out.

    Failing that, I pull out all the guns to ensure that I put the case for NEVER teaching in a school with challenging circumstances. It's career suicide for everyone and always the truly dedicated, enthusiastic ones that are tempted to apply. The dreadful situation of this guy is being replicated to a lesser extent in our schools now and I think it will only get worse.

    All the time our politicians (any party) continue to avoid using the state system, the answer is to set impossible targets and then blame everyone else for not attaining them. Pretending to give extra money to pupil premium, when you've robbed it from somewhere else in the budget is also another trick to justify it.

    For politicians, my own personal bugbear is failing to deal with the significant minority of appalling parenting. Infant children playing over 18 video games and children so tired in the morning that they can't lift their heads off the desk.

    Don't dismiss me as a cynic...I was one of those enthusiasts who, after 20 years in the profession, was still extolling its virtues. I even welcomed the SATS and the National Curriculum. Now, the die are so loaded against us it's ridiculous.

    Unsuitable or offensive? Report this comment

    10 March, 2013


  • I'm so glad I retire from teaching this year....

    Unsuitable or offensive? Report this comment

    10 March, 2013


  • A student failed to turn up to his GCSE, ruined my VA and predicted grades. Turns out his mother had died from an overdose. He stayed with her until the police and Social Care kicked the door in. It didn't make the news.

    Unsuitable or offensive? Report this comment

    11 March, 2013


  • I don't understand why the very people who mean well in our communities are being used as scapegoats for everything that is wrong. It simply has to stop.

    Unsuitable or offensive? Report this comment

    11 March, 2013


  • Shall we say human resources being treated as economic resources, replaceable as machine parts?! Implication for the future of education?

    Unsuitable or offensive? Report this comment

    11 March, 2013


  • Implication for the future of society?!?

    Unsuitable or offensive? Report this comment

    12 March, 2013


  • Unsuitable or offensive? Report this comment

    14 March, 2013


  • The effectiveness of a teacher CANNOT be measured until the end of life of the learner. My teachers in the primary school through to university -"effective" or not - got me to where I am today with a PhD. It was a partnership, a collaboration whose outcome could not, in my view, be tied to one particular teacher! It is a cheap, lazy and despicable way to score an equally cheap point. Those who favour the approach adopted in this story are not being true or honest with themselves. My students are my best judge; it is strange that their opinions of "customers" (to use a market term) rarely count in these fake measurements that masquerade as true representations of reality in the classroom!

    Unsuitable or offensive? Report this comment

    14 March, 2013


Add your comment

Subscribe to TES magazine
Join TES for free now

Join TES for free now

Four great reasons to join today...

1. Be part of the largest network of teachers in the world – over 2m members
2. Download over 600,000 free teaching resources
3. Get a personalized email of the most relevant resources for you delivered to your inbox.
4. Find out first about the latest jobs in education