What Makes for an Effective Teacher?

Reflections on a controversial Los Angeles Times series that rates student achievement

by Oliver Wang

The Los Angeles Times decided to set off a powder keg this weekend.

Here's the skinny:

The Times is running a series of stories on evaluating teacher "effectiveness" in the LAUSD. Their method of evaluation is based on a "value-added" model that essentially seeks to quantify how well students improve (or decline) after a year in the classroom of particular teachers. The LAT was able to use seven years of standardized test score data to look at how students performed with different LAUSD teachers and combined with other metrics (but the testing is a huge part of it), they:

converted the scores to percentile rankings and divided them into five equal categories, from "least effective" to "most effective." Later this month, The Times will post this data on its website. Readers will be able to look up individual teachers and schools and see their value-added ratings.

This story is already a clusterfuck of epic proportions and once that data actually gets posted - listing teachers by name—it's going to balloon to, uh, an even more epic clusterfuck.

Predictably, the teacher's union is going nuts, as well they should from a self-preservation p.o.v. This is the latest—and perhaps the most potentially damaging—attack on the credibility of their members, in a hostile, anti-teacher climate where "educational reform" has become a new mantra (but few seem to have great ideas on what that actual means).

The moment you introduce a method by which teachers can be ranked in simple order and in the short term, I can guarantee you, principals and other admins are going to get swamped by parents who will be trying to jockey to get their kids into the classrooms of the "most effective" teachers. We're talking stampede.[1] This will also undoubtedly fan the flames for those arguing for merit-based hiring/firing and/or pay decisions. The district (or better said, the union) has been the subject of several unflattering news stories that point out how incredibly difficult it seems to be to get rid of unwanted teachers. Now that parents and politicians will have a chart to point to[2], there's going to be even more calls for the district and union to dump the chaff and rethink its hiring and training procedures. Again.

Couple of things just to get out of the way. First, the district takes a lot of hits for being unwieldy, mismanaged, bloated, inconsistent, etc. and all of that is probably true. The power plays between the mayor's office, the district board, and other civic leaders is like a French period drama without all the wigs. All this is happening against a backdrop of nearly 700,000 students and over 80,000 employees, in a post-Prop 13 budgetary environment. In short, there's no easy answers here at all. I'm not saying the district deserves a pass but let's just say that running the LAUSD efficiently and effectively probably isn't as easy as running Bell into the ground.

Second of all, before some fruit fly in the comments tries to argue that the pink elephant in the room is illegal immigration, I think that's a whole 'nother conversation, and one that requires more discussion than just indiscriminate scapegoating that fails to take into account A) budget cuts to public school education over the last few decades in CA, especially after Prop 13 rejiggered property tax (a heavy base for school funding) and the ability to raise taxes to help fund education (and everything else in the state), B) the social mission of public education for all students, regardless of their (or their parents') immigration status, and C) how much of the district's woes are due solely to the size of its student population. It may be a conversation worth having but it's separate from this issue.

Back on topic: to me, the real issue here is a core problem that exists throughout the educational system, not just K-12 but certainly in college too:

Should teachers be evaluated for their teaching? Yes.
How? Umm....

In other words, I think almost anyone would agree that teachers—especially tasked with such an important job as they have—absolutely need to be given feedback and be evaluated for their effectiveness as teachers. I can't imagine anyone suggesting that acquiring a teaching credential means you're suddenly not subject to the same kind of on-the-job scrutiny as any employee, anywhere would be.

But coming up with a reasonable and rationale set of criteria to accomplish said evaluation is fraught with challenges.

The Times is using—as one of their baselines (albeit not their only one) —standardized testing of the students. Test scores go up after a year, in theory, that's meant to reflect "teaching effectiveness." But first of all, how can you devise a test whose results really speak to teaching effectiveness? And second, if teachers teach to the test, that may improve their standing on a numerical scale but is it improving the quality of education?

People like (I think "tolerate" may be a better word) standardized testing in schools because they like how it produces a simple score at the end of it. If you can quantify your problem, "fixing" it becomes a matter of finding ways to raise or lower that score. But as people on both sides of the teaching line will tell you, teaching effectiveness isn't like your credit rating. So much of what goes into making a good teacher is intangible and unquantifiable. The best teacher in my life was my high school Latin teacher. My Latin is terrible (then and now) but he was inspirational and nurturing in ways that transcended whatever could be measured on an AP test or my report card. I would wager that many, if not most, people had teachers they think of that way.

But hey, before we start standing on our desks, yelling "Oh captain, my captain" let's also note that you can't improve education simply by identifying the "special" teachers. That's not ensuring equal access to quality education for all students. Nor is it likely that one can quantify their qualities into a means through which students end up getting higher test scores (if that's the main metric you're using to evaluate effectiveness).

I wish I had some easy answer to any of this but I don't. I don't know how we're supposed to find a way to evaluate teachers that don't rely *solely* on:

  • Standardized tests that conflate student performance with teaching effectiveness and whose disproportionate impact may end up detracting from classroom education.

  • Student evaluations that may be based on criteria that also has little to do with effectiveness (in my experience, evaluations often measure the ease of a class rather than its educational content).

  • Peer reviews (which I think are actually highly useful) that could be too easily corrupted in schools rank with favoritism and power struggles.[3]

    I suspect what we'll have to work towards will be some awkward combination of all of the above (and more) that will be worked out through very painful trial-and-errors. I'm not a scholar of education by training so if any of you know of what's worked in other (though almost certainly to be smaller) districts, please share.

    I can't really condone what the Times is doing because I don't see what the short-term benefit of this is for anyone: not students, parents, admins and certainly not teachers. However, I'm also a proponent of the idea that data is neutral—it's what we do with it that's meaningful—so I can't also argue, "the Times should have sat on this data."[2] I prefer sunshine, even when it burns.

    And it could be that, in the long term, blowing things up is partially how you can rebuild a better system (but lord, does that sound Polyanna-ish).

    (I have more to say about this research but I think it'd work better in a Part 2 post).

    [1] Navigating the LAUSD's maze of schools, charters, magnets and honors programs is practically a full-time job for some parents. They're constantly pouring over public test scores, parent testimonials, graduation rates, etc. to figure out what schools to send their kids to, what teachers to request, and what the best Google map routes will get them to what is usually going to be different schools in different neighborhoods as their kids matriculate. It's byzantine but the result can be a really incredible education. However, it requires a level of resources, focus, research—and luck—that few parents are necessarily willing or able to pull together.

    Not everyone is up for it, like me and my wife. We moved to a city that has excellent K-12 schools (and is surprisingly diverse, with a non-majority student population). What is very telling is that one of my neighbors actually works for LAUSD but he moved to our town so his kids could attend the school district here. You do the math.

    Of course, the median rental and housing costs in our town is probably a lot higher than it is to live in parts of LA that fall under the LAUSD's purview. We basically "bought" our way to a more elegant solution but it's obviously not a solution of equal access.

    [2] What's interesting to me here is that this Times story really highlights the seduction of data, especially data that can be visually represented by a graph or chart. I'm not immune to this. By training, I'm actually much more of a qualitative researcher, but I do like quantified data that's organized in such a way where comparisons between X and Y can be laid out in a series of numbers of percentages. Of course, like other seductions, just because it looks/feels good, doesn't mean it's actually good for you (or society). As I note above, I do think data is neutral (generally) but because quantifiable data is so seductive, it leads to particular applications that often emphasize only the quantified portions (the clean, numeric score) but minimize the qualitative (the messy tangle of factors what went into creating that score).

    [3] It occurs to me that the fact that each evaluation method is open to corruption or misuse just reveals a certain level of cynicism (on my part, though I suspect, for many others) when it comes to schooling. It's a very Murphy's Law attitude: whatever will go wrong, will inevitably do so and I do wonder if this pessimism has been conditioned by experience, impression, or a combination thereof. America's schools were once (and I think, still are) the envy of the rest of the world yet why does it seem like the prevailing attitude is that it's all gone to hell in Pee Chee folder?