I raised Levine's point about site visits to Walsh in today's press call, and she was blunt: "This is where my temper flares a little bit," Walsh said of the criticism. She's open to hearing ideas of how NCTQ could make more than 1,100 site visits and have time for more than just sitting in on a lecture of two -- which might not even yield much information about a program's quality.
Leaving deeper evaluations up to accreditation authorities or state agencies doesn't necessarily mean low-quality programs will be identified and forced to improve. In 2011, the U.S. Secretary of Education Arne Duncan said that in the prior 12 years over half the nation's states had not rated a single teacher preparation program as inferior, a statistic he called "laughable." Those remarks came as part of the Obama administration's attempt to turn up the heat on teacher prep, including calling for a $185-million plan to push states to improve training programs, provide incentives to shut down inferior ones, and make state licensing exams more challenging.
Just how uncooperative were some ed schools in NCTQ's requests for data? Walsh said the organization had to hire attorneys in nine states. One of those cases goes to trial later this week -- the University of Missouri contends its syllabi are intellectual property. That's the argument the Minnesota State Colleges and Universities system made -- and lost -- in court last winter, Walsh said.
To Cory Koedel, a researcher at National Center for Analysis of Longitudinal Data in Education Research, the criticism of NCTQ's rankings is an example of "the perfect being the enemy of the good." There's no question the NCTQ report breaks new ground, Koedel said, and the fact that people might want them to have done more "shouldn't take away from all of the progress they've made." He noted that NCTQ went to significant lengths to find alternate means of collecting data when a school declined to provide it outright.
"I sure hope that the pushback about incomplete data isn't coming from the same people and programs who are withholding the data in the first place," said Koedel, an assistant professor in in the economics department at the University of Missouri.
One ed school which did provide full cooperation was the University of Michigan in Ann Arbor. Dean Deborah Lowenberg Ball told me that "we didn't think this was a particularly useful way to learn about teacher preparation, but we didn't have any reason to withhold the information."
The University of Michigan earned 2.5 stars on the NCTQ rankings, but Ball said she's not particularly concerned about that. She's more interested in whether the conversation will move beyond the controversy and into a meaningful dialogue about "what good teacher preparation really requires." An independent, in-depth review of the nation's ed schools hasn't yet been attempted and would be welcomed, Ball said. But that would mean moving beyond rating a school on its syllabi and focusing on how teachers perform in the classroom during the crucial first few years in the profession, and then finding the best way to support them, Ball said.
"If this report can get people to care about that, I'll be happy," Ball said. "If all it does is get people to point fingers ... that won't advance the conversation at all."
This post also appears at The Educated Reporter, an Atlantic partner site.