A Georgia judge just finalized prison sentences for eight Atlanta educators, the latest development in a drama that has transfixed the city since 2011. That year, state investigators found that almost 180 teachers and principals had illegally tampered with scores on state tests, both by telling students the right answers and by secretly correcting the work students handed in.
Once held up as a model of urban school reform, Atlanta now gets cited as an example of everything wrong with test-based accountability. Under intense pressure to meet impossible targets, and the guidance of a superintendent who fostered what investigators called "a culture of fear," educators cracked—or so the story goes.
But Atlanta students, who are overwhelmingly African-American and low-income, may have actually made genuine progress over the decade when the cheating took place. "It is possible that if they did cheat, they didn't have to cheat, because they were really were improving," says Peggy Carr, acting commissioner for the National Center for Education Statistics.
Since 2002, Atlanta test scores have steadily risen on a federal test that's low-stakes and hard to game. It's called the National Assessment of Educational Progress (NAEP), and it's been used since the 1970s to track national trends in math and reading mastery. "It's sort of an outside indicator, almost a second opinion," says Carr, whose office administers the test.
During the period in which the cheating took place—it could date back to 2001—former Superintendent Beverly Hall pursued all manner of school-reform strategies. She expanded charter schools, broke large high schools into small, themed schools, and raised millions of philanthropic dollars to support underresourced schools.
And Hall set up an aggressive accountability system that revolved around state test scores. Her administration set targets for every school in the district, doled out bonuses when targets were met, and humiliated and intimidated educators when targets were not met. Hall later told investigators that she had to set high targets due to the federal No Child Left Behind law. (Hall was among the educators indicted in connection with the cheating scandal, but she died in March 2015 of complications from breast cancer.)
Hall's administration asked schools to reach a higher target each year and would not accept failure. Slowly but surely, school leaders started to encourage cheating. "Teachers who conducted themselves ethically but failed to achieve required results were sanctioned," the state investigation found.
Early in her tenure, Hall also signed up Atlanta to participate in a new NAEP test called the Trial Urban District Assessment. She later told investigators that she wanted external validation of the district's progress. "We wanted an independent third party," she said—an assessment that the district did not administer, grade, or proctor, and that students were randomly assigned to take.
The NAEP data show mixed success. Atlanta fourth- and eighth-graders still score below the national average in math and reading. The achievement gap between black and white students remains as wide today as it was a decade ago. And score growth seems to have slowed in recent years.
But scores have been increasing in both subjects, and for both grades. Since 2003, the share of Atlanta fourth-graders scoring "basic" or higher on the NAEP reading exam rose from 37 to 57 percent. In math, the share rose from 50 to 72 percent.
When the cheating scandal broke, Hall and her backers pointed to the NAEP data as evidence that Atlanta had been doing something right. But the scandal didn't just discredit the city's state test scores; it cast doubt on students' improvements on the federal test, too.
Could Atlanta have cheated on NAEP, as PolitiFact Georgia has asked? Schools could conceivably have given test administrators a list of students that didn't include low-performers. Carr says her office never noticed any strange shifts that would indicate that kind of manipulation was going on.
Changing demographics might also have affected scores. From 2000 to 2014, the share of white students in the district rose from 6 to 15 percent, and the share of Hispanic students rose from 2 to 7 percent. But those shifts don't explain the gains made by low-income and African-American children. African-American eighth-graders, for example, achieved a 21-point gain in math scores between 2003 and 2011.
"The bottom line in this entire discussion is that for the most part the gains for Atlanta schools from 2003 to 2011 are for real!!!!" former Education Department official Marshall Smith wrote in a blog post for the Atlanta Journal Constitution.
The NAEP data may prove that Atlanta students know more math and reading than they did a decade ago, but they don't prove that Hall was doing something right. The tests aren't designed to reveal why students are learning; they're just designed to measure what students know. It's possible that one of Hall's reforms spurred the test-score gains, or that the gains occurred in spite of her administration.
"It's reasonable to believe that the kids were still learning, and that those [teachers] who weren't cheating were also doing their best teaching," says Gregory Cizek, a professor at the University of North Carolina-Chapel Hill School of Education. "So I wouldn't be surprised at all if kids were improving. It's just that some of the gains were not legitimate."
Next America's Education coverage is made possible in part by a grant from the New Venture Fund.