A Georgia judge just finalized prison sentences for eight Atlanta educators, the latest development in a drama that has transfixed the city since 2011. That year, state investigators found that nearly 180 teachers and principals had illegally tampered with scores on state tests, both by telling students the right answers and by secretly correcting the work students handed in.
Once held up as a model of urban school reform, Atlanta now gets cited as an example of everything wrong with test-based accountability. Under intense pressure to meet impossible targets, as well as the guidance of a superintendent who fostered what investigators called "a culture of fear," educators cracked—or so the story goes.
But Atlanta students, who are overwhelmingly African American and low-income, may have actually made genuine academic progress over the decade when the cheating took place. "It is possible that if they did cheat, they didn't have to cheat because they were really were improving," said Peggy Carr, the acting commissioner of the National Center for Education Statistics.
Since 2002, test scores in Atlanta have steadily risen on a federal test that doesn't have a lot of stakes and is hard to game. It's called the National Assessment of Educational Progress, or NAEP, and it's been used since the 1970s to track data on national trends in math and reading proficiency. "It's sort of an outside indicator, almost a second opinion," said Carr, whose office administers the test.
During the period in which the cheating took place—which could date back to 2001—Beverly Hall, who was superintendent at the time, implemented an array of school-reform strategies. She expanded charter schools, broke large high schools into small, themed academies, and raised millions of philanthropic dollars to support schools lacking in resources.
Slowly but surely, school leaders started to encourage cheating. "Teachers who conducted themselves ethically but failed to achieve required results were sanctioned," the state investigation found in 2011.
Early in her tenure, Hall also signed up Atlanta to pilot a new NAEP test designed for urban school districts. She later told investigators that she wanted external validation of the district's progress. "We wanted an independent third party," she said—an assessment that the district did not administer, grade, or proctor, and that students were randomly assigned to take.
The NAEP data shows mixed success. Atlanta fourth- and eighth-graders still score below the national average in math and reading. The achievement gap between black and white students remains as wide today as it was a decade ago. And score growth seems to have slowed in recent years.
But scores have been increasing in both subjects, and for both grades. Since 2003, the share of Atlanta fourth-graders scoring "basic" or higher on the reading exam rose from 37 to 57 percent. In math, the share rose from 50 to 72 percent.
When the cheating scandal broke, Hall and her backers pointed to the NAEP data as evidence that Atlanta had been doing something right. But the scandal didn't just discredit the city's state test scores; it cast doubt on students' improvements on the federal test, too.
Could Atlanta have cheated on the NAEP, as PolitiFact Georgia asked back in 2011? Schools could conceivably have given test administrators a list of students that didn't include low-performers. Carr says her office never noticed any strange shifts that would indicate that kind of manipulation was going on.
Changing demographics might also have affected scores. From 2000 to 2014, the share of white students in the district rose from 6 to 15 percent, while that of Hispanic students rose from 2 to 7 percent. But those shifts don't seem to explain the gains made by low-income and African American children. African American eighth-graders, for example, achieved a 21-point gain in math scores between 2003 and 2011.
"The bottom line in this entire discussion is that for the most part the gains for Atlanta schools from 2003 to 2011 are for real!!!!" Marshall Smith, a former Education Department official, wrote in a 2013 blog post for the Atlanta Journal Constitution.
The NAEP data may prove that Atlanta students are more proficient in math and reading than they were a decade ago, but they don't mean that Hall was doing something right. The tests aren't designed to reveal why students are learning; they're just designed to measure what students know. It's possible that one of Hall's reforms spurred the test score gains, or that the gains occurred in spite of her administration.
"It's reasonable to believe that the kids were still learning, and that those [teachers] who weren't cheating were also doing their best teaching," said Gregory Cizek, a professor at the University of North Carolina-Chapel Hill's School of Education. "So I wouldn't be surprised at all if kids were improving. It's just that some of the gains were not legitimate."