Fifty years ago—on June 4, 1965—President Lyndon Johnson gave a remarkable address at Howard University that launched America’s experiment with affirmative-action policies in education and employment. The speech outlined the guiding metaphor that has long provided the moral basis for taking affirmative steps to address this nation’s legacy of egregious discrimination.

Following on the passage of the 1964 Civil Rights Act, Johnson's speech suggested it was not enough to outlaw discrimination prospectively. “You do not take a person who, for years, has been hobbled by chains and liberate him, bring him to up to the starting line of a race and then say, ‘You are free to compete with all the others,’ and still justly believe that you have been completely fair.” Failing to take action, he said, would simply freeze into place the effects of centuries of brutality toward black Americans.

The sprint metaphor and the call for compensation were not original to Johnson. In his 1964 book, Why We Can’t Wait, Martin Luther King, Jr. wrote, “It is obvious that if a man is entered at the starting line of a race three hundred years after another man, the first would have to perform some impossible feat in order to catch up with his fellow runner.”

But what was the appropriate remedy? The Urban League’s Whitney Young had called for “a decade of discrimination in favor of Negro youth.” But in his Howard University address, Johnson conspicuously avoided a call for racial preferences, as news articles at the time noted. The president articulated a different path: class-based programs that recognize would disproportionately benefit the victims of racial oppression but also acknowledge that racism is not the only source of unequal opportunity in American society. He called for a number of social-welfare programs—those focusing on jobs, housing, and healthcare—to support economically disadvantaged people of all races. Johnson staffer Daniel Patrick Moynihan, who coauthored the Howard University address with speechwriter Richard Goodwin, told me in a 1995 interview, “It always seemed to me that you would take care of this race problem in the context of the class problem.” In September 1965, Johnson issued Executive Order 11246 calling for federal contractors to take “affirmative action to ensure that applicants are employed, and that employees are treated during employment, without regard to their race, creed, color or national origin (emphasis added).”

In this, too, Johnson was following King’s example: In Why We Can’t Wait, King called for a “Bill of the Rights for the Disadvantaged” as the proper response to centuries of racial discrimination. Because of the effects of racism, blacks would disproportionately benefit from class-based efforts, but, King wrote, “It is a simple matter of justice that America, in dealing creatively with the task of raising the Negro from backwardness, should also be rescuing a large stratum of the forgotten white poor.”

King was also deeply concerned that racial-preference policies would divide America’s progressive coalition of working-class whites and blacks. In early 1964, he wrote to a freelance editor who was helping him write Why We Can’t Wait: “It is my opinion that many white workers whose economic condition is not too far removed from the economic condition of his black brother, will find it difficult to accept a ‘Negro Bill of Rights,’ which seeks special consideration to the Negro in the context of unemployment, joblessness, etc., and does not take into sufficient account of their plight (that of the white worker).”

After King’s 1968 assassination, however, racial rioting in urban areas helped spur the adoption of explicit racial-preference programs in employment, government contracting, and university admissions. Backlash against these programs was strong, particularly from working-class whites, just as King had predicted. Today, according to a survey of the Public Religion Research Institute, 60 percent of white working-class people say they think discrimination against whites is as big a problem as discrimination of blacks is, a finding that only makes sense if they consider affirmative action a form of discrimination.

Bayard Rustin, one of King’s lieutenants, described the problem in a 1987 article in The New Republic. “Any preferential approach postulated on racial, ethnic, religious, or sexual lines will only disrupt a multicultural society and lead to a backlash,” he wrote. By contrast, “special treatment can be provided to those who have been exploited or denied opportunities if solutions are predicated on class lines, precisely because all religious, ethnic, and racial groups have a depressed class who would benefit.”

Anger over racial preferences arose in the 1990s, and President Bill Clinton flirted with the idea of shifting the focus of affirmative action from race to class, suggesting, “I want to emphasize need-based programs where we can because they work better and have a bigger impact and generate broader support.” But civil-rights and women’s groups protested and Clinton backed off.

Opponents of affirmative action didn’t give up, however. Instead, they shifted their attention from legislation to court challenges and direct voter referenda, where in subsequent years they prevailed in repealing racial preferences in both blue and red states. Between 1996 and 2012, eight states, from California to Michigan and Nebraska, home to more than one-quarter of the U.S. population, banned racial preferences in admissions to public universities, often by voter referendum.

These states did not give up on diversity, but rather found new paths to achieve it. In a majority of the eight states states, universities instituted class-based admissions preferences, increased financial-aid budgets, and formed new partnerships with disadvantaged high schools. A number of universities, such as U.C. Berkeley and UCLA, dropped legacy preferences for the children of alumni, placed an increased emphasis on high-school class rank over test scores, and beefed up transfers from community colleges to four-year institutions. In a 2012 analysis of 10 leading public universities that created alternatives, my colleague Halley Potter and I found that seven were able to produce comparable or greater levels of African American and Latino representation without resorting to racial preferences. Of course, the black and Latino beneficiaries are much different under these programs—think the young Sonia Sotomayor or Michelle Obama, not Obama’s teenage daughters—which is, as a matter of basic fairness, as it should be.

Meanwhile, the courts have also been pushing universities away from racial preferences to alternatives like preferences for needy students. In 2013, the U.S. Supreme Court’s Fisher v. University of Texas decision placed on universities “the ultimate burden of demonstrating, before turning to racial classifications, that available, workable race-neutral alternatives do not suffice.” And new lawsuits challenging affirmative-action policies at Harvard University and the University of North Carolina are beginning to make their way through the courts. The Harvard suit emphasizes a new wrinkle in the affirmative-action wars: Asian Americans, who themselves have been victims of discrimination throughout U.S. history, appear to be particularly disadvantaged by racial-preference policies.

Unlike an issue such as gay marriage, where a generational shift has led to greater support, young Americans are less supportive of affirmative action than their parents were. In a 2014 MTV survey of 14-to-24-year-olds, roughly three-fourths of whites and two-thirds of people of color opposed racial preferences “regardless of historical inequities.” Similarly, in a 2012 Georgetown University survey of young adults ages 18 through 25, only one in five supported racial preferences to make up for past discrimination. By contrast, Americans of all ages enthusiastically support—by a two-to-one margin—preference programs based on income rather than race.

Still, critics ask: Is the country ready to move beyond racial preferences when skin color so patently continues to matter in American society? Is a nation in which the headlines are dominated by the disgraceful deaths of unarmed black men—from Eric Garner to Walter Scott to Freddie Gray—really in a position to get beyond race? Not if moving beyond race means repealing civil-rights laws. Explicit race-based laws still play a crucial role in fighting ongoing discrimination in housing, employment, education, voting, and the criminal-justice system, as will police body cameras and more resources for civil-rights enforcement.

Racial preferences in education are a different matter. Factoring in skin color at select private colleges is worth the equivalent of a 310 SAT-point boost (on a 400-1600-point scale) for black students over white students, according to one study. As Johnson, King, and Moynihan recognized, class-based alternatives to these programs don’t ignore the continuing role of race; they leverage the reality of discrimination to promote the kind of racial and ethnic diversity that America’s colleges need. Take the “Texas Top 10 percent” plan, under which the graduates of every high school with the highest grades are automatically eligible for admissions to UT Austin. In 2004, that plan produced as much African American and Latino representation as the use of racial preferences previously had, precisely because racial segregation among Texas high schools remains a pervasive problem.

The Harvard University professor Robert Putnam’s new book, Our Kids, reports that interracial marriage is rising while cross-class marriage is falling. Single-parent family status is increasingly a class rather than a racial phenomenon. The academic achievement gap by race used to be twice as large as the income gap was; now, precisely the opposite is true. In a recent panel discussion with Putnam, Barack Obama noted that racial segregation, which is slowly declining, is being replaced by rising “class segregation.”

The attacks on racial affirmative action, including those from voters and from conservative judges, may represent an opportunity to pick up the progressive thread of thought first developed a half century ago by Johnson, King, Moynihan, and Rustin, filling a need for economic affirmative action that has only grown stronger over time.