Since its launch in 1983, the U.S. News and World Report’s annual college rankings have sought to compare institutions using a series of quantifiable metrics, including acceptance rates and alumni donations, that have increasingly come under scrutiny. In 2013, President Obama argued that the rankings actually incentivize colleges to “game the numbers and in some cases, [get rewarded] for raising costs,” encouraging schools to invest extra money in activities such as alumni outreach and in turn theoretically raise tuition. Yet, according to Obama, colleges motivated by these grading systems, largely continued to neglect one key measure: student outcomes. Since then, he’s pledged to change the way colleges are ranked by shifting the focus from institutional prestige to students’ actual academic experience.

Earlier this September, the president unveiled his much-anticipated “College Scorecard” as part of that pledge. The new tool, however, isn’t exactly a ranking system. Instead, it gives users access to extensive federal data on the student-debt and attendance-cost data for more than 7,000 U.S. higher-ed institutions allowing them to compare institutions; students can use the Scorecard to list and sort schools based on the institutions’ on-time graduation rates, school size, and salary after attending, among other factors. The new system—which is based on nearly two years of federal data—doesn’t translate these numbers into ratings.

Thus, as opposed to the ambiguous and often inconsistent definitions and grading categories used by popular ranking systems such as those from U.S. News or Princeton Review, the College Scorecard, backers say, emphasizes each college’s discrete value—or, as the U.S. Department of Education puts it, getting the most “bang for your educational buck.” Proponents say it makes students’ academic and employment outcomes more transparent for prospective consumers, allowing prospective students and their families to spend their money efficiently and set aside adequate financial support for their children’s postgraduate studies

“Everyone should be able to find clear, reliable, open data on college affordability and value—like whether they’re likely to graduate, find good jobs, and pay off their loans,” Obama said in a speech earlier this month. The tool’s interface, he added, was developed in collaboration with teachers, students, and parents.

Colleges fare differently on the Scorecard depending on the metric. Institutions such as State University of New York Downstate Medical Center, Kentucky’s Berea College, and California’s Samuel Merritt University top lists when considering students’ post-graduation earnings and average annual cost, while Ivy Leagues such as Harvard, Princeton, and Yale perform well when sorted by graduation rate. The tool also provides comprehensive profile pages for each college or university, detailing the percentage of its students paying down debt and the typical total debt per student, on top of information ranging from demographic breakdowns to lists of popular academic programs. Advocates are touting the launch of the College Scorecard as key to filling in long-standing gaps in data about college institutions.

Still, as utilitarian as the tool sounds, the lack of rating systems for distinct categories means it’s tricky to compare the institutions head to head. While one college might report that its graduates earn especially high salaries, for example, its graduation rate could be substantially lower than another that offers relatively lower returns. And though the College Scorecard certainly garnered substantial public support, a number of college administrators bashed the Obama initiative as misleading and oversimplified; it discounted gains, they said, that weren’t monetary in nature.

Regardless of critics’ motives, their protests evoke larger discussions about the difficulties of assessing higher-ed quality. It’s impossible to capture an institution’s value, the college experience, and its impact on students with a single metric. And perhaps therein lay the danger in the perpetual dominance of the U.S. News rankings, which were (and continue to be) taken by many, as some kind of gospel.

In response to the success of U.S. News, as well as the backlash against it, there has been a massive proliferation of new approaches to college rankings and scorecards. The Princeton Review compiles rankings that integrate student survey responses—with categories like “Best Classroom Experience” and less serious ones such as “Best Stone-Cold Sober Schools.” Money created its own barometer that examines the quality, affordability, and outcomes at different institutions that are seen as having the top return on investment. Washington Monthly’s includes a Peace Corps Rank, a ROTC rank, and a Bachelor’s to Ph.D. rank. ProPublica released a ranking system last week that looked at how colleges were supporting their poorest students, scrutinizing the proportion of low-income students at different schools using Pell Grants and the average discount they’ve received. And perhaps, most momentously, The Chronicle of Higher Education has developed an interactive, which allows users to “Make Your Own College Rankings,” based on variables like “Fat Paychecks,” and “Prestige,” values culled from other publications’ classifications.

The College Scorecard is the federal government’s foray into the mix. “The status quo serves some colleges and the companies that rank them just fine,” said Obama, “But it doesn't serve our students well—and that doesn't serve any of us well.”

It ultimately helps add more dimension to a potential answer for the thorny question around the “value” of an education at different colleges. Its data, reviewed in conjunction with that of existing ranking systems builds a more holistic picture of what different colleges and universities have to offer—providing greater guidance on one of the toughest, often most costly, and hopefully, most rewarding decisions many students need to make.