Though consumers have not been able to glimpse the actual algorithms for setting scores, some basic contours of credit scoring are intelligible: Don’t run up too much debt, and don’t be late on payments. But by 2009, financiers were scrutinizing more data, in ways completely opaque to scored consumers. Buy “little felt pads that stop chair legs from scratching the floor”? You might be rewarded with a higher credit line, or lower interest-rate offers. Purchase a beer at a billiards bar? Expect the opposite.
America’s obsession with scoring has gone far beyond credit. As Pam Dixon and Bob Gellman documented in their World Privacy Forum report “The Scoring of America,” thousands of scores exist, ranging from measurements of one’s reliability as an employee to one’s “medication adherence” as a patient. Most of these scores aren’t publicly available, and it’s obvious why: Most people don’t appreciate being scored by strangers, and might protest if they knew these secret evaluations were going on. The recent backlash against Peeple, an app aspiring to be the “Yelp for People,” showed just how strong the reaction could be once a feedback system got a critical mass of attention.
Yet the trend toward scoring continues, and in China, it has gone into overdrive. Jay Stanley, a senior policy analyst at the ACLU’s Speech, Privacy & Technology Project, has summarized a series of disturbing news stories on China’s “Planning Outline for the Construction of a Social Credit System.” As Stanley observes, “Among the things that will hurt a citizen’s score are posting political opinions without prior permission, or posting information that the regime does not like.” The scoring system will also penalize or reward individuals for what their friends post. A plan for basing individuals’ credit on the records of their friends, so far only conceptualized by Facebook, appears set for widespread implementation abroad.
The Chinese scoring systems are projected to probe well beyond political orthodoxy, too. According to one Chinese firm’s press release, its “Sesame Credit” service is “conducting a test program with a Chinese online dating site that allows suitors to check their potential dates’ credit ratings to make sure they are not meeting someone who is dishonest or untrustworthy.” Given the recent finding that “the higher your credit score, the higher your chances of a lasting relationship,” it’s easy to imagine a similar service influencing the black-boxed algorithms of American dating sites. Or it could simply be integrated into a new iteration of Google Glass, replacing 20th-century “beer goggles” with the pristine clarity of “credit score microscopes.”
Back in 2004, Daniel Solove, a professor at the George Washington University Law School, presciently observed that, “It is ever more possible to create an electronic collage that covers much of a person’s life—a life captured in records, a digital biography composed in the collective computer networks of the world.” There may be scenarios where that information could be put to good use, informing health professionals or quantified-self enthusiasts. But once the competitive dynamism of scoring enters the data mix, those positive uses become comparatively less important, and perhaps more distant. Spurious comparisons abound, correlation becomes mixed up with causation, and self-fulfilling prophecies are inevitable.