After all, "Rapper's Delight," by Sugar Hill Gang, the first radio hit that contemporaries considered hip-hop, was released 12 years prior, in 1979. Rick Rubin founded Def Jam Records in 1982. Run-D.M.C. released King of Rock and performed in Live Aid in 1985, and the Beastie Boys came out with Licensed to Ill in late 1986. The first music show dedicated to rap, the perfectly-named "Yo! MTV Raps," debuted in 1989, and some historians look back on the program as the baptism of hip-hop as a mainstream genre. "Hip-hop did not just appear out of the blue [in 1991]," said Matthias Mauch, a co-author of the paper. "We see a rising trend towards less chord-orientated sound a few years earlier."
But for chart-watchers and Top 40 radio listeners, rap clung to the fringes until the early 1990s. It might have gotten its critical boost from an unlikely ally: Billboard's statistical method. (I promise this is an interesting story about methodology, perhaps the only one I know.)
In the middle of the 20th century, Billboard started publishing the Hot 100, which lists the most popular songs in the country, and the Billboard 200, which does the same for albums. For decades, these lists were put together using methods that would offend even the most careless statistician, as I explained in "The Shazam Effect." Billboard had few ways to truly measure what albums were being sold in stores or played on the radio. Instead, they relied on an honor system, by asking record stores and DJs to self-report the most popular musicians of the moment. Both parties had reasons to lie, and not just because the labels would pressure radio stations and record stores to play the hand-picked hits.
Imagine, for example, that you're a record-store owner in the 1980s. You’ve just sold out of AC/DC, but you're still stocked with Bruce Springsteen albums. The honest thing to do would be to tell Billboard that AC/DC is the band of the moment. But the smart thing to do would be to tell Billboard that Bruce Springsteen is selling like crazy. That way, the Boss's album flies up the chart, which encourages fans of popular music to come back to the store and empty the shelves still holding his records.
So, for many years, Billboard wasn't a perfect mirror of American tastes. It was warped by label preferences and record-store inventories. It often over-counted songs that labels preferred (like rock) and under-counted genres they were indifferent toward (like country and rap).
But in 1991, this changed. First, Nielsen ended the record-store charade by releasing SoundScan, which used point-of-sales data from cash registers in stores. Finally, Nielsen had timely information on which albums were really selling. Around the same time, Billboard switched from trusting radio stations’ self-reports to monitoring airplay through a third party. The Hot 100 chart changed from a political document to a statistical register, honestly tracking the songs Americans were really buying and listening to.
Within a few weeks, N.W.A. replaced R.E.M. on the charts. The swapping of acronyms—out with soft rock, in with hip-hop—was a harbinger. In the early 1990s, the "hip-hop/rap" genre exploded to become, by far, the most common genre of music on the Billboard Hot 100 charts for two decades (see the graphs below). These methodology changes “helped new artists, particularly in hip-hop,” said Silvio Pietroluongo, director of charts at Billboard.
When British researchers studied 30-second segments of 17,000 songs in the Billboard Hot 100 between 1960 and 2010, they looked at features like chord-changes and "timbral" sounds, like drums and voices. They used this analysis to determine how certain styles and genres go into and out of fashion. For example, they discovered that the frequency of dominant-seventh chords common in jazz and the blues declined by about 75% between 1960 and 2009, while the frequency of hit songs without prominent chord structures, as in much of rap, tripled between the late 1980s and the mid-1990s. (See the graph to the right.)