Revisiting 'Dark Social'

In the data-rich world of Internet journalism, every publisher wants to understand where the audience is coming from. At The Atlantic, our analytics team prepares a variety of charts and graphs that gives us a good sense of the different sources of traffic to our websites.

Take this rainbow of lines that captures the incoming audience to TheAtlantic.com in a recent month. The blue line tracks users who type in our Web address or bookmark our site directly; the purple line follows those who arrived via search; the red line reports links from other sites; the green line monitors referrals from social media.

That seems clear enough. But it turns out we (and no doubt other publishers) are only getting part of the picture. What's missing in this data--or, to be more accurate, what's captured in this data but not broken out in any useful way--is traffic that comes from sharing but is not generated by the familiar pillars of the so-called sharing economy: Facebook, Twitter, Reddit, LinkedIn, Digg, StumbleUpon.

The stories that fall between the cracks are those that are passed around in casual ways, outside the social media super-structure, millions of times a day. "Most of the time," writes Alexis Madrigal, "someone Gchatted someone a link, or it came in on a big email distribution list, or your dad sent it to you."

Alexis, who is the tech editor at TheAtlantic.com, gave a name to this black hole of traffic: dark social. These sorts of referrals are not broken out in the chart above. But they represent a large part of the audiences we all receive. At The Atlantic, our director of analytics, Adam Felder, estimates that dark social accounts for about one-quarter of all referrals to our sites.

So where's it coming from? Since those four lines in the graph add up to all of our traffic, readers arriving under cover of dark social are stealing market share from one of the other categories. Specifically, dark social is coming out of typed/bookmarked category, which is the catch-all bucket for sources of traffic that can't really be traced. Adjusting for that fact suggests the true power of user sharing: nearly half of The Atlantic's traffic, as an example, is coming through the combination of traditional and dark social.

None of this surprised Alexis. Growing up in rural Washington, he spent a lot of time in the pre-Twitter social world: bulletin boards, ICQ, and other virtual hangouts of tech-minded mid-90s adolescents. So more than a decade later, he wasn't buying the idea that social networks had somehow birthed a new social Web. Sharing was sharing; the tools were just getting better.

Then Alexis saw how Chartbeat was dividing visitors who showed up without referrer data into two categories. The first group was people who were going to a home page or landing page. The second was people going to any other page. This second set, Chartbeat figured, were users following a link, because nobody actually types in http://www.theatlantic.com/technology/archive/2013/05/what-the-man-who-first-summited-everest-thought-of-the-first-american-to-orbit-earth/276305/, for example. Chartbeat called them direct social. Madrigal was ecstatic. "They'd found a way to quantify dark social, even if they'd given it a lamer name!"

So why should we care about dark social? As editors and publishers, we need a clear understanding of where our audiences are coming from. There's a false sense of security in believing a greater portion of our audience is coming in through typed/bookmarked than is really the case. Our audience, it turns out, may not be loyal repeat visitors after all.

On the other hand, if there are more visitors coming in through the social side door--the Facebook/Twitter axis plus everyday dark social--well, that tells us something about the content we are creating: It works! People are sharing it. People want others to see it. In the sharing economy, quality wins. And if that sharing economy is bigger than we realized, quality matters more than ever.

Our new understanding of dark social comes just as analytics gurus are coming to realize there's something quirky going on with search referrals. BuzzFeed, in a post titled "Where Did All the Search Traffic Go," notes that traffic from search engines to digital publishers has dropped 30 percent from September 2012 to April 2013. Some of that is an actual decline in the power of search as the sharing economy takes off. But some, alas, is the result of incomplete data.

As Digiday's Jack Marshall explains, Safari and Firefox - in certain circumstances--are not passing along accurate referral information when users click through from search. That means search is undercounted, and no one knows quite how much. Danny Sullivan, the search engine expert and consultant, has dubbed this delta, of course, Dark Google.

The challenge now is to find a way out of all this darkness. If we can't see where we are, we can't know where we're going.

This post also appears at Folio, where Cohn writes a bimonthly column.