The Pentagon Enters the Social Web With a Call for Memetrackers

The Department of Defense's tech incubator is looking for a few bright minds to revolutionize how the military uses social networks

Do you spend hours a day on Facebook? Can you sniff out Twitter memes before they become full-fledged trending topics? Good news: the Pentagon is looking for someone like you.

The Defense Advanced Research Projects Agency (DARPA), the DoD progenitors of revolutionary tech like passive radar and the Internet, is calling for research applications of social media to strategic communication. According to an agency announcement (PDF), DARPA is looking to shell out $42 million in funding for "innovative approaches that enable revolutionary advances in science, devices, or systems." The general goal of the Social Media in Strategic Communication (SMISC) program is to develop a new science of social networks built on an emerging technology base.

In short, the Pentagon wants to up its intelligence game to keep pace with the constantly expanding, interwoven latticework of connections in the social space. The military wants to be able to track the formation, development and spread of ideas and concepts, use linguistic clues to ferret out purposeful or deceptive misinformation, and use sentiment analysis and opinion mining to extrapolate, for example, where the next Arab revolution might take place, or identify credible (or debunked) threats reverberating across cyberspace.

This certainly isn't the federal government's first foray into using social media for intelligence gathering. The FBI began using social networking sites to gather information about the whereabouts of fugitives, even combing through Facebook and Twitter for clues as to the location of notorious Boston gangster James "Whitey" Bulger. But as David Streitfeld reported in the New York Times, such methods of information collection and analysis are far from systematized:

Social networks can allow the military not only to follow but also to shape the action. In its 37-page solicitation, Darpa described how a would-be high-technology lynching was foiled: "Rumors about the location of a certain individual began to spread in social media space and calls for storming the rumored location reached a fever pitch. By chance, responsible authorities were monitoring the social media, detected the crisis building, sent out effective messaging to dispel the rumors and averted a physical attack on the rumored location."

(Is this a reference to Osama bin Laden or someone much more obscure? Were the "responsible authorities" trying to put off an attack because the individual was not at the location, or because he was? Darpa officials did not return e-mails requesting comment.)

The crisis was formed, observed, understood and diffused entirely within social media, the solicitation noted. But the success of the authorities was a fluke, the result of "luck and unsophisticated manual methods."

Luckily for DARPA, applicable research into semantic analysis of the social Web has been underway for years in the private sector. At Indiana University in Bloomington's School for Informatics and Computing, the independent research of Professors Johan Bollen, Fil Menczer, Alex Vespignani, and Sandro Flammini at the Center for Complex Networks and Systems has become increasingly intertwined and integrated over time into a discipline they call "computational social science." Menczer heads a project called Truthy, which tracks the flow of information (and misinformation) over social networks and has been used to ferret our search engine spamming. Vespignani uses mood-tagged Twitter messages to anticipate the spread of biological agents during diseases outbreaks. Bollen's was leading a project focused on tracking the correlations between messages sent in the social space -- tweets, status updates, and the like -- and fluctuations in the Dow Jones Industrial Average. Bollen found that by categorizing millions of Twitter posts into various mood categories (happiness, kindness, alertness, sureness, vitality and calmness), he could anticipate fluctuations in the Dow with an 87% chance of accuracy. Bollen's research garnered the attention of the New York Times Magazine, which helped introduce the idea of social media as social index to the general population.

"All of a sudden we had a whole universe that was out there that was not visible," Vespignani said when I first discussed his research with him in February. "Now, we can look at our society by looking at ourselves. We can look at these systems that are more complex than physical or biological laws, than the stars and microbes itself."

"We can predict that there'll be a dangerous pandemic in the next 30 days, it's much harder to track the immediate reaction of the society that tends to change the collective behavior," explained Menzcer. "There's a feedback loop between forecasting and actual reactions induced in people. The real-time monitoring of social media allows you to measure how behavior changes from moment to moment, which provides us with better indicators to model behavior in the future."

There are limitations, of course: The online world isn't necessarily a perfect reflection of offline events. As of the end of 2010, a mere eight percent of Americans were on Twitter, and their demographics skew towards the affluent and tech-savvy, although other ubiquitous data networks like Facebook and Google can be used to track information in the same manner. Linguistic and symbolic barriers could prevent the Pentagon from accurately assessing the vast stores of information distilled from the social Web, although IARPA -- a DARPA offshoot tied to Intelligence rather than Defense -- has been building a "metaphor program" to overcome the potential linguistic challenges towards to effective analysis of particular ethnocultural ecosystems:

"The study of language offers a strategic opportunity for improved counterterrorist intelligence, in that it enables the possibility of understanding of the Other's perceptions and motivations, be he friend or foe," the two authors of Computational Methods for Counterterrorism wrote. "As we have seen, linguistic expressions have levels of meaning beyond the literal, which it is critical to address. This is true especially when dealing with texts from a high-context traditionalist culture such as those of Islamic terrorists and insurgents."

But the potential for future advances in the social sciences -- and, in turn, the military, are abundant. "The social sciences have been struggling against biases," noted Menczer. " But we can use so much data that we get a much better image than a cross-section of the population. There are millions upon millions of experiments that now we can think about because of access to this data." For intelligence analysts looking for new ways to think about the vast troves of information circulating through the social Web, the right program for performing those experiments could be the difference between the fog of war and a sure victory against enemies foreign and domestic.

Image: bitsandpixels/Flickr.

.>