THE NERDS ARE INSIDE THE BUILDING
The strange truth is that campaigns have long been low-technologist, if not low-technology, affairs. Think of them as a weird kind of niche startup and you can see why. You have very little time, maybe a year, really. You can't afford to pay very much. The job security, by design, is nonexistent. And even though you need to build a massive "customer" base and develop the infrastructure to get money and votes from them, no one gets to exit and make a bunch of money. So, campaign tech has been dominated by people who care about the politics of the thing, not the technology of the thing. The websites might have looked like solid consumer web applications, but they were not under the hood.
For all the hoopla surrounding the digital savvy of President Obama's 2008 campaign, and as much as everyone I spoke with loved it, it was not as heavily digital or technological as it is now remembered. "Facebook was about one-tenth of the size that it is now. Twitter was a nothing burger for the campaign. It wasn't a core or even peripheral part of our strategy," said Teddy Goff, Digital Director of Obama for America and a veteran of both campaigns. Think about the killer tool of that campaign, my.barackobama.com; It borrowed the my from MySpace.
Sure, the '08 campaign had Facebook co-founder Chris Hughes, but Hughes was the spokesperson for the company, not its technical guy. The '08 campaigners, Slaby told me, had been "opportunistic users of technology" who "brute forced and baling wired" different pieces of software together. Things worked (most of the time), but everyone, Slaby especially, knew that they needed a more stable platform for 2012.
In 2008, Facebook was about one-tenth of the size that it is now. Twitter was a nothing burger for the campaign. It wasn't a core or even peripheral part of the strategy.
Campaigns, however, even Howard Dean's famous 2004 Internet-enabled run at the Democratic nomination, did not hire a bunch of technologists. Though they hired a couple, like Clay Johnson, they bought technology from outside consultants. For 2012, Slaby wanted to change all that. He wanted dozens of engineers in-house, and he got them.
"The real innovation in 2012 is that we had world-class technologists inside a campaign," Slaby told me. "The traditional technology stuff inside campaigns had not been at the same level." And yet the technologists, no matter how good they were, brought a different worldview, set of personalities, and expectations.
Campaigns are not just another Fortune 500 company or top-50 website. They have their own culture and demands, strange rigors and schedules. The deadlines are hard and the pressure would be enough to press the t-shirt of even the most battle-tested startup veteran.
To really understand what happened behind the scenes at the Obama campaign, you need to know a little bit about its organizational structure. Tech was Harper Reed's domain. "Digital" was Joe Rospars' kingdom; his team was composed of the people who sent you all those emails, designed some of the consumer-facing pieces of BarackObama.com, and ran the campaigns' most-excellent accounts on Facebook, Twitter, Tumblr, video, and the like. Analytics was run by Dan Wagner, and those guys were responsible for coming up with ways of finding and targeting voters they could persuade or turn out. Jeremy Bird ran Field, the on-the-ground operations of organizing voters at the community level that many consider Obama's secret sauce . The tech for the campaign was supposed to help the Field, Analytics, and Digital teams do their jobs better. Tech, in a campaign or at least this campaign or perhaps any successful campaign, has to play a supporting role. The goal was not to build a product. The goal was to reelect the President. As Reed put it, if the campaign were Moneyball, he wouldn't be Billy Beane, he'd be "Google Boy."
There's one other interesting component to the campaign's structure. And that's the presence of two big tech vendors interfacing with the various teams -- Blue State Digital and NGP Van. The most obvious is the firm that Rospars, Jascha Franklin-Hodge, and Clay Johnson co-founded, Blue State Digital. They're the preeminent progressive digital agency, and a decent chunk -- maybe 30 percent -- of their business comes from providing technology to campaigns. Of course, BSD's biggest client was the Obama campaign and has been for some time. BSD and Obama for America were and are so deeply enmeshed, it would be difficult to say where one ended and the other began. After all, both Goff and Rospars, the company's principals, were paid staffers of the Obama campaign. And yet between 2008 and 2012, BSD was purchased by WPP, one of the largest ad agencies in the world. What had been an obviously progressive organization was now owned by a huge conglomerate and had clients that weren't other Democratic politicians.
One other thing to know about Rospars, specifically: "He's the Karl Rove of the Internet."
One other thing to know about Rospars, specifically: "He's the Karl Rove of the Internet," someone who knows him very well told me. What Rove was to direct mail -- the undisputed king of the medium -- Rospars is to email. He and Goff are the brains behind Obama's unprecedented online fundraising efforts. They know what they were doing and had proven that time and again.
The complex relationship between BSD and the Obama campaign adds another dimension to the introduction of an inside team of technologists. If all campaigns started bringing their technology in house, perhaps BSD's tech business would begin to seem less attractive, particularly if many of the tools that such an inside team created were developed as open source products.
So, perhaps the tech team was bound to butt heads with Rospars' digital squad. Slaby would note, too, that the organizational styles of the two operations were very different. "Campaigns aren't traditionally that collaborative," he said. "Departments tend to be freestanding. They are organized kind of like disaster response -- siloed and super hierarchical so that things can move very quickly."
Looking at it all from the outside, both the digital and tech teams had really good, mission-oriented reasons for wanting their way to carry the day. The tech team could say, "Hey, we've done this kind of tech before at larger scale and with more stability than you've ever had. Let us do this." And the digital team could say, "Yeah, well, we elected the president and we know how to win, regardless of the technology stack. Just make what we ask for."
The way that the conflict played out was over things like the user experience on the website. Jason Kunesh was the director of UX for the tech team. He had many years of consulting under his belt for big and small companies like Microsoft and LeapFrog. He, too, from an industry perspective knew what he was doing. So, he ran some user interrupt tests on the website to determine how people were experiencing www.barackobama.com. What he found was that the website wasn't even trying to make a go at persuading voters. Rather, everyone got funneled into the fundraising "trap." When he raised that issue with Goff and Rospars, he got a response that I imagine was something like, "Duh. Now STFU," but perhaps in more words. And from the Goff/Rospars perspective, think about it: the system they'd developed could raise $3 million *from a single email.* The sorts of moves they had learned how to make had made a difference of tens, if not hundreds of millions of dollars. Why was this Kunesh guy coming around trying to tell them how to run a campaign?
From Kunesh's perspective, though, there was no reason to think that you had to run this campaign the same as you did the last one. The outsider status that the team both adopted and had applied to them gave them the right to question previous practices.
Tech sometimes had difficulty building what the Field team, a hallowed group within the campaign's world, wanted. Most of that related to the way that they launched Dashboard, the online outreach tool. If you look at Dashboard at the end of the campaign, you see a beautifully polished product that let you volunteer any way you wanted. It's secure and intuitive and had tremendously good uptime as the campaign drew to a close.
But that wasn't how the first version of Dashboard looked.
The tech team's plan was to roll out version 1 with a limited feature set, iterate, roll out version 2, iterate, and so on and so forth until the software was complete and bulletproof. Per Kunesh's telling, the Field people were used to software that looked complete but that was unreliable under the hood. It looked as if you could do the things you needed to do, but the software would keep falling down and getting patched, falling down and getting patched, all the way through a campaign. The tech team did not want that. They might be slower, but they were going to build solid products.
In the movie version of the campaign, there's probably a meeting where I'm about to get fired and I throw myself on the table
Reed's team began to trickle into Chicago beginning in May of 2011. They promised, over-optimistically, that they'd release a version of Dashboard just a few months after the team arrived. The first version was not impressive. "August 29, 2011, my birthday, we were supposed to have a prototype out of Dashboard, that was going to be the public launch," Kunesh told me. "It was freaking horrible, you couldn't show it to anyone. But I'd only been there 13 weeks and most of the team had been there half that time."
As the tech team struggled to translate what people wanted into usable software, trust in the tech team -- already shaky -- kept eroding. By Februrary of 2012, Kunesh started to get word that people on both the digital and field teams had agitated to pull the plug on Dashboard and replace the tech team with somebody, anybody, else.
"A lot of the software is kind of late. It's looking ugly and I go out on this Field call," Kunesh remembered. "And people are like, 'Man, we should fire your bosses man... We gotta get the guys from the DNC. They don't know what the hell you're doing.' I'm sitting there going, 'I'm gonna get another margarita.'"
While the responsibility for their early struggles certainly falls to the tech team, there were mitigating factors. For one, no one had ever done what they were attempting to do. Narwhal had to connect to a bunch of different vendors' software, some of which turned out to be surprisingly arcane and difficult. Not only that, but there were differences in the way field offices in some states did things and how campaign HQ thought they did things. Tech wasted time building things that it turned out people didn't need or want.
"In the movie version of the campaign, there's probably a meeting where I'm about to get fired and I throw myself on the table," Slaby told me. But in reality, what actually happened was Obama's campaign chief Jim Messina would come by Slaby's desk and tell him, "Dude, this has to work." And Slaby would respond, "I know. It will," and then go back to work.
In fact, some shakeups were necessary. Reed and Slaby sent some product managers packing and brought in more traditional ones like former Microsoft PM Carol Davidsen. "You very much have to understand the campaign's hiring strategy: 'We'll hire these product managers who have campaign experience, then hire engineers who have technical experience -- and these two worlds will magically come together.' That failed," Davidsen said. "Those two groups of people couldn't talk to each other."
Then, in the late spring, all the products that the tech team had been promising started to show up. Dashboard got solid. You didn't have to log in a bunch of times if you wanted to do different things on the website. Other smaller products rolled out. "The stuff we told you about for a year is actually happening," Kunesh recalled telling the Field team. "You're going to have one login and have all these tools and it's all just gonna work."
Perhaps most importantly, Narwhal really got on track, thanks no doubt to Davidsen's efforts as well as Josh Thayer's, the lead engineer who arrived in April. What Narwhal fixed was a problem that's long plagued campaigns. You have all this data coming in from all these places -- the voter file, various field offices, the analytics people, the website, mobile stuff. In 2008, and all previous races, the numbers changed once a day. It wasn't real-time. And the people looking to hit their numbers in various ways out in the field offices -- number of volunteers and dollars raised and voters persuaded -- were used to seeing that update happen like that.
But from an infrastructure level, how much better would it be if you could sync that data in real time across the entire campaign? That's what Narwhal was supposed to do. Davidsen, in true product-manager form, broke down precisely how it all worked. First, she said, Narwhal wasn't really one thing, but several. Narwhal was just an initially helpful brand for the bundle of software.
In reality, it had three components. "One is vendor integration: BSD, NGP, VAN [the latter two companies merged in 2010]. Just getting all of that data into the system and getting it in real time as soon as it goes in one system to another," she said. "The second part is an API portion. You don't want a million consumers getting data via SQL." The API allowed people to access parts of the data without letting them get at the SQL database on the backend. It provided a safe way for Dashboard, the Call Tool (which helped people make calls), and the Twitter Blaster to pull data. And the last part was the presentation of the data that was in the system. While the dream had been for all applications to run through Narwhal in real time, it turned out that couldn't work. So, they split things into real-time applications like the Call Tool or things on the web. And then they provided a separate way for the Analytics people, who had very specific needs, to get the data in a different form. Then, whatever they came up with was fed back into Narwhal.
It's just change management. It's not complicated; it's just hard.
By the end, Davidsen thought all the teams' relationships had improved, even before Obama's big win. She credited a weekly Wednesday drinking and hanging out session that brought together all the various people working on the campaign's technology. By the very end, some front-end designers who were technically on the digital team had embedded with the tech squad to get work done faster. Tech might not have been fully integrated, but it was fully operational. High fives were in the air.
Slaby, with typical pragmatism, put it like this. "Our supporters don't give a shit about our org chart. They just want to have a meaningful experience. We promise them they can play a meaningful role in politics and they don't care about the departments in the campaign. So we have to do the work on our side to look integrated and have our shit together," he said. "That took some time. You have to develop new trust with people. It's just change management. It's not complicated; it's just hard."
WHAT THEY ACTUALLY BUILT
Of course, the tech didn't exist for its own sake. It was meant to be used by the organizers in the field and the analysts in the lab. Let's just run through some of the things that actually got accomplished by the tech, digital, and analytics teams beyond of Narwhal and Dashboard.
They created the most sophisticated email fundraising program ever. The digital team, under Rospars leadership, took their data-driven strategy to a new level. Any time you received an email from the Obama campaign, it had been tested on 18 smaller groups and the response rates had been gauged. The campaign thought all the letters had a good chance of succeeding, but the worst-performing letters did only 15 to 20 percent of what the best-performing emails could deliver. So, if a good performer could do $2.5 million, a poor performer might only net $500,000. The genius of the campaign was that it learned to stop sending poor performers.
Obama became the first presidential candidate to appear on Reddit, the massive popular social networking site. And yes, he really did type in his own answers with Goff at his side. One fascinating outcome of the AMA is that 30,000 Redditors registered to vote after President dropped in a link to the Obama voter registration page. Oh, and the campaign also officially has the most tweeted tweet and the most popular Facebook post. Not bad. I would also note that Laura Olin, a former strategist at Blue State Digital who moved to the Obama campaign, ran the best campaign Tumblr the world will probably ever see.
With Davidsen's help, the Analytics team built a tool they called The Optimizer, which allowed the campaign to buy eyeballs on television more cheaply. They took set-top box (that is to say, your cable or satellite box or DVR) data from Davidsen's old startup, Navik Networks, and correlated it with the campaign's own data. This occurred through a third party called Epsilon: the campaign sent its voter file and the television provider sent their billing file and boom, a list came back of people who had done certain things like, for example, watched the first presidential debate. Having that data allowed the campaign to buy ads that they knew would get in front of the most of their people at the least cost. They didn't have to buy the traditional stuff like the local news, either. Instead, they could run ads targeted to specific types of voters during reruns or off-peak hours.
According to CMAG/Kantar, the Obama's campaign's cost per ad was lower ($594) than the Romney campaign ($666) or any other major buyer in the campaign cycle. That difference may not sound impressive, but the Obama campaign itself aired more than 550 thousand ads. And it wasn't just about cost, either. They could see that some households were only watching a couple hours of TV a day and might be willing to spend more to get in front of those harder-to-reach people.
Goff described the Facebook tool as "the most significant new addition to the voter contact arsenal that's come around in years, since the phone call."
The digital, tech, and analytics teams worked to build Twitter and Facebook Blasters. They ran on a service that generated microtargeting data that was built by Will St. Clair. It was code named
Täärgus Taargüs for some reason. With Twitter, one of
the company's former employees, Mark Trammell, helped build a tool that could specifically send individual users direct messages. "We built an
influence score for the people following the [Obama for America] accounts and then cross-referenced those for specific things we were trying to target,
battleground states, that sort of stuff." Meanwhile, the teams also built an opt-in Facebook outreach program that sent people messages saying,
essentially, "Your friend, Dave in Ohio, hasn't voted yet. Go tell him to vote." Goff described the Facebook tool as "the most significant new addition to the voter
contact arsenal that's come around in years, since the phone call."
Last but certainly not least, you have the digital team's Quick Donate. It essentially brought the ease of Amazon's one-click purchases to political donations. "It's the absolute epitome of how you can make it easy for people to give money online," Goff said. "In terms of fundraising, that's as innovative as we needed to be." Storing people's payment information also let the campaign receive donations via text messages long before the Federal Elections Commission approved an official way of doing so. They could simply text people who'd opted in a simple message like, "Text back with how much money you'd like to donate." Sometimes people texted much larger dollar amounts back than the $10 increments that mobile carriers allow.
It's an impressive array of accomplishments. What you can do with data and code just keeps advancing. "After the last campaign, I got introduced as the CTO of the most technically advanced campaign ever," Slaby said. "But that's true of every CTO of every campaign every time." Or, rather, it's true of one campaign CTO every time.
That next most technically advanced CTO, in 2016, will not be Harper Reed. A few days after the election, sitting with his wife at Wicker Park's Handlebar, eating fish tacos, and drinking a Daisy Cutter pale ale, Reed looks happy. He'd told me earlier in the day that he'd never experienced stress until the Obama campaign, and I believe him.
He regaled us with stories about his old performance troupe, Jugglers Against Homophobia, wild clubbing and DJs. "It was this whole world of having fun and living in the moment," Reed said. "And there was a lot of doing that on the Internet."
"I spent a lot of time hacking doing all this stuff, building websites, building communities, working all the time, " Reed said, "and then a lot of time drinking, partying, and hanging out. And I had to choose when to do which."
We left Handlebar and made a quick pitstop at the coffee shop, Wormhole, where he first met Slaby. Reed cracks that it's like Reddit come to life. Both of them remember the meeting the same way: Slaby playing the role of square, Reed playing the role of hipster. And two minutes later, they were ready to work together. What began 18 months ago in that very spot was finally coming to an end. Reed could stop being Obama for America's CTO and return to being "Harper Reed, probably one of the coolest guys ever," as his personal webpage is titled.
But of course, he and his whole team of nerds were changed by the experience. They learned what it was like to have -- and work with people who had -- a higher purpose than building cool stuff. "Teddy [Goff] would tear up talking about the President. I would be like, 'Yeah, that guy's cool,'" Reed said. "It was only towards the end, the middle of 2012, when we realized the gravity of what we were doing."
Part of that process was Reed, a technologist's technologist, learning the limits of his own power. "I remember at one point basically breaking down during the campaign because I was losing control. The success of it was out of my hands," he told me. "I felt like the people I hired were right, the resources we argued for were right. And because of a stupid mistake, or people were scared and they didn't adopt the technology or whatever, something could go awry. We could lose."
And losing, they felt more and more deeply as the campaign went on, would mean horrible things for the country. They started to worry about the next Supreme Court Justices while they coded.
"There is the egoism of technologists. We do it because we can create. I can handle all of the parameters going into the machine and I know what is going to come out of it," Reed said. "In this, the control we all enjoyed about technology was gone."
We finished our drinks, ready for what was almost certainly going to be a long night, and headed to our first club. The last thing my recorder picked up over the bass was me saying to Harper, "I just saw someone buy Hennessy. I've never seen someone buy Hennessy." Then, all I can hear is that music.