Healthcare.gov: Code Developed by the People and for the People, Released Back to the People

Healthcare.gov

As the first website to be demonstrated by a sitting President of the United States, Healthcare.gov already occupies an unusual place in history. In October, it will take on an even more important historic role, guiding millions of Americans through the process of choosing health insurance.

How a website is built or designed may seem mundane to many people, but when the site in question is focused upon such an important function, what it looks like and how it works matter. Last week, the United States Department of Health and Human Services (HHS) relaunched Healthcare.gov with a new appearance and modern technology that is unusual in federal-government websites.

"It's fast, built in static HTML, completely scalable and secure," said Bryan Sivak, chief technology officer of HHS, in an interview. "It's basically setting up a web server. That's the beauty of it." What makes such an ambitious experiment in social coding more unusual is that the larger political and health-care policy context that it's being been built within is more fraught with tension and scrutiny than any other arena in the federal government.

The implementation and outcomes of the Affordable Care Act -- AKA "Obamacare" -- will affect millions of people, from the premiums they pay to the incentives for the health care they receive. "The goal is get people enrolled," said Sivak. "A step to that goal is to build a health insurance marketplace. It is so much better to build it in a way that's open, transparent and enables updates. This is better than a big block of proprietary code locked up in a CMS [content management system]."

Healthcare.gov

Thinking differently about a .gov
The new site has been built in public for months, iteratively created on Github using cutting edge open-source technologies. Healthcare.gov is the rarest of birds: a next-generation website that also happens to be a .gov.

"We needed to evolve from the previous site but didn't want a total departure," said Ed Mullen, a user experience designer who has worked on Healthcare.gov since it was first launched, in an interview. "The web has changed dramatically in that time. Part of adapting to that [change] has been creating a site that really understands consumers. Today, consumers are doing all kinds of things across the web. We're comparing ourselves to Rdio and similar services. We want to be aligned with the current thinking of the Web."

The people that helped to build the new Healthcare.gov are unusual: Instead of some obscure sub-contractor in a nameless office park in northern Virginia, the site was iteratively created by a cross-disciplinary team of developers and editors at HHS, and contractors at Teal Design, Edward Mullen Studio, and Development Seed, a scrappy startup in a garage in the District of Columbia.

"This is such a lean site," said Jon Booth, head of the web and new media group at the Centers for Medicare and Medicaid Services (CMS), in an interview. "HHS had a blanket contract when we when awarded this. Aquilent got creative and brought people on with powerful skills, like Ed and Jessica, a designer at Teal Media, and Development Seed. Most of my team is working on this site; we have internal UX, information architects, designers, developers, and infrastructure people that stood up the cloud environment. Their collaboration is one of the high points of this process."

The involvement of Development Seed drove specific technology choices that led to substantial improvements in design and function. The startup first made its mark in the DC tech scene consulting on Drupal, an open source content management system that has become popular in the federal government over the past several years. Recently, Development Seed has been pushing the limits of lightweight Web design, open data-driven maps and open-source code.

"This is our ultimate dogfooding experience," said Eric Gundersen, the co-founder of Development Seed, in an interview. "We're going to build it and then buy insurance through it."

"The work that they're doing is amazing," said Sivak, "like how they organize their sprints and code. It's incredible what can happen when you give a team of talented developers and managers and let them go."

The new Healthcare.gov will fill a yawning gap in the technology infrastructure deployed to support the mammoth law, providing a federal choice engine for the more than 30 different states that did not develop their own health-insurance exchanges, but the site is just one component of the insurance exchanges. Others may not be ready by the October deadline. According to a recent report from the Government Accountability Office, the Department of Health and Human Services' (HHS) Centers for Medicare & Medicaid Services (CMS) is behind in implementing key aspects of the law, such as training the workers who will help people navigate the process, certifying the plans that will be sold on the exchanges, and determining the eligibility of consumers for federal subsidies. Despite all this, HHS expressed confidence to the GAO that exchanges will be open and functioning in every state on October 1.

On that day, Healthcare.gov will be the primary interface for Americans to learn about and shop for health insurance, as Dave Cole, a lead developer at Development Seed, wrote in a blog post this March. Cole, who served as a senior advisor to the United States chief information officer and deputy director of new media at the White House, was a key part of the team that moved WhiteHouse.gov to Drupal. As he explained, the code will be open in two important ways:

First, Bryan [Sivak] pledged, "everything we do will be published on GitHub," meaning the entire code-base will be available for reuse. This is incredibly valuable because some states will set up their own state-based health insurance marketplaces. They can easily check out and build upon the work being done at the federal level. GitHub is the new standard for sharing and collaborating on all sorts of projects, from city geographic data and laws to home renovation projects and even wedding planning, as well as traditional software projects.

Moreover, all content will be available through a JSON API, for even simpler reusability. Other government or private sector websites will be able to use the API to embed content from healthcare.gov. As official content gets updated on healthcare.gov, the updates will reflect through the API on all other websites. The White House has taken the lead in defining clear best practices for web APIs.

Putting open source to work
According to Sivak, his team didn't get directly involved in the new Healthcare.gov until November 2012.

After that, "we facilitated the right conversations around what to build and how to build it, emphasizing the consumer-facing aspects of it," he said. "The other part was to figure what the right infrastructure was going to be to build this thing."

That decision is where this story gets especially interesting, if you're interested in how government uses technology to deliver information to the people it serves.

Government websites have not, historically, been sterling examples of design or usability. Unfortunately, in many cases, they're also built at great expense, given the dependence of government agencies on contractors and systems integrators, and use technologies that are years behind the rest of the web.

Healthcare.gov could have gone in the same direction, but for the influence of its young chief technology officer, an "entrepreneur-in-residence" who had successfully navigated the bureaucracies of the District of Columbia and state of Maryland. "Our first plan was to leverage Percussion, a commercial CMS that we'd been using for a long time," said Sivak. "The problem I had with that plan was that it wasn't going to be easy to update the code. The process was complicated. Simple changes to navigation were going to take a month."

At that point, Sivak did what most people do in this new millennium when making a technology choice: He reached out to his social networks and went online. "We started talking to people about a better way, including people who had just come off the Obama campaign," he said. "I learned about the ground they had broken for the political space, from A/B testing to lightweight infrastructure, and started reading about where all that came from. We started thinking about Jekyll as a platform and using Prose.io."

After Sivak and his team read about Development Seed's work with Jekyll, they contacted the startup directly. After a little convincing, Development Seed agreed to consult on one more .gov project.

"A Presidential Innovation Fellow used same tech we're using for several of their projects," said Cole. "Bryan [Sivak] heard about it and talked to us. He asked where we would go. We wanted to be on Github. We knew there were performance and reliability benefits from building the stack on HTML."

Jekyll, for those who are unfamiliar with web-development trends, is a way for developers to build a static website from dynamic components. Instead of running a traditional website with a relational database and server-side code, using Jekyll enables programmers to create content like they create code. The end result of this approach is a site that loads faster for users, a crucial performance issue, particularly on mobile devices. "Instead of [running] farms of application servers to handle massive load, you're basically slimming down to two," said Sivak. "You're just using HTML5, CSS, and Javascript, all being done in responsive design. The way it's being built matters. You could in theory do the same with application servers and a CMS, but it would be much more complex. What we're doing here is giving anyone with basic skills to basic changes on the fly. You don't need expensive consultants."

That adds up to cost savings. Sites that are heavily trafficked -- as Healthcare.gov can reasonably expected to be - normally have to use a caching layer to serve static content and add more server capacity as demand increases.

"When we worked with the World Bank, they chose a plan from Rackspace for 16 servers," said Gundersen. "That added tens of thousands of dollars, with a huge hosting bill every month."

HHS had similar strategic plans for the new site, at least at first. "They were planning 32 servers, between staging, production and disaster recovery, with application servers for different environments," said Cole. "You're just talking about content. There just needs to be one server. We're going to have two, with one for backup. That's a deduction of 30 servers."

Why was Development Seed able to succeed in selling this approach to coding and collaboration with a federal agency and other contractors, in contrast to traditional systems integrators? Or to put it another way, what could a mapping startup teach the world about making government work better? Quite a lot, as it turns out.

"It helps that we're running a cloud SaaS business," said Cole, referring to MapBox. "If you're not doing your own products and using your own products, you're not going to have a sense of what the future is. You can put Wordpress into Amazon but then you'll have problems; it wasn't designed to be a cloud application. In this case, the code is completely cloud-ready and able to be scaled through a CDN [content delivery network], through Akamai. Understanding that comes from running a cloud company."

While Jekyll eliminates the need for a full-blown content management system for Healthcare.gov (and with it, related costs) people managing the site still need to be able to update it. That's where Prose.io comes in.

Prose.io is an open-source content editor developed by Development Seed that gives non-programmers a clean user interface to update pages. "If you create content and run Jekyll, it requires content editors to know code," said Cole. "Prose is the next piece. You can run it on your on own servers or use a hosted version. It gives access to content in a CMS-like interface, basically adding a WYSIWYG [What You See Is What You Get] skin, giving you a text editor in the browser."

To non-technical users, Prose.io looks much like the standard "What you see is what you get" interface, familiar from Wordpress or Microsoft Word, with a couple bells and whistles, such as mobile editing.

"You can basically preview in live," said Cole. "You usually don't get a full in-browser preview. The difference is you have this with no backend CMS. It's just a directory and text files, with a web interface that exposes it. There are no servers, no infrastructure, and no monthly costs. All you need is a need free web app and Github. If you don't want to use that, use Git and Github Enterprise."

Prose was built by Development Seed to enable them to pass management of code off to the owners of websites. That's exactly how it's being used here.

"We were doing Jekyll sites and wanted to turn them over to clients who could then maintain them themselves," said Cole. "It's just a wrapper around the workflow, with design aesthetic to make it clean and something in the backend that's technical."

The technology behind Healthcare.gov also delivered on a crucial feature: a version geared toward a growing Spanish-speaking population that is quite likely to access the site on a mobile device.

"When you look at providing a Spanish translation or a mobile site, it becomes really complicated trying to have a site render in different ways, in different configurations," said Cole. "A CMS needs a different version for each one. That's not true here, with a responsive, easy-to-iterate upon design. A change is a quick process, not altering any application code."

The HHS tech team is now singing the praise of that choice.

"One of the strong attractions for working in Jekyll is that you can have multiple versions of everything," said Booth. "We can provide the exact same experience and mobile capacity on the Spanish website as on the English version."

While the front end of Healthcare.gov is distributed over Akamai, the back end of the site will be be hosted in a secure cloud.

"The servers are hosted in Terramark, a cloud computing firm that's a subsidiary of Verizon," said Booth. "When we got involved, Terramark had already been selected as the vendor. We inherited that; it was our first major cloud deployment. It's wonderful, compared to the traditional 'buy a lot of boxes and get the servers set up.' Percussion was fine as a CMS but the scalability issue was huge for us, really overarching."

Combining these two approaches finally realized some of the more aspirational rhetoric about the potential of "cloud computing" to deliver better savings and services that has bounced around Washington over the past four years.

"For us, it's a combination of Terramark as data center and Akamai as content distribution network," said Cole. "For the relaunch, no consumer traffic hits Terramark at all, with the exception of search queries. We have completely pushed the website out to Akamai, which gives us a lot of flexibility. This is by far the fastest site we've ever built. We wanted to make sure that this site is not adding any overhead, is as lightweight as it can be."

Embracing citizen-centric design
"Thinking differently" about this site went beyond technical choices for architecture or how rules and regulations were instantiated in code. If you visit Healthcare.gov on a smartphone or tablet, you'll notice how quickly it loads, how clean the design is on mobile, and how the site renders to fit the size of your screen.

That's no accident, as Mullen and Booth explained.

"Across the board, there was agreement that it needed to be all about the consumer," said Mullen. "That was the mantra from the beginning. It comes down to understanding the audience, from thinking about the reading level of all content to writing clear, declarative sentences."

Focusing on the consumer -- or the citizen -- as the primary driver for design and technical decisions may sound like common sense but it's not a principle that appears to have animated the construction of many .gov websites over the past two decades. In recent years, however, design principles are redefining how online government is created and delivered, with notable results.

"We knew, building this marketplace from scratch, that we needed to have consistency," said Mullen. "We had to make sure all efforts work together, from outreach, engagement, education, to enrollment. We knew there was an opportunity to make some changes and largely to simplify. The first time around, the site was a pretty big success. When it came time to reassess, what was the core mission? It boiled down to being a strictly consumer site."

"Before we made any decisions on what the site was going to be, we did a lot of research into the population we were going to try to reach," explained Booth. "One of our goals was to reach a much younger demographic," said Booth. "That pointed us to doing mobile. Mobile is growing, representing about 10 percent of the traffic to medicare.gov now."

The goal of speaking clearly to the people that HHS is trying to reach drove a series of choices for design and content.

"One of the things we saw was the need to explain the project really clearly to people, not to introduce other topics that might be confusing," said Booth. "The marketplace program was separate. We've moved away other parts, like the text of the law or implementation timeline. It's important to keep those materials online but on other properties. Policy guidelines also were moved over to other websites. We needed something that could speak clearly to the people we're trying to reach."

In design, choosing what not to include is as important as what elements do appear. The involvement of Teal Design early on and Edward Mullen throughout, focusing on user experience, made a difference in how Healthcare.gov looks and works on a mobile device today.

Code for the people, with the people
Performance, citzen-centric design, and management aside, there's a deeper importance to how Healthcare.gov is being built that will remain relevant for years to come, perhaps even setting a new standard for federal government as a whole: updates to the code repository on Github can be adopted by every health insurance exchange using this code. (The only difference between different state sites is a skin with the state logo.) "We have been working in the .gov space for a while," said Gundersen. "Government people want to make the right decisions. What's nice about what Bryan [Sivak] is doing is that he's trying to make sure that everyone can learn from what HHS is doing in real-time. From a process standpoint, what Bryan is doing is going to change how tech is built. FCC is watching the repository on Github. When agencies can collaborate around code, what will happen? The amount of money we have the opportunity to save agencies is huge. Think about servers alone."

The impact of this reduction in infrastructure on costs was a point he made repeatedly.

"You can't underestimate the cost of complexity," said Gundersen. "If you think the servers are expensive, consider the team required to maintain them."

With Healthcare.gov, something quite important happened that could also be lost in the mix: Code developed for the people, by the people, was released back to the people.

"There's an important legal detail here," said Cole. "Public domain doesn't apply to works for hire and it doesn't apply to code developed by a contractor. Healthcare.gov and other code was developed under BSD, the most permissive license possible."

While the website, funding from the federal government paid for the improvement of a series of open source software tools that it itself used.

"It's nice when government doesn't just take from open source but contributes code back to the community," said Gundersen.

The choice that HHS made in working with Development Seed meant that upgraded code for that tool was released on Github for the use of the public, over the course of the project.

"All of those contributions were made in real-time on Github as they were developed," said Cole. "The code now pulls in Google Analytics to analyze content for popularity."

"We got 10,000 authenticated users through Github and more through repository," said Cole. "Recently, we did a complete redesign, adding mobile and responsive development. HHS paid for that development. They also paid for enhancement of other open source tools that people are already using." Collaboration and cascading updates aren't an extra, in this context: they're mission-critical.

"This was a really distributed team, with design done by another party," said Cole."It started with a 'lock in' that laid out architecture and design, then tech followed. We wanted something really agile, to get it right first and keep iterating. After four months of development, we put together a site that's responsive and multilingual, and did so scalably."

Sivak said that he expects the new site to continue to be improved iteratively over time, in response to how people are actually using it. "We're going to be collecting all kinds of data," said Sivak. "We will be using tools like Optimizely to do A/B and multivariate testing, seeing what works on the fly and adapting from there. We're trying to treat this like a consumer website. The goal of this is to get people enrolled in health care coverage and get insurance. It's not simple. It's relatively complex process. We need to provide a lot of information to help people make decisions. The more this site can act in a consumer-friendly fashion, surfacing information, helping people in simple ways, tracking how people are using it and where they're getting stuck, the more we can improve."

Sivak is a fan of the agile development methodology that has become part of start-up development everywhere, including using analytics tools to track usage and design accordingly.

Notably, the HHS team has been using feedback to drive design choices from the beginning, applying the "lean startup" model to development.

"We did rounds of usability testing, starting early on with a card sort for taxonomy," said Booth. "We built out wireframes and did testing on look and feel. From my point of view, that was informed by a mix of designers, then getting user feedback."

This approach may be maturing in the private sector but still feels novel in some parts of government.

"One thing that has been nice that isn't always there on federal projects was a lot of support from leadership to look at user feedback, watch what users tell us and then go from that," he said. "It's not about what someone's favorite color might be."

The work of the team at HHS earned some praise from their collaborators at Development Seed.

"It's not a case where they needed developers," said Cole. "They just needed to adapt an agile workflow, have a good conversation and manage requirements dynamically, not create a huge spreadsheet of what they wanted and come back in two months.

Open by default
Using Jekyll and Prose.io to build the new Healthcare.gov is the latest chapter in government IT's quiet open source evolution. Across the federal government, judicious adoption of open source is slowly but surely leading to leaner, more interoperable systems.

An open, agile approach to development gave Sivak and his team transparency into the status of the project through issue trackers on Github.

"Put yourself in Bryan's perspective, with his ability to know what's going on," said Gundersen. "He could just open up an issue tracker and look at what's been done, not wait for a briefing."

Talking at length with HHS' CTO, it's clear that he's happy with the process and outcomes, though he emphasized repeatedly that the site will continue to improve. "The thing that Git is all about is social coding," said Sivak, "leveraging the community to help build projects in a better way. It's the embodiment of the open source movement, in many ways: it allows for truly democratic coding, sharing, modifications and updates in a nice interface that a lot of people use."

Sivak has high aspirations that publishing the code for Healthcare.gov will lead to a different kind of citizen engagement. "I have this idea that when we release this code, there may be people out there who will help make improvements, maybe fork the repository, and suggest changes we can choose to add," he said. "Instead of just internal consultants who help build this, we suddenly have legions of developers."

The long game for the code behind Healthcare.gov may be quite interesting, given the status of exchanges in the states. If their health insurance exchanges aren't ready or robust, they can simply pull this code and adopt it. While states and federal government sharing code might make for fertile fodder for constitutional scholars and philosophical discussions of federalism in the future, for the moment, the stakeholders all just need something that works.

"This is multi-lingual, 508-compliant, and hits on mobile," said Gundersen. "So many states and fed agencies can look at this as part of a new, different way of building websites. Part of that is process-based, from the start, using tech that is faster and more flexible."

In fact, Booth says that there have already been a couple of states that have asked CMS if they can pull the code on their sites.

"We don't have to ask if you're on Microsoft, Unix or Application Server Pages," he said. "It's on Github and you can pull it down. Open source is not always applicable but when it is, it's very powerful."

Not everything is innovative in the new Healthcare.gov, as Nick Judd reported at TechPresident in March: The procurement process that led to Development Seed is complicated, with some potential conflicts of interest present. The end result, however, is a small startup in a garage in DC collaborating with the federal government to relaunch one of the most important federal websites of the 21st century in a decidedly 21st-century way: cheaper, faster and scalable, using open source tools and open standards.

"This is the responsive Web of structured data," said Mullen. "Create once, publish everywhere."

That's a huge win for the American people, and while the vast majority of visitors to Healthcare.gov this fall will never know or perhaps care about how the site was built, the delivery of better service at lowered cost to taxpayers is an outcome that matters to everyone.

"Open by design, open by default," said Sivak. "That's what we're doing. It just makes a lot of sense. If you think about what should happen after this year, all of the states that didn't implement their systems, would it make sense for them to have code to use as their own? Or add to it? Think about the amount of money and effort that would save."