Sex, Lies, and Videogames

What if a computer program combined the action and graphics of a video game with the emotional power of great art? The result could revolutionize interactive entertainment—and even change the meaning of “play”

Michael Mateas is an assistant professor in the computer- science department of the University of California at Santa Cruz, where his duties include launching a new undergraduate-degree program in games. He wears two earrings and keeps his bushy brown hair tied back in a long ponytail. His body is small and his head is large, so from a distance one could almost mistake him for a boy. His pale green eyes are piercingly intense, though their intensity is leavened by his beaming smile. He thinks of himself as equal parts artist and computer scientist, and he manages to look both roles.

Stern, by contrast, is so average-looking that he is hard to describe: medium height, thinning brown hair, soft features, an introvert’s undemonstrative manner. He could vanish into any American crowd. Nonetheless, as the three of us talked it was Stern who emerged as the dominant personality, partly because he has an artist’s fierce sense of aesthetic rectitude. Economy, elegance, formal coherence: these are personal matters to him.

Stern is thirty-six and lives in Portland, Oregon. He grew up in various cities along the East Coast. Mateas is forty and grew up in Carson City, Nevada. In some respects, their childhoods ran in parallel. Both discovered video games as children, in the 1970s, when the very first games appeared. They haunted the arcades in the malls; they pounced on the Atari 2600 console when it appeared, in 1977. Not content with playing games, they soon began programming them. At fourteen, Mateas wrote an adventure game. Stern, whose brother kept a pet rabbit named Bonny, made a game called Bonny Attack, in which the player flew the rabbit around and dropped turds and urine on jumping cats. In a high-school essay, Mateas announced his intention to become a big-time designer building games on a “new kind of digital logic circuit based on three-valued logic.” Stern, meanwhile, was getting interested in film and computer animation. Using a Handycam, he began making movies that blended live action with animation.

Then, in college, they both lost touch with video games. “I would still play them,” Stern says, “but they started to feel a little juvenile. I was getting into filmmaking, stories with real characters, adult characters, about psychology and emotion, and games weren’t addressing those things. Once you get into your late teens or early twenties, you realize there’s a lot more out there in terms of art and literature and you lose interest in action-oriented entertainment.”

Mateas decided he would be a scientist, pursuing his longtime dream of artificial intelligence, and he went for his doctorate. Stern was rejected by film schools and wound up taking a job at a game studio. His work on the Petz games kindled his interest in artificial intelligence, the essential ingredient of believable characters, whether animal or human. Mateas’s work on artificial intelligence, meanwhile, had rekindled his interest in games. The AI dream was about building believable virtual people, and games seemed the ideal stage to test them on.

By the time their paths crossed, their thinking had already converged. They soon began plotting their anti-game. Instead of making a game about action figures in elaborate but childish game-worlds, they would make a story about adult characters and adult relationships. Instead of firing bullets at the characters, the player would fire words. The player would talk to the characters—in ordinary English, input with a keyboard rather than a joystick. And the characters would talk back, to each other and to the player. This meant—and they gulped to think of it—that their game would need to speak and understand natural language. That, in itself, is one of the great challenges in AI. But they didn’t intend to stop there.

Conventional games create vast, immersive physical environments. The new game would all take place in a single indoor space, like a black-box theater stage. Instead of taking fifty hours to play, their game would take twenty minutes. Instead of advancing through levels without telling a story, the game would provide a compact, complete dramatic experience, like a one-act play. “We envisioned something where you could come home from work and play it from beginning to end, just like you come home from work and watch a half-hour television show,” said Mateas. “You could come home and have a half-hour interactive-drama experience. It’s complete in itself, it takes you on an arc. It entertains. But then the next day, you could come home from work and play it again and make something different happen.” Instead of offering the player menus of quests or options, their game would seem to flow as naturally as life.

When Mateas, still a graduate student, told his adviser what they intended, the adviser replied that such a game would take a team of ten people ten years to build. The technology didn’t exist. Commercial game design often employs teams of dozens, and here were two guys, one a grad student and the other self-employed (Stern eventually quit his job to work on the game full-time), expecting to build a whole new kind of game with their own four hands and no budget to speak of.

Before they could build the game, they had to build a programming language in which to write it. They spent more than two years constructing what they called ABL (for “A Behavior Language”), which encodes and controls virtual actors. “The actors’ minds are written in ABL,” Mateas explains. ABL itself has a sort of mind: enough artificial intelligence to decide how a particular character might, for example, simultaneously mix a drink, walk across the room, and yell at her husband, as a human actor could do.

That done, they built, again from scratch, another piece of AI, which they call a drama manager. It is a sort of artificial dramaturge and director, which looks at what the player and characters are doing and makes plot and dialogue choices intended to ratchet up and then release dramatic tension. Then they built a natural-language engine, which “listens” to what the player types in, looking for emotional and dramatic cues that the in-game characters can react to.

The game, by now, packed massive amounts of experimental technology under its hood, but what would it be about? They needed to create an intense drama in a confined space and with only a few characters. Influenced by Edward Albee’s play Who’s Afraid of Virginia Woolf? and also by several movies (Steven Soderbergh’s sex, lies, and videotape, Woody Allen’s Husbands and Wives, and Ingmar Bergman’s Scenes From a Marriage), they decided to drop the player into a marital crisis. They hired actors to record five hours of dialogue, raw material from which the drama manager would build twenty minutes of game play.

In the end, they accomplished, they reckon, about 30 percent of what they had hoped to do. “We shot for the stars in hopes of getting to the moon,” says Stern, “and we made it into orbit.” In July 2005, standing together over Stern’s computer in Portland, they pressed the button that “shipped,” over the Internet, a new game called Façade.

Presented by

Jonathan Rauch is a correspondent for The Atlantic and a guest scholar at the Brookings Institution.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register with Disqus.

Please note that The Atlantic's account system is separate from our commenting system. To log in or register with The Atlantic, use the Sign In button at the top of every page.

blog comments powered by Disqus


A Stop-Motion Tour of New York City

A filmmaker animated hundreds of still photographs to create this Big Apple flip book


The Absurd Psychology of Restaurant Menus

Would people eat healthier if celery was called "cool celery?"


This Japanese Inn Has Been Open For 1,300 Years

It's one of the oldest family businesses in the world.


What Happens Inside a Dying Mind?

Science cannot fully explain near-death experiences.

More in National

More back issues, Sept 1995 to present.

Just In