Beyond Space Invaders

Jonathan Rauch, author of "Sex, Lies, and Video Games," talks about a new generation of innovative and emotionally complex video games.

SCENE: A modern apartment, sparsely decorated. An abstract painting hangs on the wall above the sofa, and an Ikea shelving unit stands in the corner. The city skyline can be seen through sliding glass doors that open onto a small balcony. Trip, a blond man in his early 30s, stands in the doorframe.

TRIP (Nervously) Hey! I thought I heard someone out here! Great to see you! It’s been a while.

The next line is yours. Trip is a character in Façade, a new videogame that borrows its ambiance from the modern theater. Unlike most popular games, which transport players to fantasy settings or the NBA playoffs, Façade takes place in what could be the apartment next door. Its residents, a typical urban couple named Grace and Trip, are meant to be your close friends. Like the characters in Who’s Afraid of Virginia Woolf?, they speak to each other in terse, angry sentences, each suppressing layers of mounting rage. Instead of slaying a cartoon dragon, your mission is to sit on their sofa and talk with them about their failing marriage.

From a programmer’s point of view, there are few tasks more daunting than creating believable human dialogue. Most videogame designers shy away from the challenge altogether, throwing their energy into crafting elaborate virtual landscapes—perfectly rendered castles, galaxies swirling with three-dimensional orbs, basketball hoops that resound with a gratifying swoosh. But the characters in these worlds remain primitive and flat, as Jonathan Rauch discovered when he played a widely acclaimed videogame called The Elder Scrolls IV: Oblivion:

The game enfolded me in lush, cinematic landscapes…. Looking down I saw grass rendered in granular detail; looking up I saw skies swept with feathery clouds…. The illusion was magical. But then it would all collapse. Approaching one of the characters, I would click for dialogue. The character would give a little canned speech introducing itself. In response to another click, it would mouth several bits of prerecorded dialogue. State-of-the-art games render action and environment with eerie realism and genuine aesthetic distinction. But their characters are dolls, not people.

When Rauch sat down to play Façade, he experienced something entirely new. Grace and Trip welcomed him into their apartment, and the three of them immediately began to chat. The couple asked subtle questions about relationships and peered into his face, waiting for a response. When Rauch became flirtatious, Trip threw him out of the apartment. When Rauch helped Grace understand her frustrated artistic ambitions, the pair thanked him for coming over. “You—I think you helped us,” Trip told him in a reflective voice. Leaving the imaginary apartment, Rauch felt strangely moved by his role in their reconciliation.

Façade, which is available for free on the Web, is an early prototype of a larger project that Mateas and Stern call “The Party.” Instead of two computer-generated characters, The Party will have ten. The player will be the co-host of a dinner party, and the plot will be rife with drama and intrigue. Characters will lure one another into illicit business deals and ill-advised romances. There will be sex and violence, but the driving force will be dialogue. As with Façade, the characters will “think” by virtue of an invented language called ABL (shorthand for “A Behavior Language”). As the dialogue swings back and forth, another Mateas-Stern technology called a “drama manager” will ratchet up the tension and steer the storyline toward its ultimate denouement.

Within the videogame industry, Mateas and Stern’s project is generally greeted with skeptical curiosity. One designer with the videogame megafirm Electronic Arts told Rauch that Façade “kind of works…. But then you try the next step and bam! You hit a wall and the wrong thing happens.” Rauch acknowledges that the game is riddled with kinks, and wonders whether Mateas and Stern will ever achieve their ultimate goal of transforming the computer into a de facto playwright and the animated characters into expert actors with the range and intelligence of real human beings.

Yet if Mateas and Stern are successful, or even close to successful, Rauch predicts that the potential of videogames will expand exponentially. He considers other recent innovations such as a game called Spore, which allows players to create and download complex non-human creatures. Combining these two innovations—realistic dialogue and vast creative capacity—Rauch imagines a game in which players will invent and download characters as rich and memorable as Holden Caulfield or Don Quixote. Instead of shooting at space invaders or hunting for hidden treasure, players could spend an afternoon exchanging witticisms with Sir John Falstaff or exploring Mr. Spock’s Vulcan civilization. The result, he believes, would be a world of open-ended drama that might surpass Aeschylus’s wildest imaginations.

Rauch, an Atlantic correspondent, is also a senior writer for National Journal and the author of several books, most recently Gay Marriage: Why It Is Good for Gays, Good for Straights, and Good for America. He lives in Washington, D.C., where he is a guest scholar at the Brookings Institution. We spoke by telephone on August 14.

Jennie Rothenberg

When Michael Mateas and Andrew Stern sat down to create their new game, you write that they resolved to put a “not” in front of every current video-game convention. What are some of those conventions?

Most serious, full-scale games are very large and take place in big alternative universes. Mateas and Stern decided to create a game you could play from beginning to end in twenty minutes. Most video games go from level to level but don’t have any dramatic shape—you just proceed upward as you get better at playing. Mateas and Stern inverted that by trying to create a compact experience with a true dramatic arc: a beginning, a middle, an end, and a revelation, in a sense, a dramatic release. Most games are about manipulating objects: slaying creatures, firing bullets, solving puzzles. Mateas and Stern decided they would make a game about people and characters. In a standard video game, it’s very easy to kill someone but virtually impossible to talk to them. In their game, they decided it would be the other way around.

The point of most video games is to take you somewhere you normally couldn’t go—to the floor of Madison Square Garden, playing against Kobe Bryant, or on a quest through an imaginary Tolkienesque kingdom. But what’s the point of a game where you visit ordinary people and talk about the same things you might talk about with your friends?

You could ask the same question about many movies or plays. Why would you want to go see a form of entertainment where a lot of what happens is dialogue? We take part in dialogue all the time every day. The answer, of course, is that humans have an insatiable appetite for characters, stories, and plots. Those are areas where traditional video games tend to come up short.

The idea here, presumably, is to be able to teach a computer the fundamentals of reading, writing, animation, and story design so that it can give you a more compelling artistic experience than when you’re sitting passively in a movie theater. The vision these guys have is to take all the drama of a movie or a play and add to it the interactivity of a video game, so you’re the one making stuff happen. Imagine visiting Victorian London and being part of a Sherlock Holmes mystery. It gets pretty exciting when you think about it that way.

The “drama manager” Mateas and Stern created sounds very complex. It can read subtle cues from typed-in words and figure out how to use them to move the story along. Can you tell me a bit more about the technology behind this? How, for instance, does a character know you’re angry? Do certain words tip her off?

Façade has several layers of new technology going on. It has language recognition. It doesn’t have true language understanding yet, but if you type in a sentence, the computer will try to get a sense of whether it’s a hostile statement or a friendly statement and which character it’s directed at. Then it tries to use that statement. The way it does that is through the drama manager, a kind of artificial intelligence that stands back and looks at what’s going on at any given moment. Then it says, “All right. Given what’s happened so far, how can I shape this into a drama? Now’s the time to ratchet up the tension, now’s the time for the revelation, now’s the time for the conclusion.”

Have Mateas and Stern programmed in every possible situation they can imagine? Does the drama manager just say, “Okay, he’s getting angry, and I know where to go from here”?

That turns out to be very difficult; there’s no way you can script every possible scenario in advance. You’d have to write an enormous game with terabytes of data to encompass everything a player might say or everything a character might do.

This gets to the third level of artificial intelligence they’ve got going on. They’ve preprogrammed some basic dramatic possibilities. But these possibilities are like beads, and the drama manager sequences the beads. It works with a language called ABL, which stands for “A Behavior Language.” This gives the drama manager a way to control the character without knowing everything in advance. It will let the character say something while also walking and mixing or holding a drink at the same time.

So ABL and the drama manager, working together, can animate the characters on the fly to react to what you’ve said in real time, and then sequence the dramatic possibilities—which have been supplied in the game—to create a compelling response.

Mateas and Stern chose to model their game after dramatic works like Who’s Afraid of Virginia Woolf? and Scenes from a Marriage. What is it about these stories that lends itself to the technology they had in mind?

First of all, they both liked all of those works of art. Second, they needed a contained dramatic environment. There’s no way you could start a project like this with multiple scenes and many characters. An intense marital argument is a very good, natural way to create an intense drama in a short space of time with only two characters in one room. For technical reasons, that was a very good place to begin.

There’s also the fact that you’re walking into a drama that’s already taking place without you, so your role is secondary. Would it be harder to create a game where the player was one of the two people having the marital argument?

I’m sure it would be. That’s a very astute observation. The game comes pre-supplied with dramatic tension. It’s not like you’re walking into any old room where there are two people there and they say, “Hi, how are you?” and you say, “Hi, how are you?” and you just have to hope something interesting happens. The moment you walk in the door—in fact, before you walk in the door, from outside—you can already hear them arguing. The whole world of Façade is pregnant with dramatic possibility. And the player’s role is to catalyze that.

One of the main criticisms of Façade is that it’s intriguing, it’s sometimes touching, but it’s not really fun. And that’s true. But people have to remember that this is a research project. If somebody figures out how to teach a computer to create a dramatic world, create realistic characters, and allow people to interact with them, the limits of using that in ways that are fun are mind-boggling.

Stern feels that video games should span a wider range of genres—that there should be not only action games but also drama games and comedy games. Do Stern and Mateas plan to create a “comedy manager” in the near future?

Their next game, The Party, is meant to be a comic melodrama. The goal is to set up complicated, farcical situations in which improbable things can happen. It doesn’t involve a comedy manager, but it does involve a lunatic, Desperate Housewives kind of story.

But humor is much subtler than drama. You can speak a language fluently and still not fully understand the nuances of its jokes. It’s hard to imagine a computer spinning out comedy in any effective way.

That could be true. A great deal of the interest in this project lies in exactly that. How difficult will it be to teach a computer to write a comedy or a mystery or a tragedy or a love story? If Mateas and Stern succeed, we’ll learn a lot about people and about stories. Even if they fail, we’ll learn something.

Could this technology be applied in other ways? For instance, could an online dating service use it to screen a user’s tastes and personality? Could a therapist use it to teach clients how to interact with different kinds of people?

If I recall, Mateas told me that before he went back to grad school, what got him involved in games again was working on training games for corporations—games that simulated environments employees would find themselves in, dealing with each other in a company. So there are obvious applications for games that can create and animate powerful characters.

Your article describes a second game called Spore, which takes a totally different approach than Façade. Spore doesn’t use dialogue or human emotions, but it allows players to create sophisticated creatures and universes. Why did you feel this game was relevant to an article about emotional, drama-centered video games?

I thought Spore was interesting and important. Although it’s very, very different from what Mateas and Stern are trying to do, it’s another effort to break out of the box that video games are currently in. It does that by allowing players to do something much richer than shooting at things and finding their way through levels. It uses computer intelligence to give the player immense power to design creatures. The computer animates them and then animates a whole universe.

The concept behind Spore is interestingly complementary to Façade, but it’s almost the opposite in some respects. Façade and The Party are meant to be short games and to have a compact dramatic shape. Spore is a massive universe that you’ll be able to explore forever. The game actually downloads other people’s creatures and planets for you. Based on the types of creations you’re coming up with, it decides you’ll probably like certain other ones that are floating around out there. Spore is basically a public creation, which is a breathtaking concept. It’s very open ended.

What interested me about these two games is that if you extend each of them outward by a factor of ten to 100—figure fifteen to twenty-five years out—and see where they intersect, you’ll find a game that gives players immense power to create characters and personalities and put them in planets or worlds that are pregnant with dramatic possibility. Based on whether you like comedy or mystery or romance, the computer could then create episodes for you and around you in these worlds. You could go back in time or into the future. If you add these two games together, you get this almost unlimited capacity to create characters and dramas. You have a player who is much more of a true creator and controller than anything we’ve seen in entertainment so far.

Are you envisioning a game where players design complex human beings the way Spore players invent creatures now? So you could say, “I want this man to be twenty-eight years-old, just out of graduate school, in a grumpy mood,” and then let the drama take on a life of its own?

You could do that, or you could go far beyond that. Most people probably won’t be great at inventing characters, but some people will. Every so often, somebody will create a character that takes off on the Internet, and people everywhere will download it into their own games and their own worlds. People might buy and sell characters on eBay. You’d still have off-the-shelf stuff by brilliant game designers. But you could wind up with a new culture that centers on these extraordinary characters, the equivalent of Sherlock Holmes or Mr. Spock or Harry Potter or what have you. If you use your imagination, there’s almost no limit.

If Mateas and Stern are able to achieve their ultimate goal, what will be the implications? There are plenty of humans who aren’t able to read social cues properly, let alone write a brilliant screenplay. If a computer can do all of these things better than we can, will it alter the way we think of our own intelligence—not to mention theater and literature?

Yeah. And it’s a long way from full realization. As people will see if they play Façade, it really is a very rough prototype. There are only two characters, and they really don’t understand most of what you say. Their intelligence is extremely rudimentary. It’s far above what you see in most ordinary commercial video games, but they still won’t get it if you announce that you’re pregnant, for example. When I did that, the characters thought I was flirting. They did not understand pregnancy. So they’re still extremely limited, and they’re still not very intelligent people.

But the thing to realize is that they’re the very first cut by two guys working entirely on their own. We might one day look back at Façade the same way we look back at Space Invaders or Asteroids today. We look at those games and say, “Wow, that was genius in its day, but is it ever primitive compared to what we’re doing now graphically!” These guys are hoping that Façade will, in the not-too-distant future, look like the very beginnings of something far more powerful.

It would be nice to think that a game like this could actually enrich real-life relationships rather than supplant them with artificial intelligence. Do you think it’s possible to learn more about human nature from playing a game like Façade or The Party?

That’s exactly what the creators say they’re going for. The ideal would be to sit down and play a game where, in a reasonable span of time, you’d potentially have a very powerful experience that would change your feelings about yourself and the world. We know Shakespeare could do that. Even a lot of lesser playwrights have been able to do that. When a play or a film transforms you in that way, you don’t forget the experience. And there’s no reason a game can’t do it as well. But it will raise the catharsis to an even higher level of intensity—because you’ll be part of the story.