The World Is Not Enough: Google and the Future of Augmented Reality



A book in The Future.

It is The Future. You wake up at dawn and fumble on the bedstand for your (Google) Glass. Peering out at the world through transparent screens, what do you see? 

If you pick up a book, do you see a biography of its author, an analysis of the chemical composition of its paper, or the share price for its publisher? Do you see a list of your friends who've read it or a selection of its best passages or a map of its locations or its resale price or nothing? The problem for Google's brains, as it is for all brains, is choosing where to focus attention and computational power. As a Google-structured augmented reality comes closer to becoming a product-service combination you can buy, the particulars of how it will actually merge the offline and online are starting to matter. 

To me, the hardware (transparent screens, cameras, batteries, etc) and software (machine vision, language recognition) are starting to look like the difficult but predictable parts. The wildcard is going to be the content. No one publishes a city, they publish a magazine or a book or a news site. If we've thought about our readers reading, we've imagined them at the breakfast table or curled up on the couch (always curled up! always on the couch!) or in office cubicles running out the clock. No one knows how to create words and pictures that are meant to be consumed out there in the world.

This is not a small problem.

* * *

I'm sitting with Google's former maps chief John Hanke in the company's San Francisco offices looking out at the Bay's islands and bridges, which feel close enough to touch. We're talking about Field Trip, the new Android app his 'internal startup' built, when he says something that I realize will be a major theme of my life for the next five or 10 years. Yours too, probably. 

But first, let me explain what Field Trip is. Field Trip is a geo-publishing tool that gently pushes information to you that its algorithms think you might be interested in. In the ideal use case, it works like this: I go down to the Apple Store on Fourth Street in Berkeley and as I get back to my car, I hear a ding. Looking at my phone, I see an entry from Atlas Obscura, which informs me that the East Bay Vivarium -- a reptilian wonderland that's part store, part zoo -- is a couple of blocks away. That sounds neat, so I walk over and stare at pythons and horned dragons for the next hour. Voila. "Seamless discovery," as Hanke calls it. 

Dozens of publishers are tagging their posts with geocodes that Field Trip can hoover up and send to users now. Hanke's team works on finding the right moment to insert that digital information into your physical situation. 

And when it works well, damn does it work well. 

It's only a slight exaggeration to say that Field Trip is invigorating. That is to say: It makes life more interesting. And since I switched back to my iPhone after a one-week Android/Field Trip test, it's the one thing that I really miss. 

At first, I was tempted to write off this effort as a gimmick, to say that Field Trip was a deconstructed guide book. But the app is Google's probe into the soft side of augmented reality. What the team behind it creates and discovers may become the basis of your daily reality in five or 10 years. And that brings me back to Hanke's comment, the one you could devote a career to. 

"You've got things like Google Glass coming. And one of the things with Field Trip was, if you had [Google Glass], what would it be good for?" Hanke said. "Part of the inspiration behind Field Trip was that we'd like to have that Terminator or Iron Man-style annotation in front of you, but what would you annotate?"

There's so much lurking in that word, "annotate." In essence, Hanke is saying: What parts of the digital world do you want to see appear in the physical world?

If a Field Trip notification popped up about John Hanke, it might tell you to look for the East Bay hipster with floppy hair almost falling over his eyes. He looks like a start-up guy, and admits to being one despite his eight years at Google. He refers to its cofounders like old college friends. ("Sergey was always big on, 'You should be able to blow stuff up' [in Google Earth].") Not a kid anymore, Hanke sold an early massively multiplayer online gaming company to the legendary Trip Hawkins in the '90s, then co-founded Keyhole, which became the seed from which Google's multi-thousand person map division grew. 

When maps got too big for Hanke's taste, he "ultimately talked with Larry" [Page], and figured out how to create an "autonomous unit" to play with the company's geodata to create novel, native mobile experiences. This is Google's Page-blessed skunkworks for working on this very specific problem. They are Google but they have license to be unGoogle.

"You don't want to show everything from Google Maps. You don't want to show every dry cleaner and 7-Eleven in a floating bubble," Hanke said. "I want to show that incremental information that you don't know. What would a really knowledgeable neighborhood friend tell you about the neighborhood you're moving through? He wouldn't say, 'That's a 7-Eleven. That's a fire hydrant.' He would say, 'Michael Mina is opening this new place here and they are going to do this crazy barbecue thing.' " 

Some companies, like Junaio, are working on augmented-reality apps that crowdsource location intelligence through Facebook Places and FourSquare checkins. Hold up your phone to the world and it can tell you where your friends have been. It's a cool app, and certainly worth trying out, but there isn't much value in each piece of information that you see. The information density of of that augmented reality is low, even if it is socially relevant. If you're opting into 24/7 AR through something like Glass, that cannot be the model.

* * * 

Google had previously offered up a vision of how Glass might be used in a video they released earlier this year to pretty much universal interest. 

But consider the cramped view of augmented reality you see here. What information is actually overlaid on the world? 

  • The weather
  • The time
  • An appointment
  • A text message
  • Directions
  • Interior directions (within a bookstore? Right.)
  • A location check on a friend
  • A check in

You can see why Google would put this particular vision out there. It's basically all the stuff they've already done repackaged into this new UI. Sure, there's a believable(ish) voice interface and a cute narrative and all that. But of all the information that could possibly be seamlessly transmitted to you from/about your environment, that's all we get? 

I'm willing to bet that people are going to demand a lot more from their augmented reality systems, and Hanke's team is a sign that Google might think so, too. His internal startup at Google is called Niantic Labs, and if you get that reference, you are a very particular kind of San Francisco nerd. The Niantic was a ship that came to California in 1849, got converted into a store, burned in a fire, and was buried in the city. Over the next hundred and twenty-five years, the ship kept getting rediscovered as buildings were built and rebuilt at its burial site. Artifacts from the ship now sit in museums, but a piece of the bow remains under a parking lot near the intersection of Clay and Sansome in downtown San Francisco. 

Jump to comments
Presented by

Alexis C. Madrigal

Alexis Madrigal is the deputy editor of He's the author of Powering the Dream: The History and Promise of Green Technology. More

The New York Observer has called Madrigal "for all intents and purposes, the perfect modern reporter." He co-founded Longshot magazine, a high-speed media experiment that garnered attention from The New York Times, The Wall Street Journal, and the BBC. While at, he built Wired Science into one of the most popular blogs in the world. The site was nominated for best magazine blog by the MPA and best science website in the 2009 Webby Awards. He also co-founded Haiti ReWired, a groundbreaking community dedicated to the discussion of technology, infrastructure, and the future of Haiti.

He's spoken at Stanford, CalTech, Berkeley, SXSW, E3, and the National Renewable Energy Laboratory, and his writing was anthologized in Best Technology Writing 2010 (Yale University Press).

Madrigal is a visiting scholar at the University of California at Berkeley's Office for the History of Science and Technology. Born in Mexico City, he grew up in the exurbs north of Portland, Oregon, and now lives in Oakland.

Get Today's Top Stories in Your Inbox (preview)

Why Do People Love Times Square?

A filmmaker asks New Yorkers and tourists about the allure of Broadway's iconic plaza

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus


Why Do People Love Times Square?

A filmmaker asks New Yorkers and tourists about the allure of Broadway's iconic plaza


A Time-Lapse of Alaska's Northern Lights

The beauty of aurora borealis, as seen from America's last frontier


What Do You Wish You Learned in College?

Ivy League academics reveal their undergrad regrets


Famous Movies, Reimagined

From Apocalypse Now to The Lord of the Rings, this clever video puts a new spin on Hollywood's greatest hits.


What Is a City?

Cities are like nothing else on Earth.



More in Technology

Just In