The 3D Future of Your Smartphone Camera

Which, if nothing else, is good for robots.

A matte black robotic camera the size of a 1980s lunchbox sits atop a tripod. On its face, there are three 2D cameras and three 3D sensors. There's a handle on one end sprouting from the left side of the device.

Its handler hits a button on an iPad app, and it begins to move. Over the next 24 seconds, it will make six stops as it spins 360 degrees. The imagery data will be sent to the a room full of servers, turned from a point cloud into a textured 3D mesh, and returned to the tablet.

After a single scan, the basic features of the room are already visible on the screen. We move the camera to another spot, and hit record again. More details fill in. 

Matterport iPad software (Alexis Madrigal)

After four to six more scans and twenty minutes in the cloud, the system will have created a textured, photorealistic 3D model of the room. It's dimensionally accurate, too, so you could, for example, measure the ceiling height or the distance between walls. For gestalt realism, I'd put the 3D models it creates somewhere in-between a Playstation 2 game and a Playstation 3 game. 

All in, it might take half an hour or an hour to turn a physical space into a photorealistic virtual space. Doing something like this in the past would have taken orders of magnitude longer.

This is the Matterport system, and it debuts today at $4,500, or about the cost of a professional digital camera like the Canon 5D Mark III and a nice lens. The necessary access to the cloud platform runs an extra $50-$150 a month, depending on usage.

This is a tool for professionals. But so were the original lug-about laptops or those cell phones shaped like bricks. In the next few years, Matterport's CEO Bill Brown told me, every picture you take will have a three-dimensional component. 

"The company started with the long-term vision that there is going to be a 3D sensor in everyone's pocket," Brown said. "We know that we're going to get there ... We think it's as big as the transition from film to digital."

Matterport CEO Bill Brown (Alexis Madrigal)

Brown's been working on 3D imaging since his days at Canesta, which created the technology that eventually came to power the latest-generation of Microsoft's Kinect controller

Along with the co-founders of Matterport, Matt Bell and Dave Gausebeck, he's obsessed with the idea that you'll have a 3D camera in your mobile device soon. 

They've even had a chance to play with Google's early incarnation of that vision, the mysterious Project Tango, which purports to create simple 3D maps of a space with sensors embedded in the phone. "These sensors allow the phone to make over a quarter million 3D measurements every second, updating its position and orientation in real-time, combining that data into a single 3D model of the space around you," the project site says.

Because Matterport's system is modular, they were able to connect the Tango into their cloud system and capture a decent model of Gausebeck's garage using just the phone. 

"Google has pushed the envelope on the technology faster than people thought was possible," Brown said. He pointed out that Intel, too, has been investing in 3D motion sensing, and said that he thought a company would announce a mobile device with 3D imaging built in before the end of the year.

For Brown, it's obvious that people would want 3D imaging built into their phones.

"There is no reason that anyone would take a picture that is just 2D. When you take a shot, you'll capture the 3D data. If nothing else, your 2D shots will be better," he said. "Today you walk and take a video. If you were capturing the 3D data, you're able to reconstruct anything you've walked through. You've created an environment by putting all of the 3D underneath the 2D data." 

I will admit that it still feels strange to me. I'm not sure what I would do with a library of 3D models of the places I've been. But the ability to do such a thing in the next few years with a consumer device still floors me. 

For now, Brown sees three groups of people who might be interested in Matterport's pro offering: The documenters, the promoters, and the visualizers.

Documenters might include people in the construction industry, architects, or insurance claims adjusters. These people need accurate measurements and records of spaces. And that's what Matterport can provide. 

Promoters are an obvious market, too. We're talking realtors or people renting spaces (AirBnb hosts, say). This is already a market that's served by the kinds of professional photographers who might buy the Matterport camera. 

Visualizers are the most heterogeneous group. It includes anyone who wants to play with a space for some reason, everyone from a forensic analyst to an interior designer who wants to show clients how their house might change with an accent wall. 

Admittedly, these are all niche applications. But Brown believes that only by getting out into markets can they really understand the problems and opportunities for 3D scanning. The cost and size of 3D imaging technology are both shrinking and therefore opening up new opportunities, but what those opportunities might be for regular consumers remains unclear. 

For companies like Google, though, the stakes are clear. 3D sensing gives devices—or robots—"a human-scale understanding of space and motion." Our world is 3D, and they want machines to be able to see it like we do. 

Jump to comments
Presented by

Alexis C. Madrigal

Alexis Madrigal is the deputy editor of He's the author of Powering the Dream: The History and Promise of Green Technology. More

The New York Observer has called Madrigal "for all intents and purposes, the perfect modern reporter." He co-founded Longshot magazine, a high-speed media experiment that garnered attention from The New York Times, The Wall Street Journal, and the BBC. While at, he built Wired Science into one of the most popular blogs in the world. The site was nominated for best magazine blog by the MPA and best science website in the 2009 Webby Awards. He also co-founded Haiti ReWired, a groundbreaking community dedicated to the discussion of technology, infrastructure, and the future of Haiti.

He's spoken at Stanford, CalTech, Berkeley, SXSW, E3, and the National Renewable Energy Laboratory, and his writing was anthologized in Best Technology Writing 2010 (Yale University Press).

Madrigal is a visiting scholar at the University of California at Berkeley's Office for the History of Science and Technology. Born in Mexico City, he grew up in the exurbs north of Portland, Oregon, and now lives in Oakland.

Get Today's Top Stories in Your Inbox (preview)

CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus


CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity


Is Technology Making Us Better Storytellers?

The minds behind House of Cards and The Moth weigh in.


A Short Film That Skewers Hollywood

A studio executive concocts an animated blockbuster. Who cares about the story?


In Online Dating, Everyone's a Little Bit Racist

The co-founder of OKCupid shares findings from his analysis of millions of users' data.


What Is a Sandwich?

We're overthinking sandwiches, so you don't have to.


Let's Talk About Not Smoking

Why does smoking maintain its allure? James Hamblin seeks the wisdom of a cool person.



More in Technology

Just In