Zillow's Got Cool Tech and Hot Stock, But Does It Actually Work?


Zillow, the home price voyeurism site, has had a remarkably hot debut on NASDAQ. Priced at $20 per share yesterday, the stock popped to more than $40 in the company's first day of trading.

The company sells subscription services to more than 10 million local real estate people. That brought in $13 million in 2010. Throw in $17 million selling ads on the website in that year and you've got a decent business. Particularly given that the subscription sales are booming right now. The company was even almost profitable last year, posting just a $6.8 million operating loss.

The whole company is built atop what they call a database of real estate information that gets crunched by algorithms to come up with estimates ("Zestimates") for home prices among other things. Want to know if your house is worth more than the Jones's, even if neither of them is up for sale? Just ask Zillow.

"Our algorithms will automatically generate a new set of valuation rules based on the constantly changing universe of data included in our database," the company says. "This allows us to provide timely home value information on a massive scale. Three times a week, we create more than 1.2 million unique valuation models, built atop 3.2 terabytes of data, to generate current Zestimates on nearly 100 million U.S. homes."

Mmmm... Delicious Big Data! But I've got a nagging suspicion that Zillow's numbers -- for all their algorithmic intelligence -- aren't quite as accurate as they'd like us to think. The company flags the issue in its IPO registration.

We may in the future be subject to disputes regarding the accuracy of our Zestimates and Rent Zestimates.

We provide our users with Zestimate and Rent Zestimate home and rental valuations. A Zestimate is our estimated current market value of a home based on proprietary automated valuation models that apply advanced algorithms to analyze our data; it is not an appraisal. A Rent Zestimate is our estimated current monthly rental price of a home, using similar automated valuation models that we have designed to address the unique attributes of rental homes. Revisions to our automated valuation models, or the algorithms that underlie them, may cause certain Zestimates or Rent Zestimates to vary from our expectations for those Zestimates or Rent Zestimates. In addition, from time to time, users disagree with our Zestimates and Rent Zestimates. Any such variation in Zestimates or Rent Zestimates or disagreements could result in distraction from our business or potentially harm our reputation and could result in legal disputes.

But that's really the only mention of this possible problem. A group at the University of Arkansas has tried to assess how good Zillow's information is over the past couple of years. First, they compared Zillow's data to the data from MLS, another real estate site, and found "a large discrepancy in information between the two." More recently, they looked at if data contributed by Zillow users improved things. They found that home entries got more complete but "the accuracy of Zillow's edited facts is not high."

On a purely anecdotal level, it seems like every time I talk with a homeowner (which I'm not) about what Zillow says about her property, she launches into some discussion about how the site gets a bunch of stuff wrong about her home. I'm curious: how would you rate the quality of the information in Zillow's "living database" about your home?

Update, 12:53 p.m.: Zillow keeps their own records of Zestimate accuracy, too, comparing their estimates with actual sale prices all the way down to the county level. It turns out that how close Zillow usually gets to the sale price depends a lot on where you live. In Denver, there's almost a 90 percent change that an actual sale price is within 20 percent of the Zestimate. At the other end of the spectrum is Dallas-Ft. Worth, where there is only a 71 percent chance that the sale price falls within that band around the Zestimate. In any case, good to see Zillow publicizing these numbers.

Jump to comments
Presented by

Alexis C. Madrigal

Alexis Madrigal is the deputy editor of TheAtlantic.com. He's the author of Powering the Dream: The History and Promise of Green Technology. More

The New York Observer has called Madrigal "for all intents and purposes, the perfect modern reporter." He co-founded Longshot magazine, a high-speed media experiment that garnered attention from The New York Times, The Wall Street Journal, and the BBC. While at Wired.com, he built Wired Science into one of the most popular blogs in the world. The site was nominated for best magazine blog by the MPA and best science website in the 2009 Webby Awards. He also co-founded Haiti ReWired, a groundbreaking community dedicated to the discussion of technology, infrastructure, and the future of Haiti.

He's spoken at Stanford, CalTech, Berkeley, SXSW, E3, and the National Renewable Energy Laboratory, and his writing was anthologized in Best Technology Writing 2010 (Yale University Press).

Madrigal is a visiting scholar at the University of California at Berkeley's Office for the History of Science and Technology. Born in Mexico City, he grew up in the exurbs north of Portland, Oregon, and now lives in Oakland.

Get Today's Top Stories in Your Inbox (preview)

What Is a City?

Cities are like nothing else on Earth.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus


CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity


Is Technology Making Us Better Storytellers?

The minds behind House of Cards and The Moth weigh in.


A Short Film That Skewers Hollywood

A studio executive concocts an animated blockbuster. Who cares about the story?


In Online Dating, Everyone's a Little Bit Racist

The co-founder of OKCupid shares findings from his analysis of millions of users' data.


What Is a Sandwich?

We're overthinking sandwiches, so you don't have to.


Let's Talk About Not Smoking

Why does smoking maintain its allure? James Hamblin seeks the wisdom of a cool person.



More in Technology

Just In