Google's New Algorithm Incorporates Human Feedback About Quality Sites

Tucked inside Steven Levy's interview with Google engineers Amit Singhal and Matt Cutts, there's a fascinating detail about how the company recently improved its search algorithm.

They used human feedback.

Here's how they described their methodology to Levy in the Wired piece.

Singhal: We wanted to keep it strictly scientific, so we used our standard evaluation system that we've developed, where we basically sent out documents to outside testers. Then we asked the raters questions like: "Would you be comfortable giving this site your credit card? Would you be comfortable giving medicine prescribed by this site to your kids?"
Cutts: There was an engineer who came up with a rigorous set of questions, everything from. "Do you consider this site to be authoritative? Would it be okay if this was in a magazine? Does this site have excessive ads?" Questions along those lines.
Singhal: And based on that, we basically formed some definition of what could be considered low quality.

So, to simplify. They asked human beings, "Is this site high or low quality?" Then looked for patterns in their data that could be used to identify the sites that humans rated poorly.

Now, that last operation -- "basically [forming] some definition of what could be considered low quality" -- seems like the tough part, and we don't get any details about it. But it does answer a key question I've had. How did they pick out low quality sites? Simple! They asked some people. (The future is now.)

Presented by

Google Street View, Transformed Into a Tiny Planet

A 360-degree tour of our world, made entirely from Google's panoramas

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus

Video

Google Street View, Transformed Into a Tiny Planet

A 360-degree tour of our world, made entirely from Google's panoramas

Video

The 86-Year-Old Farmer Who Won't Quit

A filmmaker returns to his hometown to profile the patriarch of a family farm

Video

Riding Unicycles in a Cave

"If you fall down and break your leg, there's no way out."

Video

Carrot: A Pitch-Perfect Satire of Tech

"It's not just a vegetable. It's what a vegetable should be."

Video

The Benefits of Living Alone on a Mountain

"You really have to love solitary time by yourself."

More in Technology

Just In