This week, Google admitted that its infamous brainteasers -- e.g.: "How much should you charge to wash all the windows in Seattle?" -- are awful at predicting who will be a good employee.
Your reaction might be: um, FINALLY. And, sure, thinking about windows-per-housing-unit isn't the most direct way to assess engineering skill or creativity. But Google's flawed strategy was the answer to another brainteaser: What's the best way to hire great employees, anyway? People are complicated, organizations are complicated, matching people and organizations is complicated, and it's extremely difficult to predict who will be brilliant and who will be a bust.
"Years ago, we did a study to determine whether anyone at Google is particularly good at hiring," Laszlo Bock, Google's senior vice president for people operations, told LinkedIn. "We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship. It's a complete random mess."
A complete random mess.
Brainteasers didn't matter. Colleges didn't matter. GPAs? Yeah, those didn't matter, either. The ability to hire well turned out to be utterly random for everybody. Except for one guy, Bock said, "who was highly predictive because he only interviewed people for a very specialized area, where he happened to be the world's leading expert."
This "specialist" exception is as interesting to me as the "complete random mess" rule. To see why, let's talk about the NFL and the NBA.
Picking football stars is hard. First-rounders tend to be better than second-rounders, but superstars are quirky and elusive. A football draft study this year from the Worcester Polytechnic Institute found that second-round picks have nearly the same production as first-round picks at less than half the price. Indeed, the best football players aren't often the first few selected. Sometimes, they're not even the first 100 picked. Tom Brady was a 6th rounder. So was Redskins running back Alfred Morris. Some of the best defensive players are big surprises.
Predicting the future of professional athletes is awfully hard, but then again, in basketball, the best players are considerably more predictable. The stars of the NBA Finals -- Tim Duncan and LeBron James -- both went 1st. Most of the games superstars were among the first five players selected in their draft.
Why are basketball stars so much easier to predict than football stars? For starters, isolating the metrics that will determine future talent is particularly hard in a complex, team-oriented sport like football. One reader, A_Lee, responded to my post persuasively:
The obvious reason is that football requires much more teamwork and unit cohesion than basketball, and the rules of each game make it so. The second reason is much smaller sample size. In football, the success or failure of an individual player is far more dependent on the production of the entire team or unit ... It can be difficult to accurately assess the skill of an individual player, because his performance is greatly affected by the performance of the people around him.
So, what in the world does this have to do with Google?
Well, when you're hiring for a small, specialized unit, the metrics and variables that will determine the person's success are clearer and more finite. As in basketball, perhaps it's easier to isolate the skills that will determine a star at the next level.
But Google isn't like basketball. It's not a bunch of sales people running "iso" plays against their clients. It's a huge, complex organization where many employees are asked to juggle multiple accounts and manage busy and complicated relationships with other companies and colleagues. It's more like football, where the metrics of future success are considerably murkier and evidence of true talent might not emerge until the employee is on the right team.
Maybe that sounds like a totally facile connection. But it's almost exactly what Bock told LinkedIn. "After two or three years, your ability to perform at Google is completely unrelated to how you performed when you were in school, because the skills you required in college are very different," Bock said. "You're also fundamentally a different person. You learn and grow, you think about things differently. [We] want people who like figuring out stuff where there is no obvious answer." Perhaps that's why drafting at Google has turned out to be so devilishly difficult. You don't know what kind of Googlers you've got until they're already inside Google.