Yes, Virginia, markets are efficient
There are a lot of students who walk through the doors of the University of Chicago's Graduate School of Business ready to contest the validity of the Efficient Markets Hypothesis. Few, however, manage to walk out without having "drunk the Kool-aid". It turns out to be very, very hard to maintain the belief that you can foretell the future against the pointed questioning of some of the world's smartest economists.
I was one of the Kool-Aid drinkers. I then went and spent some time living with a money manager who was an acolyte of Warren Buffett. I greeted more than one sunrise during our heated games of "dueling regressions". (This story is dedicated to Matt Zeitlin, who previously held the title for nerdiest blog confession.)
Now ever since Michael Lewis' article on index fund management came out, I have been experiencing something perilously close to PTSD. Some would disagree, but I think it's a pretty good article. It is, to be sure, grossly simplistic. But as any financial journalist will tell you, it is very hard to get a 15,000 word treatise on the EMH, with math, past an editor. And if you do, no one will read it.
Lewis completely glosses over distinctions between various forms of the efficient markets hypothesis, bizarrely simplifies arguments about the various premia on asset classes (those looking for a solution to this riddle might start with the word "liquidity"), and tells a suspiciously pat morality tale about a stock-jammer-turned-sainted-investment-advisor. But he gets the big thing right. The world would be a better place if we all took home the point of his sermon:
You can't beat the market. YOU can't beat the market. YOU CAN'T BEAT THE MARKET.
It doesn't matter which version of the EMH is correct. It doesn't matter if the behavioral finance guys are correct. You--adorable, clever, hardworking little you--are mathematically just as likely to underperform the market as outperform it. You would do better to go to Vegas and sit down at the $25 blackjack table with a firm resolve to walk away as soon as probability has varied a few hundred dollars in your favor.
And the guy you're paying to manage your money? Same deal. Statistically, in fact, he will give you a lower return than a broad market index, because of his salary and trading fees.
Hedge fund managers may make money. But it's a pretty safe bet that they do it in one of two ways: they develop systems that let them trade on information faster than everyone else, so that they can make money in the fraction of a second before the price changes; or their longer lock-ups allow them to trade in asset classes where markets are less efficient than large-cap stocks. But the experience of mutual funds over long periods of time indicates that everyone who thought they were a genius turned out to be lucky, not smart; the current spate of meltdowns seems to be sending the same message about hedge funds. And when there are detectable algorithms that let you make excess returns, they tend to get detected, whereupon they no longer work.
The stupidest thing you can possibly do with your money is what most Americans do: either buy and sell individual stocks, or carefully analyze the list of mutual funds to see what the "best" (i.e. highest return) ones are, and plow all your money into them. You see the fine print at the bottom that says "past returns no guarantee of future results"? They mean it. In fact, those guys are not only not particularly likely to end up in the top funds next year; they may actually be destined to underperform. That's because the manager has a "strategy" and whatever that strategy was worked well last year. Since it is no longer last year, that strategy is likely to be counterproductive.
You can't beat the market, and neither can the jerk on the phone trying to sell you stocks. Put your money in the broadest possible index fund (being young and having no children, I'm all equity with a 70/30 split between domestic and international; your mileage may vary). Then leave it there. Don't even peek. Throw the statements away unopened. Rebalance once a year to keep your money at your target allocation, and otherwise don't think about it. If you want the thrill of gambling, go to Vegas. At least they'll give you free drinks.