Felix Salmon and Matt Yglesias are both upset that all private schools are considered charities for tax purposes. And rightly so. Not only is it questionable whether some of them contribute to the public good, they might (as Matt says) actually detract from it by drawing "parents with resources and social capital out of the public school system and contributing to its neglect."
The one thing I would add is this: If you agree with the Yglesias/Salmon logic (as I do), there is no reason to halt that logic at the doors of the private schoolhouse. There is, for example, my favorite hobby horse: The tax deduction governing charitable contributions, which can be claimed for donations to a ludicrously broad variety of organizations. I've said this before, but the relevant portion of the tax code let's you claim it for donating to a...
corporation, trust, or community chest, fund, or foundation [...] organized and operated exclusively for religious, charitable, scientific, literary, or educational purposes, or to foster national or international amateur sports competition (but only if no part of its activities involve the provision of athletic facilities or equipment), or for the prevention of cruelty to children or animals.
Which, to understate matter, covers a lot of organizations -- international amateur sports leagues? -- of questionable public value. (Now if only there were some powerful elected official interested in changing the rules governing charitable deductions...)
Anyway, the big questions here are "what is the public interest?" and "who gets to define it?" -- and they are both fairly intractable questions. What can be said? I'm of the opinion that a narrower definition would be a better expression of the public interest than the law that is currently on the books, and I think most people, when confronted with the current law, would agree. Or at least I hope so.