Rather, private insurers have costs that Medicare doesn't have within the agency. Private insurers bill. Medicare does too, but the IRS has its own budget--hell, its own courts--which don't show up on Medicare's balance sheet. Private insurers negotiate with suppliers. Medicare does too, but most of the negotiation takes place between lobbyists and Congressmen who again, do not show up on Medicare's balance sheet. The Federal government has all sorts of these little items which relieve government agencies of reporting certain costs. But the costs remain.
My guess would be that these explicit costs are still lower than Medicare's. But then there are implicit costs to government fiat that markets don't have. As Tyler Cowen points out, taxation has deadweight losses, and Medicare is a tax on employment, which is something we are particularly anxious not to suppress right now.
The final point is that while people commonly think of administrative costs as "wasted", in fact, they are an important part of the market system. As Alex Tabarrok points out, and I have myself from time to time, many of the arguments in favor of national health care are literally socialist. And no, I am not using that term to apply to "anyone who is in favor of redistribution" or "government programs". But consider the following common arguments:
1. National health care will be cheaper because we will reduce administrative overhead
2. National health care will reduce wasteful competition in the form of me-too drugs
3. National health care will reduce wasteful competition in the form of advertising and other marketing expenses
4. National health care will allow us to rationally distribute care to where it does the most good rather than the current messy, wasteful hodge-podge
5. National health care will use resources for production instead of profits
6. National health care will achieve economies of scale in purchasing and record-keeping
7. People will not overuse free goods because there are hard limits to desired consumption
These were all arguments advanced in favor of socialism. Contrary to popular conservative belief, socialists were not unfamilier with either the incentive problems of communism (people will not work hard if there's no benefit to doing so) or the Hayekian argument about the value of prices, aka the Socialist Calculation Problem. Rather, smart socialists thought that they could overcome these problems with a combination of status competitions (Hero of the Soviet Union, Second Class) and massive efficiencies gained by wringing all that fragmented, wasteful competition out of the system. Economists who would be ashamed to make these sorts of arguments about Proctor and Gamble or the used car market suddenly start parroting these things as if they hadn't been thoroughly discredited by the last seventy years.