The Four Percent Fallacy
Sure the defense budget is large, the saying goes, but in percentage of GDP terms it's lower than it has been for much of the 20th century, so what's the problem? Cato's Benjamin Friedman has a good response to this line of argument:
Percentage of GDP is useful for historical comparisons of defense’s economic burden. Carafano substitutes the question of what we can afford for what we ought to spend. The United States can afford to spend four percent of its GDP on defense; indeed we can afford to spend far more. That doesn’t mean we should. Whatever your politics, money spent on defense means money not spent on something else: private investment, deficit reduction, infrastructure, a car, etc. The problem is opportunity cost, not economic malaise.
And, indeed, there you have it. We could spend much more on the Pentagon if the objective circumstances merited doing so. But they don't. The opportunity costs are large, the need lacking, and the benefits of ever-growing military spending are small compared to the benefits of spending that money on productive investments (both private and public sector) or consumption goods.