Why <i>shouldn't</i> we punish rich people who renounce their citizenship?

We're not the Soviet Union, that's why. A society should offer a good enough deal to all of its members, even the rich ones, that they do not want to leave. If it cannot do that, it should let them pursue happiness somewhere else.

If someone wants to renounce their citizenship for any reason, good riddance to them and don't let the doorknob hit them in the ass on the way out. You don't have a right to their property to fund your government, a right of which they are "depriving" you by leaving. Taxes are the bill one pays for membership in the society. If one declines membership, one doesn't owe the taxes. The princes of Saudi Arabia are very rich, and much of their money was earned selling products to Americans. Do they owe us taxes too?

That kind of punitive taxation is, like the draft, an attempt to compel a civic emotion which cannot be manufactured by any external power. Like the draft, it increases quantity at incalculable cost to quality. The American idea is that you are not an American because you were born here; you're an American because you want to be. If people do not want to be Americans, then we are better off without them.