I don't know what's in Ferguson's head, but I do have some understanding of how the process works for people who are paid to think deep thoughts.
Stupid stuff by Ferguson, who, as you note, is not stupid. I suspect this comes from the pressure of writing a regular column for a non-academic audience, and Ferguson trying to force a shot this week when there was nothing there. You would know more about this than me, but why can't a newspaper editor say something like:
"We're hiring you to write a regular column every X, but if for some reason you come up empty that week (or month, whatever), that's OK, just let us know. We'll publish one of our outside submissions instead and just not pay you for that column. We'd rather have you write less frequently but write consistently good columns than try to force it."
When you are a young writer, editors are often skeptical of you. They generally don't believe you, and they shouldn't. Not in q "you're lying" sort of way--though there is that. But in a, "Are you really capable of doing what it takes to make this thing great" sort of way. If you are lucky this takes the form of them giving you room to write short, medium and long, but kicking the crap out of you the whole way. If you're unlucky, it just involves a lot of kicking.
Should you stay at it a while, and build some sort of name, editors generally back off. Part of that is trust--you're a known quantity. But part of it is also a change in the relationship. A guy like Niall Ferguson is a brand in a way that, say, Ian Parker isn't. I love Parker and will now take this opportunity to link, yet again, to his gorgeous Alec Baldwin piece. I am sure there are people, like me, who read the New Yorker waiting to see what Parker is going to do next. But not the way people wait to hear what Ferguson has to say. For whatever reason, there's a bigger audience for opinion journalism than there is for narrative journalism. Once you're a brand, your relationship with your editor often changes. It's almost--almost--irrelevant whether you are right. Masses of people will flock to hear what you have to say about the world. Plus, a lot of writers
Lastly, once you become a brand, you feel the need to feed the beast. But the beast isn't natural. You don't have something important to say each and every week. And you certainly don't have something important to say each week, at 800-1000 words. Yet the demand is still there. Look, I'm a sheep-herder in this wonderful corner of the grand meadows of Bradley-land. But I regularly get requests to speak on issues that I know very little about. I can't imagine what it's like for a star Harvard professor.
There are people whose whole business revolves around Big Ideas. For those folks, there is immense pressure to talk--and talk a lot. But "Big Ideas" are hard, and some days--nay, most days--you've got nothing. And the next thing you know, you're telling the most powerful man in the world that what he really should be doing is paying more attention to a cartoon that no one watches, anymore. Tough business, eh?