Should We Blame Congress or the White House for Deficits?

Barry Ritholtz asks an interesting question: since Congress controls the budget and presidents set the agenda, can we say that one is more responsible for our deficits? He doesn't answer the question so much as tease readers with these two really interesting graphs that plot our deficits next to each party's control of the White House or Congress. So who can we blame?

Before we get to the conclusions, here are the graphs. First we've got deficits plotted against each party's control of Congress:
And here's what it looks like under each president:
As a commenter pointed out, this seems to me like a pretty good visualization of the Unitary Executive theory. If you try to make sense of the first graph in a vacuum, you risk a serious headache. In the second graph, the line segments between each president's term tell a pretty good story. Consistent deficits under Reagan and Bush dip with the 1991 recession. The Clinton years are a rare positive line that collapses in the first few years of the Bush administration, and Obama has both inherited and piled on to a steep down-trending line.

The other point that strikes me is how consistent the orange line -- indicating government expenditures -- is in the top part of each graph. There is a lot of noise made about the spending habits of particular presidents, and indeed there should be, because government spending reflects the values of our electeds. But it's strange that from a 30,000 foot view, the rise of government spending looks like such a perfectly straight line.