I'm afraid Andrew Jackson just isn't much of a citation for me, Stephen. I have too much of a fondness for central banks, and too much of an aversion against ethnic cleansing.
But let's look at the idea on the merits. Jackson argued, "Each public officer who takes an oath to support the Constitution swears that he will support it as he understands it, and not as it is understood by others ... The authority of the Supreme Court must not, therefore, be permitted to control the Congress or the Executive when acting in their legislative capacities, but to have only such influence as the force of their reasoning may deserve."
Under what rationale, then, were the Southern politicians wrong in refusing to recognize the validity of the Brown decision? Since any politician who loses a court case probably thought his position had some legal soundness, then why should they honor any decision at all under the sun? What does a court opinion actually decide, then?
What this amounts to is that everybody becomes their own final arbiter of the Constitution, a state of legal anarchy. And this is wrong for the very simple reason that the Constitution has already made the judiciary into that arbiter.
And it's the judiciary's job to be the final arbiter of constitutionality for a very simple reason: the politicians have too many motives and opportunities to lie and cheat about it. If the politicians can declare that any court decision they didn't like was a product of bad reasoning and then ignore it, how is the judiciary actually providing any check upon the political system?
The other branches place checks upon the judiciary, through appointing the judges and also by being able to amend the Constitution in an extreme circumstance. But under Jackson's idea, what power does the judiciary have for itself as the third branch of the government? How can any minority's rights be protected if the majority can summarily call the decision illegitimate?
What this all came down to in the end, of course, was the apocryphal reaction Jackson is said to have had when the Supreme Court ruled against his policies of mistreating the Native Americans, and even if he never said it, it nevertheless accurately describes what he and Van Buren did: "John Marshall made his decision let him enforce it."
As you concluded, "How do we decide which cases are like Brown and which cases are ones like Roe, which we all seem to agree decided an issue that should have been left to politics? I think that's a very difficult question."
Under Jackson's doctrine, why does that decision even matter once you've made it? Someone else has a different opinion, and we've just declared that the Supreme Court, or really any court, does not actually have any final jurisdiction in the matter.
There will always be decisions we don't agree with. But encouraging people to question or even deny the legal legitimacy of those decisions is simply not the answer. It only creates many more problems.