After House Speaker Paul Ryan reportedly told President Trump that Republicans lacked enough votes to pass the GOP health care bill, Republicans canceled a vote on the American Health Care Act on Friday, putting the president’s promise to repeal and replace Obamacare in jeopardy.

It’s the first major setback to the president’s agenda in Congress, but Republican voters are likely to hold Republican congressional leaders, rather than the president, responsible.

Ahead of the vote, Trump said Ryan should keep his job as House speaker even if the vote was unsuccessful. The president also told Robert Costa of The Washington Post that he doesn’t “blame Paul,” in the immediate aftermath of the news that the vote had been canceled. But the White House has reportedly been gearing up to point the finger at Ryan if anything went wrong. “Behind the scenes, the president’s aides are planning to blame Ryan if there is an embarrassing defeat on a bill that has been a Republican goal for more than seven years,” Bloomberg reported earlier in the day, citing an unnamed administration official.

And if Trump and Ryan clash as a result of the outcome, Republican voters may side with the president. In February, the Pew Research Center found that a majority of Republican voters were more likely to trust Trump than Republican congressional leaders in the event of a dispute. The survey also reported that Republican voters had a far more favorable view of Trump than they did of Ryan or Senate Majority Leader Mitch McConnell. An overwhelming majority of Republican voters—at 86 percent—had a favorable view of Trump, compared to just 65 percent who held a favorable view of Ryan and just 57 percent who felt warmly toward McConnell. That suggests that Republican congressional leaders could face retribution from their base if they end up on a collision course with the president.

Ryan himself has suggested there may be retribution from voters if the the GOP cannot pass an Obamacare replacement. “Everybody running for the Congress and the House, everybody running for the Senate, and the president himself, said to the American people ‘You give us this chance, this responsibility, this opportunity with a Republican president, with a Republican Senate, a Republican House, and we will repeal and replace Obamacare,” Ryan said, describing the stakes of the health care vote earlier in the week. “If we don’t keep our promise, it will be very hard to manage this,” he said.

Of course, it’s impossible to predict how Republican voters might react if the GOP health care effort fails, and whether that reaction will meaningfully alter election outcomes. Trump and GOP leadership may not turn on one another. The president may decide to focus his ire at Democrats, despite the fact that Republicans control both houses of Congress.

One factor that might mitigate voter backlash is that Republican efforts to move ahead with a health care overhaul have proven widely unpopular. A recent Quinnipiac Poll found that a majority of American voters disapprove of the GOP plan to replace President Obama’s health-care law, the Affordable Care Act. The plan even failed to win a majority of support among Republican voters, according to the poll, which found that only 41 percent of Republicans approved. Perhaps significantly, though, a majority of Republican voters—at 64 percent—did indicate that they approved of the way Trump had handled the health care push, despite not approving of the bill he endorsed.

At a Friday afternoon press conference, Ryan spoke approvingly of Trump even as he faced reporters after the canceled vote. “The president gave his all in this effort,” the speaker said. “He’s really been fantastic.” But he conceded that they had not followed through with what they had vowed to do. “We were on the cusp of fulfilling a promise that we made … of achieving an ambition that we’ve all had for seven years,” he said.

And despite the health care bill setback, Trump could emerge relatively unscathed in popularity in the wake of a legislative defeat, even if congressional Republicans don’t.

When a reporter asked the Speaker how he, and other Republican lawmakers, would return to their constituents without having advanced their health care agenda, Ryan responded grimly on Friday: “It’s a really good question. I wish I had a better answer for you.”