A clueless woman looking for attention. Prissy and confused little girl. Not pretty enough to be a Trump girl. Meghan Milloy has been called all of that because she’s a Republican woman who plans to vote for Hillary Clinton. For standing against Donald Trump, she’s faced an onslaught of criticism on social media—and a lot of it has been sexist. “People have been coming out of the woodwork, calling me every name in the book,” Milloy said in an interview.

Trump is famous for insulting women. He rates and ranks women based on the way they look, so much so that entire articles have been devoted to chronicling his sexist comments. As Milloy has discovered, some of his fans seem to act and talk in similar ways.

The Republican presidential candidate is extremely unpopular with women, even those who could be counted on to support conservatives in past elections. These voters could be the reason Trump loses the election, perhaps in a landslide. If Republican women vote against Trump in 2016, will they permanently leave the party?

Men and women are likely to care about many of the same issues, but women may react differently to Trump’s priorities and what he has to say. Pew Research recently concluded that “women are more likely to emphasize certain issues, including the treatment of racial and ethnic minorities.” A Washington Post-ABC News poll found that women were more likely than men to say Trump doesn’t show enough respect toward people he disagrees with and to rate that as a major problem. Trump’s denigrating remarks directed at women, Muslims, immigrants, and Mexicans may be more unacceptable to female voters.

The way Trump talks may convince some women he is not fit to act as commander in chief. Asma Hasan, a Republican woman living in Denver, Colorado, who is enthusiastic about voting for Clinton, finds it “hard to imagine voting for Trump when you consider the things he’s said about women, about minorities, even kicking a baby out of his rally.” It’s not just that his remarks are so often offensive, it’s that she believes Trump’s willingness to casually say shocking things could be dangerous. “If you’re going to be president, you need to think about what the perception will be of what you’re saying, how it will come across,” she said. “That’s especially important for a world leader.”

However Republican women vote in November, Trump could do lasting damage to the party as it attempts to win over women in future elections. “I think the party is losing an entire generation of voters, especially young women,” said Jennifer Pierotti Lim, the founder of an organization called Republican Women for Hillary. If Trump wins, even anti-Trump Republican women who want to remain in the party may find it hard to do so. “If Trump is elected and we become the party of Trump, and our leaders still aren’t able to push against him in terms of policy and rhetoric, then I can’t really see myself being part of a party like that,” Pierotti Lim said.

At least some Republican women already plan to leave the party. Over the course of the election, Suzanne Alvarez, a self-described lifelong Republican who lives in Spokane, Washington, decided she couldn’t support a party that won’t disavow Trump. “I was shocked when I saw the leadership in the party standing by him,” Alvarez said in an interview. “They helped create Donald Trump by not confronting the racism and paranoia that was infecting the party. By their silence, they validated Trump, and for me that is worth leaving the party over.” Alvarez plans to vote for Clinton, and she wants to become a Democrat as a result of Trump’s rise.

Not everyone wants to abandon the GOP, though. “I think the best outcome for the country and the Republican Party is for Trump to lose, and I want to be part of rebuilding the party along conservative principles when that happens,” said Kori Schake, an advisor to John McCain during the 2008 presidential campaign who plans to vote for Clinton. “I think it would take a lot for me to leave,” said Milloy, the chair of Republican Women for Hillary. “I believe we need voices of reason to help bring the party back to a point where it can be nationally successful.”

Anti-Trump Republican women may be called traitors and sellouts for now. But part of their motivation for speaking out may be a hope that they will be well-positioned to pick up the pieces of a broken party if Trump loses in November. If Republican leaders who endorsed Trump end up discredited, conservatives who forcefully opposed Trump from the beginning could prove influential in deciding how the party moves forward.

Some have even found a few things to like about Clinton. “I think she has been traditionally more Republican in the past than Donald Trump on things like trade, business, and immigration,” Milloy said. Clinton could “be a big trade up from the current administration” from a foreign-policy perspective, Schake commented. She believes Clinton will be more likely to use military force than President Obama, and that her presidency would be “a reassurance to America’s allies in the world.” And while Bernie Sanders and his supporters hoped to push the Democratic Party in a more progressive direction during the primary, the Clinton campaign is now openly courting Republicans.

The first woman to ever be nominated for the presidency by a major U.S. political party could be swept into the White House with unprecedented support from women voters, including a number of women who have traditionally voted Republican. If that ends up happening, Clinton’s victory will likely have as much to do with her Republican opponent as it does her own merits. What may matter most is what happens after 2016, and whether the Republican women who ultimately decide not to vote for Trump abandon the GOP as a result.