Trump vodka, Trump wine, Trump chocolate. President-elect Donald Trump has spent a lifetime cultivating a personal brand that conjures decadence and luxury. His name alone costs millions of dollars to license, and much of his fortune has been generated from agreements to use the Trump name for products and real-estate developments that he doesn’t actually own. The Trump brand is so essential to the incoming president’s identity that he often touted it as an emblem of his business successes and pointed to it as one of his main qualifications for leading the country. Now the Trump Organization must face the question of whether the presidency will strengthen or weaken that brand going forward.

Trump’s commitment to building his brand has been so strong that some believed this run for president was actually a ploy to enrich himself and build interest in a potential television news network if he lost the election. Such spin-off business ventures, are—of course—no longer on the table. The Trump Organization, however, will continue operating under the direction of his children, though that arrangement has raised questions about potential conflicts of interest if they discuss business with their father.

It’s unclear precisely how much the Trump name is worth, but a large portion (anywhere from $32 million to $55 million) of his estimated $2.9 billion net worth likely comes from licensing agreements, according to Bloomberg. (Trump’s press office did not respond to a request to comment on this story.) These agreements are highly profitable, low-risk transactions for Trump, who makes money off of them with minimal effort, and without the need to invest his own capital. One example is the Trump International Golf Club in Puerto Rico, which went bankrupt, but was still a source of profit for Trump, since he earned money for his name without actually putting money into the operation. Similarly, a developer in Panama reportedly paid the Trump Organization $75 million to license the name of the Trump Ocean Club International Hotel & Tower.

Trump ultimately won the presidency, a sign that he and his brand have more fans than many people realized. But, though Trump hasn’t yet taken office, his brand has already suffered in some quarters from his divisive campaign rhetoric, with New York residents petitioning to take the name off their apartment buildings, a Toronto city councilman calling for the removal of Trump’s name from a new hotel, and a Chilean consulting firm cutting ties with the Trump Organization as it attempted to expand in South America.

Gavan Fitzsimons, a professor of marketing and psychology at Duke University, says the Trump brand will suffer even more after he takes the oath of office. Fitzsimons, who advises companies like Campbell’s Soup, Anheuser Busch, and Kimberly Clark, says it will be his most ardent supporters—not his opponents—who might inflict the most damage.

Whether or not a brand is successful depends on the associations customers have with the name, be they negative or positive. Right now, Trump has both, Fitzsimons says. “For the loyal customers—the ones who voted and supported him—this strengthens the positive association of the Trump brand,” he says. That could boost his business right now, but that will likely change when he actually takes office: Presidents can never keep everyone happy, and usually end up disappointing even their core supporters, he says. When President Obama took office in 2009, his approval rating was 67 percent, but by the end of October this year, it had dropped to 53 percent. (George W. Bush had only 34 percent of the public’s approval by the time he left office). That’s normal, says Fitzsimons, because candidates always make promises they then have to compromise on when elected. “It would have been better for Trump’s brand if he had lost,” says Fitzsimons. “Now there’s going to be a strong negative link between the Trump logo and the Trump president, and that will dampen the positive business impact for the brand.” (Of course, losing the presidency might have tarnished Trump’s reputation as a guy who wins all the time.)

As more time goes by, more negative links will start accumulating, Fitzsimons expects. Once a customer has a negative association with a brand, it’s nearly impossible to change that. It can take years to turn around someone’s negative perception, he says, pointing out how Exxon and BP are still strongly associated with oil spills that devastated the environment years ago.

There is a chance that the presidency could help Trump’s brand in the long run, but Fitzsimons figures it wouldn’t happen until about 20 years after Trump leaves office, when people start forgetting some of the decisions he made. That’s what happened with Ronald Reagan and Bill Clinton, who are now revered in American culture, despite the controversies that plagued them when they left the White House, Fitzsimons says: “For the negatives to fade into a rosy glow, it’s going to take a really long time.”

If Trump were a client, Fitzsimons says he would’ve told him not to run for president in order to save his brand. But now that he’s won, Fitzsimons says that the most important thing for Trump to remember is that his new job is much bigger than his business interests.