Yoshiyoshi Hirokawa / Getty

In 2010, when I was 24 years old, I endured six straight months of recurring strep-throat infections before I finally got the green light to have my tonsils removed. Midway through a round of antibiotics, I hauled myself into my new specialist’s office unshowered and wearing gym clothes I had collected from my floor, sweaty and rapidly losing any remaining will to sit upright. So I was not prepared for when the doctor walked into the exam room and revealed himself to be tall, broad shouldered, square jawed, and absolutely beautiful.

Embarrassment shot through my body. Why was his unplanned handsomeness allowed to stress out sick people? Why was his face that symmetrical? Why hadn’t the receptionist warned me?

A couple of weeks later, the hot doctor cut out my rotting tonsils. When he paid me a surprise visit in the recovery room, I was consumed again by the irrational belief that people at the far end of the physical-beauty bell curve should at least give the rest of us some time to compose ourselves before we have to deal with them. Instead, we’re left to walk up to a store counter, interact with someone we find arrestingly gorgeous, and pretend that nothing has happened.

I’ve chafed under this onerous expectation periodically in the intervening 10 years: There was another hot doctor, to whom I had described a rash in detail over the phone, as well as a hot mover and the occasional hot delivery guy. Every time, it was the same small sense of panic, embarrassment, and indignation. Aren’t people supposed to enjoy beauty? As it turns out, this isn’t just me being a colossal weirdo, at least according to neuroscience. Even if they don’t mean any harm, hot people can be very, very stressful.

The problem starts with brain chemistry. “When you see an attractive person, the left ventral tegmental area of the brain becomes active and will pump out dopamine,” says Helen Fisher, a biological anthropologist who studies attraction at the Kinsey Institute. “Dopamine is a stimulant to the brain, so some people might react with surprise or awkwardness.” That feeling is the weak-kneed giddiness that very attractive people can inspire, which can leave you fumbling for words and feeling off balance, even though a dopamine rush is a fundamentally pleasurable experience.

Based on Fisher’s research, which used fMRI scans to observe the brain lighting up in response to stimuli, the left ventral tegmental area (commonly referred to as the left VTA) is responsible for pleasurable reactions to beauty. Meanwhile, the right VTA provides the dopamine that fuels romantic love; the two responses are similar but neurologically distinct, which means that what people feel when they see a random pretty face isn’t necessarily a desire for romance or even sex. “The same thing probably happens when you look at a good painting,” says Fisher. “It can pump out the dopamine and perhaps make you slightly giddy.”

The left VTA appraises and appreciates what you see, but lighting up that part of the brain doesn’t necessarily make you want to interact with the person whose appearance gives you pleasure, which is why most people don’t try to ask out every hot person they see. The stress I felt wasn’t the same as a fear of rejection; my hot surgeon wasn’t even my type. Instead, I panicked because of a key difference between gazing at a painting and a hottie: A painting doesn’t judge you back.

That’s where a second, potentially more nefarious brain chemical comes in: cortisol. That’s the stress hormone that gets blamed for everything from weight gain to road rage, and Fisher thinks a cortisol spike is probably what I experienced when surprised by my extraordinarily attractive doctor. “Some people may see someone beautiful and feel very inadequate. Then cortisol would go up,” she says. A spike in the hormone can trigger a fight-or-flight response, which could be why my brain hurtled toward intense irritation and embarrassment at beautiful strangers in situations where I was at a disadvantage: when I was sick, in the middle of moving, or watching The Real Housewives of Atlanta inside my own apartment.

“It’s the context of who you are, how you feel about yourself, if you enjoy surprises—lots of things,” Fisher says. It doesn’t help that American culture tends to code physical beauty as an indicator of overall superiority, which can make the sense of inadequacy in these interactions particularly stressful.

While people’s brains certainly enjoy beauty, our appreciation is often not that straightforward, because our perceptions are also influenced by everything else about a particular interaction. Indeed, researchers have found that the adrenaline rush created by fear can make other people seem more attractive in the immediate aftermath. And if you’re already feeling good, Fisher says, suddenly encountering an attractive person can make you feel even better by triggering a dip in cortisol levels. In hindsight, that happens to me even more frequently than the panic I had with my surgeon, but humans tend to have better recall for negative memories than positive ones.

Even if hot people have the element of surprise on their side, that gets them only so far. “Good looks are important in the beginning, because it gets you to look at a person and you might go talk to them,” says Fisher. “It’s a great first signal, but mating has breaking points and escalation points.” She notes that usually, in the long run, being really hot isn’t enough to keep people attracted to someone who has a terrible personality or a bizarre worldview. Whether knowing that pretty people have problems too makes you feel better when you’re wearing a hospital gown and suddenly confronted with a sentient Ken doll is another issue.

We want to hear what you think about this article. Submit a letter to the editor or write to letters@theatlantic.com.