The average person is keeping 13 secrets right now. Five of them are secrets they’ve never told another living soul.

These stats come from a new paper published in the Journal of Personality and Social Psychology, which looked at more than 13,000 secrets over 10 different studies. The researchers asked participants if they were keeping any of 38 different common categories of secrets which ranged from infidelity to financial secrets to secret hobbies. The most common secrets that people shared with no one else were: extra-relational thoughts (thinking something romantic or sexual about someone other than your partner), romantic desire, sexual behavior, and lies.

Much of the research on secrecy that’s already been done focuses on concealment: People interact in a lab, and one person is trying to keep something from the other. But in this paper, the actual act of hiding—the moment a person makes up a lie, or changes the subject, or simply omits certain information from a conversation—proved to be only a minor part of the experience of having a secret. Instead, what seems to affect people much more is how often they think about the secret.

“We actually don’t encounter many situations where we have to hide our secrets relative to all the times a secret will just come into our thoughts, and intrude upon our thinking,” says Michael Slepian, a professor of management at Columbia Business School and the lead author on this paper.

What makes a secret a secret, Slepian and his colleagues contend, is that it’s something you intend to hide from one or more other people. Even if it never comes up, even if you never have to actively hide it, it’s still a secret. “Just because the goal of the secret is to hide it, that doesn’t mean the secret is only happening during the brief moments of when you need to hide it,” he says.

When you consider that the point of secrets is to keep them—keep them close, keep them safe, keep them inside—it makes sense that the primary way we experience secrets is alone. Across all Slepian’s studies, participants reported that their minds spontaneously wandered to their secrets far more frequently than they encountered situations where they had to actually conceal their secrets.

The idea that secrecy might be a primarily solitary experience first came to Slepian when he was running another study, which found that when people were preoccupied by their secrets, they judged hills to be steeper and distances to be longer, and thought physical tasks would take more effort. “People have this curious way of talking about secrets as laying them down or unburdening them,” he says. “We found that when people were thinking about their secrets, they actually acted as if they were burdened by physical weight. It seems to have this powerful effect even when they’re not hiding a secret in the moment.”

Jonathan Schooler, a professor of psychology at the University of California, Santa Barbara, finds Slepian’s paper “unusually innovative in terms of moving two fields forward”—namely, the study of secrecy and the study of mind-wandering. Schooler, who researches mind-wandering and wasn’t involved in the new study, says our minds might be more likely to wander to secrets than just to any old thing, referencing the social psychologist Daniel Wegner’s theory of thought suppression, which says that when we try not to think about something, we often just end up thinking about it more.

“The very fact that we’re trying to keep secrets at bay may give them extra punch,” Schooler says.

Our minds are also likely to wander to unresolved things and goals we haven’t accomplished yet, the new paper notes. “There’s something interesting about the goal to keep a secret, which is you can never fully accomplish that goal,” Slepian says. “You might encounter a conversation where you have to conceal it, but there might always be future moments down the road where you have to conceal again.”

Previous research has shown that keeping secrets is linked to lower well-being. It was thought that the reason is just that many of our secrets are negative, and thinking about negative things is a bummer. That’s probably true (though it’s worth noting that positive secrets, like surprises, likely work differently), but this study found another reason, one more specific to secrecy: Thinking of secrets means thinking of things you aren’t being open and honest about in your relationships, which makes people feel less authentic.

In one of the studies, the researchers compared secrets people were keeping from their romantic partners with negative information that wasn’t hidden. They found that the secrets made people feel inauthentic, and when they felt inauthentic, they reported being less satisfied with their lives. And the researchers never found that concealing secrets lowered well-being; only thinking about them did.

The secrets we keep pop up like Whac-A-Moles in our thoughts—chattering little rodents that won’t stay down when you hammer them. “The bad news is that even when you don’t have to hide your secret, you might still frequently think about it to the detriment of your well-being,” Slepian says. “But the good news is, if what’s most harmful is your thinking about the secret, if we could get you to think about it less, or change how you think about it, we could mitigate that negative effect.”