“My own history of yo-yo dieting started when I was 15 and lasted about three decades,” said Sandra Aamodt, a neuroscientist and the author of Why Diets Make Us Fat, on Saturday at Spotlight Health, a conference co-hosted by the Aspen Institute and The Atlantic. “I lost the same 15 pounds pretty much every year during that same period, and gained it back regular as clockwork.”

This is a classic tale—the diet that doesn’t take, the weight loss that comes right back. The most recent, extreme, highly publicized case was that of the study done on contestants from the reality show The Biggest Loser, most of whom, six years after losing 100 to 200 pounds, had gained most of it back, and had significantly slowed metabolisms.

The study provided a dramatic example of how the body fights against weight loss. And sheer force of will is rarely sufficient to fight back.

“Why is that we can’t just control our body weight?” Aamodt asked. “We all sort of think we can. You can decide whether to take that next bite of breakfast right? Why can’t we just keep that going over years? The answer is that your brain, like a number of poorly run institutions, is governed by committee.”

Some of those committee members are the brain’s reward system, which tends to find a cookie to be a better reward than a carrot stick, and the hypothalamus, which regulates what Aamodt calls the body’s “weight thermostat.” This thermostat prefers the body to be at a certain weight, called its “set point,” and if weight dips below or rises above, hunger and calorie expenditure are adjusted to try to bring it back. These systems are constantly active, Aamodt said.

Also on the committee is the executive system, which deals with planning and decision-making. “This is the area of the brain that you tend to think of as your secret weapon for weight loss,” Aamodt said. “Your secret weapon for weight loss takes a lot of vacations.” Willpower is very taxing for people—studies show that any task that you do that requires discipline and self-control makes it harder for you to resist urges later. The executive system doesn’t function as well when people are lonely, or stressed. “And guess what? It’s impaired when you’re hungry,” Aamodt said. “The basic answer to why people have so much trouble with dieting is they’re trying to use a system that tires easily to fight against brain systems that are always working, never take a day off.”

Also at the Aspen Ideas Festival, Kevin Hall, a senior investigator at the National Institute of Diabetes and Digestive and Kidney Diseases (and an author on the Biggest Loser study) described another problem: When people go on a diet and begin to lose weight, their appetite increases substantially. And so people end up eating more calories without realizing it. “If you ask people if they’re doing anything different, they’ll say no,” Hall said. They feel like they’re exerting the same amount of effort toward their diet, because their appetite is so high that they’re still hungry even if they eat a little more. And it continues—“there’s an exponential decay of diet adherence,” Hall said.

Meanwhile, metabolism slows down, and the weight comes back, with some dieters eventually weighing more than they did before the diet. This happens over the long-term, months or years later, which means that in the short-term diets still seem appealing. “When I say diets don’t work, I mean they don’t work five years later, Aamodt said. “Two months later, they work great.”

Diets can permanently mess up not just your weight, but the way you eat. One study that followed thousands of girls and boys for two years found that dieters were more likely to binge-eat. (Female frequent dieters were 12 times as likely to binge eat, male frequent dieters were 7 times as likely to do so.)

“That’s probably just a biological response to repeated starvation,” Aamodt said.

None of this is encouraging. What are people supposed to do, just stop trying to lose weight? Well, yeah, maybe. Unless you want to diet for the rest of your life (while probably still feeling hungry a lot of the time). That’s what people do, who manage to successfully keep weight off. “They commit to counting calories forever, they exercise every day,” Aamodt said. “A large number of them are fitness professionals. That’s the level of commitment that it requires.”

But you don’t have to lose weight to improve your health and lower your risk of dying. One study of people of a range of BMIs, from normal to overweight to obese, found that the more healthy habits they adopted, the less likely they were to die during the course of the study. Those habits were regular exercise, not smoking, moderate drinking, and eating at least five fruits and vegetables every day. The higher someone’s BMI, the greater the benefit they saw from the habits, and by the time all four were adopted, there was almost no difference in the mortality risk for all BMIs.

Accepting a weight you’re unhappy with and vowing to just make your body healthier and not worry about its shape is a hard thing to do in a weight-obsessed society with unattainable beauty standards. But maybe the stigma of being overweight can be lessened some by this research that empirically shows it’s not in people’s control. Instead of clinging tightly to the thinnest possible versions of ourselves, maybe we could just unclench.

Aamodt herself has reaped the benefits of this. “In 2010 my New Year’s resolution was that I was going to stop dieting for an entire year, I was going to stop weighing myself and I was going to exercise every day,” she said. “I did all three of those things. I am still doing all three of those things 6 years later.” Now, she said, she eats mindfully—eating when she’s hungry, stopping when she’s full, and listening to her body.  “This allows many of us to enjoy food for the first time in our lives, because it’s always been a battleground.”