Five years ago, on a boat off the southern coast of Sri Lanka, I met the largest animal that exists or has ever existed.

The blue whale grows up to 110 feet in length. Its heart is the size of a small car. Its major artery is big enough that you could wedge a small child into it (although you probably shouldn’t). It’s an avatar of hugeness. And its size is evident if you ever get to see one up close. From the surface, I couldn’t make out the entire animal—just the top of its head as it exposed its blowhole and took a breath. But then, it dove. As its head tilted downwards, its arching back broke the surface of the water in a graceful roll. And it just kept going, and going, and going. By the time the huge tail finally broke the surface, an unreasonable amount of time had elapsed.

For scientists who study whales—and especially the sieve-mouthed baleen whales like the blue, fin, and humpback—it’s hard to escape the question of size. “They are big!” says Nick Pyenson at the Smithsonian Institution. “As an evolutionary biologist, you always have to wonder why.”

We’re not short of possible answers. Some scientists have suggested that giant bodies were adaptations to the recent Ice Age: At a time of uncertain climate and unstable food supplies, bigger whales could store more fat, and their large bodies  allowed them to more efficiently migrate in search of the best feeding grounds. Some pointed their fingers at competition between early baleen whales, forcing some members to become giant filter-feeders. Others said that whales became big to escape from titanic killers, like the megalodon shark, or the sperm whale Livyatan. Yet others have pointed to Cope’s rule—the tendency for groups of creatures to get bigger over evolutionary time.

But for Pyenson, the secret to really understanding why the baleen whales got so big is to really nail down when they got so big. And more importantly, when did they get really big? The sheer size of the baleen whales can distract us from the fact that some are much bigger than others—there’s a considerable difference between a 20-foot minke and a 100-foot blue. When did the latter titans emerge?

To find out, Pyenson teamed up with Graham Slater and Jeremy Goldbogen to collect data on the size of baleen whales, both past and present. For extinct species, they measured fossils. For living ones, they turned to museum specimens, records of beached whales, and even data from aboriginal harvests. They then mapped these measurements onto a family tree that unites all of these species, showing how they’re related and when each group evolved.

The trio found that the really big baleen whales—those measuring 33 feet (10 meters) or more—only show up in the final act of the whale drama. When this group first appeared, around 35 million years ago, they were already big, but they only became Big with a capital B within the last 4.5 million years. What’s more, gigantic sizes evolved independently in at least three different lineages of baleen whales. One gave rise to the fins and humpbacks, a second led to the blues and seis, and a third produced the rights and bowheads.

The result of these repeated forays into gigantism is that “modern whales are dramatically larger than any whales in their evolutionary history,” says Pyenson. He has noticed this pattern for a long time, but his data show it plainly. Even the smallest living baleen whale—the amusingly named pygmy right whale—is 20 feet long, making it about the same size as average prehistoric whales. And it’s not just that the biggest members of the family have become bigger. Almost all the smallest species disappeared. The family as a whole ballooned in size.

Cope’s rule can’t explain this late-breaking increase. It’s also unlikely that factors like competition within the group or threats from huge predators or the advent of filter-feeding were responsible, since these were already part of whale life before they became giants. Instead, Pyenson says, “We think the ice ages had something to do with it.”

Around 3 million years ago, the planet entered a cycle where large glacial sheets would form in the northern hemisphere, expand all the way down to the northern U.S., and then retreat again. These cycles triggered a shift away from continuous warmth toward seasonal climates that varied over the year. That seasonality reshaped the oceans. By strengthening winds that blow from the south, it intensified the upwelling currents that bring nutrients up from the depths, specifically near the coasts of continents. And those coastal waters were also hit by run-offs from melting glaciers on land, which brought even more nutrients with them.

The result was an unprecedented boom-time for coastal waters, with nutrients feeding hordes of crustaceans and small fish—potential prey for whales. But these bonanzas weren’t evenly distributed. They were concentrated in particular, far-flung places—all-you-can-eat buffets separated by literal food deserts. And that, Pyenson says, is why the giant baleen whales evolved.

They are beautifully adapted to hunt down sparse but concentrated prey. Their huge size allows them to survive for long stretches without encountering any food. And they evolved a foraging technique called lunge-feeding, where they accelerate into a shoal of prey, open their ballooning mouths, and suck in vast volumes of water. You can see them at work in nature documentaries like Blue Planet or The Hunt, where a milling ball of doomed fish is attacked by birds, sea lions, sharks, tuna, and dolphins, before a baleen whale comes in and just swallows all the survivors. “They’re taking bites out of superorganisms,” says Pyenson. “Imagine if you had a predator that flew in and took bites out of a flock of birds.”

The bigger that baleen whales get, the more efficient lunge-feeding becomes. A blue whale, for example, can engulf 120 tons of water and around half a million calories of krill in a single mouthful. So by becoming as big as possible, the baleen whales managed to monopolize the newfound bounties of freeze-thaw planet. That’s why they survived and their smaller peers died off.

Cheng-Hsiu Tsai from the National Museum of Nature and Science in Tokyo thinks that this is unlikely to be the only factor behind the rise of the giant whales. “It seems to me that people tend to focus on external factors like environmental change and upwelling, but pay little attention to internal factors,” he says. For example, the skull of the blue whale changes dramatically as it matures, while those of species like the pygmy right whale stays the same shape. So for whatever reason, some whale lineages might be unable to reshape their bodies in the ways it would take to truly exploit rich throngs of prey. By contrast, the groups that weren’t so physically constrained could make better use of this food bonanza.

Pyenson admits that his idea is just a hypothesis, and one that can only be tested by getting more fossils of whales from the last 3 million years. And sadly, the same glacial cycles that he thinks drove the group’s evolution also led to rapidly changing sea levels, and the destruction of many whale fossils. “We have a very spotty fossil record for baleen whales,” he says.

Still “this hypothesis is a step in the right direction,” says Annalisa Berta from San Diego State University, “and it leads to an interesting speculation about the future.” Today, the oceans are becoming more acidic and starved of oxygen. Their productivity—the amount of food they harbor—is going down.  “So what will happen to the baleen whales if there is less food available?” asks Berta. “Will they adapt fast enough? It took millions of years for them to reach large size. Can they shrink in 100 years?”