For restaurants in America, it is the best of times, and it is the worst of times.

Last century’s dystopians imagined that mediocre fast-food chains would take over every square inch of the country. But in cities across the U.S., residents are claiming that the local restaurant scene is in a golden age of variety and quality. I’ve heard it in Portland, Oregon, named the best food city in America by the Washington Post; in Washington, D.C., named the best food city in America by Bon Appetit; in New Orleans, where the number of restaurants grew 70 percent after Hurricane Katrina; and in San Francisco, which boasts the most restaurants per capita in the country; and in Chicago, which has added several three-Michelin-star restaurants this decade. I live in New York, which will always lead the country in sheer abundance of dining options, but after years of visiting my sister in Los Angeles, I’m thoroughly convinced that America’s culinary capital has switched coasts.

Restaurants are such a revitalizing force in urban life that a fine meal now carries a sacred profundity. “Food has replaced music at the heart of the cultural conversation,” wrote Eugene Wei, a technologist and writer who is currently the head of video at Oculus, in a 2015 essay. “It’s hard to think of any sphere of American life where the selection and quality have improved so much as food,” the economist Tyler Cowen, who moonlights as a food blogger, wrote this year. For the first time in US history, Americans are spending more money dining out than in grocery stores.


The Great Convergence: “Food at home” spending (purple) vs. “food away from home” (brown)

USDA


But if it’s truly the golden age of restaurants, why is there such widespread concern about the state of the restaurant industry?

Last year was “the worst restaurant year since the recession,” according to QSR Magazine. Dinner “traffic,” the industry term for the number of walk-in customers, has been falling for five straight years, says market-research firm NPD Group Inc. The lunch business is in a veritable depression. Last year saw the lowest level of lunchtime traffic in four decades, according to The Wall Street Journal, which declared “lunchtime is over for the restaurant industry.” The National Restaurant Association’s top trend for 2017? “A challenging business environment.”

What the heck is going on? How can the United States be going through a restaurant renaissance and a restaurant recession at the same time? In the last few weeks, I’ve spoken with almost a dozen restaurateurs and analysts about the state of the industry. Here are four theories.

1. The good news is there are too many great restaurants to choose from. The bad news is … the same thing.

Restaurants have grown from 25 percent of food spending in the 1950s to more than half, today. The shift has been accelerating: In the last decade, spending at restaurants and bars has grown twice as fast as all other retail spending, like clothes and cars.

But today there are simply too many places to eat, according to Victor Fernandez, executive director at Black Box Intelligence, a restaurant data firm. “Half of our food dollar is now going to restaurants, but we have more supply than we have demand,” he said.

Nothing encapsulates this crisis of abundance more than the so-called “fast casual” sector, which includes high-speed, high-quality lunch spots like Chipotle and Sweetgreen. Fast casual is on fire, with total locations soaring 9 percent last year. But the category is adding new spots so quickly that same-store sales are falling more than 2 percent annually. Hungry urban consumers watch lunch spots popping up across their city, and praise new options, across-the-board quality, and a dynamic restaurant scene. But to a fast-casual franchise, “new options, dynamic restaurant scene, and across-the-board quality” are terrifying euphemisms for cut-throat competition, high churn, and wobbly same-store sales.

In this way, the golden age of restaurants is a bit like today’s golden age of TV. For television viewers, there have never been more options or, perhaps, better quality programming. But as the number of original scripted shows has soared, so has the failure rate. A new drama is now four to five times more likely to be cancelled today than it was in the late 1990s.

The golden age of restaurants is similarly beset by cancellations. The experience of being an affluent diner in New York, Washington, D.C., or San Francisco is something like gastronomic nirvana. But the number of restaurants in all three cities is now falling amidst cut-throat competition, according to NPD data. Restaurants hovering around a 3.5 rating are on the razor’s edge of success and total failure; a half-point rise in Yelp ratings for such restaurants can increase sell-out evenings by about 50 percent. These mediocre-rated restaurants operate on such slim margins that a one-dollar increase in the minimum wage can seriously increase the likelihood of closing.


Tough Crowd: Growth in number of restaurants, by city: 2015-2016

NPD Data

The decline in traffic, along with rising labor costs, has forced restaurateurs to raise their prices to pay the rent. As a result, dining out is getting more expensive. The relative price of “food away” (government-speak for restaurants and snacks) versus food at home (government-speak for groceries) has never been higher, as you can see in the graph below. Higher prices might encourage middle-class shoppers to trade down for fast food or convenience stores, or groceries. And that leads straight to the second big-picture explanation.


The Check Says What?! “Food away” price index as a share of “food at home”, 1955-2017


2. The middle class of restaurants is really struggling … unless they specialize in breakfast.

In an economy where typical middle-class jobs are disappearing and middle-class department stores are shuttering, perhaps it’s no surprise that the middle is hollowing out in the restaurant industry, as well.

For cheap eats, fast food remains a huge and growing market. On the high end, both the fine-dining and fast casual sectors are growing, even if individual restaurants are struggling to eke out market share in a hyper-competitive environment.

But then there’s the middle-class of restaurants, also known as “casual dining”—full-service restaurants where the typical check is between $15 and $25, per person. This is where the pain lives. Same-restaurant sales are falling across most of the sector’s largest chains, including Applebees, Chili’s, and Maggiano’s. Traffic at all casual dining spots fell at the fastest rate since the middle of 2009, when unemployment was screaming past 9 percent.  

What’s killing casual dining? Theories range from its core customers’ income stagnation to the restaurants’ staid decor. But there’s another, less-obvious threat: Diners are increasingly buying prepared food at places that aren’t restaurants. They’re going to convenience stores with made-to-order food, like Wawa, or grocers with prepared food, like Whole Foods, college stores, corporate cafes, community centers, and food trucks. In fact, one-third of prepared meals this year won’t come from a typical restaurant or fast-food joint. Americans already buy more food and beverages on college campus than at bars, according to the National Restaurant Association. Prepared food is everywhere, now. That’s a killer for restaurants serving the middle class.

There is an interesting exception to the death of the middle. It’s Denny’s, where same-store sales have been rising for six straight years. That chain’s secret? It might be breakfast. “The two main meals of the industry have not been doing well for a long, long time,” said Bonnie Riggs, the chief restaurant analyst with NPD. But breakfast sales are up 8 percent in the last five years. McDonald’s, too, has credited its recent bounce-back to the excitement over all-day breakfast.


Big Breakfast: Change in foot traffic by meal, since 2011


What’s so special about breakfast? Maybe harried workers no longer have the time or inclination to cook eggs at home. Maybe people just really like egg sandwiches for dinner. Or maybe breakfast is just the last meal to be devoured by the delivery revolution in food.

3. Takeout is taking over.

It might initially seem like a paradox that restaurant sales are rising while lunch and dinner foot traffic is falling. But both can be true at the same time if prices are rising and more people are “ordering in,” and eating restaurant food on their own tables, desks, and beds.

The marketplace for delivery and takeaway has exploded in the last few years, driven by the emergence of new online food platforms like UberEATS, DoorDash, Postmates, Caviar, and more. (To be fair, lots of people still order delivery using the phone app on their cell phones.) The industry leader GrubHub, an online menu and delivery company that owns Seamless, has nearly 9 million active users and growing by two million new users each year, according to the company’s latest SEC filing. Pizza is still the king of delivery, with two-thirds of the market. But it’s declining, while everything else is growing.

These so-called “online restaurant sales”—which just means ordering a meal delivery online—only account for about 2 percent of the restaurant market today, according to a Euromonitor study commissioned for Blue Apron’s public filing. But this online market is projected to grow 15-times faster than the rest of the restaurant business through the end of the decade.

The rise of delivery and takeaway has been trouble for the dinner hour, but it’s been downright apocalypse for restaurants that once relied on lunchtime visitors. The three-martini lunch was once symbolic of a good-ole-boys culture of the 1960s, where executives would disappear for hours in the middle of a day to hammer out deals over white linen and porcelain plates. But today’s symbol of office chow is the “sad desk lunch,” where midday conversations are mediated by laptop, rather than occurring across the table. I’ll leave it to the cultural anthropologists to determine what this says about us, but the upshot is busier delivery people and emptier restaurants.

4. It’s ultimately about rich city-dwellers—especially in California.

Perhaps the golden age of dining out is just a story about rich urbanites getting better access to fresh ingredients and international cuisine.

“Major metropolitan areas with lots of money and tourists are benefitting from new and unique restaurants, but once you get outside of these major cities, restaurants are struggling,” Bonnie Riggs said. “So the dichotomy comes down to the fact that people living outside cities cannot afford to go out as much.”

Indeed, the data shows that richer cities have more restaurants, and their residents spend more on dining out. In the San Francisco, Los Angeles, and New York metro areas, residents spend at least half their food budget eating out, according to Trulia. These cities are increasingly the playgrounds of wealthy, young, highly educated people who are less likely to have families. That means they have plenty of disposable income and no kids to absorb all of it. The major cities with the largest growth in Yelp listings between 2012 and 2017—a useful proxy for restaurants—included those three major metros, along with Miami, Nashville, and New Orleans, according to data prepared for The Atlantic by Carl Bialik, a data scientist at Yelp.

Perhaps nowhere is benefitting from the urban restaurant renaissance more than California cities. Of the 15 major cities where Yelp ratings have improved the most, five are in the Golden State, including number one on the list, Irvine. That’s no surprise, perhaps, given the organic revolution and the proximity to the state’s absurdly fertile Central Valley, which produces half of the country’s fruits and vegetables. But it’s another reminder that the restaurant renaissance is a surprisingly local and elite phenomenon. We are living through a golden age in food—if you can pay the bill.

Today’s restaurant renaissance is an expression of the long-term growth of leisure spending in the U.S. But there’s nothing leisurely about the restaurant business today. The superabundance of quality and variety among restaurateurs has created cut-throat competition, particularly in the fast casual sector. The price gap between grocery bills and restaurant checks has never been higher. The rise of takeout has forced restaurants to serve more diners who don’t step foot on their property. And with Blue Apron’s impending IPO and Amazon’s purchase of Whole Foods, the entire food sector may become more of a delivery business. The future of dining out might look a lot like eating in.