The thing is, that price discrimination is only supposed to work under certain very narrow circumstances. Suppose we have a two-part tariff seller, say, a movie theater selling tickets for $10 and popcorn for $5. If a competing theater opens across the street charging $12 for tickets and $2 for popcorn, you'd expect to see everybody who doesn't like popcorn stay at the first theater and everybody who does like popcorn go to the second theater. That is, a price discrimination scheme should very quickly break down in the face of perfect competition and in fact this problem is so well understood that monopoly is understood to be a scope condition. For instance, the word "monopoly" is in the title of one of the major cites on two-part tariffs.
So what about when you don't have a monopoly or perfect competition, but something in between? In theory, you don't need a perfectly competitive market with innumerable infinitesimally small price-takers in order to get something that looks a lot like a Walrasian auction. This is why industrial-organizational econ loves game theory. Once you apply a prisoner's dilemma model to price competition in a market with two sellers (duopoly) or a handful of sellers (oligopoly), it looks a lot more like a market with an infinite number of sellers (perfect competition) than it does like a market with exactly one seller (monopoly).
The thing is though that we have lots of cases of price discrimination and most of these cases occur in reasonably competitive markets, with multiple sellers and no apparent price-fixing. For instance, I previously noted that movie theaters practice two-part tariffs but let's reflect on the fact that this is a competitive industry. This raises the puzzle of why we haven't seen a chain of movie theaters compete by giving up the two-part tariff business model, which would mean cheaper popcorn but higher ticket prices.
These kinds of issues are why I was so interested when a colleague recently sent me Xavier Gabaix and David Laibson's QJE paper "Shrouded Attributes, Consumer Myopia, and Information Suppression in Competitive Markets" (ungated version). The reason the paper is important is that last phrase about "competitive markets." It shows how all sorts of stuff we already knew could go on with monopolies can also occur under competition. "Shrouded attributes" refers to hidden costs like the marked-up component of the two-part tariff. The "consumer myopia" phrase explains that this works if you make the reasonable assumption that many consumers aren't reasonable.
The logic goes that we imagine two types of consumers, myopic and sophisticated, and two types of products, "no hidden fees" (but with a high base price) versus "low price" (but which nickel and dimes you to death). The myopic customers will flock to the low price provider because they don't realize that they'll wind up buying $10 peanuts from the minibar. The sophisticated customers will go to either the "no hidden fees" or the "low price" provider based on who gives a better deal when you tally up the total cost of the basket they expect to consume. What this means from the provider's perspective is there are no customers who will pay more for a given basket of goods under a "no hidden fees" plan than they would in a "low price" plan. As such, the "low price" plan can crowd out the "no hidden fees" plan.*