Baseball's Fancy New Instant-Replay System Is Useless

If MLB really wanted to cut down on umpiring errors, it could skip the bizarre rules of "extended instant replay" and start addressing erroneous ball and strike calls.

Major League Baseball's opening day marks the official beginning of baseball's expanded instant replay. (Last week’s official games in Australia did not have the replay feature.) Theoretically, as The New York Post put it in January, “Instead of kicking dirt on umps, or getting into their face Earl-Weaver style, managers will now simply voice their displeasure about a call with a simple video challenge.”

That umpires are fallible should come as no surprise. Even the best umpires sometimes miss calls. They are, after all, only human. What is surprising, however, is MLB’s limited set of replay rules—which seem more for show than for actual effect.

According to’s official replay FAQ, a manager can use his one allotted challenge in the first through sixth innings. If he succeeds in his challenge, he gets another challenge within the first six innings. If he succeeds in his second challenge, however, he doesn’t get any more replays, since by rule a manager can never have more than two in a game (for some reason). However, if he’s out of challenges and it’s late in the game, he can still ask an umpire for a replay and be met with a resounding “Maybe!”

There are several questionable aspects of this setup. For starters, it turns replay into a strategic layer of gameplay rather than a failsafe against human arbiters. It is not at all difficult to envision a situation in which a manager chooses not to exercise his replay right on an early critical play for fear he won’t have it available to challenge a hypothetical critical play later in the game. And similarly, runs scored in innings one through six count just as much as those scored in the seventh and onward—so the distinction by inning of when manager-initiated challenges are allowed is pointless.

Further, the Weaver-esque blowout is still allowed; there’s nothing to stop a manager from throwing impotent temper tantrums while arguing. So if you were hoping that replay would speed up the game by removing pointless manager-umpire confrontations, prepare to be disappointed. Checked swings, balls and strikes, the “neighborhood play” at second base, and tag-ups cannot be reviewed. Neither can use of the infield fly rule, so it’s entirely possible the ugliest scene of the 2012 postseason could repeat itself.

But perhaps more importantly, the balls and strikes that instant replay fails to address have a far larger impact on the outcome of a given game than any particular fair/foul or safe/out call. Replay cannot address these calls—indeed, challenging individual balls and strikes would slow games to a crawl—but the process by which these calls are made indicates that umpires are just as prone as fans to get caught up in the moment and that replay alone cannot correct the problem.

How important is a single ball or strike? Incredibly important. With help from sabermetrician Erik Murphy in navigating Baseball Reference’s data, I found that in the 2013 season, batters facing a 0-1 count (zero balls, one strike) during their at-bat reached base 26 percent of the time. Their fortunes reversed dramatically with a 1-0 count, reaching base 38 percent of the time—a 12-percent difference. Given that the average MLB batter reached base 32 percent of the time last season, the difference between a single ball or strike to open an at-bat clearly has an enormous impact on that at-bat’s outcome.

Multiply that impact by the number of pitches in an at-bat, and the number of at-bats in a game, and it’s quite easy to envision correctly calling balls and strikes as the most critical part of a game, even if any single ball/strike call may not be particularly influential.

Unfortunately, data indicates that umpires do a rather poor job of calling balls and strikes, and that psychological factors interfere with their ability to be fair arbiters. In an upcoming article in Management Science, Columbia professor Jerry Kim and Northwestern professor Brayden King examine the impact of pitcher reputation on an umpire’s ability to fairly officiate balls and strikes.

Examining more than 800,000 MLB pitches in the 2008 and 2009 seasons using PITCHf/x data, the researchers find that about 14 percent of all pitches are called incorrectly by umpires—pitches inside the strike zone called balls, and vice versa.

More importantly, pitchers with an esteemed reputation (as measured by All-Star Game appearances) get the benefit of the doubt far more often than their relatively anonymous counterparts. Controlling for variables such as game situation, pitcher control, stadium, umpire, batter, and catcher, the researchers find that umpires are about 16 percent more likely to erroneously “extend” the strike zone beyond its rulebook-defined dimensions in favor of a big-name pitcher. Only a similarly big-name batter can counteract this bias.

In other words, umpires subconsciously favor elite pitchers. Former White Sox and Marlins manager Ozzie Guillen observed much the same without the rigorous review, as quoted in Bruce Weber’s 2009 book As They See ‘Em: A Fan’s Travels in the Land of Umpires.

If Roger Clemens or Pedro Martinez or Greg Maddux, Tom Glavine, is pitching, it’s a strike. Jose Cruz pitching? It’s a ball. Same way for hitting. You’re fucking Wade Boggs? That’s a ball. You’re Frank Thomas? That’s a ball. Ozzie Guillen hitting? Strike, get the fuck out.

If MLB seeks to make its officiating more accurate, it could start with addressing erroneous ball and strike calls, and helping to insulate umpires against the psychological effects of calling a pivotal play or judging a prominent player. It’s unlikely that these umpires are missing calls on purpose, but they are only human—and apparently just as prone to being awe-struck as a casual fan.

MLB should be commended for implementing replay—it’s long-needed progress. But if MLB’s goal were to improve accuracy, it would also allow for unlimited replay initiated by video-review officials, which would have the added benefit of taking the most emotionally charged decisions out of the hands of the on-field staff. An umpire’s job is already difficult enough; if the technology exists to support umpires on a game-changing fair/foul or safe/out call when 50,000 screaming fans promise to express their displeasure when the “wrong” call is made, umpires should get that support.

Professional sports are not lacking for competent review systems that remove the on-field official’s psychology from the equation. For example, the National Hockey League has used an off-ice video goal judge for years, taking the most critical plays out of the hands of referees and providing an extra step aimed at verifying that goals are being scored correctly. But as it stands now, MLB’s limited replay rules may well prevent the most memorable and publicly embarrassing calls for the sport, while doing very little to correct the larger problem of inaccurate umpiring.