Grabbing the Bullpen by the Horns: What Is the Optimal Strategy for Using Relievers?

(Note: Unless otherwise noted, WAR refers to fangraphs' calculation of the metric)

Is it better to maximize the number of times a reliever can be used or the length of his appearances? Yesterday at frangraphs.com, Dave Cameron tried to answer that question by comparing bullpen performance over the last 30 years and concluded that because there hasn’t been an aggregate improvement, teams would be better off returning to the past practice of using relievers for longer stints.

According to Cameron’s data, not only have modern bullpens failed to improve performance over the last 30 years, they also haven’t been called upon to pitch more often. Instead, the number of relief innings pitched has simply shifted from premium relievers to those filling out the roster. In other words, according to this conclusion, the Sergio Mitres of the world have become common place on major league rosters, despite little evidence to suggest that they provide some measurable derivative value.

Percentage of Batter's Faced by Relievers and Starters, Since 1982

Source: fangraphs.com

In response to Cameron's conclusion, Tom Tango wondered if the ability of extended bullpens to maintain performance implies that the best relievers (i.e., the ones around whom modern bullpen usage revolves) are actually doing better because they have to pick up the slack for the dregs? If true, this would suggest that modern bullpen tactics have been beneficial, provided they are in compliance with usage patterns based on leverage. Almost like pawns in a game of chess, it could be that modern managers are sacrificing the last men in the bullpen to set up a fool proof endgame.

Comparing data from 1982 and 2011 (the endpoints in Cameron's broad analysis), the transfer of innings from effective relievers to ineffective relievers becomes clear. In 1982, 56% of relievers had a positive WAR and faced 73% of all batters. However, in 2011, the percentages flipped as only 44% of relievers had a positive WAR and the rate of batters faced dropped to 63%. What's more, there was also no significant difference between relievers at the top of the food chain. In 1982, the 25 best relievers by WAR had an aggregate ERA of 2.57 and combined WAR of 52.7. Meanwhile, in 2011, that same elite group posted an ERA of 2.29 and a combined WAR of 48.3. There was, however, one big difference: the number of innings pitched. In 1982, the best relievers pitched almost 800 innings more, or about 30 more per pitcher. Basically, even at the top, the same tradeoff exists: slightly better performance for significantly fewer innings.

Top-25 Relievers, 1982 vs. 2012

Note: Ranking based on WAR.
Source: fangraphs.com

Based on this data, it seems as if Cameron's conclusion is sound. However, is there more to the story? Can we make valid judgments based on the aggregate numbers, or, do we need to consider context?

In sabermetric terms, one way we can measure context is by considering leverage. If managers are able to take advantage of the modern bullpen theory to deploy their best relievers at just the right time, and hold back their fodder for games that are out of hand, it would be made evident by an examination of leverage. Once again using 1982 and 2011 as our test cases, we find that the top-25 relievers in 2011 appeared in 795 games with an average leverage index of at least 1.5 (1 is considered "average pressure"), compared to 691 for the best relievers in 1992. Spread over the lot, this difference amounts to four additional high pressure deployments per season. Is that significant? Considering the innings tradeoff, it doesn't seem so. In addition, it's worth noting that the median leverage index when entering the game (gmLI) was higher for top pitchers in 1982 (1.76) than 2011 (1.58), so it doesn't appear as if managers are more strategically using their best weapons (and, it suggests the length of appearance may be watering down cumulative leverage figures).

Into the Fire: Leverage Metrics for 25 Best Relievers, 1982 vs. 2011

Note: Ranking based on WAR; gmLI is the average leverage of all points at which a pitcher enters a game. LI is leverage for all game events (1 is considered "average pressure")
Source: fangraphs.com

There are two sides to the leverage equation. Among the cream of the crop, it doesn't seem as if top relievers are being optimally deployed, but what about at the bottom? In 2011, the bottom-25 relievers were used in 212 games with leverage of 1.5 or greater and had a median gmLI of 1.08, compared to 205 games and 1.29 for the worst pitchers in 1982. Although the 1982 dregs pitched 400 more innings than their 2011 counterparts, they also had an ERA that was almost a run lower, so, all things considered, this comparison also appears to be a wash (WAR for the 1982 group was -14.7, versus -15.7 for 2011).

Admittedly, by limiting this analysis to only two seasons (1982 versus 2011), there is plenty room for error, but, at the very least, the data seems to support the conclusion that despite vastly different strategies, bullpen performance has remained the same. Of course, that doesn't make the modern approach to managing relievers worse; it just makes it different. After all, if the old and new methods achieve the same results, does it matter which one is used?

Cameron argues that it would be better to return to the "longer stint" approach because it not only saves a roster spot or two, but it also lessens the perceived value of the closer, which, by extension, would curtail the exorbitant salaries paid to the men who own the ninth inning. It is here where I disagree with Cameron.

Still Kicking: Older Relievers in the Major Leagues

Note: Based on a minimum of 40 innings pitched.
Source: Baseball-reference.com

In framing his argument, Cameron cited Bob Stanley, who pitched 168 1/3 innings in 1982, as an example of the workload shouldered by bullpens of the past. However, it's worth mentioning that after his 1982 campaign, Stanley only had four more productive seasons and retired at the age of 34. Meanwhile, Mariano Rivera is pitching better than ever at the age of 41. That's just an anecdote, but more "old" relievers are still kicking around versus 30 years ago, so maybe the modern approach is helping to prolong careers. Of course, one could argue that health is in the best interest of the pitcher, not the team, but that's a little too Machiavellian. Also, that philosophy only applies when the reliever in question is fungible. Relievers like Mariano Rivera and Jonathan Papelbon, for example, are not easily replaceable, which is why teams should be interested in keeping them healthy for as long as possible (and willing to pay them).

So, where does that leave us? It seems certain that as a group relievers are no better or worse today than they were 30 years ago. However, instead of advocating a return to the past with the goal of saving money and roster spots (after all, if not wasted on marginal relievers, they'd probably be squandered on below average position players), perhaps the focus should be on improving bullpen usage within the modern theory? The one thing we know for certain is today's relievers do pitch in more games, but, unfortunately, managers too often defer to the save rule and waste many of these appearances in low leverage situations. If managers would instead commit to shooting their best bullets at the right targets (i.e., high leverage situation regardless of inning), the current philosophy of shorter outings might prove to be the most optimal. At the very least, this hybrid approach is worth trying, especially when you consider that a return to the past approach promises little more than the status quo.