Defensive Range, Other Things

Sometime in August Chris Stewart had managed to string a few singles together, some of them coming with runners in scoring position. While this is a little bit remarkable because of the batsman in question, it's still decidedly not remarkable. Yuniesky Betancourt has 974 career hits and he's still only 30, proving that everybody who plays will get some hits, even Chris Stewart. The only reason that I mention this now is to approximate something Michael Kay said immediately after.

Chris Stewart really has been a good pick-up for the Yankees, providing great defense and catching CC Sabathia while being a solid .260 hitter!

That's not a direct quote, but the general feeling was there. The feeling that Stewart had overachieved at the plate by being a .260 hitter and because I just watched him drive in a runner from second. My response was to think that this is why people talk shit about batting average. If you see Chris Stewart -- who concluded the season at a robust .241/.292/.319 -- as a player to be praised because of a non-terrible batting average after around 100 at-bats, you deserve to be chastised for weak analysis.

*And make no mistake, that is analysis. Once you pull out a number, even if it's just batting average, and draw a conclusion based on what that number indicates, you have taken off your fan hat and put on your analyst hat.

I know that Michael Kay is an entertainer and not a number cruncher or a front office executive. His job is to make watching the game an enjoyable experience for the maximum number of fans and information distribution is on a distant back burner. Even if he wanted to say "Kenny, Chris Stewart's is about as dangerous at the plate as a sleeping butterfly" he wouldn't, which is perfectly fine.

Chris Stewart is probably a good dude and even the worst major league player is obscenely talented. I would be willing to bet that he works hard and wishes he could do more to help the team by being a better hitter. So it's only natural that the people around him look for ways not to dump on the guy. Sure his slugging percentage is lower than Jeter's batting average, but he gets big hits, he doesn't give away at bats, he knows how to move runners over. Unfortunately, that is the kind of cliched pat on the head that is usually just a euphemism for not being very good.

Which is a fine mentality for family and friends, teammates, managers, and Michael Kay. It is not fine for an analyst who seeks to provide any kind of real insight.

***

I had said that Chris Stewart probably works hard, and I think that there is a further point about that to be made. In general, people are way more forgiving of physical shortcomings than mental shortcomings. It seems to be a part of human nature to give an inordinate amount of weight to trying your best, regardless of context. Some examples:

If Raul Ibanez can't cut off a line drive before it makes it to the gap, turning a single into a triple, that's to be expected. He's sixty-eight years old for god's sake. If Raul Ibanez attempts an ill-advised slide-dive at a sinking line drive and it gets past him, turning a single into a triple, then he needs to be taken off the field immediately and to be safe we had probably better shoot Girardi too.

If a wide receiver with blazing speed runs down an overthrown deep pass but then doesn't make the catch, it's a much more costly blunder than not being fast enough to reach it.

Robinson Cano grounding into a double play and jogging down the line is abominable, but Jorge Posada sprinting at jogging speed is just part of the game.

You get the idea, hopefully. Effort and focus can change the outcome of an event -- running harder or concentrating more intently can help you to do better things -- but once the outcome has been determined, the most important thing is the result of what actually happened.

If a run prevention unit allows a baserunner, they have sacrificed some ground on the chessboard and we can attempt to apportion blame to better understand the contribution of each player. Again, the manager should encourage players to try their best and play hard and the analyst should be concerned with what the actual results were. Mental errors might be more frustrating or glaring than physical limitations, but losing a bishop is losing a bishop, so let's figure out why and do better next time.

You can't systematically improve something without being able to measure it and there's absolutely no room in such a measurement for taking it easy on Jorge Posada because he's a Yankee icon with a lot of miles on those knees.

Which is why I completely abhor fielding percentage, an utterly subjective defensive rating based on "well, did you try your best?" I won't charge you with an error on that ball you lost in the sun, because it is kind of bright out here today and I was thinking we could grab a beer later. Error assignment permeates offensive statistics too, but to a lesser degree in that it's not the whole thing. Fielding percentage does not measure if you made a good play or a bad play, it measures how often some observer has judged that you were capable of doing better and then goofed.

If that seems incredibly silly to you -- not to mention a large departure from the type of data that we want to put into our TeamImprover5000 to fairly judge players on the most important criteria -- then we are of one mind. Not only are we using grossly subjective data, what we are measuring -- screw up frequency -- is not a particularly good indicator of defensive proficiency.

All other things equal, there's a good chance that a player who is routinely charged with fewer errors is a better defenseman than one who makes more. There is a correlation between defensive blunders and defensive quality, but it's a fraction of the story. A fraction that exists far too much in Michael Kay's realm for me to buy much into its direct analytic usefulness.

***

The problem that I often have with the presentation of statistics is the lack of clarity regarding uncertainty. At their core, statistics are a sample of a population intended to give insight about the whole thing. It is impossible to do so without some margin of error and the size of that margin of error is a very important part of the picture. Here's what I mean with some completely invented figures and clear proof that I have a bias against your candidate.

Recent polls have suggested that Barack Obama is 52% likely to win the state of Ohio, plus or minus 0.5%.
Recent polls have suggested that Barack Obama is 52% likely to win the state of Ohio, plus or minus 5%.

Those two figures mean entirely different things. One is damn, the economy strangling socialists can't be stopped! The other is damn, those planet destroying, Bible thumping, right wing nutjobs still have a chance!

But you rarely see numbers like that, do you? You just see that Barack Obama is 52% likely to win Ohio. As the example that I gave was political, we can assume that the withholding was intentionally disingenuous and not a mere oversight. In baseball statistics, we would expect less deliberate manipulation of data, but the concept of a confidence interval is no less important.

Some fans and analysts flee from advanced defensive statistics because the error bars are wider than their offensive counterparts. Defense is hard to quantify, there's a lot more nuance to soak up, and that field is still very much a work in progress.

A bad response from an analyst would be to throw out all data and claim that the only way to truly talk defense is to watch the games and sacrifice lambs to the Dark Lord Satan.
A good response from an analyst is to know what uncertainty lies in the data, where it is, and how precise to expect their data to be.

With information like UZR at our disposal, saying that a player is a +10 run defender is about as incomplete as saying that Barack Obama is 52% likely to win Ohio. If you take into account career and age trends as well as sample size it's easy to imagine one player rating as a +10 plus or minus seven runs and another rating at +10 plus or minus three runs.

While such a concept is relevant to any statistical analysis, in our little world of baseball, it is most essential in measuring defense, the area in which we have the most error and the most disagreement between methods.

***

I wrote this entire post because I wanted to compare Brendan Ryan and Derek Jeter. But there were some tangents that I wanted to explore before I got to this, so you had to scroll all the way down here to read another guy on the internet say that Derek Jeter doesn't play very good defense and be indignant about it.

Before I present the numbers, there will be one final ado: Brendan Ryan is a fielding wizard who has a career .327 slugging percentage to distract people from said wizardry. Comparing a defender to him is like saying "well here's how some hitter stacks up against Miguel Cabrera." Guess what, he's probably not going to be as good as Miguel Cabrera. In addition to the aforementioned fielding excellence, Ryan was picked to compare because of some distinct similarities that will be made clear in due course.

Derek Jeter played 1186.1 innings at shortstop in 2012, seeing 320 balls in his zone. He made the play on 239 of those (74.7%) and made 61 plays outside of his zone. He was a part of 64 double plays and made ten errors.

Brendan Ryan played 1170.2 innings at shortstop in 2012, seeing 309 balls in his zone. He made the play on 252 of them (81.6%) and made 87 plays outside of his zone. He was a part of 99 double plays and made nine errors.

(Link)

It's impossible to ever say that two defenders had equal chances, but we can say with decent confidence that they were very close. They played a nearly identical number of innings, they had roughly the same number of balls considered to be in their zone, and they played behind pitching staffs with similar ground ball rates (44.9% for the Yankees, 42.9% for the Mariners).

In fact, if we were to infer either player having a slight edge in opportunity, it would have to be Jeter. And yet Ryan was leaps and bounds better at recording outs. A conservative estimate would probably be forty baserunners worth of difference. As Ryan and Jeter both have long-standing histories of this kind of range difference, we can be more confident with our findings than with the average defensive valuation.

But here's what this comparison is really about, because it's not to reopen that Jeter can of worms more than I already have. Defensive range and efficiency have long been considered a kind of neat add-on to the more traditional fielding percentage mentality of don't throw the ball into the stands.

My favorite veteran player might not make flashy highlights anymore, but he's solid and he makes all the plays he gets to.

In addition to being trite, that line of thinking has long pushed the minority over the majority as far as what is important. Of last season's qualified shortstops, Starlin Castro was the leader with twenty-seven errors. No one else had more than twenty. Since 2000, the single most errors in a season by a shortstop was Jose Valentin with thirty-six.

A very good error-avoiding season for a shortstop is one in which they only have around five to ten. A very error-prone season will result in around twenty-five to thirty. So the difference between someone particularly good and particularly bad is somewhere around twenty baserunners over the course of the season. We just put a conservative estimate at a difference of forty baserunners between Jeter and Ryan as a result of their range.

*Not all baserunners are created equal and reaching base via error creates more runs than hitting a single, but the difference is slight.