Looking for a comparison of defensive metrics

[weskelton]

I recently picked up a copy of Dewan's Fielding Bible. In the book, Bill James provides a description of a defensive metric which he calls Relative Range Factor, that is not based on PBP data. I haven't heard much discussion of RRF. I realize that using PBP data is now the preferred method, but I was wondering if anyone had done a detailed evaluation of RRF or other open data fielding metrics to see how well they correlate to their PBP counterparts.

Actually, I'd be interested in links to any studies that have been done correlating the various defensive metrics to one another.

[MB22]

Not sure if you've seen it, but here's some pretty cool stuff from Justin (no Relative Range Factor, though): http://jinaz-reds.blogspot.com/2007/...paring-of.html

[Tango Tiger]

You'll also note how well Fans' eyes do

[jinaz]

Thanks for linking it. At some point, I'd like to go back and add Rally's TotalZone and Dan Fox's SFR to the comparison, as their approaches are probably the gold standard for when we don't have hit-location data available--my bet is that they beat the pants off of the DT's for historical and/or minor league work.

Rally might have released '06 data at the time (and I just forgot about it), but much of his and Dan Fox's work on their systems occurred after I wrote up that study.
-j

[weskelton]

Jinaz,

Great article, very informative. I'm curious, what is it about Rally's Total Zone and Dan Fox's SFR that makes you convinced that they would be superior to BP's DT's?

Do you have any opinion on Relative Range Factor?

[jinaz]

Jinaz,

Great article, very informative. I'm curious, what is it about Rally's Total Zone and Dan Fox's SFR that makes you convinced that they would be superior to BP's DT's?

Just that they're based on a bit more granular data.

DT's are, as I understand them, basically just range factor, with a "fielding factor" adjustment based on balls in play, groundball/flyball rates, and the number of innings by lefties and righties by the pitching staff. This allows them to better estimate chances for each individual. But they're still based on year-end defensive totals, which are arguably not specific enough for this kind of work. Here's a link.

TotalZone & SFR are based on play-by-play data from retrosheet, and as often as possible includes a rough indication of where the ball was hit. They also know who was hitting, who was pitching, and where they were hitting/pitching. As such, they're able to estimate opportunities a bit more precisely than the DT's (or, presumably RRF). ... but probably not as well as the systems that use BIS or STATS hit-location data (though there are enough inconsistencies between the BIS and STATS data that I'm not hugely confident on this latter point).

Anyway, Rally would be in a much better position to talk about his system in greater detail than I am. I only understand the basics of how it works. But his and Dan's comparisons indicate that their systems work pretty well vs. UZR, which is good to see. I'd like to compare them using the exact same players and innings that I used in my study, just to have it be an apples to apples comparison.

I won't have time to do that in the near future, though. Unfortunately.

Do you have any opinion on Relative Range Factor?

Not really, mostly because I haven't looked at it very closely. I was much more interested in the plus/minus system when the fielding bible came out, and therefore only gave a cursory look at relative range factor. And now my copy is packed away in storage in preparation for our move this summer...

My guess, though, is that it's probably going to perform similarly to DT's, because it's based on similar data. That's just a guess though.
-j

[Tango Tiger]

Justin is right. Clay is trying to estimate what Dan and Rally already know. We have for the last 50 years knowledge of whether a pitcher and batter was left handed or righthanded when a certain shortstop was on the field. Why in the world would we try to create a function to estimate this knowledge?

Clearly, you can do so for the pre-Retrosheet years. But, it is simple laziness that it's not done for the Retrosheet years. Laziness is ok. I'm lazy about lots of stuff too, as is everyone here. Some things in life are more important. But laziness is the only justification here.

And for that reason, Dan and Rally's method will be superior to Clay's. It would be a difficult task to be able to use more knowledge but create a worse system. You'd have to make a series of systematic blunders. Dan and Rally don't do that, so, ipso facto, their system is better.

"Caaaaaaaaaaaaase closed."
-- A. Bunker