Does Anyone Else Here Think Sabermetrics is a Sham?

[Brian McKenna]

i'm not a big studier of sabermetrics but i understand the value of numbers and their unbiased appeal. i would rather hear a narrative than a numerical display but sabermetrics has no reason to defend itself. they have and continue to developed baselines for studying the game. the reasoning is sound and it has added a great deal to the understanding of the game within the game.

the baseball viewing public is obsessed with who is better/best/worst (just look around this forum that is what they talk about ad nauseam). sabermetrics is ideal here

sabermetrics is not moneyball - and moneyball is not what i keep reading it to be. it is not a magical concept to amass the ideal team - it is a means to draft and otherwise fill holes in your roster within the available resources.

many of the naysayers to sabermetrics hold up one team or one individual or one season or one playoff run and scream that it doesn't work. ridiculous, sabermetrics sets the guidelines so you know that that one team, etc. stands out on certain points --- statistics set the median line, of course, data falls above and below that line

[digglahhh]

[QUOTE=bkmckenna]the baseball viewing public is obsessed with who is better/best/worst (just look around this forum that is what they talk about ad nauseam). sabermetrics is ideal here

Yes, that is exactly why some people on this board go overboard because it is such a valuable tool for what WE do. That is different than saying it is an undeniable truth.

[digglahhh]

I'm really getting sick of people only taking what they want from my posts.

First of all, I do not dismiss sabermetrics. I have cautioned people to follow them carefully. Over the long haul, they are a better way of predicting than relying upon our subjective observations and sketchy memories. What I am saying is that IN THE POSTSEASON, they go out the window because the nature of the game is reduced to short series, and the way the game is played changes somewhat. Rotations shorten, different guys hit the pen, teams have individual match-up issues, (mostly righty or lefty staffs or bats, speedsters vs. poor throwing catchers, whatever). These are relationships that average themselves out over a season, but in a short series...

The fact that their predictive value is heavily comprimised when it comes time to crown a champ, is a big knock on the field, IMO. Sure, you can say that there is value in using SABR to get to the postseason, or that traditional analysis have their own flaws, but what kind of defense is that.

I say,"well, you know that in America we don't really have free speech."

Then you say, "well, in Iran, they'll kill you if speak blaspheme about Mohammed."

Okay, so our system is better than tyranny- where does that leave us, that's the end of the discussion?

I say, saber has its own problems. You say, it has less problems than subjective memory. So what? It is better than a crappy method, wonderful. Its also better than flipping a coin, or throwing a dart at a list of teams with a blindfold on. and...

No matter what sabermetricians say- it is not objective. Objectivity does not exist when assumptions need to be made- period. The subjectivity is just too enshrouded in advanced math for the average fan to discern.

[Twinskoop]

Honestly, most of the problems that the SABR crew has with getting their "message" out is plain ol' arrogance; a condescending attitude mixed with listening and comprehension deficiencies. Well, that, and the fact that, like most would-be scientists and specialized "experts," most of them are terrible writers.

On this thread, I see a lot of people making the same point - that while SABR is a useful tool, it is not an end-all, or panacea. Some of you might call yourselves SABR disciples, others undoubtedly consider yourselves traditionalists. And from what I can tell, most of you understand the core principles of baseball - that preventing runs and scoring runs are of equal importance, and that while statistical analysis based on past performance might be a useful tool, it can't substitute for traditional tool-projections, coaching, and in-game strategy.

We all know that a lineup of 9 LF/1B/DH types won't work, regardless of OPS. Just like a football team of 55 Tom Bradys wouldn't win any football games.

[ssbguyincognito]

A nice summary of your perspective, digglah.

What I am saying is that IN THE POSTSEASON, they go out the window because the nature of the game is reduced to short series, and the way the game is played changes somewhat. Rotations shorten, different guys hit the pen, teams have individual match-up issues, (mostly righty or lefty staffs or bats, speedsters vs. poor throwing catchers, whatever). These are relationships that average themselves out over a season, but in a short series...

This is most certainly true, but you're still missing the point. Sabermetrics represents a long-term strategy on how to make the best team. Every team has a fixed amount of resources (even the Yankees), so the question is, how do we maximize our output given our finite resources? Traditional baseball thinking tried to answer this too, but through historical, empirical, and mathematical studies, it was shown that the historical way of looking at the game is wrong. For example: teams who emphasize batting average won't score as many runs as teams who emphasize on base percentage. A fast Jose Reyes leading off isn't worth nearly as much as a fat Giambi leading off.

Sabermetrics is not trying to predict the outcome of every single game and say if we were wrong then it must be due to chance. There is chance in everything, and that chance is uncontrollable. Sabermetrics is just a newer, more scientific, more advanced way of doing the same thing traditionalists do -- analyze the game -- but we do it better.

A team cannot control the inherent randomness in the game. But it can maximize its chances of winning by getting the most output from their finite resources.

No matter what sabermetricians say- it is not objective. Objectivity does not exist when assumptions need to be made- period. The subjectivity is just too enshrouded in advanced math for the average fan to discern.

This I have an issue with, since it's blatantly not true. I'm not sure how much math you have taken, but every system of logic employs the axiomatic method. We assume certain things to be true (axioms), then try to derive much more complex theorems from these simple axioms. The point is, if you're going to assume something to be true without proof (you have to), make sure it's as simple as possible. None of this is subjective. An example of an axiom would be: the goal of an offense is to maximize the runs it scores. It's simple enough. A bad axiom is: the goal of an offense is to maximize its batting average, becuase any empirical study will show you, there is significantly less correlataion between runs and batting average than runs and other things.

[Ubiquitous]

Everything goes out the window during a short series, not just sabremetrics. Every single way to build a team has lost in the playoffs and every single way has won.

[digglahhh]

There are truths that one has to assume, I agree.

An offense's goal is maximize run production.

A defense's goal is to minimize run production for the other team.

A CS is more detrimental than an SB is valuable.

OBP is more valuable than BA.

I agree with all this stuff. The subjectivity involved is how you weight the successful completion of your respective axiomatic goal on an in individual basis. Walks are productive, okay, how productive in comparison to a single? Therein lies the questions. The same axioms apply to traditional perspectives, the goals are always the same.

Build your team however you want. I believe that sabermetrics are an important part of team building. There are many aspects of research that shoud be included in building, managing and organizing a team.

I have more of a problem with the derived metrics of measuring individual player rankings. Believe me, I would love to agree with the conclusions these metrics reach. I would love to quantify the intangibles. But I would rather leave them alone than quantify them incompletely, or with biases.

How many times do you hear people say, well system A overvalues corner IF defense, system B overrates slightly above average production and underrates great production? Many people look for ways to support their subjective claims, regardless of the research they have. They selectively choose certain metrics for certain players, make arbitrary adjustments for league strength, era, etc.

What is so harmful about not being sure? Is it an indictment of your baseball knowledge to conclude after long research I prefer Mays to Mantle, but the debate could go either way, and if you prefer Mantle that is entirely understandable? Since when is certaintude a useful tool for the advancement of knowledge?

Perhaps, I am guilty of bias in this debate. Bias not against the studies, but the way the conclusions are sometimes used.

[Twinskoop]

Sabermetrics represents a long-term strategy on how to make the best team. Every team has a fixed amount of resources (even the Yankees), so the question is, how do we maximize our output given our finite resources? Traditional baseball thinking tried to answer this too, but through historical, empirical, and mathematical studies, it was shown that the historical way of looking at the game is wrong. For example: teams who emphasize batting average won't score as many runs as teams who emphasize on base percentage.

A perfect example of how someone might go about confusing moneyball principles with advanced statistical analysis. SABR has nothing to do with the resources a team has. A team with finite resources benefits from SABR only if those SABR principles happen to be undervalued in the current market. If anything, with the publication of "Moneyball," I think we'll find those principles to be overvalued in the near future. Teams with limited resources will have to find other areas to exploit, and you better believe guys like Billy Beane will find them.

A fast Jose Reyes leading off isn't worth nearly as much as a fat Giambi leading off.

Which might be germane if they played the same defensive position. But they don't. A fat Giambi playing shortstop would be a sight to behold, though would likely cost his team a lot of runs. A less blatantly subjective analogy would be to assert that a quick defensive specialist like Doug Mientkiewicz hitting anywhere in the lineup isn't worth nearly as much as a fat Jason Giambi hitting in that same spot. And on that point, I doubt you'll find much disagreement.

[digglahhh]

What teams don't try to maximize their resourses? (please refrain from the obvious- my team)

All perspectives begin with goal of maximizing resources. Some think that SABR provides better tools with which to analyze.

[Ubiquitous]

Every team tries to maximize their resources just some don't do it so well. Some waste their resources signing top picks out of high school that never do anything. Some focus their resources on one region and find undiscovered diamonds in their backyard on the cheap. Some use their resources to sign old established veterans while other use their resources to develop youth and then let them play.

[Twinskoop]

What teams don't try to maximize their resourses? (please refrain from the obvious- my team)

All perspectives begin with goal of maximizing resources. Some think that SABR provides better tools with which to analyze.

Simple economics explains most of the discrepancy between teams that succeed in maximizing their resources and teams that don't. It is, simply speaking, a zero-sum game. Teams that don't innovate, simply trying to keep up with what everyone else is doing, counting on getting lucky, will have a harder time breaking away and actually getting the most bang for their buck. If you're doing what everyone else is doing, you're not exploiting undervalued skill-sets, and therefore, not maximizing production.

SABR is one set of tools, and a valuable set. Some SABR principles are more valuable than others, none is a panacea. Traditional tool-scouting, coaching, and strategy cannot be replaced by stat analysis.

There's a familiar basketball expression that sums up the traditional approach to scouting - "You can't coach tall." The flip-side of this idea is that there are some attributes that are coachable, and an evaluator does well to allow for that. SABR principles can only address what a player has done, not what he is capable of doing. Before the 2003 season, lots of SABR guys were in love with Erubiel Durazo, and were predicting MVP-like stats from him. Never happened, never will happen. David Ortiz went to Boston in 2003 with no fanfare from the statheads. His stats weren't SABR-friendly. That was the system he was in. But discipline, unlike power and speed, is coachable. In Boston, Ortiz is an MVP-caliber player, and a SABR darling.

All that is to say that while SABR is a good tool, it is not the only tool. Baseball is a game, not a constant. Games are fluid, and are decided by their participants, not mathematical equations.

[SABR Matt]

diglahhh seems to be of the mindset that in a short series (post-season) setting, sabermetric principles fail and traditional philosophy succeeds, which is of course 100% not true. The paradigm shifts, but you could still use sabermetrics to get an understanding of how to win more post-season games than you lose over a period of several years in the post-season.

For instance...perhaps in the regular season you need to worry some about who your fifth starter is, but in the post-season it might be more advantageous to have a dominant top three than a solid top five. It's all about putting as much value as you possibly can into each situation you're likely to face. In the regular season, someone's got to pitch at least 5 innings (generally) every day and to maximize the value you get on the mound, you need to find servicable guys to fill the back of your rotation. In the post-season...all of your innings go to your best avaiable arms.

In the regular season, you starting position players need rest so you have to have good depth at every position. In the post-season, depth is a strategic thing...used to better single-game match-ups...not to weather the attrition of a long season...meaning a DIFFERENT kind of depth is required.

This may all be true...but it doesn't mean you can't use sabermetric principles to analyze it...you would need to take many years worth of post-seasons to get a large enough sample to test on...but just because no one has studied the best way to team-build for the post-season sabermetrically...doesn't mean it can't be done.

And just because seemingly subjective traditional methodologies have produced winning teams in the past sometimes doesn't mean it always works or work any better than sabermetrically derived team-building strategies would.

There is inherent variability in attempting to apply any ideology over a scant few games. That's the way the ball bounces when you've got just 7 games to play. But...if for instance you're the Yankees and you can just about count on making the post-season every year, you most CERTAINLY can and should attempt to build a long term strategy that maximizes your chances of winning once you get there.

[jalbright]

I did no such thing.

I do not dismiss sabermetrics, I question the amount of stock certain people put in them. I question if all those who espouse them understand how they are formed and are educated enough in the advanced mathematics involved to even have a valid opinion on their relevance. I question their applicability in predicting postseason success. This is not dismissing SABR, this is constructive criticism, it is pushing you guys to address and consider diverse perspectives. This reminds me of the conservative political pundits who claim that any time somebody critisizes America is un-American, when in truth critiquing the country is the most American use of free speech possible.

I don't dismiss the study of science, although, much it too is corporate guided and largely devoid of the altruistic and humane spirit that should guide the field. Anyway, I don't need the level of weather analysis that you guys provide.

Some of my brethren in the sabermetric community want to have everyone believe that their hammer or screwdriver or wrench (pick your tool) is the only one you'll ever need. They've gone overboard, just like one could with a choice of tools. If one wants to cut ones fingernails, a sledgehammer will not provide a nice solution. But if you want to pound some metal flat, a clipper won't help much either. It's even true that if we have all the tools available to us, we can't solve everything. That's particularly true in situations where the sabermetric tools, which are largely based on probabilities, are given a limited number of trials. Playoffs are just too short for everything to balance out. The White Sox played well, but they also caught a very helpful string of breaks. Nothing can predict when such events will occur, because human beings simply aren't that predictable. On the other hand, if you understand how to get predictions for such series right 75% of the time when everyone else is struggling to beat 60%, and there are ways to exploit that knowledge, you have a definite advantage. I happen to believe sabermetrics can provide those kinds of advantages.

Even in ranking players, sabermetric approaches have the defiinite advantages of organizing our knowledge and helping us see things clearly, which is an exceptionally difficult task to accomplish with subjective information for just one comparison, much less for a long series of such comparisons. There are times that the way the numbers come out are ones that initially make us scratch out heads. Sometimes, those numbers are picking up something we've missed. Sometimes, for whatever reason, our assumptions don't balance out on a given individual, and the "system" yields a lesser answer. We can work toward refining those systems so they yield fewer and fewer results which aren't acceptable, so long as we try to be objective about what is "acceptable". I believe we've gone a long way in that regard, though we're certainly not at perfection. I'm not sure that state is achievable, but we can get closer.

Jim Albright

[SABR Matt]

Again...I hope I have not left the impression that I expect everyone to fully understand or make use of my work or any work in sabermetrics...I do however think that for people who really want to know more about how the game works...how to build teams...even how to win more championships if you have enough chances...those folks should be making an effort to learn about probability theory, sabermetrics, etc. It's perfectly fine to just enjoy the game casually...MOST of the time, even I just enjoy the game casually...my sabermetric research does not encroach much on how I watch the games. Team owners/GMs/Managers though would be better served to come to a good understanding about the science of baseball strategy than they would if they continued using the same "old hat" methods they've been using since the 60s.

[Twinskoop]

Some of my brethren in the sabermetric community want to have everyone believe that their hammer or screwdriver or wrench (pick your tool) is the only one you'll ever need. They've gone overboard, just like one could with a choice of tools. If one wants to cut ones fingernails, a sledgehammer will not provide a nice solution. But if you want to pound some metal flat, a clipper won't help much either. It's even true that if we have all the tools available to us, we can't solve everything. That's particularly true in situations where the sabermetric tools, which are largely based on probabilities, are given a limited number of trials. Playoffs are just too short for everything to balance out. The White Sox played well, but they also caught a very helpful string of breaks. Nothing can predict when such events will occur, because human beings simply aren't that predictable. On the other hand, if you understand how to get predictions for such series right 75% of the time when everyone else is struggling to beat 60%, and there are ways to exploit that knowledge, you have a definite advantage. I happen to believe sabermetrics can provide those kinds of advantages.

Even in ranking players, sabermetric approaches have the defiinite advantages of organizing our knowledge and helping us see things clearly, which is an exceptionally difficult task to accomplish with subjective information for just one comparison, much less for a long series of such comparisons. There are times that the way the numbers come out are ones that initially make us scratch out heads. Sometimes, those numbers are picking up something we've missed. Sometimes, for whatever reason, our assumptions don't balance out on a given individual, and the "system" yields a lesser answer. We can work toward refining those systems so they yield fewer and fewer results which aren't acceptable, so long as we try to be objective about what is "acceptable". I believe we've gone a long way in that regard, though we're certainly not at perfection. I'm not sure that state is achievable, but we can get closer.

For the most part, I agree with the spirit of this post. But I don't think we can overlook the fact that while we may think of the numbers that SABR metrics yield as "objective," in order for them to mean anything to us, we must apply subjective normative standards. And I think your analysis is understating the scope that many SABR folk strive for in the development of these measures. Win Shares? Runs Created? These are very clearly attempts to establish broad valuations on players who have, for the most part, very specific skill sets in a game that demands many different skills. This is like trying to develop a single tool that will trim your fingernails and pound metal (not to mention mow the grass and walk the dog).

I'm all for a more modern approach to understanding baseball statistics. When I first read Rob Neyer talk about OPS in 1997, I was knocked over. But in its evolution, it's become more of an idealogical wedge than anything, and that's sad. As you imply, it (the system of statistical analysis) just doesn't do what some seem to want it to do - and it likely never will. It can't replace tool-scouting, coaching or strategy. You just can't take the game out of the game.