"Stat Gurus" Learn Value of the Eye Test
The idea of using "advanced statistics" to better understand a sport and to more accurately evaluate teams and players has its roots in baseball. Bill James and other pioneers came up with numbers and interpretations that went well beyond the box score. There is certainly a place for that kind of creative thinking. However, the "eye test"--firsthand evaluations made by qualified observers--cannot be replaced by just looking at numbers. "Stat gurus" are often quick to dismiss the "eye test" as subjective and outmoded while at the same time ignoring the inherent limitations of their preferred methodologies.Baseball "stat gurus" are particularly proud of some of the defensive metrics that they have developed. A recent Wall Street Journal article by Andrew Beaton and Michael Salfino termed the quest for accurate defensive numbers "nothing less than the holy grail of baseball statistics." For a while, the "stat gurus" thought that they had solved this problem but now it turns out that some of the much ballyhooed "advanced" defensive statistics are not all that they were cracked up to be, particularly since teams employ defensive shifts so often that it is difficult to precisely determine how much credit a player should receive for a particular defensive play. Beaton and Salfino note that defensive shifts make "it almost impossible to assign proper credit for a great defensive play: Did a player make a play because he has incredible range, or because prescient scouting had him stationed in that area?" Beaton and Salfino conclude, "The result is something that no one would have predicted: Eyeball scouting may be more necessary than ever."
It is obvious that it is important to gather as much useful data as possible and to figure out the best way to interpret that data--but raw numbers (and even "advanced" statistics) cannot take the place of the informed eye. It is like the difference between looking at Cliffs Notes and actually studying the course material; the Cliffs Notes may tell you where to look in a general sense but they do not replace the methodical, step by step learning process. An "advanced" statistic may suggest that a certain baseball player has great defensive range or that a certain basketball player is offensively efficient but it is still essential to watch that baseball player or basketball player in action to place those numbers in context.
Basketball is an inherently more difficult sport to quantify than baseball because basketball involves 10 players moving around at once while baseball is a series of discrete actions. The fact that baseball "stat gurus" are realizing that some of their "advanced" numbers are of limited value should give pause to anyone who blindly relies on "advanced" basketball statistics, particularly regarding defense. If defensive range is hard to quantify in baseball, then it is logical to assume that it is even more difficult to quantify individual defense in basketball; a perimeter player could be horrible defensively and yet his "advanced" numbers may not look bad if he is surrounded by teammates who cover up for his mistakes.
Labels: "advanced basketball statistics", "stat gurus", eye test
posted by David Friedman @ 10:10 AM
7 Comments:
Great article. Statistics, especially advanced statistics, are a tool. Scouts, GMs, coaches, and fans should have statistics in their toolbox, but you can not rely on any one tool especially when evaluating basketball players. Because most of us fans do not have the time or ability to watch the great multitude of players in action, I think a lot of fans tend to over-rely on stats.
Basketball is one of the most beautiful sports to watch because of all the inter-related activity and constant motion on the court. One of the best tips I ever read for watching basketball to learn and see more of all the inter-related movement during any play was from Charley Rosen, "focus on one player (not the ball) and follow him throughout a series of possessions". Watching the game in this way has really increased my understanding and appreciation for the game.
I've always thought the basic misunderstanding about numerical measures of basketball is that they are the RESULTS of what has happened rather than the CAUSE. Useful? Absolutely, if kept in context. But to use a simple example: for a coach to look at numbers for the reason for his team's loss and say "Oh, we only shot 41% from two point range. We need to up our field goal percentage!" Of course, the same team may shoot 61% the next week because they got better shots or they ran more etc. The numbers told the results of many intricate factors that make basketball by far the most compelling team game. One can make the numbers look better by many different adjustments that can best be uncovered by watching the tape.
Unknown:
Thank you.
I think that the Rosen quote you are referencing comes from his book "God, Man and Basketball Jones," one of my favorite basketball books.
Mike S:
You are right; numbers can tell us at least some of what happened but often they can tell us very little about why it happened or how to make it happen (or stop it from happening, as the case may be) in the future.
I wonder if Paul Westhead wasn't an early overinterpreter of numbers. He thought if you really just got more shots up you would score more points and win most of your games when he completely underestimated how many points EVERY NBA team COULD score at a fast enough pace with little enough defense.
This is why we have the cliche "Defense wins championships". That sounds disintuitive (doesn't the team with the most points win?) but offense is a given, meaning everyone wants to score. Most kids learning to play basketball spend a lot of time shooting alone. You don't go out and practice defense alone. So the difference is shown at the highest levels by most effective defense. Do you still have to score? Of course, but top level defenders are going to negate one dimensional scorers most of the time.
Largely true to a lesser degree in other sports. Good pitching often shuts down good hitting- not good hitting overcomes great pitching. Great defensive foot ball (Seattle) can disrupt what has been high scoring offenses. The great offensive team is faced with something they had not seen before (a defense that can take away what they do best) and it totally disrupts their whole rhythm and identity as a team. The team that defines itself defensively always has their identity as long as the effort is present.
Mike S:
Defense depends on effort and toughness and positioning, things that players/teams can do every game if they are focused. Offense relies on timing and skill and finesse, traits that are more difficult to replicate on a day in, day out basis. I think that is why defense wins championships, because the great teams defend well consistently and it is more difficult to consistently play at a high level offensively. A classic example of this is game seven of the 1998 ECF between Chicago and Indiana; the Bulls could not make a shot but they defended, they crashed the offensive boards and they won a hard fought game.
Comment #1 Unknown was from me. I apologize. I didn't realize when I posted that my name would be displayed as "Unknown".
Post a Comment
<< Home