20 Second Timeout

Charley Rosen Explains Why Statistics Can be Very Misleading

The title of Charley Rosen's article about the limitations of basketball statistics says it all: The Numbers Game is Fraudulent. The two part series deserves to be read in its entirety but here are some choice excerpts to whet your appetite (the passages are not numbered in the original piece):

1) Yet even as Sabermetrics has become the latest craze in the NBA (as per the recent hiring of the stat-crunching John Hollinger by Memphis) there are many fallacies in this way of evaluating basketball players. The primary one being that most statistics tally what happens when a player either has the ball in his possession or is in close proximity to the ball. Since there are 10 players on the court, 80-90 percent of the game is ignored. 

Consider assists: Used to be that a pass-catch-shoot-and-make was the only sequence that led to an assist. Nowadays, an assist can be awarded when the pass-recipient takes one dribble and two steps before his make.

2) Points scored is another misleading number. Points racked up by players on bad teams are relatively meaningless. It’s when a player scores that really counts, plus the circumstances. Amassing stats on how many fourth quarter points somebody scores, or how many times he scores in the last two minutes of a close game, etc. are also limited evaluations. Did a player score because a defender missed an assigned rotation? Or the player grabbed a teammate’s airball and wound up with an uncontested layup? Or was it a breakaway dunker created by a teammate’s steal, offensive rebound, and or perfect outlet pass?

3) Let’s examine minutes and games played. In NBA.com, minutes are given in whole numbers. This means a guy who plays one second and one who plays eighty-nine seconds are both credited with one game and one minute played. This eighty-eight second difference could conceivably include at least four possessions, and could therefore be the difference between a totally insignificant appearance and a critical one. 

4) Now we come to my favorite group of people...According to Ed Rush, the one-time supervisor of NBA officials, the calls made by the league’s refs are correct 92 percent of the time. But that means that eight percent of fouls and infractions called are erroneous – even with late-game access to tape replays. Plus, Rush offered no approximation of how many times the refs err by not tooting their whistles... 

If the refs have a distorted view of the game, then statistics such as free-throws attempted, fouls, turnovers, and even wins and losses are anything but objective. 

So what are we left with?

Free-throw shooting. Three-point accuracy. 

To truly judge the effectiveness of any given player, disregard the numbers and disregard the ball. Instead watch what he does when he doesn’t have possession or isn’t even in the neighborhood of the ball. Does he set weakside picks? Make cuts that open up space for a teammate? Make the pass that leads to an assist pass? Box out? Make appropriate defensive rotations? Get back on defense? And so on. 

After all, in no circumstances can numbers ever measure the human spirit. 

Rosen's point about assists is particularly relevant and also easy to verify; Chris Paul is indisputably a great passer but I have repeatedly documented that his assist totals are inflated. I don't know if the NBA has unofficially broadened the definition of an assist to the extent that the statistic is rendered almost meaningless or if certain scorekeepers favor home players and/or star players but it should be obvious that if the raw box score statistics are not trustworthy then the so-called "advanced basketball statistics" must be taken with a heavy grain of salt.

Many "stat gurus" are all too willing to ignore the problematic nature of box score statistics while at the same time insisting that their proprietary "advanced" numbers are infallible but Dean Oliver is one researcher who consistently communicates in a very measured tone about the limitations of "advanced basketball statistics." Oliver's recent ESPN.com article is a welcome departure from the haughty tone and wildly outrageous claims that far too often characterize the writing produced by proponents of "advanced basketball statistics." Oliver has high hopes for the future of "advanced basketball statistics" but he also candidly admits, "The problem with player value metrics is that there is little to validate them, meaning that no metric has established itself as clearly the best. Metrics couldn't even be fairly compared. As each new metric has been developed, it has served mostly to complement traditional scouting, a way to reality check when subjective opinions formed by watching and hearing about players were going too far astray." Oliver believes that, for now, "advanced basketball statistics" are more valuable in terms of evaluating a team's effectiveness and best lineup combinations as opposed to producing allegedly definitive individual player evaluations.

Labels: "advanced basketball statistics", assists, Charley Rosen, Chris Paul, Dean Oliver

posted by David Friedman @ 2:29 AM

0 comments