ProBasketballNews.com Relaunch Kicks off With a Look at Basketball Statistical Analysis
I am pleased to report that Sam Amico's fine website ProBasketballNews.com is alive again after taking a hiatus during the offseason. I will be contributing original content to PBN.com on a regular basis but for the site relaunch we decided to run a piece that first appeared at 20 Second Timeout recently, namely my explanation of why most current forms of basketball statistical analysis should not properly considered to be a science:Sorry, Stats Analysis Not a Science (2/11/09 Note: PBN recently relaunched again and the 10/6/08 article link was destroyed, so I have reconfigured this link to simply go back to the original 20 Second Timeout post).
I don't expect that people who are making good money peddling their formulas and evaluations will stop doing so just because I have pointed out the flaws in their methodologies but it would be nice if someone who has the resources to do so would at least look into the way that certain statistics are kept, most notably assists. Wouldn't you like to know how many of Chris Paul's league-leading 925 assists last season were really assists under rule book definitions? I don't mean to single out Paul but as regular visitors to this site already know, during last year's playoffs when I was writing a recap for the Hornets' 101-82 game one victory over the Spurs I noticed some irregularities between the official play by play data regarding Paul's assists and what I had observed while watching the game. When I went back to the tape and reviewed those plays I found that several of them were improperly scored and the same thing was true when I examined a subsequent Hornets playoff game. Frankly, I find it disturbing that no prominent figures in the stats analysis community have made any kind of public response to my findings; it's as if they don't care whether or not the basic data that they are plugging into their formulas is accurate. As an example of why this is important (beyond the obvious reason that using faulty basic data leads to errors in the final stats formulas), a lot of people assert that Chris Paul "made" David West into an All-Star. No one would dispute that West benefits from playing alongside a great point guard but the reality--as I demonstrated in the two posts mentioned above and as anyone who watches Hornets' games with an educated eye understands--is that West is a phenomenally skilled player who creates a lot of his own offense because he can post up, shoot face up jumpers or face up a defender and drive to the hoop. However, when scorekeepers award Paul assists on a large number of West field goals for which Paul did not really deserve assists this not only skews Paul's rating in various stat systems but it also gives a false impression regarding West's game/skill set.
I don't have the resources (or time) to track every one of Paul's assists from last season--let alone every assist awarded in the NBA--but surely someone at one of the big media conglomerates (or one of these academics who does nothing but crunch NBA numbers all day) has the wherewithal to really study this. Instead of claiming to provide accurate player ratings for every single player in the NBA, it would be nice if one or more of these "stat gurus" actually found out how reliable the basic data is in the first place. Of course, it is highly unlikely that anybody will ever do this, for several reasons:
1) There is no money to be made in conducting such research and, indeed, if the basic data is proven to be faulty to a significant extent then some of these guys could stand to lose money because their prior work--or at least the some of the conclusions that they drew about players--would be discredited to a certain degree.
2) It would take a lot of time to do this research properly.
3) Many of these "stat gurus" probably do not even know how to properly score an assist and thus they would not be qualified to do this research in the first place!
Of course, none of this will stop the "stat gurus" from breathlessly telling us two weeks after the season started the "exact" ratings for every player in the NBA, all while making snide remarks about the GMs, coaches and scouts who make their livings doing proper, in depth player analysis.
As former NBA head coach Eric Musselman recently discussed, there are certain abilities and traits--for instance, performing well in "clutch situations"--that can only be discovered and properly evaluated by watching players perform, preferably in person:
To find these players, you have to put down the stat sheet and go into the gyms of the world and talk with them. Meet them. Look into their eyes. Most of all, you have to watch them perform under certain conditions. Seeing it on film is one thing, but seeing it up close and in person will give you a much better idea of their "clutchness." When talking with those who have "it," you'll know. You can sense it.
Clutch is an attitude, a mind-set. Without a doubt, it's something we're born with--an inherited quality. I've seen it in 10-year-old kids. How they carry themselves when a (Little League) game is on the line. It's remarkable.
I saw it in high school and in college. And I've seen it coaching in the pros--both at the minor league and NBA level.
But you can't find it on a stat sheet. Two players could have identical stat lines and one has "it" and the other doesn't.
I know that most "stat gurus" swear up and down that "clutch play" does not exist--but anyone who has competed at a meaningful level in anything knows that there is such a thing as performing well in the clutch, whether that consists of elevating one's own game under pressure or simply not having one's performance/efficiency/awareness in such situations decline as much as other people's performance/efficiency/awareness declines. The bottom line is that when the outcome of the game is in doubt, some guys want the ball and some guys don't, even if they say that they do; you can see it at any level: some players make sure that the ball winds up in their hands, while other guys come off of the screen just a tick slower than usual to make sure that the ball is not passed to them. Before Michael Jordan hit "the Shot" to beat Cleveland, he had to elude two defenders just to receive the inbounds pass; plenty of other players would have just given up and said afterwards that they would have loved to take the shot but they simply could not get open.
Statistical analysis of the sport of basketball is a wonderful idea in theory, provided that the people who are doing the analysis--and the people who are using the final product of the analysis--understand the exact nature and limitations of these numbers.
Labels: Chris Paul, David West, Pro Basketball News, Sam Amico
posted by David Friedman @ 5:42 AM
