ProBasketballNews.com Relaunch Kicks off With a Look at Basketball Statistical Analysis
I am pleased to report that Sam Amico's fine website ProBasketballNews.com is alive again after taking a hiatus during the offseason. I will be contributing original content to PBN.com on a regular basis but for the site relaunch we decided to run a piece that first appeared at 20 Second Timeout recently, namely my explanation of why most current forms of basketball statistical analysis should not properly considered to be a science:Sorry, Stats Analysis Not a Science (2/11/09 Note: PBN recently relaunched again and the 10/6/08 article link was destroyed, so I have reconfigured this link to simply go back to the original 20 Second Timeout post).
I don't expect that people who are making good money peddling their formulas and evaluations will stop doing so just because I have pointed out the flaws in their methodologies but it would be nice if someone who has the resources to do so would at least look into the way that certain statistics are kept, most notably assists. Wouldn't you like to know how many of Chris Paul's league-leading 925 assists last season were really assists under rule book definitions? I don't mean to single out Paul but as regular visitors to this site already know, during last year's playoffs when I was writing a recap for the Hornets' 101-82 game one victory over the Spurs I noticed some irregularities between the official play by play data regarding Paul's assists and what I had observed while watching the game. When I went back to the tape and reviewed those plays I found that several of them were improperly scored and the same thing was true when I examined a subsequent Hornets playoff game. Frankly, I find it disturbing that no prominent figures in the stats analysis community have made any kind of public response to my findings; it's as if they don't care whether or not the basic data that they are plugging into their formulas is accurate. As an example of why this is important (beyond the obvious reason that using faulty basic data leads to errors in the final stats formulas), a lot of people assert that Chris Paul "made" David West into an All-Star. No one would dispute that West benefits from playing alongside a great point guard but the reality--as I demonstrated in the two posts mentioned above and as anyone who watches Hornets' games with an educated eye understands--is that West is a phenomenally skilled player who creates a lot of his own offense because he can post up, shoot face up jumpers or face up a defender and drive to the hoop. However, when scorekeepers award Paul assists on a large number of West field goals for which Paul did not really deserve assists this not only skews Paul's rating in various stat systems but it also gives a false impression regarding West's game/skill set.
I don't have the resources (or time) to track every one of Paul's assists from last season--let alone every assist awarded in the NBA--but surely someone at one of the big media conglomerates (or one of these academics who does nothing but crunch NBA numbers all day) has the wherewithal to really study this. Instead of claiming to provide accurate player ratings for every single player in the NBA, it would be nice if one or more of these "stat gurus" actually found out how reliable the basic data is in the first place. Of course, it is highly unlikely that anybody will ever do this, for several reasons:
1) There is no money to be made in conducting such research and, indeed, if the basic data is proven to be faulty to a significant extent then some of these guys could stand to lose money because their prior work--or at least the some of the conclusions that they drew about players--would be discredited to a certain degree.
2) It would take a lot of time to do this research properly.
3) Many of these "stat gurus" probably do not even know how to properly score an assist and thus they would not be qualified to do this research in the first place!
Of course, none of this will stop the "stat gurus" from breathlessly telling us two weeks after the season started the "exact" ratings for every player in the NBA, all while making snide remarks about the GMs, coaches and scouts who make their livings doing proper, in depth player analysis.
As former NBA head coach Eric Musselman recently discussed, there are certain abilities and traits--for instance, performing well in "clutch situations"--that can only be discovered and properly evaluated by watching players perform, preferably in person:
To find these players, you have to put down the stat sheet and go into the gyms of the world and talk with them. Meet them. Look into their eyes. Most of all, you have to watch them perform under certain conditions. Seeing it on film is one thing, but seeing it up close and in person will give you a much better idea of their "clutchness." When talking with those who have "it," you'll know. You can sense it.
Clutch is an attitude, a mind-set. Without a doubt, it's something we're born with--an inherited quality. I've seen it in 10-year-old kids. How they carry themselves when a (Little League) game is on the line. It's remarkable.
I saw it in high school and in college. And I've seen it coaching in the pros--both at the minor league and NBA level.
But you can't find it on a stat sheet. Two players could have identical stat lines and one has "it" and the other doesn't.
I know that most "stat gurus" swear up and down that "clutch play" does not exist--but anyone who has competed at a meaningful level in anything knows that there is such a thing as performing well in the clutch, whether that consists of elevating one's own game under pressure or simply not having one's performance/efficiency/awareness in such situations decline as much as other people's performance/efficiency/awareness declines. The bottom line is that when the outcome of the game is in doubt, some guys want the ball and some guys don't, even if they say that they do; you can see it at any level: some players make sure that the ball winds up in their hands, while other guys come off of the screen just a tick slower than usual to make sure that the ball is not passed to them. Before Michael Jordan hit "the Shot" to beat Cleveland, he had to elude two defenders just to receive the inbounds pass; plenty of other players would have just given up and said afterwards that they would have loved to take the shot but they simply could not get open.
Statistical analysis of the sport of basketball is a wonderful idea in theory, provided that the people who are doing the analysis--and the people who are using the final product of the analysis--understand the exact nature and limitations of these numbers.
Labels: Chris Paul, David West, Pro Basketball News, Sam Amico
posted by David Friedman @ 5:42 AM
10 Comments:
I have noticed erroneous statistics on boxscores before, so what you are saying does not surprise me. It does ultimately come down to the judgment of the person recording the stats.
There are different definitions regarding what exactly is an assist, but most essentially state, "a pass that leads directly to a field goal."
Dribbling does not erase an assist. I can pass the ball to my SF at the top of the key, and he can immediately take 2 dribbles, attack the basket and score, and I should be credited with the assist. Same thing if we are in transition and I find him early, but he needs a dribble or two on his way to the basket.
As long as you have a continued, direct line to the field goal, there is no problem.
When a player hesitates or starts bringing out the fakes, then the opportunity for an assist should really be over.
I will definitely keep a closer eye on it this season.
You pointed the fact about Chris Paul & assist but I've noticed the same syndrom with Dwight Howard.
Some "team rebounds" transformed into rebounds in the hands of Howard. That's weird but i don't have knowledge to compare that with any others players.
Maybe that's just the way it goes with the new favorite faces of the NBA. I really don't know but i'm waiting for the next season for trying to determine that.
I think you're aware of this website but if someone wanted to complete the official boxscore with more accuracy, test "the pop corn machine" (on google). It would not tell us who is clutch or not but each stat for each player during a game divided into quarters.
David -
First of all, there have been tons of threads on problems with assists at APBR. You will see two threads on the first page. See "assist scoring bias" started by Ben F. for instance.
The WOW has an excellent treatment of assists on 115-117. They began their research thinking assists didn't matter. Their research actually produced a totally different conclusion.
Re Chris Paul - He averaged 11.9 assists at home and 11.2 on the road. So, we are talking about .7 of an assist per game here at most. This probably indicates a small measure of bias. But he also played better at home, as evidenced by higher fg%, scoring, and rebounding. So it stands to reason that his assist total might have been a little bit higher as well, irrespective of bias. Which leaves you with a very small effect. But even if you take away that whole .7 of an assist, he is still far and away the best point guard in the NBA.
Assists are subjective. Turnovers can be somewhat as well, although that comes out in the wash over time. But scoring statistics and rebounding statistics are extremely concrete. And those are the most important statistics both at the team level and the individual level.
David West is a good player, but he simply doesn't score as efficiently as elite players in the NBA at the same position. He posted a ts% of 53.5% last year. Carlos Boozer was at 58%. Amare Stoudemire was at 65.6% You also won't find him in the top 20 in rebound rate. And that isn't just Tyson Chandler. His rate has barely moved over the last four years.
As for Musselman and his post about clutch, it doesn't make much sense, even coming from a former NBA coach, perhaps especially so.
He talks about clutch, then says,"you can't find it on a stat sheet. Two players could have identical stat lines and one has "it" and the other doesn't."
This is just nonsense, pure and simple. It's exactly the kind of thing that makes "stat guys" skeptical of the wisdom of coaches.
First of all, what is clutch if Jordan, Bird, and Magic define it for you. These are three of the best players of the modern era. They were good all the time. They were better than everyone in the first quarter and the same thing happened in the fourth.
And it's not like you can't look at stats and get a crude estimate of how players are performing in high pressure situations. You can go to 82games for instance, and see the following. In 127 minutes of clutch last year, (5 minutes or less, margin of 5 points or less) Manu Ginobili scored 43.3 points per 48. He did that shooting 57.4% 44% 3pt%, and 93% ft%. He averaged 2.6 turnovers, 10.5 assists per 48, 3 steals, 7.2 rebounds, and was +18 in +/-.
Did he have less of "it" than Kobe or Lebron? I wonder what Eric Musselman would say.
JP:
You are correct that dribbling per se does not automatically remove the possibility of awarding an assist but a lot depends on context. While it is true that taking a "direct line" to score--as you call it--is part of the issue, it is also important to consider how much defensive resistance the player with the ball met along the way.
If I get a defensive rebound and throw the ball ahead to a teammate who has no defenders in front of him, even if he takes three dribbles to score an uncontested layup I should still get an assist--but if I get a defensive rebound and throw the ball ahead to a teammate who has a defender in front of him and my teammate has to use a crossover dribble to shake that defender before he scores then I should not be awarded an assist. Just making the last pass before a score is not enough to earn an assist; that pass is supposed to have significantly contributed to the scoring opportunity. If my teammate has to make a one on one move to score, then he scored on his own. However, as you indicate, if I pass to a teammate and he takes one dribble to set himself and goes straight into a shooting motion then that one dribble should not cancel out an assist.
There are two issues here:
1) Awarding an assist is inherently subjective in many cases, which in and of itself introduces a certain margin of error into stat formulas above and beyond whatever limitations may be inherent in the construction of those formulas.
2) Scorekeeper bias adds a further element of subjectivity into the awarding of assists. That bias can come in many forms: it could be for a certain player or players, it could be against a certain player or players or it could even be for or against a lenient interpretation of assist guidelines in general.
awopbopaloobopalopbamboom:
I have not noticed the issue with team rebounds that you described but it would not surprise me if it happens. For those who don't know, a team rebound is how deadball rebounds are classified in the boxscore and they are not, by definition, supposed to be added to any player's total. An example of a team rebound would be when a player misses the first of two free throw attempts.
Anonymous:
I never said that home bias is the reason that Paul's assists were incorrectly scored in the games that I looked at, so the home/road split is irrelevant. Assists are subjective in general and there may be a league-wide tendency to more generously award assists to top playmakers. Oscar Robertson and other retired players have mentioned to me that they believe that assists are more generously awarded in general now than they were in previous eras. I don't have hard data to back this up, but my subjective impression is that this started to change in the 1980s with Magic and Bird. Both were fabulous passers but I think that coincidentally or not scorekeepers also became more generous during that time.
I did not say anything about West's "efficiency" or make a judgment about whether or not he is better than Boozer, Amare or others. I simply stated that he has a skill set that enables him to score with one on one moves, both in the post and with his face up game. His scoring is not merely a product of Chris Paul's passing.
I know that "clutch" is a real hot button issue but I do believe that it exists. As I indicated, I don't merely define it by production but also by the willingness of a player to accept the responsibility of taking the last shot or trying to make a late game play. Many players shrink from that moment. I think that is part of what Coach Musselman is talking about.
It is interesting to me how stats gurus can bend numbers to whatever way suits them. Sometimes they will say that something is irrelevant because of sample size but other times they are perfectly content to rely on a small sample size. Are 127 minutes from several different games spread out over an entire season statistically significant or not? If they are significant, to what degree are they significant and with what margin of error?
I think that "clutch" play is particularly hard to quantify because it is not merely what happens in "late and close" situations. It can also be what happens at the start of the second half when one team is up by eight and the other team goes on a quick 6-0 run--or conversely, when the team that is up by eight goes on a 10-0 run to break the game open. Coaches understand that great clutch players do a lot of damage in such situations but it is hard to come up with the proper definitions to define and quantify such "clutch" production--but just because it cannot be quantified does not mean that it does not exist.
David - Clutch is by definition small sample size. You aren't going to build a statistical model out of 100 minutes of game data. But you can get a real answer to the question, how did player X perform in close and late situations last year. Certainly, you shouldn't draw any hard and fast conclusions from that data. You shouldn't say for instance, Manu will be much better in the clutch than Lebron and Kobe this coming year. But as a record of what actually happened, it has value. And I brought up Manu because I wanted you to know how exceptional he was last year. If I am doing the math right, he converted 78% of his scoring opportunities in the clutch, as defined by 82games, while scoring at the fourth highest volume in the NBA.
Musselman is just daft. The absurdity of citing Omar Minaya as an authority on clutchness. Honestly, it's hilarious.. The players Minaya has put out on the field have choked themselves out of the postseason two years in a row, staging a historic collapse two years ago. (I don't blame him for that, I like Omar, but under the circumstances he sounds like a nincompoop.)
Musselman apparently thinks that two players can put up the exact same stats, and one can have "it" and the other doesn't. That suggests to me, that "it" has nothing to do with any kind of basketball performance that can be measured statistically. Which is a little bizarre. Clutch doesn't involve shooting, scoring, or rebounding. Helpful to know.
Honestly, that article is a real dud. I think there are interesting things to be said about performance in "clutch" situations. I think it would be fair to say that wing players like Kobe and Manu do have a huge advantage in these situations relative to bigger players who don't have as many "skill sets." It's a time of the game where being multifacted really helps you a lot I think. And you could do some very interesting data mining through 82games to build a better data sample. However, Musselman says nothing of interest, just, that he thinks there is a thing called a clutch, that he can't define it, and that the three best players in the NBA the last 25 years had it.
Finally, a very interesting article by Romer, posted on Mankiw's blog, on models, statistics, and economists. It basically talks about the same debate we are having, but in the field of economics. A snippet:
"The key difference lies in the relative weight each side gives to formal models as opposed to judgment. Fundamentalists have an unswerving faith in models. Policies should always be derived from the best available model. Data should be filtered through a model. If an observation does not fit within the context of a model, it should be excluded from consideration.Realists are more conscious of the limits of models and more comfortable with a division of labor between the researcher who improves the models and the clinician who makes policy decisions. They recognize that the power of models comes precisely from a commitment to abstraction that filters out potentially important complexity."
http://gregmankiw.blogspot.com/2008/10/do-you-take-models-too-seriously.html
Owen
Owen:
As always, you make some interesting and well articulated points, even if I don't agree with all of your premises (or conclusions).
I agree that Manu is a very good clutch player. I don't need the limited data to come to that conclusion because I have seen him play a lot and I know that he not only has the skill set to create shots for himself and others but that he is fearless in such situations.
You are correct that wing players have an advantage over post players in "clutch" situations. This is an aspect of Kobe's contributions to the Lakers' three championship teams that many people underrate. They had a lot of close games in which they needed Kobe to create something for himself or a teammate and that is a role that Shaq, for all of his dominance, could not fulfill.
I don't intend to get into a whole side discussion about the Mets, but isn't it possible that Minaya understands what "clutch" is but he has simply not been able to obtain players who exemplify this quality?
I certainly think that it is possible for two players to put up the same (or very similar) numbers but to have very divergent "clutch" abilities. Skill set differences are very important here, because a player who can create his own shot or who can hurt the opponent in many different ways is more likely to be deadly in a "clutch" situation than a one dimensional player. Obviously, the exception to that is someone like a John Paxson or Steve Kerr, players who do one thing really well but depend on better players to create openings for them. Paxson and Kerr had the focus, concentration and discipline of elite players but did not have the physical tools to be All-Stars; when they played against lower level athletes (in college) they were All-American players. Larry Bird has said that any player will take a last second shot when the game is tied but only special players want to take that shot when their team is trailing (everyone says he wants to take that shot but, as I indicated before, a lot of players figure out how to not get the ball in those situations).
Using the terminology in the post that you cited, I am a "realist."
Yes, definitely a realist, Which, I hope was clear from the context, is a perfectly valid thing to be.
I guess I misunderstood the Minaya comment, or conflated it with what Musselman is saying, which is that you can see it in Little League, you can see it when you meet a person, etc. I like Omar, I think he has done a pretty good job, but as a Mets fan to hear him even say the word clutch is rage provoking.
Honestly, any time someones parenthesizes "it," well, it's hard for me to take them seriously.
Finally, Berri has written a post about the Lakers. Perhaps take a small sedative before you read it.
;-)
best
Owen
Owen:
I don't pay much attention to what Berri writes now; his methodology is flawed and therefore his conclusions are not to be taken seriously. Even the other "stat gurus"--with whom I disagree at times, too--don't take Berri's work very seriously.
I did glance at this particular article, since you mentioned it. I did a much better and more accurate look at what I called the Lakers' "three seasons" in a piece that I did for PBN during last season. As I have said repeatedly about "stat gurus," when they "like" certain numbers they run with them but when they don't like them they talk about "sample size" or make other excuses. The "sample" of games without Bynum or Gasol is small and Berri completely fails to take into account strength of schedule, home/road split, etc. He also totally ignores how Gasol's field goal percentage shot up after he joined the Lakers, a direct result of the defensive attention Kobe draws, nor does Berri take into account how much Bynum benefits from playing with Kobe. As I noted in my recap of the Lakers' first preseason game, things are a lot different for Bynum when Kobe is not on the court (and yes, I realize that was a small sample size of possessions, but the problems that Bynum had in those possessions are clearly illustrative of the limitations in his game right now and simply playing more possessions without Kobe will not help until he improves his court awareness and ability to react to double teams).
Post a Comment
<< Home