20 Second Timeout is the place to find the best analysis and commentary about the NBA.

Tuesday, March 31, 2009

If Some of the Numbers are Bogus Then How "Advanced" are "Advanced Stats"?

I have always enjoyed crunching numbers and looking at sports statistics, so I certainly am not some kind of anti-stat Luddite. I am not opposed to using basketball statistics, whether they are "archaic" box score-style stats or the so-called "advanced" stats that have been touted by various "stat gurus"; what I am opposed to is the misuse of basketball statistics, when people either rely too heavily on numbers at the expense of actually watching games and/or do not understand the significance/meaning of certain statistics.

Most of the so-called "advanced stats" are based on taking box score numbers--points, shooting percentages, rebounds, assists, steals, blocked shots, turnovers--and assigning particular weights to each category and/or converting per game numbers into per minute numbers that are adjusted for "pace" (one exception to this is plus/minus, which simply tracks the point differential when a player is on the court and compares it to the point differential when he is not on the court). It should be obvious that even if the weighting formulas are perfect--which they most certainly are not--the "advanced" stats are only as reliable as the basic data that they use. Even excluding the possibility of deliberate bias and/or incompetence on the part of scorekeepers, it should be obvious that assists, steals, blocked shots, turnovers and rebounds are all somewhat subjective in nature, so no player's box score statistics are 100% accurate. That is one reason that I criticize the way that so many "stat gurus" present their player rankings without any mention of a margin for error. Of course, the reason that "stat gurus" shy away from doing this is that they have no way of calculating a margin for error because they don't have a way to quantify how reliable the boxscore numbers may be, nor do they want to seriously consider the possibility that the weightings that they assign to those numbers may be less than perfect; if your "stat guru" competitor does not provide a margin for error but you do then in the court of public opinion you may appear to be less authoritative. I have no doubt that many teams use statistical analysis to some degree when making various decisions but you can bet that the data that they are using is more complete--and more focused on team comparisons as opposed to individual player rankings--than the information that various "stat gurus" are peddling on the open market in various books/websites. Also, many "stat gurus" seem to have thinly veiled agendas pertaining to how the game "should" be played, which kinds of players (guards or big men, rebounders or scorers, "high volume" players who are not "efficient" or "low volume" player who are "efficient," etc.) are most valuable and how competent (or, to be more precise, incompetent) they consider NBA GMs, coaches and scouts to be.

I have done several posts in which I provided detailed, play by play analysis of scorekeeping errors regarding assists. The last time that I charted assists in this fashion, 14 of the 16 assists credited to Chris Paul and Tony Parker fit the rulebook definition of an assist*. The Spurs lost 90-86 in New Orleans on Sunday night and those two players once again combined for 16 official assists (nine for Paul, seven for Parker). Here is how I would have scored those 16 plays:

Chris Paul's Nine Assists

1: David West jump shot, 9:01 1st q: Correct; this was a bit of a borderline play but I will give Paul the benefit of the doubt. West caught the ball, made a slight jab step and then went right up with the shot. It was not a straightforward catch and shoot but only about two-three seconds elapsed between the catch and the shot going up.

2: David West fadeaway jump shot, 5:22 1st q: Correct; this was also a bit of a borderline play where I am giving Paul the benefit of the doubt. West caught the ball on the left block and shot a turnaround jumper over Drew Gooden. West did not put on a whole low post clinic prior to the shot--as I have seen him do on some plays that were scored as assists for Paul (see next entry)--and he did go up for the shot pretty quickly after receiving the ball but it is hard to argue that the pass created the shot in the way that an alley oop lob or a drive and kick pass does. If this kind of pass is regularly scored as an assist then virtually every pass into the post--indeed, every pass preceding a successful shot attempt--would have to be deemed to be an assist.

3: David West jump shot, 4:41 1st q: Incorrect; this play is an egregious example of why I started charting assists in the first place and why I am inclined to consider this stat to be devalued to the point of almost being meaningless. Paul passed the ball to West at the top of the key at the 4:48 mark. West made a jab step with his right leg, then he pump faked, then he took one dribble and made a hard drive to his right before stopping and spinning to his left. The assist should already be well off the table by now but there is more. After coming to a stop, West pump faked again, did the "up" part of an up and under move but faded backwards instead of under, launching a one handed, high arcing shot over Drew Gooden. As Mike Tirico said, West"goes through the full arsenal of moves." Hubie Brown added, "That was a very pretty move. He gave Gooden at least three different moves off a jab step series and then caught him on a reverse move." Brown should have passed that comment along in a note to the official scorekeeper, because when a shot is preceded by enough post moves to be featured as a clip on an instructional video, there is no possible way that an assist should be awarded. You know what Chris Paul was doing while West was channeling Kevin McHale? He was standing well behind the three point line, watching, just like I was at home. The next time ESPN declares that Paul has broken an assist record set by Oscar Robertson, please remember this play.

4: David West jump shot, 4:21 1st q: Correct; after a screen/roll play, Paul fed West for a catch and shoot jumper.

5: Antonio Daniels three pointer, 1:59 2nd q: Correct; Paul posted up (!), drew a double team and kicked the ball out to Daniels for a catch and shoot. This is a perfect example of what an assist should be: Paul created a scoring opportunity for a teammate by drawing the double team and then he delivered the pass to the open man.

6: Julian Wright layup, 6:46 3rd q: Correct; Paul drove to the hoop, drew multiple defenders and passed to a cutting Wright, who scored a layup and was fouled, converting the three point play. This is another great example of an assist, because Paul created the scoring opportunity as opposed to simply making the last pass before a shot was attempted.

7: Julian Wright jump shot, 6:22 3rd q: Correct; once again, Paul attracted the defense and passed to Wright, who this time nailed a catch and shoot jumper. The difference between this play and assists one and two--which I consider to be correct but borderline--is that on the previous plays West was more closely guarded and had to do at least a little work prior to shooting (but he did make an immediate or close to immediate reaction to shoot and he did not put on a low post or perimeter footwork clinic) while on this play Wright simply caught and shot. The other nice aspect of this play is that Wright was open specifically because Paul attracted the attention of multiple defenders.

8: Julian Wright dunk, 6:42 4th q: Correct; Paul drew the defense and lofted a perfect alley oop to Wright, who was cutting to the hoop from the weak side. Wright pointed up in the air to signal to Paul, jumped off of two feet from just outside the restricted area and threw down a nice dunk.

9: David West jump shot, :38 4th q: Correct; this was a straightforward catch and shoot play.

Tony Parker's Seven Assists

1: Drew Gooden jump shot, 6:44 1st q: Incorrect; Gooden caught the ball on the right block at the 6:49 mark. He turned to face up his defender, made a strong drive and delivered what Hubie Brown called "a tough shot" over Hilton Armstrong. The issue is not the number of dribbles or time elapsed per se, but the fact that the assist had little to do with actually creating the scoring opportunity; it is not supposed to be the case that the pass immediately preceding a made basket is automatically deemed to be an assist.

2: Drew Gooden hook shot, 4:21 1st q: Incorrect; Gooden caught the ball on the wing, faked out a defender who was closing in on him, took two dribbles, turned 3/4 away from another defender and made a jump hook. Gooden created the shot with his one on one skills.

3: Manu Ginobili three pointer, :02 1st q: Correct; Parker drew the defense and passed to Ginobili, who immediately drained the shot.

4: Michael Finley three pointer, 5:57 3rd q: Correct; Parker drew the defense and passed to Finley, who nailed a three pointer with the shot clock running down.

5: Tim Duncan hook shot, 5:33 4th q: Correct; this is another borderline play. Parker caught the ball behind the three point line off of a back tap and fired a pass into Duncan in the paint. Duncan gathered himself with a quick dribble and a slight fake and then made a hook shot. I don't like the fact that Duncan made a bit of a one on one move before the shot but the whole sequence happened quickly and would not have been possible without Parker's bullet feed. I consider this play to be a judgment call. A "strict constructionist" might argue against awarding an assist because Duncan played a part in creating the shot but the nature of the pass--a quick feed deep into the post in heavy traffic--and the fact that it was a bang, bang play makes the awarding of an assist OK from my perspective. In contrast, if Duncan had performed multiple fakes and moves then I would be opposed to awarding an assist--even though it was a great pass--because in that case it could no longer be argued that the pass really created the scoring opportunity.

6: Matt Bonner three pointer, 3:40 4th q: Correct; just like assists three and four, this was a straightforward catch and shoot play.

7: Michael Finley three pointer, :17 4th q: Correct; this was another catch and shoot play.

Out of 16 assists credited to these two great point guards, 10 were unquestionably correct, three were unquestionably wrong and I have classified three as correct but borderline. I don't have a big issue with the borderline plays and I realize that there is some subjectivity inherent in this aspect of scorekeeping (even though there are some hard and fast rules outlining exactly what is and is not an assist) but the third assist awarded to Paul is indescribably bad scorekeeping by any standard. If the NBA is going to go back and take away one borderline rebound from LeBron James to nullify his triple double versus the Knicks then the league should certainly strike that assist from the record books.

It is worth noting that in this game Parker benefited more often than Paul did even though the game was played in New Orleans. I don't think that there is an NBA conspiracy or a New Orleans conspiracy involved with how assists are officially tracked; for whatever reason, scorekeepers are awarding assists more liberally than the rulebook says that they should. This would tend to favor players who are primarily playmakers--like Paul--but that does not mean that there is a deliberate effort to favor one player in particular. I have focused on Paul because he is the league leader, so I know that if I track him for a whole game that there will probably be a lot of plays to consider; in this post and my previous one on this subject, I also charted Parker as a "control" to see if only Paul is being credited with dubious assists.

I have now charted assists for Chris Paul in two playoff games and three regular season games; in the last two regular season games that I examined, I also charted Tony Parker's assists. The overall count in those five games shows that Paul was officially credited with 55 assists but that only 42 of those assists fit the rulebook definition (and those 42 include some borderline plays). In the two games in which I tracked Parker's assists, he was credited with 11 assists but should only have been credited with eight. The sample size with Parker is obviously small but I think that 55 official assists for Paul is a diverse enough sample of plays from which to draw at least a preliminary conclusion that he is being credited with too many assists. Paul is averaging just a shade under 11 apg this season but if he is consistently being credited with too many assists at the rate that I have observed in this sampling then he is actually averaging about 8.4 apg. If what I have charted reflects a season or career long trend then this is quite significant. Again, as I stated above, I don't think that the NBA or the Hornets are necessarily deliberately doing this but any bending of the scorekeeping rules regarding assists will obviously benefit playmakers more than other players. During the whole LeBron James triple double controversy, the NBA claimed that it regularly reviews game films and adjusts statistics that are improperly awarded; this certainly should not have been news to James, because the NBA had previously taken a triple double away from him due to a questionable assist (April 1, 2006 versus Miami). If it is true that the NBA so thoroughly supervises all of its scorekeeping (and not just games involving triple doubles) then assist number three for Chris Paul from this game simply has to be rescinded.

I should not even have to add this last paragraph, but it is almost certain that someone will read this post and conclude that I either (1) don't watch Chris Paul enough to appreciate his greatness and/or (2) I "hate" Chris Paul. Last summer, I ranked Paul as the best point guard in the NBA and put Parker fourth on my list; my charting of bogus assists does not lower my opinion of either player, because I don't rank players based solely on numbers. The reason that I am charting assists in this fashion is that I am very disappointed that the NBA and the "stat gurus" apparently do not care at all about the inaccurate and inconsistent methods used by NBA scorekeepers who produce the official assist numbers that go in the record books and form the basis of so many player ranking formulas.

--------
*For those of you who have not read my previous posts on this subject and/or do not know what the rulebook definition of an assist is, here is a passage that was posted on NBA.com in 2002 (yes, that was seven years ago but the NBA has not announced any official changes in its scorekeeping procedures regarding assists since that time):

An assist is a pass that directly leads to a basket. This can be a pass to the low post that leads to a direct score, a long pass for a layup, a fast break pass to a teammate for a layup, and/or a pass that results in an open perimeter shot for a teammate. In basketball, an assist is awarded only if, in the judgement of the statistician, the last player's pass contributed directly to a made basket. An assist can be awarded for a basket scored after the ball has been dribbled if the player's pass led to the field goal being made.

The concluding words of my December 18, 2008 post are well worth repeating here:

The rule of thumb to keep in mind is that the pass is supposed to "directly" lead to a basket. Every fake, dribble and move that the recipient makes after getting the ball makes that "direct" connection more and more tenuous. If the recipient is running down court uncontested and his teammate passes him the ball, then the number of dribbles he takes is irrelevant: he is meeting no defensive resistance and he clearly would not have scored without receiving that pass--but if a player is running down court, receives a pass, does a crossover dribble to shake one defender and then twists and turns to lay the ball up over another defender, then the pass did not really "directly" lead to the score because the scorer did most of the work. If the scorer does most of the work then the passer should not receive credit for an assist.

Labels: , , , ,

posted by David Friedman @ 4:22 AM

10 comments

10 Comments:

At Tuesday, March 31, 2009 4:36:00 PM, Anonymous Anonymous said...

How often do you think Chris Paul sets up his teammates in a situation where they end up getting fouled and don't score? Does that happen more or less often than an incorrectly credited assist? That would seem to be a far more valid concern.

At the end of the day, the data shows that Paul is the best playmaker in the NBA, which I think everyone agrees with. The data is telling the correct story. And unless there is some sort of systematic plot to favor Paul, it probably correctly portrays how much better he is than his closest competitor also.

It may be there are some incorrectly awarded assists, but at the end of the day, we know that assists are an important piece of data because we can't explain wins without them as accurately as we can with them. They aren't perfect, but they are quite useful.

I don't know, I think the best criticism that can be made of "advanced" stats remains that we don't keep enough of them.

Inquiring minds want to know, is the fact that Paul still has a higher rebound rate than Kobe this year a "bogus" statistic?



Owen

 
At Tuesday, March 31, 2009 5:46:00 PM, Blogger David Friedman said...

Owen:

The "advanced" stats do not factor in the situation that you described, so that is not affecting player rankings--nor is it making a mockery of the record book with artificially inflated numbers.

What "data" shows that Paul is the best playmaker in the NBA? The assist data that appears to be inflated by 25% or so, based on the games I have tracked? Subjectively, I do think that Paul is probably the best playmaker but we certainly do not have numerical proof of this. I love how you are a numbers guy up until the point that the numbers don't agree with you. You believe that Paul is the best player in the NBA so when I show that he is getting the benefit of completely bogus assists you are not the slightest bit interested in that piece of information--and you have the nerve to say (in previous comments) that I am analyzing basketball based on aesthetics instead of objective considerations. What a crock!

How is "data" like Chris Paul's third "assist" useful? It is just astounding that you purport to analyze the game objectively and scientifically but you do not care one bit if the basic data that you are using is flawed. Could you imagine a real scientist--a physicist or a chemist--saying that incorrect data is "useful"? That is just silly.

Your closing question is a classic example of the bait and switch. You certainly realize that you cannot mount any challenge to the data that I have provided so instead you change the subject. Kobe averages more rpg and more rpg/minute than Paul but since that is not really what you are asking about I will answer your inquiry in the spirit that it was delivered: Is the fact that Kobe has won three NBA championships and was the go-to player down the stretch in the 2008 Olympic gold medal game a bogus piece of information? What about the fact that NBA GMs still consider Kobe to be the toughest player to construct a game plan against? Is that bogus, too? How about the fact that Kobe has been by far the best player on the team with the best record in the tough Western Conference each of the past two seasons and that he has done so playing with four healthy fingers on his shooting hand last year and three healthy fingers on his shooting hand this year?

 
At Tuesday, March 31, 2009 8:16:00 PM, Anonymous J said...

I too find it rather odd that Owen blithely brings up the failure of stat-keepers to track how often players make passes that lead directly to free throws (and some points) as an "assist-like" stat (or for the failure to account for the difference in value of an assist for a 2-pt shot or 3-pt shot), and yet seems completely confident in the "advanced" stat analyses.

In any event, I do agree with you Owen that the main problem with advanced stats is that we don't keep enough of them -- i.e., "hockey-assists" or key passes-to-assist-passes, passes creating FT opportunities, shot alterations (some of these are as clear as assists -- in a game a couple weeks ago, Kobe committed a TO, but raced down the court and leaped up to swat at the ball and although he missed, the player clearly and noticeably altered his shot and missed his lay-up), etc.

Well, actually, the main problem with advanced stat analyses is that their proponents are supremely self-confident and all-but impervious to any criticism of the underlying data or weighting and interpretation of it.

And David, thanks for another in this continuing series. I find them particularly interesting. I am also a big fan of statistic analysis in baseball, but I continue to believe strongly that current statistic measures simply do not add an exceptionally great deal of value to my two favorite sports -- soccer and basketball. With its isolated, one-v-one sequences, baseball is really a completely different animal than those sports.

 
At Tuesday, March 31, 2009 11:11:00 PM, Blogger David Friedman said...

J:

As I mentioned to Pat Williams when I interviewed him--and Williams agreed with me--baseball lends itself to accurate statistical analysis much more readily than basketball because a baseball game consists of a series of discrete, one on one confrontations: the pitcher throws the ball to the batter, the batter hits the ball, a fielder catches the ball, etc. A basketball game consists of 10 players simultaneously moving and interacting with each other in various ways, which makes it much more difficult to quantify what is happening, particularly in terms of comparing players (it is easier to compare teams).

It is a shame that neither the NBA nor the "stat gurus" apparently care at all about the devaluing of the assist and about the possibility of other scorekeeping irregularities with steals, blocked shots, turnovers, etc.

Owen is apparently willing to argue until the end of time about the significance of "true shooting percentage" when comparing Kobe's skill set to LeBron's but he is completely unconcerned that Paul's numbers are inflated by "assists" that require no more skill than that possessed by any recreational basketball player. I could have passed the ball to West, stood outside the three point line and watched West go through the whole Kevin McHale low post moves clinic.

It's funny how "stat gurus" profess to only care about what can actually be measured/quantified and yet they don't care if the methods of measurement are accurate or not.

 
At Wednesday, April 01, 2009 12:30:00 AM, Anonymous Anonymous said...

The liberal assist-giving might not affect the current season's league assist ranking, but as David said, it makes a mockery of the record books.

If stat gurus would really like to rate playmaking using a single number, then assists shouldn't be what they should look for.
I'd like to see them track good passes (passes that improve the offensive situation), neutral (no advantage gained or lost), and bad(not necessarily TOs, but passes in which the receiver has to move into a disadvantageous situation and so forth).

There are also selfish assits to be aware of. Instead of taking the open layup, the PG might toss it off the backboard for the trailer in a fast break.
The correct play is to lay it in. If he missed, the trailer is usually in a good position for an offensive board anyway.
Open layup = high % made shot. IF it missed, the bigger trailer has a decent chance for a putback.
Pass to trailer = slight chance of a TO. The follow shot might not be as open as the open layup, maybe, maybe not but by taking the extra step, the defense gets to catch up. Finally the PG is most likely not as good in securing the offensive board as the bigger trailer.
This is a case when the extra pass didn't give the offense an extra advantage. Low-risk, 2 points? Or Slightly higher-risk, still 2 points?

I'm sure scouts and coaches track passes all the time, regardless of them ending up as an assist pass or not. By doing so, they can conclude that player X is a better passer than player Y.
The stat gurus will only be able to tell you that player X can accumulate more assists for his team than player Y does for his team.

A bit off topic, but a player's turnovers do not tell the whole picture. Wild passes penalizes the guy who last touched the ball.
Going after a ball that is going out of bounds might result in a turnover for you, and if you did manage to save it, you won't get any "stats" for that.
These moments are important and sometimes even pivotal in a game, but they are not tracked. "Advanced?" I don't think "basic" has been mastered yet.

Z

 
At Wednesday, April 01, 2009 5:25:00 AM, Blogger David Friedman said...

Z:

I agree with just about everything that you said. My one slight disagreement would be that I don't think that when a player tries to save a ball that is going out of bounds he is automatically charged with a turnover; I think that it would depend on whether or not his team was the last one that had possession of the ball (if a defensive player goes after a loose ball and throws it back on to the court and the offensive team retains possession I don't think that would be considered to be a turnover). In a general sense, though, you are 100% correct that turnovers can be a misleading statistic, at least on the individual level. If you look at the players who have the most turnovers, they are usually among the best players in the NBA; those are the guys who handle the ball the most. Of course, from a team standpoint it is important to keep turnovers down overall; I think that most coaches are less concerned about their superstars turning the ball over than about role players who don't handle the ball very much and have limited responsibilities but still have high turnover rates: for instance, if you are just a catch and shoot player or primarily a rebounder or defensive specialist, then you should have very few turnovers.

 
At Wednesday, April 01, 2009 2:03:00 PM, Anonymous Anonymous said...

True Hoop today has a good post about assists and references an excellent article in the Wall Street Journal.

What I should have said is this. It seems like there are far more constructive ways to approach this issue. And I think if you were a little more comfortable with advanced statistics, it would make your life a lot easier. The story you are trying to tell in general is in the data.

Owen

 
At Wednesday, April 01, 2009 4:09:00 PM, Blogger David Friedman said...

Owen:

What could possibly be more constructive than pointing out very specific instances in which data is false? That enables--or should enable--the NBA to correct this data and "stat gurus" to take this information into account when constructing their formulas. It is very interesting and revealing that you are essentially admitting that you don't care if the basic data is wrong and that you have no interest in even trying to account for this in any fashion.

Let me make this crystal clear: I am not "uncomfortable" with any kind of statistics but I am very "uncomfortable" with the misuse of statistics (or any other kind of information).

 
At Thursday, April 02, 2009 1:11:00 PM, Anonymous Anonymous said...

I actually take for granted the fact that some assists are incorrectly awarded, that some scorekeepers bend the standard sometimes. I also take for granted the fact that assists are awarded far more generously than they were historically. It's very easy to see these things looking at the data. I have pointed the latter fact out to you before. Just look at the assist rate leaders over the years and how much they have spiked since the late seventies.

I don't know, you seem to think you have made some kind of discovery here, which is just bizarre.

In other news, Kobe's ts% is down to 55.7%. Lebron is at 58.5%. Chris Paul is at 59.9%.

Owen

 
At Thursday, April 02, 2009 4:43:00 PM, Blogger David Friedman said...

Owen:

I am not claiming to have made a "discovery." I am documenting exactly how assists have been awarded in specific games. One would think that this would be of interest to the league, to fans and, most of all, to the "stat gurus" who depend on box score numbers to create their "advanced" metrics. What is "bizarre" is that the NBA apparently selectively deals with this issue, taking away triple doubles from LeBron James (correctly, I might add) but blithely ignoring even more blatant examples of faulty scorekeeping.

You seem to be completely oblivious to the concept that if assists are incorrectly awarded then this is probably also the case for steals, blocks, turnovers and, to a lesser degree, rebounds. Where does that leave your so-called "advanced" metrics?

Numbers are just one tool to use when evaluating players and the "advanced" metrics are hardly fine tuned enough to replace watching games with understanding.

I don't understand your obsession with ts% but I do know that Kobe has led the Lakers to the best road record in the NBA and that they just completed a 5-2 East Coast swing. I think that a team's road record over the course of a marathon season is much more relevant and meaningful than trying to compare players' values by looking at one shooting metric.

 

Post a Comment

<< Home