Ken Pomeroy Describes the Limitations of Plus/Minus
Plus/minus seems like a perfectly objective statistic and a very useful one for people who cannot watch every single minute of every single game (which would describe everyone, even though one blogger likes to pretend/brag that he alone has consumed every second of NBA action since sometime during the Bulls' three-peat glory days): you simply calculate the scoring differential when each player is on the court and, voila, you can see which players are having a positive impact and which players are having a negative impact; all of the hustle plays that are not captured by conventional box score numbers or even by "advanced stats" are presumably detected by plus/minus. Furthermore, adjusted plus/minus--which takes into account who else is on the court--seemingly refines the data even more.However, anyone who has looked at a lot of plus/minus data immediately realizes that it is very "noisy": some players who even the "stat gurus" know are not that good inexplicably have impressive plus/minus numbers, while some players who are clearly above average do not have outstanding plus/minus numbers. At the very least it seems obvious that one needs a very large set of data to filter out this noise. "Stat guru" Ken Pomeroy recently devised a very interesting test of the limitations of plus/minus; his work focused on the college game but can clearly be applied to the NBA game as well. You can read a detailed description of his methodologies and results in A treatise on plus/minus but his conclusion should be embraced by anyone who is attempting to analyze basketball: "It's true plus-minus captures everything that's happening, but that includes a whole lot of random things that lead to a hoop or a stop. Things that have nothing to do with the ability of the player you want to analyze. In basketball analysis, we should be filtering out randomness, not embracing it." Pomeroy notes that because the professional season is much longer than the college season there may be "limited use" for adjusted plus/minus in the NBA but even in that case one probably needs at least two full seasons of data to make any meaningful evaluations; in other words, most of the stat-based articles (about "clutch performance," player ratings, MVP rankings, etc.) that are popping up like dandelions in an untended yard are using data sets that are far too small to form the basis for sweeping, definitive conclusions (I realize that not all of these articles are using plus/minus or advanced plus/minus data but there is even less reason to trust the accuracy of Berri's numbers or Hollinger's numbers--both of which are based on subjective formulas that can be tweaked to reach whatever conclusions the author desires--then there is to trust plus/minus data that truly is objective in some sense even if it is only potentially meaningful when the data set is very large).
Pomeroy's article represents the kind of frankly honest research/experimentation that all "stat gurus" should be doing; instead of brazenly declaring that their numbers are flawless while the observations of skilled talent evaluators are hopelessly biased, "stat gurus" should be in their labs (metaphorically speaking) trying to ascertain the strengths and limitations of their beloved formulas: if more of them would do that--instead of writing articles with catchy headlines so that certain high profile entertainment providers will link to them--then they could make a real contribution to better understanding basketball.
Labels: "advanced basketball statistics", "stat gurus", Ken Pomeroy
posted by David Friedman @ 3:59 AM
8 Comments:
When you get a chance, take a look at CA Clark's article.
http://www.silverscreenandroll.com/2011/2/11/1987878/where-black-holes-meet-geometry-kobe-bryant-and-the-next-evolution-of
This is the kind of advanced stats analysis I like, not because Clark is a Laker and Kobe fan, but because he never makes the assumption that a set of stats tells everything. He's always willing to acknowledge the weakness of small sample size and noise. Also he does try to think unconventionally when looking at the numbers and trying to figure out what they mean.
I appreciate writers and analysts that can write clear and cogent articles that realize they don't know everything. That to me is what good advanced stats does, not parsing through the numbers just to find the one set that fit a preconceived bias.
Also, I've been pretty busy the last couple of weeks and what a treat it was to come on your site and see so many new articles. Nice to see you had some time to put out so many in such a short window. I always enjoy your articles.
Hey David,
I know this comment is irrelevant to your article - but I read this blog often. I've read laments on the lack of good journalism, and the ignorant reliance on statistics - but I believe I've found a place where stats are put to good use. I'm a big laker fan, and I love the laker blog - silver screen and roll. I think you would love this article in particular -
http://www.silverscreenandroll.com/2011/2/11/1987878/where-black-holes-meet-geometry-kobe-bryant-and-the-next-evolution-of.
ChowNoir/Lijo John:
The Clark article is well done and it is similar in concept to the way that I tracked assists for Chris Paul and other point guards a while back; it is important to understand that box score numbers do not tell the complete story and that even "advanced statistics"--or, more precisely, specifically "advanced statistics--can be misleading if they are not placed in the proper context.
Jeff Van Gundy and Hubie Brown have both mentioned on several occasions that Bryant is one of the best passers/playmakers in the NBA. I believe that Van Gundy once said that Bryant could lead the league in assists if Bryant played in a different offense with a different role.
One thing that is rarely brought up in discussions about Kobe, MJ and the Triangle--I don't think I have seen it mentioned anywhere other than a few of my own articles, actually--is that Scottie Pippen played the primary playmaking role for the Bulls; this enabled MJ to sprint down court and get good post position on offense. In contrast, the Lakers essentially ask Kobe to be both MJ (dominant scorer) and Pippen (leading playmaker/team leader in assists). I am not saying that Kobe is therefore better than MJ--I have stated many times that I would take MJ over Kobe--but Kobe has to walk a delicate balance between scoring and passing as a result of the combined roles that he shoulders.
What really gets me is the fact that for all of the hoopla surrounding "advanced statistics, I can't think of a single thing that they've done to advance the game of basketball in any meaningful way, particularly when they try to use it as a form of predictive analysis. The results range from redundant (telling us nothing that traditional skillset and game film analysis doesn't already reveal), to flat out wrong. When your only claim to fame is the laughable assertion that Derrick Rose getting to the freethrow line more was brought about as a result of "reading John Hollinger's statistical breakdown of his lack of freethrows", then you know you're reaching.
The only saving grace of advanced stats has been the true shooting/effective fieldgoal percentage and to a lessor degree, +/-. Outside of that, they have nothing.
I used to be a stat geek myself. But in the end, I discovered that the single best predictor of future results are gambling odds - point spreads and future book markets in offshore gambling sites.
These odds have biases and flaws, but there are very few people who can beat them consistently. And those who do generally study how betting markets work and identify public bias. It gets you a lot further than crunching numbers.
Players like Kobe Bryant and teams like the Lakers drive stat geeks crazy because they don't fit the mold. Most sensible statistical analysis is going to show that the Lakers are a fringe top-5 team and Kobe is somewhere around 5th-10th best player in the NBA right now. Yet they keep winning championships and Kobe keeps adding to his legacy - unlike Lebron, Howard, Wade, or Anthony who appear to be statistically superior.
Obviously there are factors beyond raw regular-season statistics that determine who will win the Championship. The betting lines and future book markets generally reflect this - but when they don't, experienced bettors move in for the kill.
While I've been retired from the betting scene since 2006, I enjoyed the evolution of "advanced basketball statistics" while I was active. Not only did they provide with me with another tool to analyze teams, they also helped me identify flawed reasoning and gauge public bias. They also largely led people to bet on the wrong teams, which helped me increase my profit margin.
Sharp
The point about Kobe combining the Pippen/Jordan role is not very widely acknowledged. I have definitely seen you mention it several times, David. I remember hearing Doug Collins mentioning this several times during the Shaq/Kobe era. Kobe was always the primary ball handler, and initiated the offense. Jordan had Scottie to take care of that stuff for him.
That is definitely one of the two biggest differences between Bryant and Jordan.
I've always felt the other big difference was defensively. Both are good man to man defenders (Jordan was a little more consistent, in my opinion), but Kobe also plays a lot of the Scottie Pippen role on defense as well. That is to say, Kobe is utilized a lot more as a "free safety,"/help defender than Jordan ever was. And in that respect, I consider Bryant to be the best perimeter help defender in the game today, definitely the best one since Scottie himself.
DanielSong39:
It is interesting that you mention your success with handicapping, because Henry Abbott often quotes someone who he claims is a professional gambler who is making a lot of money using "advanced basketball statistics." I have always been skeptical about that because, as you note, the "advanced basketball statistics" are not particularly accurate. I find it much more credible to believe that someone could make money in the fashion that you described--detecting the biases (stat based or otherwise) that move the betting lines and then making your wagers accordingly when you find good values.
Sharp:
Yes, you are right that Collins pointed this out back in the day but Kobe is carrying an even heavier burden now; Shaq was a legit MVP level player who could be the first scoring option while Kobe was the playmaker/top perimeter scoring option but now the Lakers rely on Kobe to be the number one scoring threat period in addition to initiating the offense. The only time MJ really had to do that was the brief period when, ironically enough, Coach Doug Collins made MJ the Bulls point guard (not in the Triangle) and MJ ran off a string of triple doubles. Under Jackson's tutelage, Pip developed into a point forward and MJ did not have to be the primary facilitator.
During the 2008 NBA Finals, Boston Coach Doc Rivers said that Kobe is the best help defender the NBA has seen since Pip.
Post a Comment
<< Home