The Counterfeit Currency of David Berri's Wages of Wins
A couple months ago, I took the Wages of Wins Journal to task for asserting, among other things, that performance-enhancing drugs do not in fact enhance performance; WoW is apparently oblivious to Ben Johnson, Marion Jones, Floyd Landis and an entire generation of East German Olympic swimmers. The latest "revelation" from WoW is that Kobe Bryant is not the best player in the NBA. Admittedly, there is a certain degree of subjectivity in calling any one person the best at a particular endeavor (except maybe in the cases of Tiger Woods or Roger Federer) but if you took a straw poll of NBA executives, coaches and players you would find that a plurality, if not a majority, would say that Bryant is the best player in the NBA. Even people who would not vote for him for MVP because his team did not win 50 games would still say that he is the best player. Of course, the WoW people are economists and economists know everything about everything, so why should they care one bit about what actual basketball experts think?The WoW writers can, pardon the pun, "wow" the casual fan because their work appears to be so in depth and some of their credentials are superficially impressive--but their in depth work in fact contains serious flaws and their credentials have nothing whatsoever to do with being able to understand sports. Anyone who believes that Kobe Bryant is not the best player in the NBA is going to point to the WoW article and say, "Aha! That proves it." Suit yourself--but be aware that the same system also led to the conclusion that Dennis Rodman was more valuable to the 1995-96 Chicago Bulls than Michael Jordan. That is the thing with statistical based analysis (as opposed to combining stats with one's own analysis and the analysis of informed observers, the method that I prefer to use)--you cannot pick and choose what you like. If you are signing on for Bryant not being the best player based on WoW's methods then you are also signing on for Rodman being more valuable than Jordan. Also, it is worth noting that when called to task about making the Rodman/Jordan statement, David Berri's initial response was to deny ever making such an assertion; that is kind of like Rafael Palmeiro wagging his finger defiantly and then flunking a drug test (maybe Palmeiro should have just said that WoW denies that performance-enhancing drugs enhance performance). As Salon.com's King Kaufman pointed out, "alas and alackaday for Berri, when you write something in a book it stays writ and the statement that Rodman was more valuable on a per-minute basis than Jordan--that is, a better player who just didn't play as many minutes--was on Page 144" of the book The Wages of Wins. The exact quote on that page is "Per 48 minutes played, Rodman's productivity even eclipsed Jordan."
By the way, earlier this year, FreeDarko.com posted a thorough refutation of WoW's methods of basketball analysis. Also, other economists are not necessarily buying what WoW is selling. Here is an extensive discussion of the flaws of WoW, as described by Dan Rosenbaum and others at APBR Metrics. If wading through those 19 pages is a bit much for you, here is a good summary of some of the flaws that Rosenbaum--a professor of economics who has also done statistical analysis work for the Cleveland Cavaliers--finds in WoW's approach (he posted the following at APBR Metrics on July 29, 2006, shortly after the WoW book was published):
One of the main points of Wages of Wins is that according to the evidence presented in the book, it appears that NBA decision-makers are irrational. They write: "It is not that people in the NBA are lazy or stupid. It is just that the tools at their disposal do not allow them to see the value of the various actions players take on the court."
The argument that supports this conclusion is they show that Wins Produced explains wins much better than NBA Efficiency, but NBA Efficiency is much more closely tied to salaries and All-Rookie Team voting (done by coaches). Thus, according to their evidence, it is irrational for teams to not be using something like Wins Produced to make their decisions.
OK, but the difficulty with this argument is that it hinges on Wins Produced better explaining wins than NBA Efficiency. Wins Produced does a great job explaining wins because of their team defense adjustment, but the authors admit that this adjustment has very little effect on their relative rankings of players. So if it doesn't matter much for the relative rankings of players, I just don't see how it can be used as a justification for the methodology. To me, that whole exercise raises a big red flag about the validity of using the prediction of team wins as a barometer for a metric for individual players.
But then if we move to another barometer--adjusted plus/minus ratings--we see that Wins Produced only performs better than NBA Efficiency if position adjustments are used for Wins Produced but not for NBA Efficiency. That significantly changes the story of much of their book. Instead of a story about NBA teams overvaluing scorers, their story becomes one that NBA decision-makers are irrational because they don't properly position adjust.
Moreover, the authors provide little justification for their position adjusting, especially in relation to how important it is to their metric. They argue that big players would have difficulty filling the roles of guards; i.e. a team could not play all centers. But if centers truly are worth more than guards as their unadjusted Wins Produced suggests, this would not be the only reaction of NBA decision-makers. Instead of playing centers at guard, what would happen would be that they would pay centers more than guards--which is precisely what does happen. So rather than proving conventional wisdom wrong, maybe the authors have provided justification for conventional wisdom.
Now I am not necessarily trying to defend NBA decision-makers as being super-rational. Lots of points made in Wages of Wins are points I agree with wholeheartedly. And I highly recommend that everyone in this APBRmetric community read this book. You will find lots that you agree with in this book and lots that will force you to think more deeply about things.
But when we go to cast stones at NBA decision-makers, we need to be sure that our own house is in order. And that is really my biggest complaint about Wages of Wins. My experience has been that NBA people often do a much better job than we give them credit for. I work for Cleveland, and last year when we traded for Flip Murray I was against it. Flip was the lowest rated two guard in my system. (And I am sure he would not be rated too highly in Wins Produced.)
But you know what? Flip did not play too badly for us. He was not a star or even a good starter, but he made some changes to how he played in Seattle and he contributed to us doing well down the stretch and in the playoffs. So in that situation if Danny had listened to me (or probably consulted Wins Produced), the team would have won fewer games.
There is a lot about this game that we in this community know well, but there is a lot we don't know well. I strongly believe that good stats work can play an important role in a well-run organization. But I vehemently disagree with those who are ready to start calling NBA people irrational because of some results from a possibly mistaken empirical analysis. We all can work on being better than that.
The most important point that Rosenbaum makes is one that should be obvious: NBA personnel executives, who have devoted their lives to the sport, have some idea of what they are doing. Sure, some of them are better at it than others, but Berri's wide-ranging broadsides questioning the competency of NBA executives have no more credibility than when Joe Sixpack declares that he could coach better than his favorite team's coach does or run a team better than his team's general manager does.
posted by David Friedman @ 5:13 AM
4 Comments:
Actually, Sports Illustrated DID have a poll of General Managers earlier this year and over 55% named LeBron James as the best player, with Kobe garnering less than 15% of the vote. As often is the case, a clueless writer has allowed his ignorant personal biases to drive his opinions at the expense of the facts.
Socrates:
I'm not sure which poll you are talking about. NBA.com annually does a preseason poll of NBA GMs and this year's edition can be found here:
http://www.nba.com/preview2007/gmsurvey_players.html
Asked who would win the 2008 MVP, 29.6% said LeBron, 22.2% said Duncan and 18.5% said Kobe. Of course, that is not at all the same as asking who is the best player. Many NBA insiders who have told me that Kobe is the best player in the league have also said that they would choose someone else as MVP because Kobe's team did not win 50 games. ABC's Mark Jackson is on record saying that Kobe is the best player in the NBA and has a chance to be the greatest player of all-time but he had Kobe third in last year's MVP race. When I asked him to explain the seeming contradiction in his stance, he told me that Kobe is the best player but the MVP should come from a contending team. So why did Jackson still have Kobe third and not off of the ballot altogether? Jackson told me that Kobe had done such an exceptional job just to get the Lakers into the postseason that he deserved MVP consideration. The GMs in your SI poll are likely guided by similar thinking. Also, Kobe had a slow start (for him) last season and then scored more ppg after the All-Star break than anyone since Wilt Chamberlain, so if the poll was from prior than March or April it came before Kobe did that.
Two more interesting things from the NBA.com GM survey: 59.3% of the GMs said that they would choose LeBron first if they were starting a team today (11.1% picked Kobe) but 34.6% (a plurality) picked Kobe as the player who forces opposing coaches to make the most adjustments (only 11.5% picked LeBron). The reason for this is age: if you were starting a team today you'd pick LeBron over Kobe because LeBron is younger. Also, 92.6% of the GMs said that Kobe is the best shooting guard in the NBA, while 74.1% said that LeBron is the best small forward.
I assume that you formed your opinion by speaking directly to people like Hubie Brown, Mark Jackson, Charles Barkley and numerous other players, ex-players, coaches and NBA executives, right? That is why you can so confidently call me ignorant--while using an alias, no less. Very impressive.
Your argument would be convincng if you disproved the methodology the WoW writers used instead of attacking the conclusions.
Anonymous:
Dan Rosenbaum and others at APBR Metrics have written many pieces that discuss the technical deficiencies in the WoW approach. That discussion is a bit numbers heavy for use here but interested readers can go over to APBR Metrics and check it out for themselves.
I fundamentally object to the WoW idea that one can better understand basketball by not watching it (and solely relying on numbers) than by watching it.
Post a Comment
<< Home