"Sports Economists: Always Wrong About Everything"
The headline is an exaggeration--slightly--but the article is a great read, a combination of insightful analysis and biting humor: "Sports Economists: Always Wrong About Everything."
It is important to understand that intelligent critics of sports economists are not Luddites who hate numbers and who do not understand statistical analysis; we are analytically-minded people who are frustrated with people who--through some combination of ignorance and bias--twist numbers to fit a preconceived narrative.
The author of the article cited above tears to shreds a lot of nonsense from a variety of sources, but he has a laser focus on Dave Berri, who I wrote about over a decade ago and then forgot about until I stumbled onto an article describing how Forbes fired him in 2018 for submitting work that was "misleading," "sloppy," "polemic," and "just bad reporting."
I long ago grew tired of reading Berri's nonsensical basketball rants, and I scarcely paid any attention to what he said about other sports, but apparently he is versatile enough to provide horrible analysis about more than just basketball. You should read the above article for all of the gory details, but here is the devastating conclusion (the Birnbaum cited is Phil Birnbaum, whose excellent work I have cited before as well):
When you've got a hammer, everything looks like a nail. Berri's hammer is regression analysis, and he goes about hitting everything he can find with it until he finds something that seems vaguely nail-like from a certain angle. Then he proclaims a group of extremely well-paid subject matter experts dumb. When challenged about this, he says things like "regressions are nice, but not always understood by everyone." He calls the protestors dumb.
This is more than a logical fallacy: it's a worldview. In a post on a cricket study by another set of authors, Birnbaum points out the assumption built into a lot of economics studies. It, like most of Berri's work, runs a regression on some data and reports back that something fails to be statistically significant:
The authors chose the null hypothesis that the managers' adjustment of HFA [home field advantage] is zero. They then fail to reject the hypothesis.
But, what if they chose a contradictory null hypothesis -- that managers' HFA *irrationality* was zero? That is, what if the null hypothesis was that managers fully understood what HFA meant and adjusted their expectations accordingly? The authors would have included a "managers are dumb" dummy variable. The equations would have still come up with 4% for a road player and 10% for a home player -- and it would turn out that the significance of the "managers are dumb" variable would not be significant. Two different and contradictory null hypotheses, both which would be rejected by the data. The authors chose to test one, but not the other.
Basically, the test the authors chose is not powerful enough to distinguish the two hypotheses (manager dumb, manager not dumb) with statistical significance.
But if you look at the actual equation, which shows that home players are twice as likely to be dropped than road players for equal levels of underperformance -- it certainly looks like "not dumb" is a lot more likely than "dumb".
The goalie example is the most illuminating here: by adjusting the parameters of your study you can arrive at radically different conclusions. I'm not sure if Berri is intentionally skewing his results to get shiny Moneyball answers, but given how dumb his justifications are for the NFL study that's the kinder interpretation. Running around saying that we don't know that the average sixth rounder isn't John Elway waiting to happen because they can't get on the field is obtuseness that almost has to be intentional. On the other hand, he does blithely state he's "not sure there is much to clarify" about his assertion that NFL general managers are on par with stock-picking monkeys when it comes to identifying quarterbacks, so he may be that genuinely clueless. (The Lions tried a stock-picking monkey. It didn't work out.)
It is very important to emphasize that Berri's critics are not people who reject statistical analysis; we are people who reject flawed statistical analysis.
Labels: "advanced basketball statistics", Dave Berri, Phil Birnbaum
posted by David Friedman @ 4:42 AM
0 Comments:
Post a Comment
<< Home