Economics is Not a Science, Nor is Basketball Statistical Analysis, Part II
Russ Roberts' recent Wall Street Journal article titled Is the Dismal Science Really a Science? makes some of the same points that I have stated forcefully for several years. Roberts--who is an economics professor at George Mason University--declares that economics is not a science because "most sciences make progress. Nobody in medicine wants to bring back lead goblets. Sir Isaac Newton understood a lot about gravity. But Albert Einstein taught us more." Roberts adds, "Facts and evidence still matter" but that instead of proving or disproving theories it seems like economists are primarily just "confirming our biases." Roberts concludes, "The economy is a complex system, our data are imperfect and our models inevitably fail to account for all the interactions. The bottom line is that we should expect less of economists. Economics is a powerful tool, a lens for organizing one's thinking about the complexity of the world around us. That should be enough. We should be honest about what we know, what we don't know and what we may never know. Admitting that publicly is the first step toward respectability."
The basketball "stat gurus" who received their training as economists should take heed of Roberts' words and likewise be "honest" about the limitations of their methodologies--but, of course, that will likely never happen, because how can Dave Berri and others sell books if they admit that all they are doing is shuffling numbers on spreadsheets to confirm their biases and that evaluating basketball players and teams is a far more complex enterprise than they care to admit?'
As I wrote in October 2008, Economics is Not a Science, Nor is Basketball Statistical Analysis. The data being used by "stat gurus" is incomplete and often inaccurate. Statistics can be a "powerful tool" (in Roberts' words) to help to understand basketball but they do not provide definitive answers: the player rankings produce by Berri or John Hollinger do not represent some absolute, objective reality but merely reflect the biases and limitations inherent in the formulas that Berri and Hollinger invented. In The Difference Between Measuring Defense in Basketball and Baseball, I made the important point that basketball statistical analysis is pseudoscience because its practitioners do not base their research on the scientific method:
- Ask a Question
- Do Background Research
- Construct a Hypothesis
- Test Your Hypothesis by Doing an Experiment
- Analyze Your Data and Draw a Conclusion
- Communicate Your Results
Labels: basketball statistical analysis, Dave Berri, John Hollinger, Russ Roberts
posted by David Friedman @ 6:39 PM
1 Comments:
Much of advanced statistics are correlational studies.
Correlational studies differ from experimental studies because correlation does NOT imply causation. The only way to make causal inferences is through, as explained, the experimental method. This is why, for so long, smoking companies were able to get away with saying that smoking may not cause cancer: just because people that smoke often have lung cancer doesn't mean smoking caisedling cancer.
Correlational studies do not imply causation because of two reasons: the directionality problem and the third variable problem. Consider the correlation between number of churches and robberies in cities. The directionality problem states that we don't know which caused which-church attendance may have increased peoples motive to steal or robberies may have increased the demand for churches. More likely, though, this correlation is due to a third variable: city size.
Advanced basketball stats are flawed in that they imply causation. Yes, they can be used as a tool to examine and compare correlations, but they cannot be put up in a list to determine who is the best player and why, like many advanced stats try to do.
Post a Comment
<< Home