Economics is Not a Science, Nor is Basketball Statistical Analysis, Part IIRuss Roberts' recent Wall Street Journal article titled Is the Dismal Science Really a Science? makes some of the same points that I have stated forcefully for several years. Roberts--who is an economics professor at George Mason University--declares that economics is not a science because "most sciences make progress. Nobody in medicine wants to bring back lead goblets. Sir Isaac Newton understood a lot about gravity. But Albert Einstein taught us more." Roberts adds, "Facts and evidence still matter" but that instead of proving or disproving theories it seems like economists are primarily just "confirming our biases." Roberts concludes, "The economy is a complex system, our data are imperfect and our models inevitably fail to account for all the interactions. The bottom line is that we should expect less of economists. Economics is a powerful tool, a lens for organizing one's thinking about the complexity of the world around us. That should be enough. We should be honest about what we know, what we don't know and what we may never know. Admitting that publicly is the first step toward respectability."
The basketball "stats gurus" who received their training as economists should take heed of Roberts' words and likewise be "honest" about the limitations of their methodologies--but, of course, that will likely never happen, because how can Dave Berri and others sell books if they admit that all they are doing is shuffling numbers on spreadsheets to confirm their biases and that evaluating basketball players and teams is a far more complex enterprise than they care to admit?
As I wrote in October 2008, Economics is Not a Science, Nor is Basketball Statistical Analysis. The data being used by "stats gurus" is incomplete and often inaccurate. Statistics can be a "powerful tool" (in Roberts' words) to help to understand basketball but they do not provide definitive answers: the player rankings produce by Berri or John Hollinger do not represent some absolute, objective reality but merely reflect the biases and limitations inherent in the formulas that Berri and Hollinger invented.
In The Difference Between Measuring Defense in Basketball and Baseball, I made the important point that basketball statistical analysis is pseudoscience because its practitioners do not base their research on the scientific method:
- Ask a Question
- Do Background Research
- Construct a Hypothesis
- Test Your Hypothesis by Doing an Experiment
- Analyze Your Data and Draw a Conclusion
- Communicate Your Results
posted by David Friedman @ 6:39 PM