20 Second Timeout

Economics is Not a Science, Nor is Basketball Statistical Analysis, Part III

Gideon Rachman, a columnist for the Financial Times, declares "The vanity of economists needs to be challenged. Above all, their claim to scientific rigour--buttressed by models and equations--must be treated much more sceptically." Rachman explains, "When things were going well for the global economy, the prestige of economists rose steadily. They were the gurus of the age of globalisation. Governments, consultancies and investment banks rushed to hire economists, who were thought to possess vital skills and information...There has been some self-examination and soul-searching within the economics profession since the onset of the financial crisis." Rachman then quotes Nobel Prize-winning economist Professor Joseph Stiglitz: "If science is defined by its ability to forecast the future, the failure of much of the economics profession to see the crisis coming should be a cause of great concern." However, Rachman rejects Professor Stiglitz' tepid suggestion that economists should search for new "paradigms" (i.e., simply create more effective mathematical models); Rachman asserts that it is "misconceived" to categorize economics as a science.

What does this have to do with basketball? Many "stat gurus" received their formal academic training as economists and/or have been heavily influenced by the type of statistical modeling that economists do. While some "stat gurus"--most notably Dean Oliver, Roland Beech and Dan Rosenbaum--are candid about the limitations of basketball statistical analysis, others (particularly Dave Berri) act as if they have discovered the mathematical Holy Grail that unlocks all of the secrets to understanding basketball at an individual and team level. The problem with adapting economic mathematical modeling for use in analyzing basketball is that economic modeling is not even effective at what it was designed to do--namely, explain how economic markets work and predict how they will perform in the future--so it is quite a stretch to believe that something that failed in its primary function will somehow magically explain the inner workings of a sport as complicated and nuanced as professional basketball.

It is certainly a worthy goal to try to create an accurate mathematical/statistical method for player/team evaluations but that process is still in its infancy regarding pro basketball and it is counterproductive for anyone to assert otherwise; instead of media members and fan bloggers sycophantically praising the likes of Berri there is a need for strong, authoritative questioning of the methodologies used by "stat gurus." For instance, there is good reason to believe that box score numbers--the raw data used by "stat gurus"-- are not accurate. What, if anything, are "stat gurus" doing to improve the quality of the box score data and/or how are they accounting for the unreliability of that data when they create their "advanced" formulas?

Even in cases where the box score data is accurate it still does not tell the complete story: if Pau Gasol sets a screen for Kobe Bryant, two defenders trap Bryant, Bryant passes to Lamar Odom at the top of the key and Odom feeds Gasol for a dunk the box score will show two points and a made field goal attempt for Gasol and an assist for Odom and nothing for Bryant--but if Bryant had not drawn two defenders then the entire play would not have been possible (or, at the very least, Odom's pass would have been contested to a greater degree, as would have Gasol's shot). I am not suggesting that a new stat should be created to describe the impact that Bryant had on that play but rather I am just citing one example of how difficult it is to accurately quantify what happens even on a relatively simple basketball play. This is why I trust "advanced" five on five data more than I trust individual player data: if a given five man unit has played a large enough sample of minutes against a representative sample of teams (two big "ifs") then their collective plus/minus number gives some indication of their effectiveness as a five man unit--but you cannot parse that data to figure out why that group was effective; to determine that you still have to watch video with an educated eye and break down what exactly happened on the court.

The "hard" sciences--like physics and chemistry--make progress because strong-minded, intelligent people question previous theories in order to either strengthen or topple those thoeries; even though Isaac Newton's Theory of Gravity provided a useful model for many centuries Albert Einstein questioned it and ultimately replaced it with a more complete theory (Special Relativity). Can any intelligent, open minded person really believe that within the past five to 10 years the "stat gurus" have "solved" basketball to the extent that their mantras should never be questioned and that all one needs to do to understand the sport is to stop watching games and start devouring spreadsheets? "Stat guru" advocates like Henry Abbott tend to refer to my articles about this issue as "cranky" and they suggest not too subtly that I am some kind of anti-stat Luddite who is stubbornly clinging to outdated notions but the reality is that the field of "advanced basketball statistics" would benefit more in the long run if its practitioners critically examined their methodologies instead of basking in Abbott's praise while snobbishly dismissing valid critiques.

As Rachman mentioned, before the global economy tanked economists were in high demand and the field garnered a lot of prestige--but the stiff wind of economic mayhem has trashed the thin facade of that Potemkin Village and the glorification of economists will likely be viewed by future generations as nothing more than strange reverence for a cult; many of the practitioners of "advanced basketball statistics" will have to become more rigorous in their methodologies and more conservative about their assertions or they will ultimately suffer the same fate, regardless of how many writers ride the short term gravy train by becoming "converts" and praising every utterance of the "high priests."

Labels: basketball statistical analysis

posted by David Friedman @ 1:40 AM

1 comments