Editing Economic model (section)

== Tests of macroeconomic predictions ==
In the late 1980s, the [[Brookings Institution]] compared 12 leading [[macroeconomic models]] available at the time. They compared the models' predictions for how the economy would respond to specific economic shocks (allowing the models to control for all the variability in the real world; this was a test of model vs. model, not a test against the actual outcome). Although the models simplified the world and started from a stable, known common parameters the various models gave significantly different answers. For instance, in calculating the impact of a [[monetary]] loosening on output some models estimated a 3% change in [[GDP]] after one year, and one gave almost no change, with the rest spread between.<ref>{{cite journal|last=Frankel|first=Jeffrey A.|title=The Sources of Disagreement Among International Macro Models and Implications for Policy Coordination|journal=NBER Working Paper No. 1925 |date=May 1986 |doi=10.3386/w1925 |doi-access=free}}</ref>

Partly as a result of such experiments, modern central bankers no longer have as much confidence that it is possible to 'fine-tune' the economy as they had in the 1960s and early 1970s. Modern policy makers tend to use a less activist approach, explicitly because they lack confidence that their models will actually predict where the economy is going, or the effect of any shock upon it. The new, more humble, approach sees danger in dramatic policy changes based on model predictions, because of several practical and theoretical limitations in current macroeconomic models; in addition to the theoretical pitfalls, ([[Model (economics)#Pitfalls|listed above]]) some problems specific to aggregate modelling are:
* Limitations in model construction caused by difficulties in understanding the underlying mechanisms of the real economy. (Hence the profusion of separate models.)
* The law of [[unintended consequence]]s, on elements of the real economy not yet included in the model.
* The [[Lag operator|time lag]] in both receiving data and the reaction of economic variables to policy makers attempts to 'steer' them (mostly through [[monetary]] policy) in the direction that central bankers want them to move. [[Milton Friedman]] has vigorously argued that these lags are so long and unpredictably variable that effective management of the macroeconomy is impossible.
* The difficulty in correctly specifying all of the parameters (through [[econometric]] measurements) even if the structural model and data were perfect.
* The fact that all the model's relationships and coefficients are stochastic, so that the error term becomes very large quickly, and the available snapshot of the input parameters is already out of date.
* Modern economic models incorporate the reaction of the public and market to the policy maker's actions (through [[game theory]]), and this feedback is included in modern models (following the [[rational expectations]] revolution and [[Robert Lucas, Jr.]]'s [[Lucas critique]] of non-[[microfounded]] models). If the response to the decision maker's actions (and their [[time inconsistency|credibility]]) must be included in the model then it becomes much harder to influence some of the variables simulated.

=== Comparison with models in other sciences ===

[[Complex systems]] specialist and mathematician [[David Orrell]] wrote on this issue in his book [[Apollo's Arrow]] and explained that the weather, human health and economics use similar methods of prediction (mathematical models). Their systems—the atmosphere, the human body and the economy—also have similar levels of complexity. He found that forecasts fail because the models suffer from two problems: (i) they cannot capture the full detail of the underlying system, so rely on approximate equations; (ii) they are sensitive to small changes in the exact form of these equations. This is because complex systems like the economy or the climate consist of a delicate balance of opposing forces, so a slight imbalance in their representation has big effects. Thus, predictions of things like economic recessions are still highly inaccurate, despite the use of enormous models running on fast computers.<ref>{{cite web|url=http://www.postpythagorean.com/FAQ.html|title=FAQ for Apollo's Arrow Future of Everything|website=www.postpythagorean.com}}</ref>
See {{slink|Unreasonable ineffectiveness of mathematics #Economics and finance}}.

=== Effects of deterministic chaos on economic models ===

Economic and meteorological simulations may share a fundamental limit to their predictive powers: [[Chaos theory|chaos]]. Although the modern mathematical work on [[chaotic systems]] began in the 1970s the danger of chaos had been identified and defined in ''[[Econometrica]]'' as early as 1958:
:"Good theorising consists to a large extent in avoiding assumptions&nbsp;... [with the property that] a small change in what is posited will seriously affect the conclusions."
:([[William Baumol]], Econometrica, 26 ''see'': [http://www.iemss.org/iemss2004/pdf/keynotes/Keynote_OXLEY.pdf ''Economics on the Edge of Chaos'']).

It is straightforward to design economic models susceptible to [[butterfly effect]]s of initial-condition sensitivity.<ref>[[Paul Wilmott]] on his early research in finance: "I quickly dropped&nbsp;... chaos theory [as] it was too easy to construct ‘toy models’ that looked plausible but were useless in practice." {{citation|first=Paul|last=Wilmott|title=Frequently Asked Questions in Quantitative Finance| publisher=John Wiley and Sons|year=2009|url=https://books.google.com/books?id=n4swgjSoMyIC&pg=PT227 |page=227|isbn=9780470685143 }}</ref><ref>{{citation|url=http://www.sp.uconn.edu/~ages/files/NL_Chaos_and_%20Macro%20-%20429%20Essay.pdf|first=Steve|last=Kuchta|title=Nonlinearity and Chaos in Macroeconomics and Financial Markets|publisher=[[University of Connecticut]]|year=2004}}</ref>

However, the [[econometric]] research program to identify which variables are chaotic (if any) has largely concluded that aggregate macroeconomic variables probably do not behave chaotically.{{Citation needed|reason=This claim and conclusions based on it need one or more reliable sources.|date=October 2023}} This would mean that refinements to the models could ultimately produce reliable long-term forecasts. However, the validity of this conclusion has generated two challenges:
* In 2004 [[Philip Mirowski]] challenged this view and those who hold it, saying that chaos in economics is suffering from a biased "crusade" against it by [[neo-classical economics]] in order to preserve their mathematical models.
* The variables in [[finance]] may well be subject to chaos. Also in 2004, the [[University of Canterbury]] study ''Economics on the Edge of Chaos'' concludes that after noise is removed from [[S&P 500]] returns, evidence of [[determinism|deterministic]] chaos ''is'' found.

More recently, chaos (or the butterfly effect) has been identified as less significant than previously thought to explain prediction errors. Rather, the predictive power of economics and meteorology would mostly be limited by the models themselves and the nature of their underlying systems (see [[Economic models#Comparison with models in other sciences|Comparison with models in other sciences]] above).

=== Critique of hubris in planning ===

A key strand of [[free market]] economic thinking is that the market's [[invisible hand]] guides an economy to prosperity more efficiently than [[command economy|central planning]] using an economic model. One reason, emphasized by [[Friedrich Hayek]], is the claim that many of the true forces shaping the economy can never be captured in a single plan. This is an argument that cannot be made through a conventional (mathematical) economic model because it says that there are critical systemic-elements that will always be omitted from any top-down analysis of the economy.<ref>{{Citation
| last = Hayek
| first = Friedrich
| author-link = Friedrich Hayek
| title = The Use of Knowledge in Society
| journal = American Economic Review
| volume = 35
| issue = 4
| pages = 519–30
| date = September 1945
| postscript = .
| jstor =1809376}}
</ref>