We know the weather reports are wrong
We have an innate desire to know what will happen. It can manifest itself in the form of very practical questions related to technology and applications of physics and biology, like "if I sharpen this stick, and throw it at one of those mammoths, can I feed my family with it?" Questions can be strategic in nature, like "if I invade Persia with my legions, will I win?" Big questions at the moment include "if we continue to pour carbon into the atmosphere at current rates, are we doomed?" and "if we increase our commitment of troops in Iraq, will we be reelected?" The path to answers has ranged from the scientific method to scapulimancy, computer simulation of prototype machinery, eminent study group reports, the I Ching, consultation of sacred texts or oracles, business forecasts, and climate change models. At one level, all of these represent a genuine effort, given beliefs and available technology, to discern the future. Depending on your cultural predispositions, some of these methods are preferred to others. For those that belong to the tribe of Aristotle and Bacon, the drive to understand has led us to a combination of formulating theories about cause and effect, and confronting them with experiment and mathematics.
In an uncertain world, what is the role of scenario or policy studies? I include here macroeconomic forecasts, as well as climate change models and the recent hybrid of the two -- studies of the economic consequences of climate change.
An important example now in play at the monent is the Stern Report. The report offers us a synthesis and interpretation of available climate and economic simulation modeling. There has been heated discussion in editorials and blogs (see the RealClimate blog) about the quality of these estimates. If the report is correct, then inaction in the face of rising levels of global warming and greenhouse gasses implies very high opportunity costs relative to the alternative path of corrective behavior. If it is wrong, corrective action can be a costly mistake.
What is the function of an inherently political report on economic and climate forecasts? For that matter, what is the function of the broader family of forecast and simulation-based policy studies? At one level, they are certainly meant to quantify the issues at hand. Viewed in this way, it really is important that the numbers be right. If we are going to make decisions, we want accurate forecasts. This seems to be the spirit behind much of the critical economic discussion (dissection?) of the Stern Report. Did we get interest rates (i.e. the discount rate for future costs and benefits) correct? (Jane Galt offers a good discussion of the discount rate issue). Are underlying assumptions about the trajectory of technical change correct? (See "Running the rule over Stern's numbers," BBC). Is opportunity cost calculated correctly?
This discussion and criticism is important, especially if the purpose of the forecasting study really is to accurately forecast. Indeed, from this starting point, the criticism of the Stern Report has been quite harsh. Richard Tol, from Carnegie Mellon University, is quoted by BBC4 saying "There is a whole range of very basic economics mistakes that somebody who claims to be a Professor of Economics simply should not make...Stern consistently picks the most pessimistic for every choice that one can make. He overestimates through cherry-picking, he double counts particularly the risks and he underestimates what development and adaptation will do to impacts." These criticims are from people who believe climate change is happening, and that something needs to be done. At one level, the criticims are valid. However, I think many of the critics have missed the point of the study. The purpose the study is not really to get an accurate prediction of exact economic costs 30 years from now. Rather, the purpose of the report is to serve as a focal point for discussion, and to provide a broad sense of alternative directions and magnitudes. It is a policy study, and hence by definition is something of a straw man. The discussion it fosters will most certainly discard many of the premises of the study, and will move in directions not covered in the original forecasts and estimates. This is ok. The role of policy studies (even when the economics are sloppy) is to serve as starting points for discussion by creating a structured, rational inventory of our uncertainties. The Stern Report creates a space for constructive argument. Arguably, reports by oracles and high priests have served a similar purpose in the past. In this sense, they are all important and useful, even if wrong.
To illustrate the point, I am going to quote Kenneth Arrow out of context. In a very personal, worldview discussion of uncertainty and the hopelessness of accurately modeling what will happen in the real world of markets, Arrow offers the following anecdote from World War II:
''Some of my colleagues had the responsibility of preparing long-range weather forecasts, i.e., for the following month. The statisticians among us subjected these forecasts to verificiation and found they differed in no way from chance. The forecasters themselves were convinced and requested that the forecasts be discontinued. The reply read approximately like this: ' The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.' '' [*]
Arrow's point is that, at one level, the weather reports really were useless because they were wrong. Yet he then goes on to say that "Accuracy of prediction is a desirable aim, but it is not the only aim of economic theory. As in meteorology, understanding is possible, desirable, and useful even when predictability is very limited." Like CGE studies of trade policy, finance ministry and Congressional Budget Office budget forecasts, and military plans, we know the precise estimates in the Stern Report will prove wrong. This does not preclude their usefulness for policy debate.
We stress accuracy and statistical robustness when we teach econometrics and modeling. We also get bogged down in debate on the merits of observation vs. simulation. Maybe, in paying attention to the accuracy of the art, we sometimes lose sight of its purpose. Accuracy is not the only important function of forecasting and numerical modeling. It may also be to create space for constructive argument. We need policy studies for planning purposes.
Further reading:
1. The Stern Report.
2. The BBC4 report, "Running the rule over Stern's numbers".
3. "BBC Finds Stern Report Wanting", (The Volokh Conspiracy)
4. Robert Mendelsohn's "A Critique of the Stern Report"
5. The RealClimate blog.
6. Jane Galt's discussion on the discount rate issue.
[*] K. Arrow (1992), "I Know a Hawk From a Handsaw," in M. Szenberg, editor, Emminent Economists: Their Life Philosophies, Cambridge University Press.
In an uncertain world, what is the role of scenario or policy studies? I include here macroeconomic forecasts, as well as climate change models and the recent hybrid of the two -- studies of the economic consequences of climate change.
An important example now in play at the monent is the Stern Report. The report offers us a synthesis and interpretation of available climate and economic simulation modeling. There has been heated discussion in editorials and blogs (see the RealClimate blog) about the quality of these estimates. If the report is correct, then inaction in the face of rising levels of global warming and greenhouse gasses implies very high opportunity costs relative to the alternative path of corrective behavior. If it is wrong, corrective action can be a costly mistake.
What is the function of an inherently political report on economic and climate forecasts? For that matter, what is the function of the broader family of forecast and simulation-based policy studies? At one level, they are certainly meant to quantify the issues at hand. Viewed in this way, it really is important that the numbers be right. If we are going to make decisions, we want accurate forecasts. This seems to be the spirit behind much of the critical economic discussion (dissection?) of the Stern Report. Did we get interest rates (i.e. the discount rate for future costs and benefits) correct? (Jane Galt offers a good discussion of the discount rate issue). Are underlying assumptions about the trajectory of technical change correct? (See "Running the rule over Stern's numbers," BBC). Is opportunity cost calculated correctly?
This discussion and criticism is important, especially if the purpose of the forecasting study really is to accurately forecast. Indeed, from this starting point, the criticism of the Stern Report has been quite harsh. Richard Tol, from Carnegie Mellon University, is quoted by BBC4 saying "There is a whole range of very basic economics mistakes that somebody who claims to be a Professor of Economics simply should not make...Stern consistently picks the most pessimistic for every choice that one can make. He overestimates through cherry-picking, he double counts particularly the risks and he underestimates what development and adaptation will do to impacts." These criticims are from people who believe climate change is happening, and that something needs to be done. At one level, the criticims are valid. However, I think many of the critics have missed the point of the study. The purpose the study is not really to get an accurate prediction of exact economic costs 30 years from now. Rather, the purpose of the report is to serve as a focal point for discussion, and to provide a broad sense of alternative directions and magnitudes. It is a policy study, and hence by definition is something of a straw man. The discussion it fosters will most certainly discard many of the premises of the study, and will move in directions not covered in the original forecasts and estimates. This is ok. The role of policy studies (even when the economics are sloppy) is to serve as starting points for discussion by creating a structured, rational inventory of our uncertainties. The Stern Report creates a space for constructive argument. Arguably, reports by oracles and high priests have served a similar purpose in the past. In this sense, they are all important and useful, even if wrong.
To illustrate the point, I am going to quote Kenneth Arrow out of context. In a very personal, worldview discussion of uncertainty and the hopelessness of accurately modeling what will happen in the real world of markets, Arrow offers the following anecdote from World War II:
''Some of my colleagues had the responsibility of preparing long-range weather forecasts, i.e., for the following month. The statisticians among us subjected these forecasts to verificiation and found they differed in no way from chance. The forecasters themselves were convinced and requested that the forecasts be discontinued. The reply read approximately like this: ' The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.' '' [*]
Arrow's point is that, at one level, the weather reports really were useless because they were wrong. Yet he then goes on to say that "Accuracy of prediction is a desirable aim, but it is not the only aim of economic theory. As in meteorology, understanding is possible, desirable, and useful even when predictability is very limited." Like CGE studies of trade policy, finance ministry and Congressional Budget Office budget forecasts, and military plans, we know the precise estimates in the Stern Report will prove wrong. This does not preclude their usefulness for policy debate.
We stress accuracy and statistical robustness when we teach econometrics and modeling. We also get bogged down in debate on the merits of observation vs. simulation. Maybe, in paying attention to the accuracy of the art, we sometimes lose sight of its purpose. Accuracy is not the only important function of forecasting and numerical modeling. It may also be to create space for constructive argument. We need policy studies for planning purposes.
Further reading:
1. The Stern Report.
2. The BBC4 report, "Running the rule over Stern's numbers".
3. "BBC Finds Stern Report Wanting", (The Volokh Conspiracy)
4. Robert Mendelsohn's "A Critique of the Stern Report"
5. The RealClimate blog.
6. Jane Galt's discussion on the discount rate issue.
[*] K. Arrow (1992), "I Know a Hawk From a Handsaw," in M. Szenberg, editor, Emminent Economists: Their Life Philosophies, Cambridge University Press.
Labels: climate change, climate models, discount rates, economic models, forecasting, global warming, policy studies, Stern Report
posted by Joe Francois at
12:22 AM
Links to this post
