There is an important new contribution to our understanding of the capabilities of multi-decadal global climate model predictions. This new paper is
Fildes, R. and N. Kourentzes, 2011: Validation and forecasting accuracy in models of climate change. International Journal of Forecasting. doi 10.1016/j.ijforecast.2011.03.008
The first author, Robert Fildes, who is a Distinguished Professor at the Lancaster University Management School, summarizes the paper in an e-mail to me as
Climate Models, Global Warming and Forecast Accuracy
With the increasing emphasis on climate forecasting, not least by the Intergovernmental Panel on Climate Change’s forthcoming assessment, a crucial element in the argument for action is the accuracy of the forecasts produced by different climate models. Accuracy matters, in terms of the validity of the climate models, the presumed policy impacts and the uncertainties around the forecasts. These uncertainties translate into massive investments in resources which are potentially wasted or mis-allocated. In the paper, “Validation and forecasting accuracy in models of climate change, now published in the International Journal of Forecasting, we (Fildes and Kourentzes) cast doubt on the effectiveness of climate models when used in 10 year ahead forecasting. In fact, the Hadley Centre model analysed could be improved by following recommendations long established from forecasting research. Does this research cast doubt on global warming? – no. But it does cast doubt on the methods being used by climate modellers to produce accurate forecasts and the levels of accuracy that are claimed.
Their paper has the abstract
“Forecasting researchers, with few exceptions, have ignored the current major forecasting controversy: global warming and the role of climate modelling in resolving this challenging topic. In this paper, we take a forecaster’s perspective in reviewing established principles for validating the atmospheric-ocean general circulation models (AOGCMs) used in most climate forecasting, and in particular by the Intergovernmental Panel on Climate Change (IPCC). Such models should reproduce the behaviours characterising key model outputs, such as global and regional temperature changes. We develop various time series models and compare them with forecasts based on one well-established AOGCM from the UK Hadley Centre. Time series models perform strongly, and structural deficiencies in the AOGCM forecasts are identified using encompassing tests. Regional forecasts from various GCMs had even more deficiencies. We conclude that combining standard time series methods with the structure of AOGCMs may result in a higher forecasting accuracy. The methodology described here has implications for improving AOGCMs and for the effectiveness of environmental control policies which are focussed on carbon dioxide emissions alone. Critically, the forecast accuracy in decadal prediction has important consequences for environmental planning, so its improvement through this multiple modelling approach should be a priority.”
Excerpts from the paper are
‘…… unfortunately, Chapter 8 of the IPCC report, “Climate models and their evaluation” (Randall et al., 2007, Section 8.1.2.3), has not taken such a clear epistemological position. In particular, its view of falsifiability based on the analysis of in-sample evidence is overly limited in the criteria it lays down for its assessment of the AOGCM models “against past and present climate”.
“In summary, the evidence provided in the IPCC report on the validity of the various AOGCMs,supplemented by much research work, mostly from scientists within the GCM community, rests primarily on the physical science of the sub-models, rather than on their predictive abilities. The models also capture the stylised facts of climate such as the El Nino and the Southern Oscillation. While the IPCC authors note that there is a considerable degree of agreement between the outputs of the various models, the forecasts do differ quite substantially, and the combined model forecasts apparently conform to recent data better than any single model. The omissions in Chapter 10 of the IPCC report and most of the subsequent research lie in the lack of evidence that the models actually produce good forecasts. There is ample testimony in the forecasting literature of the difficulties of forecasting beyond the range of data on which a model is constructed. This is tempered somewhat by the recognition that the physical submodels are supposedly robust over the increasing CO2 emissions input, and key experimental parameters in the physical laws embedded in the models should remain constant. In fact, climate modellers have raised ‘completeness’ in model building above all other criteria when evaluating the model validity. It is not a criterion that earlier simulation modellers have ever regarded as dominant (Kleindorfer et al., 1998); rather, it has often been regarded as a diversion that detracts from both understanding and forecast accuracy.”
“…the structural weaknesses in the GCM identified here suggest that a reliance on the policy implications from the general circulation models, and in particular the primary emphasis on controlling global CO2 emissions, is misguided (a conclusion which others have reached by following a different line of argument, see Pielke Sr. et al., 2009).”
“The scientific community of global climate modellers has surely taken unnecessary risks in raising the stakes so high when depending on forecasts and models that have many weaknesses.”
Climate Science: Roger Pielke Sr. 
