E-Mail Interaction With Rasmus Benestad On Regional Downscaling Of Multi-Decadal Global Climate Model Predictions For Use By The Impacts Community

Readers of my weblog know that I have been very critical of the ability of regional climate models to add any skillful information to multi-decadal global models from whatever is already present in those models [which themselves have no skill at predicting even the statistics of large scale circulation features, much less the change of these statistics due to human climate forcings].  For a discussion of this inability, see, for example,

The Huge Waste Of Research Money In Providing Multi-Decadal Climate Projections For The New IPCC Report

With respect to my viewpoint, I was invited (via skype) to give a presentation at a meeting in Japan

Pielke Sr., R.A. 2011: Is There Value-Added With Dynamic  Regional Downscaling? Final Workshop of the S5-3 Project, Tsukuba, Japan, October 18,  2011.

in which I discussed with Rasmus Benestad the science behind the concept of regional climate downscaling of multi-decadal global climate model predictions [which we define as a Type 4 downscaling in Castro et al 2005).

Rasmus and I continued our discussion via e-mail and, with his permission, I have reproduced below. For clarity, Rasmus’s comments are in regular font and my comments are in italics. Koji Dairaku and Rob Wilby, two colleagues and internationally well-respect experts on regional downscaling, were copied on all of the e-mails.

As an introduction, below is a short biographical summary of his outstanding professional credentials.

Dr. Rasmus Benestad D. Phil. (Oxon). Senior scientist, Norwegian Meteorological Institute. Experience: Leader for a project on seasonal forecasting at the Norwegian Meteorological Institute. Norwegian contact person for the European Meteorological Society. Relevant publications last 3 years:

Benestad, R.E., 2003: What can present climate models tell us about climate change? Climatic Change, 59, 311-332

Benestad, R.E., 2003: How often can we expect a record-event? Climate Research, 23, 3-13.

Benestad, R.E., 2004: clim.pact: Empirical-Statistical Downscaling for Everyone. EOS

Benestad, R.E., 2004: Tentative probabilistic temperature scenarios for northern Europe. Tellus 56A, 89-101

Benestad, R.E., 2004: Record-values, non-stationarity tests and extreme value distributions. Global and Planetary Change. Vol 44, issue 1-4, p.11-26

Benestad, R.E., 2005: A review of the solar cycle length estimates. GRL/32 doi:10.1029/2005GL023621

Benestad, R.E., 2005: Climate change scenarios for northern Europe from multi-model IPCC AR4 climate simulations, GRL/32 doi:10.1029/2005GL023401

Benestad, R.E., 2006: Can we expect more extreme precipitation on the monthly time scale? J.Clim., Vol.19, 630-637

The e-mails are reproduced below. There is some redundancy since I embedded my replies within several of the e-mails. I have highlighted key text.

From R. Pielke Sr.

Hi Rasmus

I enjoyed talking with you via skype while you were in Japan at the downscaling workshop that Koji organized.

As a follow up, I thought you might be interested in these papers below which present a viewpoint distinct from yours in terms of the value of downscaling from multi-decadal global climate model predictions.

I would be interested in discussing with you how our major concerns can be refuted.

Pielke Sr., R.A., R. Wilby, D. Niyogi, F. Hossain, K. Dairuku, J. Adegoke, G. Kallos, T. Seastedt, and K. Suding, 2011: Dealing with complexity and extreme events using a bottom-up, resource-based vulnerability perspective. AGU Monograph on Complexity and Extreme Events in Geosciences, in press. https://pielkeclimatesci.files.wordpress.com/2011/05/r-365.pdf

Pielke Sr., R.A., and R.L. Wilby, 2011: Regional climate downscaling . what.s the point? Eos Forum, submitted. https://pielkeclimatesci.files.wordpress.com/2011/10/r-361.pdf

With Best Regards

Roger

From Rasmus Benestad

Dear Roger, Rob and Koji,

Thanks for these! I think you make some good points – especially with the ‘bottom-up’ approach and thinking in terms of contextual vulnerability. It seems to me fairly obvious that this way of thinking is sensible, and also a bit surprising that these notions are not more ingrained.

I think that your papers provide an excellent starting point for discussions. Some of the points that you raise are in my view up for debate (and I don’t know the answers). It’s good to have some critical voices in the literature, but I also think that you’ve used a ‘broad brush’ in some of the criticism. Saying that, there is also another paper by Oreskes et al. (2010) – see link below – that fits in with your view.

When it comes to what’s the point of downscaling, I think the main concern is the GCMs ability to project regional climate change in ‘type 4’. I appreciate this point – at least when it comes to individual GCMs. The question is whether there is at least some information embedded in all the GCMs we have available. Can we use the CMIP3/5 to describe the range of possibilities?

A useful question, I think, is what are the real sources of information? To me, the obvious candidates seem to be empirical data and the laws of physics. Even acknowledging that the climate is extremely complicated and complex implies information, be it about the fact that the temperature may vary strongly over short distances due to very local phenomena.

I agree with several of your points, but I’m not convinced about the statements that the models are not able to simulate the NAO, ENSO, etc. But this depends on the expectations and degree of fidelity. Another question is whether the character of the NAO and ENSO will change in the future is still unknown, and I agree that if their character will change, we do not know if the models are able to predict this change.

Aside from that, I think some of the criticism is a bit exaggerated. But it really depends on what you want to look at. Some models are worse than others, but there are some models which are used in seasonal prediction and are able to provide some description of ENSO – albeit with biases. My own analysis of various GCMs also suggest that they reproduce the characteristics of the NAO too.

I think your discussion on empirical-statistical downscaling (ESD) also is a bit narrow because the field traditionally has been narrow (I guess it’s a bit analogous to the ‘top-down’ and ‘bottom-up’ discussion). To me, it involved more than just regression, and I’m getting more into predicting how the shape of the probability disntribution functions may change in a future climate (see attached paper – this is only the teoretical foundation of a new and promising method that attempts to predict extreme rainfall).

Furthermore, it is important to design ESD in such a way that minimises non-stationarity, and it is important to test the strategy to see if the design is successful. ‘Testing’ and ‘assessing’ are two key words – probably not appreciated enough. There are at least two types of tests: against the past and using quasi-reality to test for the future. Furthermore, by drawing in more information based on the past – as you point out – and improved statistical analysis, I believe that we in some cases can do better than just looking at the past. Sometimes, statistical models can be quite useful in terms of describing ‘uncertainty’.

All the best,

Rasmus

http://www2.lse.ac.uk/CATS/publications/papersPDFs/80_AdaptationtoGlobalWarming_2010.pdf http://journals.ametsoc.org/doi/abs/10.1175/2010JCLI3687.1

From R. Pielke Sr.

Hi Rasmus

Thank you for your detailed reply. Please see my responses embedded in your text.  Do I have your permission to post your e-mail and my reply on my weblog?

Best Wishes for the Holidays!

Roger

From Rasmus with my replies embedded inside of his comments

Dear Roger, Ron and Koji,

Thanks for these! I think you make some good points – especially with the  ‘bottom-up’ approach and thinking in terms of contextual vulnerability. It seems to me fairly obvious that this way of thinking is sensible, and also a bit surprising that these notions are not more ingrained.

Thank you for the encouragement. I hope we can obtain a wider acceptance of this approach.

I think that your papers provide an excellent starting point for discussions. Some of the points that you raise are in my view up for debate  (and I don’t know the answers). It’s good to have some critical voices in the literature, but I also think that you’ve used a ‘broad brush’ in some of the criticism. Saying that, there is also another paper by Oreskes et al. (2010) – see link below – that fits in with your view.

When it comes to what’s the point of downscaling, I think the main concern is the GCMs ability to project regional climate change in ‘type 4’. I  appreciate this point – at least when it comes to individual GCMs. The question is whether there is at least some information embedded in all the GCMs we have available. Can we use the CMIP3/5 to describe the range of possibilities?

Even with ensemble results, there is no evidence that Type 4 runs can provide skillful predictions on multi-decadal time scales. Your last two questions are hypotheses. Until this is properly tested, we really should not be presenting these results to the impacts community and claim they have any skill.

A useful question, I think, is what are the real sources of information? To me, the obvious candidates seem to be empirical data and the laws of physics. Even acknowledging that the climate is extremely complicated and  complex implies information, be it about the fact that the temperature may vary strongly over short distances due to very local phenomena.

The models are not fundamental physics, as only the dynamical core (such as advection, pressure gradient force, gravity) can fit in that definition. All other components of the climate models are engineering code with tunable coefficients and functions.

I agree with several of your points, but I’m not convinced about the statements that the models are not able to simulate the NAO, ENSO, etc. But this depends on the expectations and degree of fidelity.

Please provide papers that show an ability to simulate NAO, ENSO etc when run for multi-decadal time periods. I agree the models can replicate some aspects of these features when they are run in a weather prediction mode (primarily, in my view, because of the relatively slow changes in time of SSTs such that seasonal runs are a type of nowcasting for SST).

Another question is whether the character of the NAO and ENSO will change in the future is still unknown, and I agree that if their character will change, we do not know if the models are able to predict this change.

This is a key fundamental issue. If the models cannot skillfully predict CHANGES in the statistics of weather patterns, they add no value for the impacts community beyond what is available from the historical record, the recent paleo-record and worst case sequence of real world observed events. Thus, we should not be giving multi-decadal climate model climate change statistics to the impacts community and claim they have any skill.

Aside from that, I think some of the criticism is a bit exaggerated. But it  really depends on what you want to look at. Some models are worse than others, but there are some models which are used in seasonal prediction and are able to provide some description of ENSO – albeit with biases. My own analysis of various GCMs also suggest that they reproduce the characteristics of the NAO too.

Seasonal prediction is not a Type 4 application. I agree there is limited skill such as for ENSO on this time scale. The ability to faithfully predict seasonal weather, however, is a necessary condition but not a sufficient condition to then assume that multi-decadal predictions of climate is skillful. Seasonal prediction is a Type 3 application.

I think your discussion on empirical-statistical downscaling (ESD) also is a bit narrow because the field traditionally has been narrow (I guess it’s a bit analogous to the ‘top-down’ and ‘bottom-up’ discussion). To me, it involved more than just regression, and I’m getting more into predicting how the shape of the probability distribution functions may change in a future climate (see attached paper – this is only the theoretical foundation of   new and promising method that attempts to predict extreme rainfall).

I do not see how the paper you sent adds any new information in terms of predicting changes in statistics. The example from the Benestad et al 2012 paper that you enclosed is a type 2 downscaling study; see the text

“…the RCMs from ENSEMBLES, all of which had a spatial resolution of  50km and used ERA 40 as boundary conditions.”

It is not Type 4 downscaling.

Furthermore, it is important to design ESD in such a way that minimises non-stationarity, and it is important to test the strategy to see if the  design is successful. ‘Testing’ and ‘assessing’ are two key words – probably not appreciated enough. There are at least two types of tests: against the past and using quasi-reality to test for the future. Furthermore, by drawing in more information based on the past – as you point out – and improved statistical analysis, I believe that we in some cases can do better than just looking at the past. Sometimes, statistical models can be quite useful in terms of describing ‘uncertainty’.

We agree on the value of testing models in the hindcast mode. However, what is a “quasi-reality” test for the future? If you cannot have real world data to evaluate against, it is not a robust test.

Thanks again for engaging in this discussion! Please let me know if I can post.

All the best,

Rasmus

From Rasmus

to me, R, Koji

Sure – you can post my response on your blog. I hope you will let me emphasis the need for thorough testing of the models. It is important to carefully consider how these tests should be designed, and one important element is to see if the models are able to predict changes (it depends on the use of the models).

I did some evaluation of type-4 GCM skill reproducing the NAO (see attached paper 2001a), and some further work is described in various reports (I can provide you with more information, although this is ‘grey literature’). It is also important to keep in mind that the actual mode of variability that is of interest may not be exactly the NAO/ENSO, but a related pattern that covaries more strongly with the variable in question (see attached paper 2001b). In any case, ESD can be used to evaluate type 4 GCM simulations.

The same GCMs used in type 3 and type 4 runs incorporate the same set of physical processes encoded in computer lines, but they differ in terms of their initial conditions. In my mind, the fact that these are used for making ENSO prognoses suggest that they have some skill in simulating ENSO (however, the models also have some shortcomings too, such as double ITCZ, poor MJO). Furthermore, we see that the models – or their components – provide solutions which embed natural phenomena that have not been prescribed, be it the Hadley cell, westerlies, tropical cyclones, ocean currents, jets, tropical instability waves, ENSO, or the NAO. I agree that the models consist of a mixture of a dynamical core and parameterisations, these parameterisation scheemes are based on our physical understanding (representing the bulk physics) and the tuning should be restrained by observations. This means that the models are not perfect, but I think they can provide useful information.

I also agree that the bottom line is the question whether the models (in type 4 runs) can predict *changes* in the statistics of weather patterns. Again, I’ll stress the importance of tests and evaluation. And the importance of including information from other sources – very much in line with the discussion in your papers. This aspect provides the connection to the example provided by Benestad et al 2012 paper – there is such a clear pattern in the daily rain guage statistics that seems to be universal (I’ve looked at more than 30,000 rain gauges by now). Robust inter-dependencies provide us with additional information and constraints. The type of ESD can also be applied for type 3 & 4 cases.

Actually, I’d like to expand the discussion about information sources. In addition to empirical (which also includes geographical), and physics-based, there is information/constraintes from the mathematics (hence, statisticians can provide very useful contribution to climatological research). Part of this set of information is used in modelling, but should also be used in testing/evaluating the models – using information from independent sources. This can be done in many different ways, e.g. comparing spatial and temporal structures, predicting out-of-sample data.

All the best

Benestad, R.E. (2001) A comparison between two empirical downscaling strategies, Int. J. Climatology,Vol 21, Issue 13, pp.1645-1668. [DOI 10.1002/joc.703] http://onlinelibrary.wiley.com/doi/10.1002/joc.703/abstract

Benestad, R.E. (2001) The cause of warming over Norway in the ECHAM4/OPYC3 GHG integration, Int. J. Clim. 15 March Vol 21 371-387. [DOI: 10.1002/joc.603] http://onlinelibrary.wiley.com/doi/10.1002/joc.603/abstract

From R.Pielke Sr.

Hi Rasmus

Thank you for your detailed and thoughtful reply! In terms of showing value-added using downscaling (statistical and dynamic) it is crucial, in my view, to discriminate between the 4 types of downscaling. We all agree that there is very significant value added with Type 1 downscaling (NWP), and also for Type 2 and Type 3, although the value added becomes progressively less.

However, I do not see how your 2001 papers demonstrate value-added for Type 4 since the key fundamental requirement is that the models would have to skillfully predict changes in the climate statistics. The Type 4 models certainly do predict such changes, but what observational data, in hindcast of course, has been shown to agree with these predictions?

We agree on what you have written

“I also agree that the bottom line is the question whether the models (in type 4 runs) can predict *changes* in the statistics of weather patterns.”

but I do not see how the 2012 (or your earlier papers) have shown we can do this. The finding of a universal rainfall behavior is, of itself, a quite important finding, but it does not show we can predict changes in the statistics in mulit-decadal time periods. Indeed, I do not see how this could be done unless long term changes in major circulation patterns (such the PDO, ENSO, ect can be skillfully predicted.

Type 3 downscaling is distinct from Type 4 as the key variable, SSTs are prescribed in the former as an initial value, and only change relatively slowly over a season in the real world. Thus, Type 3 forecasts provide an upper bound on the skill of Type 4 runs [and Type 2 is an upper bound for Type 3].

I do completely agree with you that the models are very valuable to inform us on climate processes (e.g. the Hadley cell, ITCZ, etc). However, this does not mean they can skillfully predict the multi-decadal changes in these features. That is climate prediction and it must be validated against real-world data.

A necessary condition, of course, for skillful predictions of the multi-decadal changes in climatology is that the climate processes be faithfully replicated. As you note, there are still issues with doing that. Until this level of skill is achieved, we cannot claim any skill of predicting changes in these processes.

Thus the bottom line remains that the Type 4 model runs (including the parent global models) have not shown skill in predicting changes in climatology of weather features that influence society and the environment such as drought, floods, hurricanes, heat waves, etc. Indeed, I have concluded this is a very daunting task.

I would welcome specific examples of where Type 4 runs have been shown in hindcast to skillfully predict changes in such features as drought frequency, ect.

I will plan to post our e-mail exchanges when we have completed them. If you could, please send me the urls for the papers you have sent and referred to as that will make it easier for the reader to access them when I post.

I very much appreciate this constructive interaction!

From Rasmus

to me, R, Koji

Hi Roger,

I think you are right that we strictly do not know whether the models are able to predict the future – type 4 case – but this is also true for all predictions. Take the weather forecast for example – we do not know whether they will forecast the true situation, especially at longer ranges. Nevertheless, we are fairly confident about nowcasting, and there is a gradual decline in the model skill.

I think that the skill in these models is mostly due to both the inter-dependencies in the atmosphere/ocean and the observations (constraints). In your distinction, however, you imply that the skill is mostly due to the observations (constraints). I think that the models deserve a little more credit, and that analyses of inter-dependencies between different types of ‘fields’ (e.g. surface temperature, sea surface temperature, mean sea level pressure, geopotential heights, winds) can shed light on the intrinsic model skills.

When the GCMs (type 4) simulate the spatial structure and the statistical nature (mean, variance, interannual variability, and coupling between different fields) approximately correctly, then we see that the models are able to describe some of the most important inter-dependencies. The cited 2001 papers describe both the simulated spatial structure of the NAO and the stationarity between scales as simulated by the GCM, but this analysis is not exhaustive, so you have a point. We do not know for certain.

The hypothesis with which we are concerned is whether there are similar inter-dependencies between the greenouse gas concentrations (warmer world) and regional atmospheric phenomena. The presence of strong inter-dependencies in both temporal and spatial dimensions, such as shown in the 2012-paper, provides useful information for type 4 predictions.

Working for a meteorological service, I have to provide practical and useful information for e.g. decision making. We need to live with uncertainties and we need to specify the unknowns. This is nothing new, and large sums of money is used to plan for uncertain outcomes – the most obvious example is a country’s defence (we do not know if the country will need the army and the arms). The same deal with mitigation and climate adaptation – we need to think risk analysis. Therefore I think that your idea concerning ‘bottom-up’ strategies and ‘contextual vulerability’ is so valuable.

All the best,

Rasmus

From R. Pielke Sr.

Hi Rasmus

Thank you for your detailed and thoughtful reply! In terms of showing value-added using downscaling (statistical and dynamic) it is crucial, in my view, to discriminate between the 4 types of downscaling. We all agree that there is very significant value added with Type 1 downscaling (NWP), and also for Type 2 and Type 3, although the value added becomes progressively less.

However, I do not see how your 2001 papers demonstrate value-added for Type 4 since the key fundamental requirement is that the models would have to skillfully predict changes in the climate statistics. The Type 4 models certainly do predict such changes, but what observational data, in hindcast of course, has been shown to agree with these predictions?

We agree on what you have written

“I also agree that the bottom line is the question whether the models (in type 4 runs) can predict *changes* in the statistics of weather patterns.”

but I do not see how the 2012 (or your earlier papers) have shown we can do this. The finding of a universal rainfall behavior is, of itself, a quite important finding, but it does not show we can predict changes in the statistics in mulit-decadal time periods. Indeed, I do not see how this could be done unless long term changes in major circulation patterns (such the PDO, ENSO, ect can be skillfully predicted.

Type 3 downscaling is distinct from Type 4 as the key variable, SSTs are prescribed in the former as an initial value, and only change relatively slowly over a season in the real world. Thus, Type 3 forecasts provide an upper bound on the skill of Type 4 runs [and Type 2 is an upper bound for Type 3].

I do completely agree with you that the models are very valuable to inform us on climate processes (e.g. the Hadley cell, ITCZ, etc). However, this does not mean they can skillfully predict the multi-decadal changes in these features. That is climate prediction and it must be validated against real-world data.

A necessary condition, of course, for skillful predictions of the multi-decadal changes in climatology is that the climate processes be faithfully replicated. As you note, there are still issues with doing that. Until this level of skill is achieved, we cannot claim any skill of predicting changes in these processes.

Thus the bottom line remains that the Type 4 model runs (including the parent global models) have not shown skill in predicting changes in climatology of weather features that influence society and the environment such as drought, floods, hurricanes, heat waves, etc. Indeed, I have concluded this is a very daunting task.

I would welcome specific examples of where Type 4 runs have been shown in hindcast to skillfully predict changes in such features as drought frequency, ect.

I will plan to post our e-mail exchanges when we have completed them. If you could, please send me the urls for the papers you have sent and referred to as that will make it easier for the reader to access them when I post.

I very much appreciate this constructive interaction!

Best Wishes for the Holidays!

Roger

Rasmus’s Reply

Sure – you can post my response on your blog. I hope you will let me emphasis the need for thorough testing of the models. It is important to carefully consider how these tests should be designed, and one important element is to see if the models are able to predict changes (it depends on the use of the models).

I did some evaluation of type-4 GCM skill reproducing the NAO (see attached paper 2001a), and some further work is described in various reports (I can provide you with more information, although this is ‘grey literature’). It is also important to keep in mind that the actual mode of variability that is of interest may not be exactly the NAO/ENSO, but a related pattern that covaries more strongly with the variable in question (see attached paper 2001b). In any case, ESD can be used to evaluate type 4 GCM simulations.

The same GCMs used in type 3 and type 4 runs incorporate the same set of physical processes encoded in computer lines, but they differ in terms of their initial conditions. In my mind, the fact that these are used for making ENSO prognoses suggest that they have some skill in simulating ENSO (however, the models also have some shortcomings too, such as double ITCZ, poor MJO). Furthermore, we see that the models – or their components – provide solutions which embed natural phenomena that have not been prescribed, be it the Hadley cell, westerlies, tropical cyclones, ocean currents, jets, tropical instability waves, ENSO, or the NAO. I agree that the models consist of a mixture of a dynamical core and parameterisations, these parameterisation scheemes are based on our physical understanding (representing the bulk physics) and the tuning should be restrained by observations. This means that the models are not perfect, but I think they can provide useful information.

I also agree that the bottom line is the question whether the models (in type 4 runs) can predict *changes* in the statistics of weather patterns. Again, I’ll stress the importance of tests and evaluation. And the importance of including information from other sources – very much in line with the discussion in your papers. This aspect provides the connection to the example provided by Benestad et al 2012 paper – there is such a clear pattern in the daily rain guage statistics that seems to be universal (I’ve looked at more than 30,000 rain gauges by now). Robust inter-dependencies provide us with additional information and constraints. The type of ESD can also be applied for type 3 & 4 cases.

Actually, I’d like to expand the discussion about information sources. In addition to empirical (which also includes geographical), and physics-based, there is information/constraintes from the mathematics (hence, statisticians can provide very useful contribution to climatological research). Part of this set of information is used in modelling, but should also be used in testing/evaluating the models – using information from independent sources. This can be done in many different ways, e.g. comparing spatial and temporal structures, predicting out-of-sample data.

All the best

Rasmus

R. Pielke Sr. Reply

Hi Rasmus

Thank you for your detailed reply. Please see my responses embedded in your text. Do I have your permission to post your e-mail and my reply on my weblog?

Best Wishes for the Holidays!

Roger

Rasmus  wrote with my reply embedded with italics

Dear Roger, Ron and Koji,

Thanks for these! I think you make some good points – especially with the ‘bottom-up’ approach and thinking in terms of contextual vulnerability. It seems to me fairly obvious that this way of thinking is sensible, and also a bit surprising that these notions are not more ingrained.

Thank you for the encouragement. I hope we can obtain a wider acceptance of this approach.

I think that your papers provide an excellent starting point for discussions. Some of the points that you raise are in my view up for debate (and I don’t know the answers). It’s good to have some critical voices in the literature, but I also think that you’ve used a ‘broad brush’ in some of the criticism. Saying that, there is also another paper by Oreskes et al. (2010) – see link below – that fits in with your view.

When it comes to what’s the point of downscaling, I think the main concern is the GCMs ability to project regional climate change in ‘type 4’. I appreciate this point – at least when it comes to individual GCMs. The question is whether there is at least some information embedded in all the GCMs we have available. Can we use the CMIP3/5 to describe the range of possibilities?

Even with ensemble results, there is no evidence that Type 4 runs can provide skillful predictions on multi-decadal time scales. Your last two questions are hypotheses. Until this is properly tested, we really should not be presenting these results to the impacts community and claim they have any skill.

A useful question, I think, is what are the real sources of information? To me, the obvious candidates seem to be empirical data and the laws of physics. Even acknowledging that the climate is extremely complicated and complex implies information, be it about the fact that the temperature may vary strongly over short distances due to very local phenomena.

The models are not fundamental physics, as only the dynamical core (such as advection, pressure gradient force, gravity) can fit in that definition. All other components of the climate models are engineering code with tunable coefficients and functions.

I agree with several of your points, but I’m not convinced about the statements that the models are not able to simulate the NAO, ENSO, etc. But this depends on the expectations and degree of fidelity.

Please provide papers that show an ability to simulate NAO, ENSO etc when run for multi-decadal time periods. I agree the models can replicate some aspects of these features when they are run in a weather prediction mode (primarily, in my view, because of the relatively slow changes in time of SSTs such that seasonal runs are a type of nowcasting for SST).

Another question is whether the character of the NAO and ENSO will change in the future is still unknown, and I agree that if their character will change, we do not know if the models are able to predict this change.

This is a key fundamental issue. If the models cannot skillfully predict CHANGES in the statistics of weather patterns, they add no value for the impacts community beyond what is available from the historical record, the recent paleo-record and worst case sequence of real world observed events. Thus, we should not be giving multi-decadal climate model climate change statistics to the impacts community and claim they have any skill.

Aside from that, I think some of the criticism is a bit exaggerated. But it really depends on what you want to look at. Some models are worse than others, but there are some models which are used in seasonal prediction and are able to provide some description of ENSO – albeit with biases. My own analysis of various GCMs also suggest that they reproduce the characteristics of the NAO too.

Seasonal prediction is not a Type 4 application. I agree there is limited skill such as for ENSO on this time scale. The ability to faithfully predict seasonal weather, however, is a necessary condition but not a sufficient condition to then assume that multi-decadal predictions of climate is skillful. Seasonal prediction is a Type 3 application. I think your discussion on empirical-statistical downscaling (ESD) also is a bit narrow because the field traditionally has been narrow (I guess it’s a bit analogous to the ‘top-down’ and ‘bottom-up’ discussion). To me, it involved more than just regression, and I’m getting more into predicting how the shape of the probability distribution functions may change in a future climate (see attached paper – this is only the theoretical foundation of new and promising method that attempts to predict extreme rainfall).

I do not see how the paper you sent adds any new information in terms of predicting changes in statistics. The example from the Benestad et al 2012 paper that you enclosed is a type 2 downscaling study; see the text

“…the RCMs from ENSEMBLES, all of which had a spatial resolution of 50km and used ERA 40 as boundary conditions.”

It is not Type 4 downscaling.

Furthermore, it is important to design ESD in such a way that minimises non-stationarity, and it is important to test the strategy to see if the design is successful. ‘Testing’ and ‘assessing’ are two key words – probably not appreciated enough. There are at least two types of tests: against the past and using quasi-reality to test for the future. Furthermore, by drawing in more information based on the past – as you point out – and improved statistical analysis, I believe that we in some cases can do better than just looking at the past. Sometimes, statistical models can be quite useful in terms of describing ‘uncertainty’.

We agree on the value of testing models in the hindcast mode. However, what is a “quasi-reality” test for the future? If you cannot have real world data to evaluate against, it is not a robust test.

Thanks again for engaging in this discussion! Please let me know if I can post.

All the best,

Rasmus

http: // www2.lse.ac.uk/CATS/publications/papersPDFs/80_AdaptationtoGlobalWarming_2010.pdf http: // journals.ametsoc.org/doi/abs/10.1175/2010JCLI3687.1

Rasmus wrote

Hi Roger,

I think you are right that we strictly do not know whether the models are able to predict the future – type 4 case – but this is also true for all predictions. Take the weather forecast for example – we do not know whether they will forecast the true situation, especially at longer ranges.

Since their time horizon is short, we have millions of validation tests of weather predictions on the hours to days to several week time scales. Even for seasonal prediction, we have hundreds of tests.

However, this is not true for multi-decadal climate predictions. I am in favor (and have advocated for) assessing the limits of skillful predictability, but this is distinct from providing forecasts decades from now for the impacts community.

Nevertheless, we are fairly confident about nowcasting, and there is a gradual decline in the model skill.

Even with NWP, the statistically evaluated decline of forecast skill is generally exponential with the rate of drop off depending on the variable.

I think that the skill in these models is mostly due to both the inter-dependencies in the atmosphere/ocean and the observations (constraints). In your distinction, however, you imply that the skill is mostly due to the observations (constraints). I think that the models deserve a little more credit, and that analyses of inter-dependencies between different types of ‘fields’ (e.g. surface temperature, sea surface temperature, mean sea level pressure, geopotential heights, winds) can shed light on the intrinsic model skills.

It is the combination of observations and the physical rules of the models which provide the skillful forecasts. Models are essential for this (much of my career has involved working with the models. :-)  ).  However ,with Type 4 downscaling (and for their parent models), the real world observations are mostly forgotten (exceptions being the deeper ocean temperatures, the terrain, land-water boundaries, etc). We are relying on models to create changes in the statistics of weather due to forcing such as added CO2, yet have no way to validate the predictions except in a hindcast mode.

The challenge is to show, in this hindcast mode, that changes in local and regional climatology can be skillfuly predicted as a response to human and natural climate forcings. This has not been done.

When the GCMs (type 4) simulate the spatial structure and the statistical nature (mean, variance, interannual variability, and coupling between different fields) approximately correctly, then we see that the models are able to describe some of the most important inter-dependencies. The cited 2001 papers describe both the simulated spatial structure of the NAO and the stationarity between scales as simulated by the GCM, but this analysis is not exhaustive, so you have a point. We do not know for certain.

The successful simulation of the spatial structure and the statistical nature of the climate system is, unfortunately, only the first requirement (and as you correctly note even this has not been done completely). For the impacts community to have confidence in  multi-decadal local and regional predicted changes in climate, the models must show skill in these predictions. They have not.

Thus, our bottom-up, resource-based approach becomes, in our view, the preferred approach as we can assess thresholds beyond which key resources are threatened. In terms of climate, we can use historical, recent paleo-record, worst case sequence of events, and even sensitivity experiments (e.g. +5% relative humidity arbitrarily prescribed in NWP runs) to estimate plausible impacts with today’s and estimated future societal conditions.

The hypothesis with which we are concerned is whether there are similar inter-dependencies between the greenouse gas concentrations (warmer world) and regional atmospheric phenomena. The presence of strong inter-dependencies in both temporal and spatial dimensions, such as shown in the 2012-paper, provides useful information for type 4 predictions.

The 2012 paper is not a Type 4 prediction. We all agree that added CO2 will have an effect [I actually feel the biogeochemical effects are the larger concern). However, there is no skill on Type 4 and, in my view, we are misleading the impacts community by providing these forecasts.

Working for a meteorological service, I have to provide practical and useful information for e.g. decision making. We need to live with uncertainties and we need to specify the unknowns. This is nothing new, and large sums of money is used to plan for uncertain outcomes – the most obvious example is a country’s defence (we do not know if the country will need the army and the arms). The same deal with mitigation and climate adaptation – we need to think risk analysis. Therefore I think that your idea concerning ‘bottom-up’ strategies and ‘contextual vulerability’ is so valuable.

The assessment of risk is one reason that I have concluded that the top-down IPCC approach is much too narrow of an approach. Thank you for the positive feedback on the bottom-up approach!

Rasmus wrote

to me

You’re welcome. I think we agree on many issues, but might have sligtly different views on others. That’s a good thing.

R, Pielke Sr wrote

Hi Rasmus

I will work to post our e-mail exchanges. Others should benefit from our interactions. :-)

Best Wishes!

source of image

Comments Off on E-Mail Interaction With Rasmus Benestad On Regional Downscaling Of Multi-Decadal Global Climate Model Predictions For Use By The Impacts Community

Filed under Interviews

Comments are closed.