The InterAcademy Council Review Of The IPCC is performing an evaluation of the procedures and process of the InterGovernmental Panel on Climate Change and is soliciting public comments by July 1 2010. I plan to submit comments and encourage readers of my weblog who are climate scientists to also provide input to them. The online IAC Review Panel questionnaire is available on-line here. Next week, the IAC will meet in Montreal Canada. The members of the IAC are listed here.
Below, with permission, I have posted the insightful comments provided by Marcel Crok.
Questionnaire on IPCC Processes and Procedures Input by Marcel Crok
Name
Marcel CrokAffiliation
1. What role(s), if any, have you played in any of the IPCC assessment processes?
None; As a science journalist I interviewed a lot of climate scientists from all sides of the spectrum, from strong believers to strong sceptics and everything in between.2. What are your views on the strengths and weaknesses of the following steps in the IPCC assessment process? Do you have any recommendations for improvement?
a. Scoping and identification of policy questions
A crucial question is whether the focus of IPCC was wrong from the start by focusing purely on the potential role of man-made global warming. We need science historians to figure that out. The growing certainty in the four IPCC reports that CO2 is the main culprit reflects more the demand for a clear answer from policy makers than the advancement of the scientific evidence for the role of CO2, in my opinion.b. Election of bureau including working group chairs
Working group chairs are always strong believers in the AGW hypothesis. This will have impact on the supposed neutrality of the assessment. There should be more balance in the choice of these chairs.c. Selection of lead authors
Here the inclusion of scientists with different perspectives is even more important. John Christy was the only outspoken sceptic who contributed to WG1 of AR4, but only as a contributing author. Without having scientists with different perspectives in the lead author teams, the danger of scientists confirming their own beliefs is very real.
A second big problem is that lead authors are promoting their own work in the chapters and are often asked to give an ‘objective’ opinion about controversies in which they themselves are involved. A clear example of this in AR4 is Phil Jones who promotes his HadCrut3 graph meanwhile ignoring literature (Michaels/McKitrick 2004, 2006) that raises serious questions about the reliability of this graph. A second example is Briffa as lead author of chapter 3, where he, as a member of the Hockey Team, defends the hockey stick, and ignores the divergence problem (i.e. the fact that his temperature reconstruction based on tree rings is going down after 1960).
A third problem is that IPCC didn’t seem to take resignations of authors, like Paul Reiter and Chris Landsea, very serious. They are both quite critical about the relation between AGW and malaria and hurricanes respectively. It’s highly disturbing when such well-recognized experts are so disappointed by the process that they resign, but IPCC did nothing at all to investigate this or to make clear to the world what happened exactly.
d. Writing of working group reports
See e.
e. Review processes
As scientists with criticism about aspects of the AGW hypothesis are not in the lead author teams, their role is limited to that of the expert reviewers. So how the lead authors deal with comments on the first and second draft is crucial for the final outcome. It’s a big improvement that all comments and reactions of the lead authors have been made public during and after the publication of AR4. However, reading the comments and the reactions of the authors, one can only get the impression that most of the lead authors are completely unwilling to let criticism on the AGW hypothesis through to the final report. Look for comments by McIntyre and McKitrick on chapter 3 and 6. So, in principle the process is fine and quite transparent, but in practice it fails for the simple reason that lead authors are unwilling to accept criticism on their own work and views.
Finally, the role of the review editors is very important. IPCC procedures state: “where significant differences of opinion on scientific issues remain, such differences are described in an annex to the Report.” When their reports were finally made public after requests from David Holland (see http://climateaudit.org/2008/04/01/ipcc-review-editors-comments-online/) it turned out that 25 of the 26 review editors of WG1 signed a standard form letter. Only one, John Mitchell of chapter 6, sent in some comments. None of the review editors felt it necessary that differences of opinion should be described in an annex. Given the substantial criticism from expert reviewers this can only mean that the editors are fully on the side of the lead authors. Again, a lack of balance in the choice of these people leads to a less objective assessment than promoted by the IPCC itself.
f. Preparation of the Synthesis report, including the Summary for Policy Makers
It’s unfortunate that the SPM is released before the report itself. Journalists have to accept claims in the SPM without having access to the full report. This is strange and would not be accepted in other fields, let alone in industry.
g. Adoption of report by the IPCC plenary
IPCC could be much more open towards the media in that stage. As far as I know all final meetings are closed for the media. Slightly off topic: in 2009 I wanted to attend a WG1 meeting of IPCC lead authors on Hawaii. I asked Susan Solomon by email, but the request was refused. Later an IPCC lead author told me this is just for practical reasons. They don’t want these meetings to be too big. But given the recent criticism about IPCC, more openness in this regard should be a step forward. Holding closed meetings gives the impression – that we also get from the climategate emails – that IPCC authors are discussing tactics how to deal with ‘the sceptics’ instead of just doing the job the best they can.
h. Preparation of any special reports
No opinion
3. What is your opinion on the way in which the full range of scientific views is handled?This is in my opinion one of the biggest failures of the IPCC reports. There are currently more than a handful of hypotheses around in the literature to explain the recent warming. Some of this literature is mentioned in the IPCC reports, but only to be dismissed immediately. It should be a high priority for IPCC to form a separate team of really independent scientists (senior scientists from other fields) to make an inventory of the alternative views that are described in the literature. In the next assessment report there should be much more attention for these alternative hypotheses. To mention a few:
1) The role of clouds and water vapor as negative feedback instead of positive (Lindzen, Spencer)
2) The role of oceans (ENSO, PDO, AMO etc.) (Spencer, Swanson/Tsonis, Compo/Sardeshmukh)
3) The role of the sun which itself has different sub-hypotheses:
3a Based on Total Solar Radiation (Scafetta, Soon)
3b Based on the role of UV in the stratosphere (Van Loon/Labitzke)
3c Based on the relation between Cosmic Rays and clouds (Svensmark/Christensen, Veizer/Shaviv)
4) the role of other human forcings than CO2, e.g land use changes, soot and nitrogen deposition, the biogeochemical effect of CO2. In this respect I should mention this interesting essay by Roger Pielke sr and a large group of scientists:https://pielkeclimatesci.files.wordpress.com/2009/12/r-354.pdf
They mention three hypotheses of which only one can be true. Their preferred hypothesis is hypothesis 2a: Although the natural causes of climate variations and changes are undoubtedly important, the human influences are significant and involve a diverse range of first-order climate forcings, including, but not limited to, the human input of carbon dioxide (CO2). Most, if not all, of these human influences on regional and global climate will continue to be of concern during the coming decades. If true this is crucial information for policy makers, because it means that a focus on CO2 only will never lead to effective policy decisions.
4. Given the intergovernmental nature of IPCC, what are your views on the role of governments in the entire process?
Governments are trusting the IPCC far too much. The IPCC reports have a monopoly position. This is acceptable when enough checks and balances are incorporated in the production of the reports. There are no real checks and balances however, not in the meaning these terms have in the industry. The IPCC is relying on the peer review of journals, but as every scientist know, this is an easy to pass filter, especially when an article confirms the current paradigm.
Also, IPCC has no rules for data archiving and sharing. This resulted in Phil Jones not making his raw temperature data available, even when critics asked for this data via FOI requests. IPCC nevertheless uses these data as one of the basic pillars in their global warming edifice.
Thirdly, IPCC has no rules for conflicts of interest. Lead authors can work for industry or environmental organizations. The chairman of the IPCC can work for banks and even emission trading firms without breaking IPCC rules. In any other field this would be highly disturbing but so far governments haven’t paid attention to this in the IPCC.
5. Given that IPCC assessments consider a vast amount of literature, what are your views and suggestions for improvement on the sources of data and the comprehensiveness of the literature used, including non-peer-reviewed literature?
My answer is limited to WG1. Instead of trying to be complete, this report should focus much more on the really crucial issues: What are the natural and anthropogenic forcings that influence the climate, what are the relevant feedbacks? How can we decide whether feedbacks are positive or negative? How good are the models? Which models are the best and for what reasons? Many of these crucial issues are now buried deep inside the report instead of highlighted in the summary. Progress would be much faster if supporters of AGW and sceptics would be forced to sit together at the table and discuss where the consensus is and where the discrepancy. This worked when the satellite temperature of Spencer and Christy were challenged by Mears and Wentz; it worked when hurricane specialists sat around the table to come up with a consensus statement.
6. What are your views and suggestions regarding the characterization and handling of uncertainty in each of the working group reports and the synthesis report?The currently used terminology (likelihood) is misleading. A claim like it’s very likely (90% certainty) that recent warming was due to anthropogenic greenhouse gases gives a false sense of certainty. This 90% is not based on statistical calculations but on expert judgement, but few people outside IPCC realise this.
7. What is your view of how IPCC handles data quality assurance and quality control and identification and rectification of errors, including those discovered after publication?
As I said in answer to question 4, IPCC should pay much more attention to data availability, quality and control. They could simply require that when articles are mentioned in the report, data and source code should be archived. It’s impossible for lead authors to check all the data themselves. This is not necessary however. They should focus first on data on which important claims are based, like the global average temperature, the hockey stick, cloud data, water vapor data etc.
It should be possible to rectify errors after publication. The problem is that errors are often subjective. McKitrick (of Michaels/McKitrick 2004 and 2006) and De Laat (of De Laat/Maurellis 2004/2006) were very disappointed by the way IPCC misrepresented their work in chapter 3. What was written in the final report was not even in the second draft, so it was not reviewed at all. After the publication of AR4 De Laat and McKitrick had no means to correct the misrepresentation of their work. It’s not directly obvious how one can prevent this. It should be possible, however, for scientists to complain to IPCC when they feel their work is misrepresented.
8. What is your view of how IPCC communicates with the media and general public, and suggestions for improving it?
As I wrote earlier, in my opinion IPCC should be more open to the media. I see no reason why WG1 meetings or other meetings should be closed for journalists. This conflicts with their supposed open and transparent process.
9. Comment on the sustainability of the IPCC assessment model. Do you have any suggestions for an alternative process?
I don’t think the current approach will be sustainable, especially now when more and more journalists and policy makers are beginning to realize that the process is much more one sided than they thought.
As an alternative I like the proposal of John Christy to start a Wiki-IPCC. Small and balanced teams of scientists working together on the crucial topics I mentioned before. This has the advantage that they can update their consensus statement whenever there are important developments in the observations or in the scientific literature.
10. Do you have any suggestions for improvements in the IPCC management, secretariat, and/or funding structure to support an assessment of this scale?The IPCC is officially policy neutral. However in their public statements both the chairman and the secretary make clear policy suggestions all the time. In terms of Roger Pielke jr, is the IPCC an honest broker or an issue advocate? In my opinion they should operate as an honest broker, but as this moment they are clearly an issue advocate.
The scale of the assessment should be reduced. Given the huge uncertainties in WG1, a lot of the information in WG2 and WG3 is pure speculation. WG1 should focus more on the crucial issues, and the size of the WG2 and WG3 reports could be greatly reduced.
11. Any other commentsI hope the IAC review panel will listen to all parties in the debate, i.e. including well informed critics like McIntyre, McKitrick, Pielke sr etc. So far I am pleased by the openness of the IAC review panel. I would encourage the panel to make all the information they acquire (submissions, interviews, documents) available on their website.
Climate Science: Roger Pielke Sr. 