Gavin Schmidt has an informative and candid summary of the grant review process in his post on Real Climate
Reflections on funding panels
His overview accurately summarizes the issues with the review process including, very importantly, how this process can be very biased. His comments on the process also apply to mail in-reviews that are solicited by funding agencies.
I have excerpted several of his statement on this issue for further comment, as it illustrates why bias has, unfortunately, permeated this funding process.
Gavin wrote
Funding is highly competitive and, depending on the call, only between 10 and 20% of proposals will be funded. Choosing which proposals get funded relies on the good judgement of the program managers in the most part, but they are helped enormously by external reviewers and panels.
The “good judgement” of the program managers includes who they select as reviewers and, even with very good reviews, which proposals they decide to fund. The program managers themselves clearly have biases also.
I have documented the failure (in my view) of the review process with respect to my NSF proposals in the posts
NSF Decision On Our Request For Reconsideration Of A Rejected NSF Proposal On The Role Of Land Use Change In The Climate System
Is The NSF Funding Process Working Correctly?
In the later post I recommended that
[t]he short answer to this question is NO. I recommend the following changes in the NSF review procedures to improve the process:
present ALL proposal abstracts, anonymous reviews of both accepted and rejected proposals and program managers decision letters (or e-mails) on line for public access
present the date of submission and final acceptance (or rejection) of the proposal.
My suggestion has been ignored. Gavin Schmidt’s next excerpt is
Having someone on the panel whose expertise covered the specific topic of a proposal can be polarizing. That is, such a proposal was either more likely to shine or be dismissed than a proposal for whom no-one is as intimately knowledgeable.
This is actually quite a serious criticism. This candid admission indicates that bias is regularly acted on in the panel reviews.
Gavin continues
Conflicts of interest exist – proposals can come in from a previous student of a panel member, or a current colleague or close collaborator. However, in all such cases, the conflicted person is asked to leave the room and not participate in the discussion on that proposal. This works well in avoiding “less objective” criteria in funding.
Conflicts of interest, however, also include “someone on the panel whose expertise covered the specific topic” since as Gavin “wrote such a proposal was either more likely to shine or be dismissed than a proposal for whom no-one is as intimately knowledgeable.” This is a clear example of exercising a bias in making a funding recommendation.
Gavin writes further
[The] [r]eputation of the proposers as people capable of good science goes a long way to judging the feasibility of a proposal.
This quite a remarkable admission since while the statement that the “reputation of the proposers as people capable of good science goes a long way to judging the feasibility of a proposal”, seems appropriate, it is actually a statement of bias. In the polarized environment of climate science, “good science” has come to mean climate science that fits in the worldview of the reviewers. Kevin Trenberth’s recent labeling of the paper
R. S. Knox, David H. Douglass 2010: Recent energy balance of Earth International Journal of Geosciences, 2010, vol. 1, no. 3 (November) – In press doi:10.4236/ijg2010.00000
as “rubbish” is just one example (see).
My own experiences with the review process, including a panel in which both Gavin Schmidt and I appeared, supports the view that “good science” as Gavin presents, means climate science that he agrees with. In my post
Protecting The IPCC Turf – There Are No Independent Climate Assessments Of The IPCC WG1 Report Funded And Sanctioned By The NSF, NASA Or The NRC (and see also)
I wrote an a National Research Council Study Planning Meeting titled
The Attribution of Solar Forcing to Recently Observed Climate Change
The meeting outline included the text
“In order to understand the solar influence on climate and the atmosphere, it is essential to also understand the contributions of volcanic aerosols, as well as anthropogenic greenhouse gases and tropospheric aerosols, and other human influences such as land use changes, all of which contribute to the observed climate. Furthermore, because there is growing evidence that responses of the climate system to these various influences likely engages and modifies existing circulation patterns, it is necessary to understand pervasive climate processes (e.g., ENSO, NAO, QBO) and centers of action, and their responses to radiative forcings.”
Gavin was a vocal attendee at the meeting, who clearly felt that I did not do “good science”. The “strawman proposal” to convene a formal meeting on this subject was rejected by Gavin and most of the others at this meeting (who clearly, as Gavin wrote on Real Climate, involved colleagues “on the panel whose expertise covered the specific topic of a proposal can be polarizing…. such a proposal was either more likely to shine or be dismissed.” In the case of the panel I was on, the proposal for “dismissed”.
As I concluded in my post on this meeting
“Except for Judith Lean, Art Charo and myself, however, there was no support for the Strawman proposal. The proposal for a formal NRC Panel was rejected by the others, unless it was very narrowly focused, such as on “decadal forecasts”. The agency representatives (from NASA and the NSF) were similarly not willing to support such a study.
The reason, undoubtedly preordained before we even met on that Monday, is that a significant number of the members of the Committee were (and presumably still are) active participants of the IPCC assessment, as documented above.
Thus, the intensity of the dismissive and negative comments by a number of the committee members, and from even several of the agency representatives, with respect to any view that differed from the IPCC orthodoxy, made abundantly clear, that there was no interest in vesting an assessment of climate to anyone but the IPCC.
The IPCC is actually a relatively small group of individuals who are using the IPCC process to control what policymakers and the public learn about climate on multi-decadal time scales. This NRC planning process further demonstrates the intent of the IPCC members to manipulate the science, so that their viewpoints are the only ones that reach the policymakers.
If the NSF, NASA and the NRC are going to appoint and accept recommendations by groups with a clear conflict of interest to protect their turf [in this case the IPCC], they will be complicit in denying all of us a balanced presentation of the physical science basis of climate change, including the role that humans have. “
Thus, Gavin Schmidt provides us insight into why the funding process itself has become polarized such that only those who accept the IPCC perspective (as Gavin does) are doing “good science” and can be funded.
Climate Science: Roger Pielke Sr. 