Category Archives: The Review Process

New Paper “Estimating Annual Numbers Of Atlantic Hurricanes Missing From The HURDAT Database (1878-1965) Using Ship Track Density” By Vecchi and Knutson 2011

Chris Landsea has alerted us to a new paper

Vecchi, Gabriel A., and Thomas R Knutson, March 2011: Estimating annual numbers of Atlantic hurricanes missing from the HURDAT database (1878-1965) using ship track density. Journal of Climate, 24(6), doi:10.1175/2010JCLI3810.1.

The abstract reads (highlight added)

“This study assesses the impact of imperfect sampling in the presatellite era (between 1878 and 1965) on North Atlantic hurricane activity measures and on the long-term trends in those measures. The results indicate that a substantial upward adjustment of hurricane counts may be needed prior to 1965 to account for likely ‘‘missed’’ hurricanes due to sparse density of reporting ship traffic. After adjusting for the estimate of missed hurricanes in the basin, the long-term (1878–2008) trend in hurricane counts changes from significantly positive to no significant change (with a nominally negative trend). The adjusted hurricane count record is more strongly connected to the difference between main development region (MDR) sea surface temperature (SST) and tropical-mean SST than with MDR SST. These results do not support the hypothesis that the warming of the tropical North Atlantic due to anthropogenic greenhouse gas emissions has caused Atlantic hurricane frequency to increase.”

Comments Off

Filed under Climate Change Metrics, The Review Process

My Comments On “NSB/NSF Seeks Input on Proposed Merit Review Criteria Revision and Principles”

I have reproduced below my comments to the National Science Board  and National Science Foundation on the merit review process.

 

I am writing this e-mail to comment on

“NSB/NSF Seeks Input on Proposed Merit Review Criteria Revision and Principles”

First, my research credentials are summarized at

http://cires.colorado.edu/science/groups/pielke/

http://cires.colorado.edu/science/groups/pielke/people/pielke.html

I have had quite negative experiences with NSF respect to climate proposals in recent years. I have posted several weblog discussions on my experience as summarized in

http://pielkeclimatesci.wordpress.com/2011/06/27/nsbnsf-seeks-input-on-proposed-merit-review-criteria-revision-and-principles/

Based on my experience, I have concluded that the review process lacks sufficient accountability. To remedy this deficiency, I have the following recommendations

-Guarantee that the review process be completed within 6 months [my most recent land use and climate proposal was not even sent out for review until 10 months after its receipt!).

-Retain all e-mail communications indefinitely (NSF staff can routinely delete e-mails, such that there is no record to check their accountability).

-Require external independent assessments, by a subset of scientists who are outside of the NSF, of the reviews and manager decisions, including names of referees. This review should be on all accepted and rejected proposals.

Information on my experiences with NSF climate research are provided in these weblog posts

My Experiences With A Lack Of Proper Diligence And Bias In The NSF Review Process For Climate Proposals
http://pielkeclimatesci.wordpress.com/2011/05/26/my-experiences-with-a-lack-of-proper-diligence-in-the-nsf-review-process-for-climate-proposals/

Is The NSF Funding Untestable Climate Predictions . My Comments On A $6 Million Grant To Fund A Center For Robust Decision.Making On Climate And Energy Policy.
http://pielkeclimatesci.wordpress.com/2011/03/02/is-the-nsf-funding-untestable-climate-predictions-my-comments-on-a-6-million-grant-to-fund-a-center-for-robust-decision%e2%80%93making-on-climate-and-energy-policy/

The National Science Foundation Funds Multi-Decadal Climate Predictions Without An Ability To Verify Their Skill
http://pielkeclimatesci.wordpress.com/2010/10/21/the-national-science-foundation-funds-multi-decadal-climate-predictions-without-an-ability-to-verify-their-skill/

NSF Decision On Our Request For Reconsideration Of A Rejected NSF Proposal On The Role Of Land Use Change In The Climate System
http://pielkeclimatesci.wordpress.com/2010/06/11/nsf-decision-on-our-request-for-reconsideration-of-a-rejected-nsf-proposal/

Is The NSF Funding Process Working Correctly?
http://pielkeclimatesci.wordpress.com/2010/05/18/is-the-nsf-funding-process-working-correctly/

I would be glad to elaborate further on the lack of diligence and bias by the NSF review process with respect to climate research.

Sincerely

Roger A. Pielke Sr.

Comments Off

Filed under The Review Process

Candid Discussion Of The Grant Review Process By Gavin Schmidt At Real Climate

Gavin Schmidt has an informative and candid summary of the  grant review process in his post on Real Climate

Reflections on funding panels

His overview accurately summarizes the issues with the review process including, very importantly,  how this process can be very biased. His comments on the process also apply to mail in-reviews that are solicited by funding agencies.  

I have excerpted several of his statement on this issue  for further comment, as it illustrates why bias  has, unfortunately,  permeated this funding process.

Gavin wrote

Funding is highly competitive and, depending on the call, only between 10 and 20% of proposals will be funded. Choosing which proposals get funded relies on the good judgement of the program managers in the most part, but they are helped enormously by external reviewers and panels.

The “good judgement” of the program managers includes who they select as reviewers and, even with very good reviews, which proposals they decide to fund.  The program managers themselves clearly have biases also.

I have documented the failure (in my view) of the review process with respect to my NSF proposals in the posts

NSF Decision On Our Request For Reconsideration Of A Rejected NSF Proposal On The Role Of Land Use Change In The Climate System

Is The NSF Funding Process Working Correctly?

In the later post I recommended that

[t]he short answer to this question is NO. I recommend the following changes in the NSF review procedures to improve the process:

  • present ALL proposal abstracts, anonymous reviews of both accepted and rejected proposals and program managers decision letters (or e-mails) on line for public access

  • present the date of submission and final acceptance (or rejection) of the proposal.

  • My suggestion has been ignored.  Gavin Schmidt’s next excerpt is

    Having someone on the panel whose expertise covered the specific topic of a proposal can be polarizing. That is, such a proposal was either more likely to shine or be dismissed than a proposal for whom no-one is as intimately knowledgeable.

    This is actually quite a serious criticism.  This candid admission indicates that bias is regularly acted on in the panel reviews.

    Gavin continues

    Conflicts of interest exist – proposals can come in from a previous student of a panel member, or a current colleague or close collaborator. However, in all such cases, the conflicted person is asked to leave the room and not participate in the discussion on that proposal. This works well in avoiding “less objective” criteria in funding.

    Conflicts of interest, however,  also include “someone on the panel whose expertise covered the specific topic” since as Gavin “wrote such a proposal was either more likely to shine or be dismissed than a proposal for whom no-one is as intimately knowledgeable.” This is a clear example of exercising a bias in making a funding recommendation.

    Gavin writes further

    [The] [r]eputation of the proposers as people capable of good science goes a long way to judging the feasibility of a proposal.

    This quite a remarkable admission since while the statement that the “reputation of the proposers as people capable of good science goes a long way to judging the feasibility of a proposal”, seems appropriate, it is actually a statement of bias. In the polarized environment of climate science, “good science” has come to mean climate science that fits in the worldview of the reviewers. Kevin Trenberth’s recent labeling of the paper 

    R. S. Knox, David H. Douglass 2010: Recent energy balance of Earth  International Journal of Geosciences, 2010, vol. 1, no. 3 (November) – In press doi:10.4236/ijg2010.00000

     as “rubbish” is just one example (see).

    My own experiences with the review process, including a panel in which both Gavin Schmidt and I appeared, supports the view that “good science” as Gavin presents, means climate science that he agrees with. In my post

    Protecting The IPCC Turf – There Are No Independent Climate Assessments Of The IPCC WG1 Report Funded And Sanctioned By The NSF, NASA Or The NRC (and see also)

    I wrote an a National Research Council  Study Planning Meeting titled

    The Attribution of Solar Forcing to Recently Observed Climate Change

    The meeting outline included the text

    “In order to understand the solar influence on climate and the atmosphere, it is essential to also understand the contributions of volcanic aerosols, as well as anthropogenic greenhouse gases and tropospheric aerosols, and other human influences such as land use changes, all of which contribute to the observed climate. Furthermore, because there is growing evidence that responses of the climate system to these various influences likely engages and modifies existing circulation patterns, it is necessary to understand pervasive climate processes (e.g., ENSO, NAO, QBO) and centers of action, and their responses to radiative forcings.”

    Gavin was a vocal attendee at the meeting, who clearly felt that I did not do “good science”. The “strawman proposal” to convene a formal meeting on this subject was rejected by Gavin and most of the others at this meeting (who clearly, as Gavin wrote on Real Climate, involved colleagues “on the panel whose expertise covered the specific topic of a proposal can be polarizing…. such a proposal was either more likely to shine or be dismissed.”  In the case of the panel I was on, the proposal for “dismissed”.

    As I concluded in my post on this meeting

    “Except for Judith Lean, Art Charo and myself, however, there was no support for the Strawman proposal. The proposal for a formal NRC Panel was rejected by the others, unless it was very narrowly focused, such as on “decadal forecasts”. The agency representatives (from NASA and the NSF) were similarly not willing to support such a study.

    The reason, undoubtedly preordained before we even met on that Monday, is that a significant number of the members of the Committee were (and presumably still are) active participants of the IPCC assessment, as documented above.

    Thus, the intensity of the dismissive and negative comments by a number of the committee members, and from even several of the agency representatives, with respect to any view that differed from the IPCC orthodoxy, made abundantly clear, that there was no interest in vesting an assessment of climate to anyone but the IPCC.

    The IPCC is actually a relatively small group of individuals who are using the IPCC process to control what policymakers and the public learn about climate on multi-decadal time scales. This NRC planning process further demonstrates the intent of the IPCC members to manipulate the science, so that their viewpoints are the only ones that reach the policymakers.

    If the NSF, NASA and the NRC are going to appoint and accept recommendations by groups with a clear conflict of interest to protect their turf [in this case the IPCC], they will be complicit in denying all of us a balanced presentation of the physical science basis of climate change, including the role that humans have. “

    Thus, Gavin Schmidt provides us insight into why the funding process itself has become polarized such that only those who accept the IPCC perspective (as Gavin does) are doing “good science” and can be funded.

    Comments Off

    Filed under The Review Process