Everybody with the slightest interest in the subject should look at the executive summary, but the whole document is well worth reading.
In most media accounts, attention has focused on what appears to be
the implicit recommendation that Rajendra Pachauri, the IPCC's current
head, should step down. The committee says the chief (and other key
people) should serve one term, and Pachauri is already into his second.
I am no fan of Pachauri, and I think it would be far better if he went,
but this is less important than repairing other aspects of the panel's
governance and methods. Among the new report's many recommendations, I
would stress the following.
Recommendation: The IPCC should establish an Executive
Committee to act on its behalf between Plenary sessions. The membership
of the Committee should include the IPCC Chair, the Working Group
Co-chairs, the senior member of the Secretariat, and 3 independent
members, including some from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place. [My emphasis.]
Broadening the range of perspectives is vital. Aside from creating
an executive committee with outsider representation, one of the best
ways to do this is through the IPCC's review process. The report looks
at this in detail, recognising that many of the panel's critics have
focused on it. Review Editors play a key role in the IPCC's work -- or
ought to. They oversee the IPCC's Lead Authors' treatment of reviewers'
comments on their drafts. The IAC explains:
Two or more Review Editors for each chapter oversee the
review process, ensuring that review comments and controversial issues
are handled appropriately. However, the Lead Authors have the final say on the content of their chapter.
With the tight schedule for the revision process, authors do not
always consider the review comments carefully, potentially overlooking
errors in the draft report that might have been caught...
A near-universal observation -- made in presentations,
interviews, and responses to the questionnaire -- was the need to
strengthen the authority of the Review Editors to ensure that authors
consider the review comments carefully and document their responses.
With the tight schedule for completing revisions, authors do not always
do an adequate job of revising the text and Review Editors do not
always require them to explain why they rejected a comment... Although
a few such errors are likely to be missed in any review process,
stronger enforcement of existing IPCC procedures by the Review Editors
could minimize their numbers. This includes paying special
attention to review comments that point out contradictions,
unreferenced literature, or potential errors; and ensuring that
alternate or dissenting views receive proper consideration.
Recommendation: The IPCC should encourage Review Editors to fully
exercise their authority to ensure that reviewers' comments are
adequately considered by the authors and that genuine controversies are adequately reflected in the report. [My emphases.]
Enabling and requiring Review Editors to do their job is critical.
Partly, as the report says, this is a matter of resources. But the
report also acknowledges that encouraging Review Editors to fully
exercise their authority is not enough by itself to make the review
process fully independent. Further thought will need to be given to
Although implementing these recommendations would
greatly strengthen the review process, it would not make the review
process truly independent because the Working Group Co-chairs, who have
overall responsibility for the preparation of the reports, are also
responsible for selecting Review Editors. To be independent, the
selection of Review Editors would have to be made by an individual or
group that is not engaged in writing the report, and Review Editors
would report directly to that individual or group (NRC, 1998, 2002).
Despite the desirability of an independent review, it is not clear
what scientific body has the recognized legitimacy and capacity to
carry out such a large task. At the NRC, a special group called the
Report Review Committee carries out this function on behalf of the
institution. The Report Review Committee is made up of approximately 30
members of the National Academy of Sciences, National Academy of
Engineering, and Institute of Medicine, is staffed by individuals from
outside the program units, and reports directly to the NRC Governing
Board. One option for the IPCC would be to appoint a small group of
experts who would report directly to a new Executive Committee (see
"IPCC Management Structure" in Chapter 4) to serve a similar function
for the IPCC. Another option would be to engage an international
scientific body to provide such services for the IPCC.
An important related issue is the selection of authors.
Although the individuals who corresponded with the
Committee were generally supportive of the author teams chosen, few
knew why some authors are chosen and others are not, and the selection
criteria seemed arbitrary to many respondents. The absence of a
transparent author-selection process or well-defined criteria for
author selection can raise questions of bias and undermine the
confidence of scientists and others in the credibility of the assessment
(e.g., Pielke, 2010a). The IPCC has no formal process or criteria for
selecting authors, although some Working Group Co-chairs established
their own for the fourth assessment, considering factors such as
scientific expertise and excellence, geography, gender, age, viewpoint,
and the ability to work in teams. Establishing such criteria and
applying them in a transparent manner to all Working Groups would
alleviate some of the frustrations voiced by the community.
Recommendation: The IPCC should establish a formal set of criteria
and processes for selecting Coordinating Lead Authors and Lead Authors.
The report has this to say about "handling the full range of views".
An assessment is intended to arrive at a judgment of a
topic, such as the best estimate of changes in average global surface
temperature over a specified timeframe and its impacts on the water
cycle. Although all reasonable points of view should be considered,
they need not be given equal weight or even described fully in an
assessment report. Which alternative viewpoints warrant mention is a
matter of professional judgment. Therefore, Coordinating Lead Authors
and Lead Authors have considerable influence over which viewpoints will
be discussed in the process. Having author teams with diverse
viewpoints is the first step toward ensuring that a full range of
thoughtful views are considered.
Equally important is combating confirmation bias -- the tendency of
authors to place too much weight on their own views relative to other
views (Jonas et al., 2001). As pointed out to the Committee by a
presenter [John Christy of the University of Alabama, Huntsville] and
some questionnaire respondents, alternative views are not always cited in a chapter if the Lead Authors do not agree with them.
Getting the balance right is an ongoing struggle. However, concrete
steps could also be taken. For example, chapters could include
references to all papers that were considered by the authoring team and
describe the authors' rationale for arriving at their conclusions.
Recommendation: Lead Authors should explicitly document that a range of scientific viewpoints has been considered,
and Coordinating Lead Authors and Review Editors should satisfy
themselves that due consideration was given to properly documented
alternative views. [My emphases.]
The report has a lot to say about the way the IPCC has characterised
and communicated uncertainty. For instance, commenting on one document
(Working Group 2 Summary for Policymakers), it says:
The Working Group II Summary for Policy Makers has been
criticized for various errors and for emphasizing the negative impacts
of climate change. These problems derive partly from a failure to
adhere to IPCC's uncertainty guidance for the fourth assessment and
partly from shortcomings in the guidance itself. Authors were urged to
consider the amount of evidence and level of agreement about all
conclusions and to apply subjective probabilities of confidence to
conclusions when there was high agreement and much evidence. However, authors
reported high confidence in some statements for which there is little
evidence. Furthermore, by making vague statements that were difficult
to refute, authors were able to attach "high confidence" to the
statements. The Working Group II Summary for Policy Makers contains
many such statements that are not supported sufficiently in the
literature, not put into perspective, or not expressed clearly. [My emphasis.]
The IPCC has no formal policy on conflicts of interest. The report says this should change.
Recommendation: The IPCC should develop and adopt a
rigorous conflict of interest policy that applies to all individuals
directly involved in the preparation of IPCC reports...
In developing such a policy, the IPCC may want to consider features of the NRC policy. These include:
- Distinguishing between strong points of view (i.e., biases) that
can be balanced and conflicts of interest that should be avoided unless
determined to be unavoidable
- Differentiating between current conflicts, where the candidate's
current interests could be directly and predictably affected by the
outcome of the report, and potential conflicts of interests
- Considering a range of relevant financial interests...
- Judging the extent to which an author or Review Editor would be
reviewing his or her own work, or that of his or her immediate employer
- Examining indications of a fixed position on a particular issue
revealed through public statements (e.g., testimony, speeches,
interviews), publications (e.g., articles, books), or personal or
- Maintaining up-to-date confidential disclosure forms and
participating in regular, confidential discussions of conflict of
interest and balance for the major components of each report. [My
The IPCC's communications efforts come in for criticism too.
IPCC's mandate is to be policy relevant, not policy
prescriptive. However, as noted above, IPCC spokespersons have not
always adhered to this mandate. Straying into advocacy can only hurt IPCC's credibility. [My emphasis.]
I think it is a pity that the IAC will not be publishing the
questionnaire responses from the hundreds of experts they approached,
except apparently in aggregated form with names withheld. That hardly
conforms to a spirit of openness. Perhaps somebody will assemble a list
of links to those responses that have already found their way on to the
web. That would be a useful service.