The Chronicle of Higher Education
The Chronicle of Higher Education: Live Discussions

Colloquy

A Scientific Graph Stands Trial

Wednesday, September 6, at 12 noon, U.S. Eastern time

The Topic

Advocates on both sides of the global-warming debate have focused recently on what's commonly known as the "hockey-stick graph," published seven years ago as part of a paper arguing that the 1990s were probably the warmest decade in at least a millennium. Although there is a mass of other evidence for global warming, environmentalists find in the graph a stark illustration of the need to curb greenhouse-gas emissions, while opponents of stricter regulation are quick to point out flaws in the graph, and use it to criticize how climate science is conducted.

Are those critics right that climate science is too cozy a field to adequately review its own work? Has climate science become so politicized that it has lost its credibility in Washington and across the country? How might scientists better present the results of future climate research in a way that avoids such bitter debates?



The Guest

Gerald North is a professor of atmospheric sciences at Texas A&M; University at College Station. His professional interests include climate analysis, climate and hydrological modeling, satellite remote sensing and mission planning, and statistical methods. He was chairman of the National Research Council panel that studied the hockey-stick issue and produced the report Surface Temperature Reconstructions for the Last 2,000 Years. He testified at the first of the recent hearings on climate science.

A transcript of the chat follows.

Richard Monastersky (Moderator):
    Welcome to the Chronicle's live chat with Gerald North, a professor of atmospheric sciences at Texas A&M; University at College Station. Dr. North chaired the National Research Council panel that studied the hockey-stick issue and produced the report "Surface Temperature Reconstructions for the Last 2,000 Years."

I want to thank Dr. North for joining us. Please send in your questions. Don't be bashful.

Gerald North:
    My name is Gerald North and I was the Chairman of the National Research Council Committee on Surface Temperature Reconstructions for the Last 2000 Years. The report of this committee was released in July and is available at the National Academies website. It will be available as a hardback in a 8-10 weeks.

You can find out more about me by logging in to the Department of Atmospheric Sciences website: www.met.tamu.edu. You can also find a lecture I presented a week ago at Andy Dessler's blog website: http://sciencepoliticsclimatechange.blogspot.com/ My e-mail is g-north@tamu.edu.

The committee was composed of 12 experts in various aspects of climate research and statistics. Its purpose was to examine the methology of these surface temperature reconstructions that have been published over the last decade and assess how much we know about this subject. The committee did not do any original research, but simply relied upon the published literature and the wisdom of those on the committee. The committee solicited input from the community and held an open session with about a dozen speakers on different aspects of the problem. The report was assembled quickly (three months) and was refereed by 12 anonymous referees and two additional anonymous monitors who saw to it that every question or criticism was addressed. In addition to the charge above the committee was asked to discuss the relevance of these studies to anthropogenic global warming.

I look forward to participating in this session sponsored by the Chronicle of Higher Education.

Question from Richard Monastersky:
    I wonder if you could summarize the general conclusion of the National Research Council report with respect to the 1999 study by Mann, Bradley, and Hughes--the basis of the hockey stick curve. Critics of Dr. Mann see the NRC report as a vindication of their position that the hockey stick study was wrong. While many articles in the press have described the NRC report as offering qualified support for Dr. Mann and his 1999 study.

Gerald North:
    The Mann et al study of 1999 draws conclusions that are more optimistic about our ability to estimate paleotemperatures than we would. We say that the averages over three decades are likely to be the warmest in the last 400 years. We also say that the average over the last three decades are plausibly the warmest over the last 1000 years. The statements by Mann et al. were a bit stronger than these. However, if you look at reconstructions of later reports by other investigators, you will find that they generally fall within the error bars of Mann et al., especially if you place error bars on the other studies comparable to those in the Mann et al. study.

Question from Patrick Frank, Stanford University:
    The original hockey stick has been shown not just flawed but wrong. Why was the NAS committee unable to clearly state that? A more basic question: There is no analytical theory that can extract a growth temperature from tree ring widths or tree ring densities. On what scientifically valid grounds, therefore, can anyone possibly "calibrate" a tree ring series using temperature vs. time data?

Gerald North:
    I am not a tree ring expert, but we had an expert on our panel. We also listened to several experts and I read most of Fritts's book on the subject. I concluded that there is something to the signals extracted from tree rings. I suppose we disagree on this matter. Tree rings have been studied for about a century now and I believe that they are probably the best indicator we have at this time. Other proxies will undoubtedly become better as they are developed. When this happens we will have a check on the tree rings. So far the tree ring estimates of surface temperature changes do agree with borehole temperatures in ground and ice and with glacial length data for the last 400 years.

Question from Jennifer Ruark:
    What do you think of Rep. Joe Barton's plan to have the National Research Council ask a group of scientists who do not conduct climate research to judge whether the field is too insular? What about his request that the U.S. Government Accountability Office examine scientists' data-sharing policies?

Gerald North:
    This strikes me as a rather strange request, but I suppose Rep. Barton has the right to do so. I also suppose the NRC/NAS has a right to refuse to do it. I am sure they make their own rules about how they select committee members.

Question from Richard Byrne, Chronicle of Higher Education:
    In the story on the graph that we published this week, the researcher who gave the "hockey stick" graph its catch nickname observed that it "was a colossal mistake" to highlight that graphic as the star witness in the case for global warming. Do you agree? How can we better educate the media and the public about what such research proves and does not prove about climate change?

Gerald North:
    The question of how we educate the media is difficult -- not just on science issues. All you have to do is watch any of the network or cable outlets and you find attention grabbing stories rather that the information that people need. There is a tendency for excitement in science reporting. This means the over reporting of weird or extreme claims by some scientists. Even on the better interview programs it is common to excite arguments between opposing extremes rather than examing the underlying consensus among scientists. Reporting should not always be entertainment.

Question from Richard Monastersky:
    At the Congressional hearings, there was a lot of criticism of the peer-review system. Edward Wegman from George Mason U. and many members of Congress said that the system had failed in vetting the original study by Michael Mann and his colleagues. Do you think that criticism is valid?

Gerald North:
    I believe the peer review system works well, but it sometimes slips. In the case of the Mann et al. papers I think the papers were properly accepted by the journals. One must keep in mind that refereeing papers is a difficult and thankless job. You can spend lots of time on these papers and still overlook some fine point. The system usually corrects these errors in due course via the scientific debating mechanisms. Referees cannot redo the research in most cases. They can only estimate the importance of the paper and vouch for its plausibility.

Question from Richard Monastersky:
    I'd like to ask a bit about the recent Congressional hearings on the hockey stick. Was this your first time testifying before Congress and was it what you expected? Did the level of scientific ignorance or knowledge among the members of Congress surprise you?

Gerald North:
    It was the second time for me. I was asked to appear before a small group of senators about 15 years ago and that was very encouraging. But this time proved to be a disappointment. This issue is so polarized politically that it is impossible to simply inform the elected representatives. I was definitely under the impression that they were twisting the scientific information for their own propaganda purposes. The hearing was not an information gathering operation, but rather a spin machine. Of course, there were comic aspects. Some of the questions showed appalling ignorance, but we cannot expect our representatives to be scientists. There was a huge variation in the preparation of the various members.

Richard Monastersky (Moderator):
    I want to thank Stephen McIntyre for joining the discussion. Mr. McIntyre testified at the recent Congressional hearings and has been a sharp critic of the hockey stick.

Question from Stephen McIntyre:
    The NRC Panel stated that strip-bark tree forms, such as found in bristlecones and foxtals, should be avoided in temprature reconstructions and that these proxies were used by Mann et al. Did the Panel carry out any due diligence to determine whether these proxies were used in any of the other studies illustrated in the NRC spaghetti graph?

Gerald North:
    There was much discussion of this matter during our deliberations. We did not dissect each and every study in the report to see which trees were used. The tree ring people are well aware of the problem you bring up. I feel certain that the most recent studies by Cook, d'arrigo and others do take this into account. The strip-bark forms in the bristlecones do seem to be influenced by the recent rise in CO2 and are therefore not suitable for use in the reconstructions over the last 150 years. One reason we place much more reliance on our conclusions about the last 400 years is that we have several other proxies besides tree rings in this period.

Question from Randy Kluver, Texas A&M; University:
    Greetings from College Station! Dr. North, one of the questions that surrounds this debate is the insularity of the climate specialists, and a subsequent inability to look beyond certain preconceived frameworks. From your role in this panel, do you think there is any truth at all to that contention?

Gerald North:
    I believe the insularity thing is a red herring. This field is quite broad from a disciplinary stand point. Of the 12 members on the panel, I did not know more than half before we started. Later I found about the same when I learned of the referees of the report. The fact is that we come from atmospheric sciences, oceanography, geology, geography, etc. These are groups that don't get together on a regular basis. All seem to have come to the same conclusion however.

Question from Stephen McIntyre:
    Is it your view that meaningful error bars can be obtained from calibration period residuals using Mann et al methodology, considering both their regression and principal components methodology?

Gerald North:
    This is a difficult question (you always pose difficult questions!). My own view (not necessarily the committee's) is that the verification period is misleading. I do not think there is enough data in that period to really nail down the matter. There are also the questions about using the mean (low frequency) versus the variations (higher frequency) parts in the verification procedure. Personally, I like the way Mann did it better than the others, because it is the long term stuff we want to check on. But this is a personal opinion. The fact is, there is no one way to do this -- especially when we have so little data. That is why the committee was reluctant to put error bars on the early part of the record (or even the late part).

Richard Monastersky (Moderator):
    We're past the half-way point in our chat. If you've been itching to ask a question, don't hold back. Now is the time to send it in.

Question from Richard Monastersky:
    Do you think there is a misconception about the way peer review works? Scientists have often said to me that getting a paper through peer review does not mean the paper is without fault or even mistakes. They often talk about science as being self correcting--that further studies will reveal mistakes, problems, or cases where an author extrapolated too far from the data.

Gerald North:
    I do think the peer review system works, not perfect, but it works for our purposes. It does not take too much time, screens out bad stuff, but inevitably controversial stuff gets through. This fertilizes debate, and the right answer emerges over time.

Question from Richard Monastersky:
    Dr. Wegman, who wrote a report for the House energy committee, criticized the paleoclimate community of scientits for being too insular. Do you think that is true? If so, I wonder if it is true of all subdisciplines of science?

Gerald North:
    As in an earlier answer, I do not believe it is too insular. The cross-disciplinary aspects of the research actually make it less insular than in many disciplines. As you quoted in the Chronical article, I think it is a contact sport.

Question from Concerned public servant scientist:
    Is uncertainty systemtically under-represented in quasi-scientific information that is fed to policy makers? The absence of "confidence envelopes" on multiproxy temperature reconstructions, the absence of statistics in analyses of various impact time-series data, etc. seem to indicate this is the case. Isn't this the primary flaw in statements regarding "unprecedented trends" in warming and warming impacts? If we are going to get robust estimates of the A in AGW (anthrogopgenic global warming) - and adapt to them - don't we need to model those uncertainties, rather than ignore them?

Gerald North:
    This is a good question. The problem is that in these kinds of reconstructions, the errors are not always quantifiable as they are in purely statisical sampling errors where we can really quantify the error margins. Here we are really into the unknown and the biases are not well understood. Hence, in our graphic we indicated that as we recede into the past we used a grayer shading of the whole illustration to indicate the murkiness of the issue. In some areas such as model simulations we can quantify the sampling error component of the error, but even there it is difficult to quantify the whole error variance.

Question from Kristin Dell, SUNY Geneseo:
    Did you get the feeling that congress had already made up their minds about the hockey-stick graph before your appearance?

Gerald North:
    yes

Question from Jennifer Ruark, CHE:
    The history of the hockey-stick graph shows the tendency of both journalists and activists to oversimplify scientific findings. Can climate scientists do anything to avoid having their research misrepresented? Did they, for example, respond effectively to The Wall Street Journal's misleading representation of the graph?

Gerald North:
    Most scientists are busy with their work and most do not read the Wall Street Journal editorial page. It was sent to me by a friend and I immediately saw the misleading graphics. But I have to admit that like most of us I am very busy and tend to shy away from such public affairs. Perhaps we should all take the time to write letters to the editor. The few I have every written have not seen the light of day.

Question from Richard Monastersky:
    At one of the hearings, there was a suggestion that scientists should have an independent statistician check their work before it is published. Is this a reasonable proposal?

Gerald North:
    I think this is unnecessary. I work with statisticians often, have published many papers with them and I even have an NSF grant with several. But I do not think this is necessary for the bulk of work in paleoclimatolgy.

Question from Georg Hoffmann, LSCE, Paris:
    Wouldn't we all be better off when MBH were right and so millenial variability is rather small. Larger variability indicates more sensitivity of the Earth system and so also more sensitivity to Greenhouse gases. If we would triple, for example, MBH variability, is this still in agreement with variability on longer timescales such as the last glacial or the Holocene?

Gerald North:
    This is a good point. The Mann et al. studies probably made the handle of their hockey stick a bit too straight. Some of the later studies showed more low frequency variability. Of course, you are right, the larger the natural variability the larger the sensitivity of climate to external perturbations. So we might prefer the Mann et al. hockey stick to the 'spaghetti graphic' of our report. I cannot answer the last question on my feet. It does appear that present moderate sensitivities are in pretty good agreement (cf., recent papers by James Hansen) with the last glacial max.

Question from Richard Monastersky:
    There has been some criticm about the refereeing of Dr. Wegman's report to Congress. He said his report was peer reviewed but I have read some critics saying there were substantial errors in the report. Is there anything in the report that you saw in need of correction?

Gerald North:
    The Wegman report was sent out to several readers just before its release. This was quite a different procedure from the NRC one. But we must not be too harsh here since the NRC has a time honored framework and a very competent staff for this kind of vetting and this was not available to Dr. Wegman. This is why requests for future Congressional reports on science are best sent to the National Academies. Moreover, it might not be best for the Congress to interfere or micromanage how these studies are conducted. This is of course assuming Congress really wants to obtain the best possible assessment.

Question from Concerned pulic servant scientist:
    Dr. North, I, like you, have the utmost confidence in dendroclimatologists such as Dr. Malcolm Hugues, co-author on the original hockey-stick paper. But given the importance of bristlecone/foxtail pines (and Dr. Gordon Jacoby's Gaspé cedars) to the "global" temperature reconstruction, what is one to make of the fact that these chronologies not been updated for what is now 8 years? If new samples have been taken - and I understand they have - why do you think the data have not been published? Doesn't this suggest to you that they are dragging their heels? Why would they do that?

Gerald North:
    Sorry, I do not know these individuals more than acquaintances. Hence, I cannot answer any questions about motivation. I can say, however, that if they could prove the hockey stick or spaghetti graphics wrong, I am sure they would jump to the opportunity -- and what scientist wouldn't?

Question from Ernie Linsay, Wilmington College:
    What is your personal opinion - has there been global warming due to human intervention?

Gerald North:
    Yes, it is my opinion that there is a human origin to the recent warming. The reason this is so widely accepted by the community is that we have good data for the last 150 years for both the temperatures and the 'forcings' (volcanoes, greenhouse gases, aerosols, and for the last 30 years we have solar measurements). When we put the forcing into our models (even with their uncertainties) we are able to link the cause and effect pretty certainly. Over the longer period we do not have all this information. This is why the hockey stick is really not so relevant to the question of anthropogenics.

Question from Gavin Schmidt, NASA GISS:
    When dealing with science that is so actively politicized, certain results will get taken out of their context and iconified independently of anything the original researchers do. This has happened throughout the history of the climate change discussion (ask Ben Santer or Peter Doran for instance). How can the community work to prevent that from happening, if indeed it can, and when it does happen, is it possible to de-politicize such an icon?

Gerald North:
    We all would like to know the answer to this question. If you can get any clues let me know. I am well aware of what Santer went through. This game can get very vicious.

Question from Dick Schneider, Wake Forest University:
    I am not a scientist. I teach law. I'm wondering whether the technical objections to the hockey stick really affect the ultimate conclusions with respect to U.S. policy. That is, are the objections substantial enough to give comfort to those who either deny or minimize the likelihood of anthropogenically-driven climate change?

Gerald North:
    The minor technical objections serve as a weapon for those special interests who want to delay any action on GW. In politics it is not what is true that matters, but rather what is perceived to be true by the broader public. Most scientists are not 'lawyers' representing a client. They are not accustomed to these kinds of tactics and do not often do a good job with them. But many of our representatives and their staffs are very skilled in that position of serving a client.

Question from Joel McDade, bystander:
    Greetings Dr. North: I am curious what you thought of the primary part of the Wegman Report, that dealing with the statistical issues in Mann, et al. Specifically, the statement (or similar), "Incorrect mathematics + correct result = bad science." I must say that the NAS Report appeared, to me, to find fault with the Mann methodology but then went on to seemingly endorse the result. The later was the media's take, anyway. TIA

Gerald North:
    There is a long history of making an inference from data using pretty crude methods and coming up with the right answer. Most of the great discoveries have been made this way. The Mann et al., results were not 'wrong' and the science was not 'bad'. They simply made choices in their analysis which were not precisely the ones we (in hindsight) might have made. It turns out that their choices led them to essentially the right answer (at least as compared with later studies which used perhaps better choices).

Richard Monastersky (Moderator):
    We've reached the end of our discussion with Dr. North. I want to thank him for participating and also thank those of you who sent in questions.