Polling for ID

The latest pro-Intelligent Design Zogby poll is just one in a series of questionable surveys.

Chris Mooney; September 11, 2003

Polls about evolution get me depressed. First of all, these polls tend to paint a sorry picture of the state of science education in the United States. And not only that. They also underscore just how malleable survey results can be -- and thus, how easily partisans can manipulate polls to demonstrate what the public "thinks."

Consider the most basic evolution polls: A series of surveys conducted over the years by the Gallup Organization and the General Social Survey. Gallup's polls show that 46 percent of respondents (on average) believe "God created man pretty much in his present form at one time within the last 10,000 years" and another 38 percent believe "Man has developed over millions of years from less advanced forms of life, but God guided this process, including man's creation." Just 10 percent believe "Man has developed over millions of years from less advanced forms of life. God had no part in this process," while 6 percent don't know. But the General Social Survey asked a different question. These polls found (on average) that 14 percent of Americans consider the statement "human beings developed from earlier species of animals" to be definitely true and another 29 percent consider it probably true, while 15 percent said "probably not true" and 33 said "definitely not true" (9 percent didn't know).

43 percent of Americans considering evolution to be at least probably true (General Social Survey) doesn't seem so hard to reconcile with 48 percent of Americans believing in some form of evolution, guided or unguided (Gallup). But now consider a 2001 Gallup poll that used very different wording, explicitly mentioning "evolution" instead of speaking of life forms having "developed" over time. When the question took this form--"Would you say that you believe more the theory of evolution or the theory of creationism to explain the origin of human beings, or are you unsure"--the respective results were 28 percent, 48 percent, and 14 percent, with 10 percent saying they didn't know. Here the level of affirmative support for evolution came out dramatically lower, an effect that seems attributable to question wording and the differing choices presented to poll respondents.

Clearly, figuring out what Americans "think" about evolution depends on what questions you ask. As sociologists Otis Dudley Duncan and Claudia Geist--upon whom I have relied for many of the aforementioned polling results--note in a forthcoming article in Reports of the National Center for Science Education, "The evidence does not justify the assumption that respondents will always be logically consistent in their responses to different questions. (Why should that be a surprise?)"

It shouldn't. But understanding the malleability of public opinion--and the way that malleability can be used to serve various political agendas--provides a helpful framework for considering the latest evolution polling controversy, which arose in New Mexico. It also sheds light on problems with a recently released survey of beliefs about evolution and Intelligent Design theory (ID) in Texas, which ID advocates have injected into ongoing biology textbook adoption discussions in the state.

Bad Design

But first to New Mexico. In July, the state chapter of the Intelligent Design Network released results from a Zogby online poll that purportedly showed support for the teaching of Intelligent Design theory among scientists at Los Alamos and Sandia National Laboratories in New Mexico. But it quickly became clear that the Zogby poll didn't play by standard scientific rules: Numerous problems infected the poll design. For instance, of 16,000 people at Sandia, Los Alamos, and various New Mexico universities supposedly invited to respond to the survey, only 248 did so, a ridiculously small response rate. In a statement, Sandia National Laboratory president C. Paul Robinson even termed the poll a "bogus mini-survey." The Intelligent Design Network has now taken a curious stance: The group stands by the poll but says it won't cite it any more, according to a letter obtained by New Mexicans for Science and Reason.

Skepticism about this poll should have arisen instantly from the fact that Zogby International conducted it on behalf of an advocacy group with an express interest in the outcome. The Intelligent Design Network, after all, has lobbied in New Mexico to make state education standards more favorable to the ID perspective.

In fact, ID proponents have used Zogby to gather data favorable to their viewpoint on several other occasions, most recently in Texas. Such polls have now become a recurrent factor in state level battles over evolution and Intelligent Design, from Ohio to New Mexico to the Longhorn State. And though none of the polls have suffered from such severe problems as the latest New Mexico poll, troublesome leading questions have recurred across all of these surveys with little variation in wording. As I demonstrate below, one question even contains arguably false information.

Collectively, these polls serve as powerful evidence of just how easily public opinion can be doctored to serve political goals.

First, though, we need a little context about John Zogby's polling operation. As I have documented in The American Prospect magazine, Zogby International has a long history of preparing polls for political advocacy organizations. Not surprisingly, these polls tend to show results favorable to the client's political interests. Thus Zogby polls for right wing groups show public support for Clinton's impeachment and president Bush's policies, Zogby polls for the libertarian Cato Institute show the popularity of Social Security privatization, and polls for more liberal groups like the Doris Day Animal League and the National Environmental Trust humor these groups' agendas.

To be fair, Zogby doesn't just poll for interest groups; he also polls for politicians, the media, and private companies. And he isn't the only pollster to cross the streams in this way. But some of Zogby's rules for conducting interest group polling should be questioned. For example, Zogby hasn't always fully divulged the sponsors of particular polls. Furthermore, while Zogby ultimately controls question wording, the decision about whether to make the poll public rests with the group funding it, which means that unfavorable results can be suppressed.

But usually, that's the last thing that interest groups who pay good money for polling want. Instead, these groups hope to tout numbers from a distinguished pollster to advance their policy objectives. In the case of polls about evolution and Intelligent Design, they have certainly done so. In August of 2001, Zogby conducted a national poll for the Seattle-based Discovery Institute, the nation's leading advocate of Intelligent Design theory, which proceeded to cite the survey regularly. At least according to the version made public by Discovery, the poll asked four questions, two of which recur (with slight variation) in subsequent Zogby-ID polls, including the New Mexico and Texas versions:

Which of the following comes closest to your own opinion? A: Biology teachers should teach only Darwin's theory of evolution and the scientific evidence that supports it. B: Biology teachers should teach Darwin's theory of evolution, but also the scientific evidence against it.

Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree with the following statement: "When Darwin's theory of evolution is taught in school, students should also be able to learn about the scientific evidence that points to an intelligent design of life."

The answer to the first question is a no-brainer for anyone who believes in open-mindedness, no matter what they think about evolution. Sure enough, the various Zogby-Intelligent Design polls have shown overwhelming support for option B. Yet the question presumes the existence of "scientific evidence" that contradicts Darwin's theory, and thus automatically biases respondents towards the Intelligent Design perspective.

The next question strays even farther into the realm of manipulation, suggesting directly that a body of scientific evidence "points to an intelligent design for life." Respondents to this question have almost all chosen "strongly agree" and "somewhat agree"--not surprisingly, given its wording. But since no scientific evidence supports ID, the premise of the question is absurd, and asking it manipulative. Don't take my word on this: Trust the American Association for the Advancement of Science, which released a strongly worded resolution last October stating firmly that ID is not science and lacks any scientific evidence in its favor.

Do It Like They Do at the Discovery Institute

In my American Prospect article that criticized Zogby's interest group polling, I pointed out this problem with Zogby's national Discovery Institute poll. Following my article, Discovery Institute president Bruce Chapman wrote in to the magazine with a critique. Chapman made three points. The poll in question, he said, was not centrally about ID, but rather about "the appropriateness of students learning the scientific evidence against as well as for Charles Darwin's theory." Chapman also claimed that I defined ID unfairly--drawing upon definitions promulgated by ID critics like Eugenie Scott of the National Center for Science Education--and that I neglected to note that other polls had confirmed the Discovery-Zogby poll's finding:

Worst, Mooney neglects to mention what happened in Ohio last spring as the state school board considered the issue of how to teach evolution. There the Zogby organization followed up its national poll with a statewide poll (also commissioned by us). Not only did the Ohio poll results closely resemble the national poll results, they later were effectively confirmed by a statewide survey produced by a company employed by The (Cleveland) Plain Dealer.

Every bit of this, it turns out, is hokum. First of all, it's hard to see how a poll with the word "intelligent design" in the second question doesn't involve intelligent design. Secondly, I trust the American Association for the Advancement of Science's definition of ID more than I trust the definition of its proponents. But the third point--about another poll in Ohio backing up the Discovery poll--is most astonishing.

Here's the Cleveland Plain Dealer poll (at least as publicized by the Intelligent Design Network). The first thing you will notice, when comparing it with Zogby's polls on Intelligent Design (here, here, here, and here), is that the Plain Dealer version asks extensive background questions to determine what respondents actually know about the issues before soliciting their opinion. In the process, the poll reveals that 45 percent of respondents were "not familiar" with the concept of Intelligent Design in the first place, a fact suggesting the public may be very vulnerable to leading questions about this subject.

Furthermore, not one of the Cleveland Plain Dealer questions suggests that current scientific evidence supports Intelligent Design theory. The closest question was the following:

Currently, the Ohio Board of Education is debating new academic standards for public school science classes, including what to teach students about the development of life on Earth. Which position do you support:

8% - Teach only evolution;

8% - Teach only intelligent design;

59% - Teach both;

15% - Teach the evidence both for and against evolution, but not necessarily intelligent design;

9% - Teach nothing about human development;

1% - Not Sure (NOT READ)

Clearly this avoids the chief pitfall of the Zogby poll and does not suggest that scientific evidence supports ID. And even in interpreting this result, ID theorists have engaged in fuzzy math, claiming that "74% of Ohioans said that evidence for and against evolution should be taught" (scroll down past the end of the survey for this claim), a result that must have been obtained by adding together the 59 percent and 15 percent number above. Yet these numbers are arguably incompatible, since one shows support for teaching ID and one does not.

Furthermore, all of these polls face the same problem: By presenting ID as a rival to evolution and asking whether "both sides" should be taught, they appeal to the public's sense of fair play. Yet this framing gives ID far too much credit. If you asked the public whether students should be taught "both sides" when it comes to other well established scientific theories, like general relativity or plate tectonics, the response would likely be the same.

Texas-Sized Polling Problems

Yet another Zogby-Discovery Institute survey has recently been publicized, presumably in the interest of influencing state board of education biology textbook adoptions in Texas. Once again, fuzzy math adorns the effort to promote the survey, in this case of a particularly embarrassing sort. A Discovery Institute press release about the poll contains the following howler: "By nearly a five-to-one margin, 75% of Texas residents say the state board of education should approve biology textbooks that teach both Darwin's theory of evolution and the scientific evidence against it." Last I checked, a five to one margin--or 83.33 percent--differs rather significantly from 75 percent.

Versions of the two problematic questions cited above also appear in this survey, with predictable results. In addition, new questions have been asked, specifically tailored to the current situation in Texas. Once again, these questions use wording that forces poll respondents to answer in a way that supports the Discovery Institute worldview. For example, respondents were asked to agree or disagree with the following two statements:

Texas law requires that textbooks be (quote) "free from factual errors." Should the state board of education apply this standard to how biology textbooks present Darwin's theory of evolution?

Texas law requires students to analyze, review, and critique scientific explanations, including hypotheses and theories, as to their strengths and weaknesses using scientific evidence and information. Should the state board of education apply this standard to how evolution is presented in textbooks?

These questions, too, amount to "no brainers." That's especially so if respondents know little about Intelligent Design theory and how its proponents use appeals to open-mindedness and scientific questioning to undercut evolution. As expected, respondents ratified the statements above by overwhelming margins. Indeed, the most startling thing about the poll was the fact that a largish 20 percent minority of respondents actually seem to think that biology textbooks should contain factual errors. Who the hell are these people?

Polling as Pseudoscience

Taken as a whole, polls about evolution certainly do suggest that we need much better science education in this country. But polls about evolution and Intelligent Design conducted by Zogby International on behalf of the Discovery Institute and its ilk show something else as well. Not only do Americans need to understand the difference between evolution and its pseudoscientific rivals, they also need to grasp that polling is prone to its own form of pseudoscience. The leap between a given survey finding and changes in public policy tends to be fraught in any case--something that polling experts repeatedly warn against. Such warnings have even more force when self-interested advocacy groups fund scientifically questionable surveys.

Feedback | Reverse links for this page | Translate this page