Peer Review, Publication in Top Journals, Scientific Consensus, and So Forth

In following the discussion of global warming and related issues in the press and the blogosphere, I have been struck repeatedly by the assumption or expression of certain beliefs that strike me as highly problematical. Many writers who are not scientists themselves are trading on the prestige of science and the authority of scientists. Reference to “peer-reviewed research” and to an alleged “scientific consensus” are regarded as veritable knock-out blows by many commentators. Yet many of those who make such references appear to me to be more or less ignorant of how science as a form of knowledge-seeking and scientists as individual professionals operate, especially nowadays, when national governments―most notably the U.S. government―play such an overwhelming role in financing scientific research and hence in determining which scientists rise to the top and which fall by the wayside.

I do not pretend to have expertise in climatology or any of the related physical sciences, so nothing I might say about strictly climatological or related physical-scientific matters deserves any weight. However, I have thirty-nine years of professional experience―twenty-six as a university professor, including fifteen at a major research university, and then thirteen as a researcher, writer, and editor―in close contact with scientists of various sorts, including some in the biological and physical sciences and many in the social sciences and demography. I have served as a peer reviewer for more than thirty professional journals and as a reviewer of research proposals for the National Science Foundation, the National Institutes of Health, and a number of large private foundations. I was the principal investigator of a major NSF-funded research project in the field of demography. So, I think I know something about how the system works.

It does not work as outsiders seem to think.

Peer review, on which lay people place great weight, varies from being an important control, where the editors and the referees are competent and responsible, to being a complete farce, where they are not. As a rule, not surprisingly, the process operates somewhere in the middle, being more than a joke but less than the nearly flawless system of Olympian scrutiny that outsiders imagine it to be. Any journal editor who desires, for whatever reason, to reject a submission can easily do so by choosing referees he knows full well will knock it down; likewise, he can easily obtain favorable referee reports. As I have always counseled young people whose work was rejected, seemingly on improper or insufficient grounds, the system is a crap shoot. Personal vendettas, ideological conflicts, professional jealousies, methodological disagreements, sheer self-promotion, and a great deal of plain incompetence and irresponsibility are no strangers to the scientific world; indeed, that world is rife with these all-too-human attributes. In no event can peer review ensure that research is correct in its procedures or its conclusions. The history of every science is a chronicle of one mistake after another. In some sciences these mistakes are largely weeded out in the course of time; in others they persist for extended periods; and in some sciences, such as economics, actual scientific retrogression may continue for generations under the misguided (but self-serving) belief that it is really progress.

At any given time, consensus may exist about all sorts of matters in a particular science. In retrospect, however, that consensus is often seen to have been mistaken. As recently as the mid-1970s, for example, a scientific consensus existed among climatologists and scientists in related fields that the earth was about to enter a new ice age. Drastic proposals were made, such as exploding hydrogen bombs over the polar icecaps (to melt them) or damming the Bering Strait (to prevent cold Arctic water from entering the Pacific Ocean), to avert this impending disaster. Well-reputed scientists, not just uninformed wackos, made such proposals. How quickly we forget.

Researchers who employ unorthodox methods or theoretical frameworks have great difficulty under modern conditions in getting their findings published in the “best” journals or, at times, in any scientific journal. Scientific innovators or creative eccentrics always strike the great mass of practitioners as nut cases―until their findings become impossible to deny, which often occurs only after one generation’s professional ring-masters have died off. Science is an odd undertaking: everybody strives to make the next breakthrough, yet when someone does, he is often greeted as if he were carrying the ebola virus. Too many people have too much invested in the reigning ideas; for those people an acknowledgment of their own idea’s bankruptcy is tantamount to an admission that they have wasted their lives. Often, perhaps to avoid cognitive dissonance, they never admit that their ideas were wrong. Most important, as a rule, in science as elsewhere, to get along, you must go along.

Research worlds, in their upper reaches, are pretty small. Leading researchers know all the major players and what everybody else is doing. They attend the same conferences, belong to the same societies, send their grad students to be postdocs in the other people’s labs, review one another’s work for the NSF, NIH, or other government funding organizations, and so forth. If you do not belong to this tight fraternity, it will prove very, very difficult for you to gain a hearing for your work, to publish in a “top” journal, to acquire a government grant, to receive an invitation to participate in a scientific-conference panel discussion, or to place your grad students in decent positions. The whole setup is tremendously incestuous; the interconnections are numerous, tight, and close.

In this context, a bright young person needs to display cleverness in applying the prevailing orthodoxy, but it behooves him not to rock the boat by challenging anything fundamental or dear to the hearts of those who constitute the review committees for the NSF, NIH, and other funding organizations. Modern biological and physical science is, overwhelmingly, government-funded science. If your work, for whatever reason, does not appeal to the relevant funding agency’s bureaucrats and academic review committees, you can forget about getting any money to carry out your proposal. Recall the human frailties I mentioned previously; they apply just as much in the funding context as in the publication context. Indeed, these two contexts are themselves tightly linked: if you don’t get funding, you’ll never produce publishable work, and if you don’t land good publications, you won’t continue to receive funding.

When your research implies a “need” for drastic government action to avert a looming disaster or to allay some dire existing problem, government bureaucrats and legislators (can you say “earmarks”?) are more likely to approve it. If the managers at the NSF, NIH, and other government funding agencies gave great amounts of money to scientists whose research implies that no disaster looms or no dire problem now exists or even that although a problem exists, no currently feasible government policy can do anything to solve it without creating greater problems in the process, members of Congress would be much less inclined to throw money at the agency, with all the consequences that an appropriations cutback implies for bureaucratic thriving. No one has to explain all these things to the parties involved; they are not idiots, and they understand how the wheels are greased in their tight little worlds.

Finally, we need to develop a much keener sense of what a scientist is qualified to talk about and what he is not qualified to talk about. Climatologists, for example, are qualified to talk about the science of climatology (though subject to all the intrusions upon pure science I have already mentioned). They are not qualified to say, however, that “we must act now” by imposing government “solutions” of some imagined sort. They are not professionally knowledgeable about what degree of risk is better or worse for people to take; only the individuals who bear the risk can make that decision, because it’s a matter of personal preference, not a matter of science. Climatologists know nothing about cost/benefit considerations; indeed, most mainstream economists themselves are fundamentally misguided about such matters (adopting, for example, procedures and assumptions about the aggregation of individual valuations that lack a sound scientific basis). Climate scientists are the best qualified people to talk about climate science, but they have no qualifications to talk about public policy, law, or individual values, rates of time preference, and degrees of risk aversion. In talking about desirable government action, they give the impression that they are either fools or charlatans, but they keep talking―worst of all, talking to doomsday-seeking journalists―nevertheless.

In this connection, we might well bear in mind that the United Nations (and its committees and the bureaus it oversees) is no more a scientifc organization than the U.S. Congress (and its committees and the bureaus it oversees). When decisions and pronouncements come forth from these political organizations, it makes sense to treat them as essentially political in origin and purpose. Politicians aren’t dumb, either―vicious, yes, but not dumb. One thing they know above everything else is how to stampede masses of people into approving or accepting ill-advised government actions that cost the people dearly in both their standard of living and their liberties in the long run.