Drop Errors and the Trouble with Peer Review

In product development there are two kinds of errors. A “go” error occurs when the green light is given to a product that eventually fails. The Edsel, a $250 million write-off by the Ford Motor Company in 1959, is one example. The “drop” error occurs when an idea that could have been highly profitable is eliminated from further consideration. How do we know that the idea could have been profitable? In a free market dropped ideas have the habit of being picked up by someone else. Chester Carlson’s invention was dropped by such notables as General Electric, 3M, Kodak, RCA, and IBM, but picked up by the small Haloid Company. In 1961 Haloid changed its name to Xerox. Even go errors in a free market often get corrected; just a few years after the Edsel fiasco, Ford rolled out a better idea called the Mustang.

Peer review is the process by which millions of dollars of government money are handed out to researchers in medicine and the physical sciences; the process by which recognition, promotion, and tenure are determined for professors, especially those in the “softer” sciences who do not need or use grants for their research; and one of the criteria—numbers of peer-reviewed journal articles, for example—used to determine accreditation for universities.

Peer review, a “blind” process in which the names of author and evaluator are concealed from each other, requires two or three so-called peers to read a paper or proposal to judge the quality of actual or proposed research before acceptance. As such, peer review is a product development process that protects only against go errors. It is at best quality control that insures accuracy and reliability of research done. At worst it holds back innovation through drop errors. Since there is no free market in scholarly research—today’s government-university-science complex is a severely hampered market—dropped ideas either never get a hearing or take many more years than they otherwise should to surface.

Medical researcher and long-time critic of peer review, David Horrobin, argued that the peer-review process, which developed in its current form largely as a screening device after World War II, has perhaps improved the accuracy and reliability of conventional research published in medicine, but it has done so at the price of innovation. Prior to World War II, unknown researchers could submit papers to journals only with the endorsement of a published author. The editor would then decide whether or not to publish. Peer review was ad hoc and not common. It was the growth of government involvement in education and, especially, the government’s lavishing of money on research that called for the blind-review screening process.

In a paper titled “The Philosophical Basis of Peer Review and the Suppression of Innovation” Horrobin urged that more unconventional and innovative research be encouraged by journal editors. When a reviewer questioned the need for such a statement, Horrobin produced eighteen incidences of medical innovations rejected by the peer-review process. In 1995 Horrobin’s paper was cited by the US Supreme Court as support for the argument that some “well-grounded but innovative theories” may not be published in peer-reviewed outlets.

Horrobin’s solution to divvying up grant money was to give funds equally to all researchers and let each work on whatever his or her interests indicated. Prior to 1960, said Horrobin, this interest-as-guide process was essentially how funding was distributed in the UK and more innovation in medicine resulted in those years than in the years since 1960. Horrobin approved of government involvement in and funding of research, but the analogy to free markets in his solution—bottom-up, self-interested choice by researchers—versus central planning—top-down, “expert” direction by peer reviewers—cannot be escaped.

Never mind that Socrates and Galileo were badly treated by their peer reviewers or that frauds and hoaxes sometimes dodge the quality controllers or, further, that you may want to cite Ayn Rand and Ludwig von Mises but can’t figure out how to get past your peer-review gatekeepers, the real problem of peer review is the severely hampered market in scholarly research. What would a truly free market in scholarly research be like?

First, publishers of journals and scholarly books would have to earn a profit from their buyers and not live off the donations of their authors or other benefactors. Some university presses, for example, are now publishing what are called “supported books,” which means someone, usually the author’s department, must contribute one or two thousand dollars to the publication of the author’s book. And at least one commercial press requires authors to do their own copy edit and provide camera-ready typeset text; this can add up to two thousand dollars or more that authors must fund. Twenty-five dollars per page, charged to authors or their departments, has long been the going rate for published papers in some fields. (In some quarters today this method of getting into print would be called subsidy or vanity publishing.)

In addition, the so-called nonprofits, which finance a portion of today’s research and journals, are in fact creatures of the tax system and must, despite their descriptive name, show an excess of donations over expenses lest their organization become some philanthropist’s very expensive hobby. Under laissez-faire, in the absence of tax write-offs and the guilt and ignorance of economics that wealthy business people tend to exhibit, there probably would be far fewer such organizations than exist today.

Second, there would be no government money to dangle in front of researchers and no government-owned or -regulated universities filled with bureaucratized product lines (curricula designed by committee), bureaucratized sales reps (the professors), or bureaucratized performance evaluations (those mounds of paper, which include lists of published research, that must be produced for promotion, tenure, and most every other consideration). All of this distorts the market and probably encourages the overabundance of pretentious minutia that fills today’s overabundance of academic journals.

Under laissez-faire, only the market would decide who produces what and who gets what in scholarly output. Indeed, the market for this research might not differ much from the product development market in automobiles. Private, profit-making firms, both traditional businesses and universities, would finance the work and effectively and efficiently produce market-satisfying results. Portions of the results might be published in profit-making journals and books, much of it perhaps not.

Yes, there might be some Edsels created by this free-market development process and there might still be some delayed acceptances of Xeroxes, but there also might be a lot more Mustangs! Absent the government-encouraged gatekeepers and other hurdles that must be jumped in order to get into the market, researchers who cannot find an outlet will be free to start their own journals, publishing companies, businesses, or even universities. The hampered market today, which includes the “golden handcuffs” of tenure, makes this quite difficult.


See related posts on the Mises blog: 1, 2.