The Ethics and Epistemology of Peer Review

In a previous post, I argued that academic peer review is a gatekeeping process brought about by the post-World War II growth of government involvement in research and scholarship. Though it may control quality in a narrow, conventional sense, one significant consequence of this process is the suppression of innovation. The present post takes a look at the underlying ethics and epistemology of peer review.

Medical researcher David Horrobin, whom I quoted in the previous post, says that critics of peer review “are almost always dismissed in pejorative terms such as ‘maverick,’ ‘failure,’ and ‘driven by failure.’” Lest those epithets be ascribed to me, I hasten to say that I have had some success in the process and that I am not denigrating anyone who uses it to advance his or her career. The process nonetheless does have serious flaws.

Most significant of its flaws is the view that peer review must be blind in order to maintain objectivity, that is, to prevent bias from entering the process. However, as the British Medical Journal, which has not used blind peer review since 1999, points out, “A court with an unidentified judge makes us think immediately of totalitarian states and the world of Franz Kafka.” Objectivity is the fallacy-free perception and communication of what the object of cognition is, and bias means that some other factor, such as irrelevant preconceived notions, whether formed by emotion or by reason, interferes with this perception and communication. Lack of objectivity stems from a failure to perceive reality accurately.

Neither blind nor open peer review can guarantee this accuracy. Indeed, anonymity removes the need for care and responsibility when commenting on someone else’s work. How many ill-mannered or ill-thought-out remarks would be made about submitted papers if reviewers knew that the papers’ authors will know their names and how to contact them? Being allowed to hide behind anonymity is an invitation to scurrilous behavior. This is why the objectivity of legal systems in free societies demands that witnesses, whether supporters, accusers, or expert testifiers, be identified. Contrary to the conventional wisdom of peer review, objectivity requires at minimum that the process be open.

Objectivity, at root, is an epistemological concept and the failure to perceive and communicate accurately is a function of how one uses one’s mind in the processes of perceiving and communicating. Neither anonymity nor openness will improve this. The most important requirement of objectivity while reviewing someone else’s work is a constant awareness of one’s preconceived notions. The most significant one to watch out for is “This is not how I would have written the paper; it should therefore be changed to . . .”

As one journal editor said, no doubt with some exaggeration, all of his reviewers of so-called empirical papers recommend rejection and those of theoretical papers insist that the papers be “recreated in the reviewers’ own images.” And another editor complained that reviewers have turned into wannabe co-authors, requiring extensive revisions and writing comments that are sometimes as long or longer than the original articles. Clearly, decentering, to use Piaget’s term, meaning the ability to consider other points of view or to appreciate the perspectives of others, is needed by some, perhaps many, reviewers.

Once it has been established that a paper meets a journal’s editorial guidelines and philosophy, that is, that the topic of the paper is appropriate for the journal, then it is the author’s objective that should guide evaluation. Decentering in reviewing, or editing or criticism, means accepting the premises of the author and recommending improvements in execution. The reviewer’s personal preferences on the topic, including agreement or disagreement with the author’s basic premises, should be set aside. The author’s paper is the reality to be adhered to in the reviewing process; interference from irrelevant, previously formed emotional associations and intellectual beliefs destroys the objectivity of the process.

A reviewer, of course, may strongly disagree with the editorial guidelines and philosophy of a journal or with the objective of a paper, but then such a reviewer should either decline to be a reviewer or come to terms with the principle of objectivity. Much suppression of innovation in the peer review process probably stems from the failure of reviewers to distinguish their personal philosophies and preferences from those of the authors they are reviewing. When reviewer and author disagree, the reviewer either demands conformity or recommends rejection.

The issue of objectivity in reviewing (or editing or criticizing) is similar to the so-called problem of taste in art. Is this work of art bad art or is my reaction to it just my taste? Artists have an aim for their art and their execution of that aim makes it either good or bad art. Whether one likes a particular work of art, though, depends on many other factors, including emotional associations and intellectual beliefs. Therefore, as Ayn Rand points out, it is not a contradiction to say “This is a good work of art, but I don’t like it,” and vice versa. The same can be said in reviewing scholarly work, namely, “I don’t like or agree with this paper, but it is well done.”

The reviewer, editor, or critic who can make this last statement is one who exhibits objectivity. When looked at from the standpoint of epistemology, whether the process is blind or open is beside the point.