In Defense of Regulatory Peer Review

Abstract

The debate over application of peer review to the regulatory decisions of administrative agencies has heated up in the last year. Part of the larger and controversial sound science movement, mandating peer review for certain types of agency decisions has recently been championed by the White House and proponents in Congress. Indeed, this past January the Office of Management and Budget finalized guidelines requiring peer review for large classes of agency activities. These initiatives have not gone unchallenged, and a fierce debate has resulted between those who claim peer review will strengthen the scientific basis of agency decisions and those who contend that peer review will politicize and burden agency activities.

While peer review is fast becoming an integral and controversial part of agency behavior, it has received remarkably little scholarly attention. This article presents a comprehensive and current examination of peer review and is supported by new empirical data. To help sharpen our analysis, we conducted a nationwide survey of environmental law practitioners - lawyers who regularly practice on behalf of or before agencies with substantial regulatory missions. Most advocates of regulatory peer review argue that agencies regularly overstate the extent of scientific support for their policy decisions and that peer review will help correct that problem. Its critics contend it will unduly slow down agency decision-making. Our survey results suggest that those who actually practice regulatory law believe both of those propositions are likely. The challenge, therefore, is how to derive the benefits of regulatory peer review while minimizing its costs.

Based on our survey, our evaluation of research from a wide variety of fields, and our own experience, which includes co-author J.B. Ruhl's tenure as the lawyer on a National Academy of Sciences committee that conducted the first high-profile peer review under the Endangered Species Act, we believe that peer review can assist the transparency and legitimacy of agency decisions by sharpening the line between scientific support and policy judgment in agency decision-making. Yet neither we nor anyone else who has entered the debate can say whether this benefit outweighs the costs that would result from adding such a procedure to agency processes, for the simple reason that nobody has produced robust empirical evidence to answer three basic questions - do agencies regularly overstate the scientific support for their decisions; if so, does this practice make a difference in terms of the policy merits of their decisions; and if so, does regulatory peer review provide a cost-effective means of correcting the practice?

This gap in empirical data points to our proposed solution. We argue that regulatory peer review should take advantage of another practice of science - random sampling - in order to serve a diagnostic function in addition to its quality improvement function. By applying rigorous peer review to the science component of a small number of selected regulatory decisions, regulatory peer review could (a) help in defining the scope of the problem of agency overstatement of scientific support and (b) induce agencies to pay more attention to clearly articulating where science ends and policy judgment begins in the justification of their decisions. We provide the details of the proposal in the closing section of the manuscript.