Friday, September 30, 2011

Prometheus Unbound: Releasing the New Edition of the FJC Reference Manual on Scientific Evidence

Two days ago, the National Academy of Sciences released a third edition of the Federal Judicial Center’s Reference Manual on Scientific Evidence. I listened to the webcast of the unveiling as an insider and an outsider. An insider in that I co-authored two of the chapters. An outsider in that until the Manual eventually emerged, I seemed poised on the exterior side of the event horizon of a singularity into which drafts disappeared and time slowed.

Wednesday's unveiling included remarks from the two co-chairs of the NAS committee assembled to commission and supervise the writing of the manual—a group of five judges and five science professionals (a physician, a toxicologist, an engineer, a statistician, and an epidemiologist). Judge Gladys Kessler explained that in 1993, Daubert v. Merrell Dow Pharmaceuticals established “the gatekeeping role” of federal judges.

Certainly, the majority opinion, penned by Justice Blackmun in Daubert, is famous for this metaphor, but if federal judges were not gatekeepers before 1993, waht were they? Surely not sheep. The Daubert opinion draws heavily on prior case law regarding the Federal Rules of Evidence, substituting for the previously dominant “austere” requirement of general acceptance in the scientific community a multifaceted inquiry into “evidential reliability.” Under either legal standard, however, judges are gatekeepers.

Although Judge Kessler correctly suggested that Daubert’s reliability standard (borrowed from earlier court of appeals cases) goes beyond mere relevancy, the same can be said for the general-acceptance standard (announced in a court of appeals case in 1923) that it displaced. Thus, the notion that judges were not gatekeepers for scientific evidence until 1993 always has struck me as odd. (For more on this legal history and the meaning of Daubert, see Kaye et al. (2011).)

Dr. Jerome Kassirer fielded a number of questions from the virtual and physical audiences. One came from a forensic scientist or analyst in Florida who wanted to know if there were any “practicing forensic scientists” on the editorial committee. The response, that the Manual relied on the very detailed 2009 NRC report on forensic science in its treatment of the forensic sciences, missed the subtext of the question. The practicing forensic science community has been bashing the 2009 report for not accurately depicting the knowledge base of forensic identification techniques (other than DNA evidence). The criticism often takes the form of complaints that the committee lacked enough forensic scientists. (For a rejoinder from the co-chair of that committee, see Edwards (2010).)

Another question was why there was no chapter on digital forensics. The answer referred to the failure of the designated author to produce a manuscript that the committee thought would be useful or intelligible to judges. This probably was not the only chapter to fall by the wayside. Indeed, the problems encountered with such chapters may be part of a more complete answer than the one given to the interlocutor who asked whether an 11-year gap between editions was not a bit much.

A final question to which I alerted included a little speech about the importance of Bayesian inference. The questioner wanted to know why the Manual did not mention Bayes’ rule. Evidently, the questioner was not updating his prior beliefs with any data, for the chapters on statistics, DNA evidence, and medicine have substantial discussions of Bayes' rule. A better question would have been why there is not more discussion in the epidemiology chapter or why the presentation in the medicine chapter is so garbled. But that is a subject for another day.


Harry T. Edwards, 2010, The National Academy of Sciences Report on Forensic Sciences: What it Means for the Bench and Bar, Presentation at the Superior Court of the District of Columbia Conference on The Role of the Court in an Age of Developing Science & Technology, Washington, D.C., May 6, available at NAS Report on Forensic Science.pdf.

David H. Kaye et al., 2011, The New Wigmore, A Treatise on Evidence: Expert Evidence, 2d ed., New York, NY: Aspen Pub.

National Research Council Committee on the Development of the Third Edition of the
Reference Manual on Scientific Evidence ed., 2011, Reference Manual on Scientific Evidence, Washington DC: National Academies Press.

No comments:

Post a Comment