The PCAST Report on Forensic Science: "A Roadmap for Defense Lawyers"

Yesterday, U.S. Court of Appeals judge Alex Kozinski announced in the Wall Street Journal that after examining "the scientific validity of forensic-evidence techniques—DNA, fingerprint, bitemark, firearm, footwear and hair analysis," the President’s Council of Advisors on Science and Technology (PCAST) "concludes that virtually all of these methods are flawed, some irredeemably so." The report, he predicted in this op-ed on "Rejecting Voodoo Science," will "immediately influence ongoing criminal cases, as it provides a road map for defense lawyers to challenge prosecution experts."

Here I will merely point to some large landmarks on this map. Later, I hope to critically explore the report's seemingly tendentious use of phrases like "validity," "reliability," and "scientifically meaningless." In haec verba, the "scientific findings ... concerning foundational validity of six forensic feature comparison methods" are as follows:

• DNA analysis of single-source samples or simple mixtures of two individuals, such as from many rape kits, is an objective method that has been established to be foundationally valid (P. 147).
• DNA analysis of complex mixtures based on CPI [Combined Probability of Inclusion]-based approaches has been an inadequately specified, subjective method that has the potential to lead to erroneous results. As such, it is not foundationally valid (P. 148).
• Objective analysis of complex DNA mixtures with probabilistic genotyping software is relatively new and promising approach. ... At present, published evidence supports the foundational validity of analysis, with some programs, of DNA mixtures of 3 individuals in which the minor contributor constitutes at least 20 percent of the intact DNA in the mixture and in which the DNA amount exceeds the minimum required level for the method (P. 148).
• [B]itemark analysis does not meet the scientific standards for foundational validity, and is far from meeting such standards (P. 148).
• [L]atent fingerprint analysis is a foundationally valid subjective methodology—albeit with a false positive rate that is substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis. [¶] Conclusions of a proposed identification may be scientifically valid, provided that they are accompanied by accurate information about limitations on the reliability of the conclusion—specifically, that (1) only two properly designed studies of the foundational validity and accuracy of latent fingerprint analysis have been conducted, (2) these studies found false positive rates that could be as high as 1 error in 306 cases in one study and 1 error in 18 cases in the other, and (3) because the examiners were aware they were being tested, the actual false positive rate in casework may be higher (P. 149).
• [F]irearms analysis currently falls short of the criteria for foundational validity, because there is only a single appropriately designed study to measure validity and estimate reliability (P. 150).
• [T]here are no appropriate empirical studies to support the foundational validity of footwear analysis to associate shoeprints with particular shoes based on specific identifying marks (sometimes called “randomly acquired characteristics). Such conclusions are ... not scientifically valid. (P. 150).

It seems safe to predict that the phrase "foundational validity" and the basis for the report's conclusions will be the subject of heated debate. More on that later.

Index to comments and cases on the PCAST report