PCAST Recommends More Forensic Science R&R (Research & Reform)

This is a near real-time bulletin on a meeting of the President's Council of Advisors on Science and Technology (PCAST) to consider a working group's recommendations regarding forensic science. Eric Lander presented the working group's draft report to the Council for a vote. He made the following comments (not quite verbatim):

The report is a natural follow-on to 2009 NRC Report on strengthening forensic science. We've been seeing a shift in the forensic science community toward empirical measurement of accuracy. The FBI has produced a series of truly elegant papers ... black box studies and white box studies on latent fingerprint identification. These show an error rate of 1/600, or, if you include a confidence interval, as you should, 1/300. 1/

The report lays out criteria for determining scientific validity and applies them to DNA analysis of mixed samples, bitemarks, latent fingerprinting, firearms, footwear, and, to a lesser extent, to hair analysis. It makes the following recommendations:

• Ongoing assessment of foundational validity for forensic feature assessments performed annually by the national Institute of Standards and Technology (NIST);
• A leadership role for NIST in transforming subjective methods into objective feature-comparison methods;
• NIST should improve the process by which the Organization of Scientific Area Committees (OSAC) develop or approve standards by using a metrology resource committee;
• The White House Office of Science and Technology Programs should develop an R&D strategy for forensic science;
• The FBI laboratory should undertake additional black box studies of subjective assessments and implement routine blind proficiency testing in case flow; 
• The FBI's Uniform Language for Testifying and and Reporting standards should entail statements based on empirical tests of validity;
• Federal judges should take into account the appropriate scientific criteria for judging validity and should ensure that testimony about the probative value of the comparisons is consistent with empirical studies.
• Judicial organizations should provide guidance and education for judges should on how to do this;
• There should be more federal funding for forensic-science research to make the transformation to objective methods and to improve the use of subjective procedures in the interim.

In response to a question about the nature of the objective methods being sought, Lander referred to recent, impressive progress in computerized facial recognition.

PCAST unanimously adopted the report (subject to final edits that will not change the substance of the recommendations).  There was no mention of when a report will become public.

Note

1. Editorial comment: Of course, a two-sided interval is equally consistent with a smaller false positive rate for individual examiner judgments, and the rate for blindly verified positive identifications would be smaller.

Index to this blog's comments (and to cases) on the PCAST report