Wednesday, October 31, 2012

Florida Trial Court Excludes the Opinion of a Latent Fingerprint Examiner — Maybe

Last week, Miami-Dade Circuit Court Judge Milton Hirsch issued an “order” stating that when the time comes in a burglary case, he will exercise his “common sense” to fulfill his “gatekeeping function” for scientific evidence under Florida law to prevent “excessive and unsupportable claims made by fingerprint examiners.” Order on Defendant's Motion in Limine, State v. Borrego, Nos. F12-101 & F12-7083, at 16 (Fla. Cir. Ct. Oct. 25, 2012) [cited as ODMIL]. One would not think that this promise would be horribly out of line.

Yet, the Miami Herald reported that outraged prosecutors now “vow to appeal” this “rare and controversial legal move.” David Ovalle, Miami-Dade Judge Rules Fingerprint Evidence Should Be Restricted, Miami Herald, Oct. 28, 2012. So what, exactly, is the shocking legal move here? Reviewing the three documents filed so far in the case, the judge's "order" looks more like a vague campaign pledge than a concrete judicial order amenable to interlocutory review.

The Defendant’s Motion

The defense asked for a specific ruling. It filed a pretrial motion for an order limiting the examiner’s testimony “to the similarities and dissimilarities he observed.” Defendant’s Motion in Limine, State v. Borrego, Nos. F12-101 & F12-7083 (Fla. Cir. Ct.) [cited as DMIL]. The ghost of U.S. District Court Judge Pollak’s perceptive (but then disowned) initial opinion in United States v. Llera Plaza rises just in time for Halloween. Andy Newman, Judge Rules Fingerprints Cannot Be Called a Match, N.Y. Times, Jan. 11, 2002.

In addition, the public defender asked for a series of “thou shalt nots. One would think that these would have been superfluous if the court made the first ruling. Nonetheless, the defense wanted an order prohibiting the analyst from speaking of a “match” or “identification,” and from stating his “level of confidence in his own testimony,” and from revealing or suggesting that a second examiner verified the match. DMIL at 2.

The defense did not rely on the general acceptance standard for scientific evidence that Florida follows. It could have. The Florida Supreme Court has applied the standard in a manner that resembles the direct inquiry into scientific validity mandated for federal courts in Daubert v. Merrell Dow Pharmaceuticals, and even a long history of use in police laboratories is not conclusive proof of general acceptance when a broader cross-section of the scientific community expresses doubts. David H. Kaye et al., The New Wigmore on Evidence: Expert Evidence (2d ed. 2011).

Instead of raising this threshold objection, however, the defense contended that a good fingerprint examiner is no better than a juror in forming a categorical opinion on the basis of the similarities and differences between an exemplar and a latent print DMIL at 2-3. This claim in highly problematic. Existing research may not be extensive, but it does support the view that trained examiners can outperform the laity. See Fingerprinting Error Rates Down Under, June 24, 2012.

Also implausibly, the defense argued that “match” necessarily means “absolute certainty” and somehow reached the conclusion that in recommending an end to testimony of “a source attribution to the exclusion of all others in the world,” DMIL at 4, the NIST report on latent fingerprinting supported the contention that no testimony about a “match” should be allowed. See Government-sponsored Report on Latent Fingerprint Work in Criminal Investigation and Prosecution, Feb. 18, 2012.

The defense advanced several other peculiar arguments. It maintained that the expert was unqualified to attribute a print to Borrego, even tentatively, because he lacked training or education in “population statistics or probabilities.” DMIL at 5. This qualifications argument has no force of its own. The real argument in this part of the public defender's memorandum is that fingerprint examiners do not follow the practice of DNA analysts of reporting probabilities “based on established scientific principles.” Id. at 6. A course in statistics and probability would not solve this problem. The gravamen of the complaint is not really the education of examiners. It is the practice of using personal judgment instead of a generally accepted statistical model.

Finally, the defense suggested, with no legal analysis, that “due process” and the “constitutional right to trial by jury, rather than trial by ‘expert’” justified “an order limiting Womack’s testimony to the parameters [sic] described herein.” Id at 8. In light of the normal opportunity to challenge excessive or dubious claims before a jury, however, the Fifth and Sixth Amendments do not add much, if anything, to the evidentiary argument.

The Prosecutor’s Reply

Rather than respond to any of these overblown arguments, the prosecution filed the State’s Motion to Disqualify Judge, State v. Borrego, Nos. F12-101 & F12-7083 (Fla. Cir. Ct. Oct. 15, 2012) [SMDJ]. (With unintended humor, the Miami Herald’s website refers to this as a “motion to rescue Judge Hirsch from the case.”) The motion stated that Judge Hirsch, in another case, had suggested that the prosecutor read his writings on fingerprint evidence and then said he would recuse himself if the state moved for his disqualification. On this basis, the prosecutor wrote that she entertained “a reasonable belief that Judge Hirsch will not be fair and impartial in ruling on any motions on fingerprint testimony.” SMDJ at 2.

Judge Hirsch was not swayed. He summarily denied the request. His Order Denying Motion to Disqualify Judge, State v. Borrego, Nos. F12-101 & F12-7083 (Fla. Cir. Ct. Oct. 25, 2012), is unedifying. But then again, the state’s theory that a judge must or should disqualify himself because he has written something on a subject or has stated a willingness to recuse himself in another case seems flimsy. These statements do not mean that a judge is incapable of making a fair ruling. Still, the state’s argument, if raised on appeal, might gain more traction if (as the Miami Herald reported) “Hirsch issued his order Thursday before prosecutors could write their reply to Borrego’s defense request to restrict the testimony of the fingerprint expert.”

The Order on the Fingerprint Testimony

The Order on Defendant’s Motion in Limine stretches across 17 pages. The court prepared and issued this sprawling “order” without a reply memorandum from the state. For that matter, neither does the document discuss the defendant’s argument about the education of the expert in probability and statistics. It surveys the history of science and scientific evidence, Florida’s adoption and application of the standard of general acceptance for such evidence, and a judge’s role in excluding evidence.

Then it turns to fingerprinting in literature and law. The court states that no problem arises when a fingerprint analyst merely displays the similarities and differences between two images. ODMIL at 13. This might be so if the examiner did not present himself as a forensic scientist, did not speak of the “science” of fingerprinting, did not refer to “the scientific method,” and insisted that he was present as little more than a photographer of images with no more skill than any juror to make the comparison. But even with all these fangs removed from the expert testimony, the admissibility of such testimony on ordinary relevance grounds is open to question. What intelligent use can jurors who know nothing about the variability of impressions of fingerprint features make of the images that are said to coincide sufficiently to be incriminating? Kaye et al., supra. Yet, the court seems to regard it as axiomatic that “[w]hen blow-ups, photographs, or other reproductions of fingerprints are admitted into evidence, the truth-seeking function of trials is advanced.” ODMIL at 14-15.

The Order castigates the most extreme testimony that fingerprint examiners once provided. Testimony that an identification can be made “to the exclusion of every other fingerprint in the history of the world” is “unsupportable.” Id. at 14. Testimony that “the error rate associated with their work, or with fingerprint examination in general, is zero” is“worse than wrong.” Id. Yet, the ensuing discussion of error rates from “human imperfections” is itself rather confused. The Order defines “error rate” as “false positives plus false negatives over total population.” ODMIL at 14. Whatever this means, it is not an “error rate” suitable for presentation to a jury. Kaye et al., supra. Moreover, contrary to the implication in the Order, it is not the “measurable error rate” used in “DNA analysis, the gold standard in forensic evidence.” Id. at 14. Like fingerprint examiners, DNA analysts do not normally present a rate of errors from “human imperfections,” and few of them would claim that human errors are impossible or never occur.

Thus, only this much of the Order is clear: The expert may not state that there is no chance at all that someone other than Radames Borrego left the fingerprints, that the association between Mr. Borrego and the latent prints is absolutely certain, or that mistakes are impossible. But beyond these limitations—which should be part of the profession’s own standards anyway—the Order is incredibly murky. It states that
a trial court must ... protect the integrity of the truth-seeking function from pollution and misdirection due to excessive and unsupportable claims made by fingerprint examiners. For a fingerprint witness to testify, "I direct the jury's attention to the arch appearing here, and the loop appearing here" is one thing; for a fingerprint witness to testify, "I have concluded that this fingerprint matches that of the defendant to the exclusion of all other fingerprints in the history of the world" is a very different thing. And in between these two very different things lie a thousand nuances and gradations of testimony. The trial judge must apply Frye, and Ramirez [a Florida Supreme Court case excluding testimony not generally accepted among toolmark analysts about the marks on cartilage], and his gatekeeping function, and his common sense, to each one of them when and as they are offered in evidence. [¶] And that is exactly what I intend to do at the trial of the case at bar.
Id. at 16.

The Bottom Line

So what do the 17 pages mean for the “thousand nuances and gradations” of testimony that are the subject of allusion rather than analysis? Would a description of relevant features followed by a qualitative statement of likelihoods be acceptable? Cf. Going South with Shoeprint Testimony, July 14, 2012. How about a statement that Mr. Borrego cannot be excluded as the source, although most people could be? Would the judge's "common sense" allow the observation that the prints in question are far more consistent with each other than randomly selected ones? May the examiner opine that the prints “match” as that term is used in the field, while adding that this match does not mean no one else in the world also might have a finger that would produce a matching image? Cf. Who Is Nelson Acosta-Roque? (Part III).

The operative part of the Order is not helpful here. It reads, “Defendant's motion in limine is respectfully GRANTED only to the extent of the foregoing order” ODMIL at 16. It is fine to enliven opinions with poetry, as this one does, but some precision in evidentiary rulings would be more useful to the parties.

References
Acknowledgments: Thanks to Professor Joelle Moreno for calling the case to my attention.
Posted by DH Kaye at 10:55 PM 0 comments
Labels:

Wednesday, October 17, 2012

More on Semrau: The Other Daubert Factors

In United States v. Semrau, the U.S. Court of Appeals for the Sixth Circuit upheld the exclusion of a defendant’s “unilateral” fMRI testing for conscious deception. Previously, I focused on the court’s discussion of error rates. Known error rates implicate admissibility under both Federal Rule of Evidence 702 and Federal Rule 703. (Rule 702 is the locus of the scientific validity standard adopted in Daubert v. Merrell Dow Pharmaceutics, and Rule 703 states the common law, ad hoc balancing test for virtually all evidence.) I do not think the opinion is as clear as it could have been on which error rate pertained to what. Nevertheless, it is encouraging that the court recognized that two parameters are necessary to describe the accuracy of a procedure that classifies items or people into two categories (liar or truth teller).

But Daubert's list of factors extends beyond error rates, and the Semrau court’s handling of the other Daubert subissues also merits a mixed review. First, the court suggested that fMRI lie detection satisfied Daubert’s criteria for testing and peer review. It referred to “several factors in Dr. Semrau's favor,” namely:
“[T]he underlying theories behind fMRI-based lie detection are capable of being tested, and at least in the laboratory setting, have been subjected to some level of testing. It also appears that the theories have been subjected to some peer review and publication.” Semrau, 2010 WL 6845092, at *10. The Government does not appear to challenge these findings, although it does point out that the bulk of the research supporting fMRI research has come from Dr. Laken himself.
The suggestion that these factors favor the defendant treats Daubert’s references to testing, peer review, and publication rather superficially. That a scientific theory is “capable of being tested” tells us almost nothing about the validity of the theory. The theory that in the year 2075, the moon will turn into a blob of green cheese is capable of being tested, but that does not help validate it today. Likewise, the mere existence of peer reviewed publications means nothing without examining the content of the publications and the reactions to them in the scientific literature.

The court came closer to addressing the true Daubert issue of whether peer reviewed publications,  considered as a whole, validate a technique or theory when it responded to defendant’s argument that the district court was overly concerned with the realism of validity studies.  In that context, the court of appeals quoted the caveat in one fMRI study that:
This study has several factors that must be considered for adequate interpretation of the results. Although this study attempted to approximate a scenario that was closer to a real-world situation than prior fMRI detection studies, it still did not equal the level of jeopardy that exists in real-world testing. The reality of a research setting involves balancing ethical concerns, the need to know accurately the participant's truth and deception, and producing realistic scenarios that have adequate jeopardy.... Future studies will need to be performed involving these populations.
But even this mention of the content of one study does not explain why the experiments are inadequate to demonstrate validity. Why would it be harder to detect a lie that has grave consequences to the subject of the laboratory experiment or field study than one that has more trivial consequences?

Second, the court of appeals wrote that the “controlling standards factor” had not been satisfied because “[w]hile it is unclear from the testimony what the error rates are or how valid they may be in the laboratory setting, there are no known error rates for fMRI-based lie detection outside the laboratory setting, i.e., in the ‘real-world’ or ‘real-life’ setting.” But what does the realism of laboratory experiments have to do with the existence of a clear protocol for gathering and interpreting data? Naturally, if a test is not standardized, it is hard to ascertain its error rate—a point that has been prominent in debates over fingerprinting. And, if the tester departs slightly from the standard test protocol, the probative value of the test should be questioned under Rule 403. But the issue of external validity should not be confused with the issue of whether standards are in place for administering a test.

Finally, the court implied that without realistic field testing, there could be no general scientific acceptance of a method of lie detection in the forensic setting. This may be true, but all empirical studies pertain to particular times, places, and subjects. Deciding what generalizations are reasonable or generally accepted depends on understanding the phenomena in question. Can laboratory experiments alone show that certain factors tend to affect the accuracy of eyewitness identifications? For years, many experimental psychologists seemed willing to accept forensic applications of laboratory results that lacked complete realism. The ability of fingerprint examiners to match true pairs of prints and exclude false pairs of prints can be demonstrated in laboratory studies with artificially created pairs. Applying the error rates from such laboratory experiments to actual forensic setting could well be hazardous, but the experiments still prove that there is information that analysts can use to make valid judgments. In that situation, it is doubtful that raising the stakes of a judgment will render the technique invalid.

The Semrau court does not pinpoint the source of its discomfort with pure laboratory experiments. As we have just seen, a court should not assume that laboratory experiments never can establish validity of a technique as applied to casework. However, in the case of fingerprint identification, it seems clear enough that the prints do not change depending on whether they are deposited in the course of a crime or produced at another location. The fMRI data might well be different when generated under fully realistic circumstances. As a result, proving that there is detectable brain activity specific to conscious deception under low stakes conditions might not establish that the same pattern arises under high stakes conditions. Without a generally accepted theory of underlying mechanisms to justify extrapolations to the usual conditions of casework, low stakes laboratory findings may not suffice show general acceptance of validity under those conditions.

In sum, Semrau should not be read as establishing that the existence of testability carries any significant weight in favor of admission, that publication in a peer reviewed journal necessarily demonstrates validity, that a lack of complete realism in laboratory studies proves that there  are no “controlling standards” in practice, or that only field studies can establish general acceptance.

These concerns about the wording of the opinion notwithstanding, the problem of generalizing from the laboratory studies to the conditions of the Semrau case are substantial, and the court’s conclusion is difficult to dispute.
Posted by DH Kaye at 3:35 PM 0 comments
Labels: ,