Scott Henson, "a former journalist turned opposition researcher/political consultant, public policy researcher and blogger who is presently Executive Director of the Innocence Project of Texas" 1/ and whose Grits for Breakfast blog is a great source of information with a Texas flavor, made an important comment:It is entirely possible that DNA testing will soon exonerate Manning in the Miller-Steckler murder case (to the extent of showing that the hair in the car was not his). As of May, 2015, this had not happened. If it does, it would mean that either the original determination of the racial characteristics of the hair was wrong -- something that the FBI has not conceded -- or that the determination was correct but that Manning was not the person whose hairs were in the car -- something that the criminalist never purported to resolve.
Overstating the scientific findings by experts or parties is a major problem. The Santae Tribble case is a glaring example of prosecutorial overclaiming. The Assistant US Attorney told the jury that "[t]here is one chance, perhaps for all we know, in 10 million that it could [be] someone else’s hair." But published estimates of the probability that hairs from different individuals will be classified as microscopically similar range from 1/4500 to 1/10 (with the latter being much more plausible). Closing arguments like this are therefore unacceptable. If courts permit hair evidence to be used, then a prosecutor who does so has a duty to use it only for what what it is worth. In Tribble, the prosecutor claimed that the hairs had matching features and then overclaimed that the probability of this match was negligible if it was someone else's hair.Perhaps that's "something that the criminalist never purported to resolve," but posing that implication is of course the reason the prosecutor presented his or her testimony. When they're reviewing these cases, the issue isn't just whether the forensic examiner erred but also if prosecutors overstated or misused the evidence to ask the jury to draw invalid conclusions. (In Texas' review of state-level examiners, with which I'm more familiar, they examine both the forensic testimony AND the prosecutor's closing arguments.)
If the DNA testing comes back excluding Manning, IMO it'd be fair to say that the use of hair testimony in his case has been "severely undermined," since it would have treated exculpatory evidence as inculpatory.
The odd thing about the Manning case in which hair evidence played a role, however, is that the overclaiming did not relate to the claim. The overclaiming that the FBI spotted in its hair evidence review project pertained to the ability to identify individuals by a combination of many physical characteristics of hair. But the expert expressly stated that that could not be done in this case and only made a claim about race.
Could the prosecutor have gone beyond the expert's testimony about race and "overstated or misused the evidence to ask the jury to draw invalid conclusions"? Yes, that could be. The FBI's hair review project is limited to the reports and testimony of FBI hair examiners. It will be interesting to see whether states implement hair evidence review projects and what their scope will be.
Whether or not there was any relevant overclaiming in Manning, it is true that a DNA test showing that the hair is not Manning's would severely undermine the state's case. But it would not necessarily undermine the validity of the examiner's testimony that the hair had certain "racial characteristics." (Whether, as the DOJ letter indicated, racial classifications are valid to begin with is, of course, a different question.) 2/
To see that disproof of an inference from scientific evidence that arises from better scientific evidence does not make the original evidence invalid, let's switch from hair to serology for a moment. Sometimes hair can be analyzed for ABO blood groups. As a thought experiment, suppose that the testimony had not been about race but was only that both Willie Manning and the hair in the car were blood group B, and that Manning was among the 10% or so of the U.S. population whose ABO type is consistent with this finding. Would a later mitochondrial DNA test that excluded Manning severely undermine this ABO testimony? It certainly would contradict the conclusion that Manning left hairs in the car. It would change the hair evidence from inculpatory to exculpatory.
But it would not show that ABO testing is invalid in general or that the value of the test was overstated in this case. It would be a reason to exonerate Manning but not a reason to denominate the original evidence as scientifically invalid or as having exceeded the limits of science. In this respect, it would be no different than incontrovertible evidence that Manning was incarcerated when the murders were committed -- powerful evidence of innocence but no evidence of invalidity or overclaiming.
NOTES
- Scott Henson, Who Is This Guy?, Grits for Breakfast, Jan. 6, 2004, http://gritsforbreakfast.blogspot.com/2005/01/about-me.html
- For a short, sympathetic review of the theory of racial classification of hairs, see Edward J. Imwinkelried, Forensic Hair Analysis: The Case Against the Underemployment of Scientific Evidence, 39 Wash. & Lee L. Rev. 41, 49-50 (1982).
Thanks for such a thoughtful and detailed response. I regret I can only agree with caveats. And before going further, these opinions are my own, not any position endorsed by my employers, since you mentioned them, nor possibly anyone else!
ReplyDeleteA few perhaps-disconnected, certainly too-lengthy thoughts, in no particular order:
First, since the FBI said the testimony "included additional statements that exceeded the limits of science," and neither of us has read the transcript, I wonder if perhaps you dismiss the possibility of overstatement by the examiner too quickly?
More importantly, though, I think you're too focused on deciding "who is to blame?" as opposed to "how does error occur?" If the forensic analyst did nothing wrong, the prosecutor did nothing wrong, the cop did nothing wrong, ditto for defense counsel, but an innocent person was convicted, it's still worth understanding how the mistake was made.
I think error can arise at trial from a variety of means, including context that may not be immediately apparent from a textual analysis, even if we had the examiner's verbatim testimony in front of us. Hair analysis isn't a science developed independently like serology, it's something cops started doing to try to accuse people, which is a horse of a different color.
The 2009 NAS report called for separation of crime labs from police departments for a reason: Historically, forensic analysts considered themselves part of the prosecution team. Until relatively recently, a culture of independence from the department was not valued like it might be today. Understandably. After all, they're part of the same department, sometimes work in the same building, and the defense can't solicit their services. It wasn't just a perception they were part of the cop/prosecution team, it was the truth.
Hair is almost never the primary accusing evidence, especially in older cases before mitochondrial DNA testing. It's something gathered at the scene that may not be examined for months. Meanwhile, police develop suspects based on other evidence and the forensics are often used as an add on, frequently to bolster a weak case. (To appropriate from Andrew Lang, prosecutors use hair forensics like a drunk uses a lamp post, for support rather than illumination.)
The reason the prosecutor put the hair examiner on the stand was to suggest the defendant was in the car, and because the other, better evidence hadn't yet proved it. It's part of the prosecution's Big Show, lending a faux "scientific" veneer to shaky cases.
Finally, since this isn't actual science but more tradecraft - visually comparing and making what are ultimately informed but subjective judgments - I think the comparison to serology is flawed. We may know for sure how many people have blood type B and be confident the test can give that data. But we cannot know how accurate were these subjective hair comparisons because it's not like there's a standardized test where their conclusions can be duplicated.
A high-powered plaintiff's lawyer once told me that Daubert means something different on the civil and criminal sides. Where the money stakes are high and both sides can pay for counsel and experts, he said, this sort of non-probative testimony gets sidelined, or at least sufficiently countered.
Thanks for these further observations. I agree with almost all of them.
ReplyDeleteI assumed that the "additional statements that exceeded the limits of science" according to the DOJ letter of May 4 relate to the examiner's racial classification of the hairs (the crux of his testimony) and that all pertinent statements were quoted in the published opinions. But maybe there were worse statements. If so, the copy of the FBI report attached to the letter should have enumerated them, but I have not seen that report, the transcript, or the examiner's original report. Without more complete disclosure, I can't exclude the possibility that the examiner said something like "the hairs in the car definitely came from a black individual" or "the probability that those hairs came from an African American is 85%." To my knowledge (limited as it is), rigorous, published scientific studies do not support such statements.
I also agree that it's well worth understanding how mistaken convictions come about, that crime labs should be independent of police and prosecutors, that microscopic hair analysis is only weakly probative in general, that there are cogent arguments for excluding it in the first place (even though so many courts rejected them), and that physical hair analysis (as well as latent print identification, handwriting comparisons, toolmark comparisons, etc.) differ from serology in important ways.
Of course, there is no shortage of overstated testimony from serologists in cases of known wrongful convictions. And, there could be cases of actual innocence with entirely accurate testimony from them as well (as in my hypothetical version of Manning). I think it is useful to distinguish between these two situations in studying wrongful convictions. All scientific evidence has its limits. If the evidence meets a basic threshold so as to merit admission, it can be presented for what it is worth, and it should not be presented for more than that. When results are presented without overclaiming, it is not correct to describe the method that generated those results (whether it is comparative microscopy, latent print analysis, serology, or anything else) as “invalid”—not that you did this—but it often happens in the media and in litigation. Whether a technique is scientifically valid is a distinct question from whether it is applied correctly in a particular case and whether the results are presented for what they are worth. Overclaiming can be considered a type of invalidity (as in “It is is scientifically invalid to go beyond the bounds of what can be scientifically supported”), but it might call for a different remedy than does the use of a technique that lacks validity in all its forensic applications.