False Justice and Prosecutors' Fallacies

False Justice, by Jim and Nancy Petro, is an engaging, first-person tale of a former Ohio Attorney General's involvement in correcting false convictions as well as a summary and refutation of, as the book's subtitle puts it, "Eight Myths that Convict the Innocent." 1/ The book reveals the frustrations that lawyers in the innocence movement know all too well, and it wisely warns prosecutors, police, and the public of pernicious fallacies about criminals and the criminal justice system.

But the book perpetuates a fallacy of a different sort -- a statistical fallacy often called, in legal circles, "the prosecutor's fallacy." 2/ With some types of trace evidence (particularly DNA evidence), it is feasible to estimate the probability of a match between a defendant and the trace evidence given that a suspect is not the source of the trace at the crime scene. We can write this coincidental match probability as Pr(Match | ~Source).

The problem is that the judge or jury wants to know the probability of a DNA match given that a suspect is not the source of the DNA: P(~Source | Match). These two conditional probabilities are conceptually distinct. Sometimes they can be numerically identical or very close to one another, but other times they are not even close. The statistical or logical fallacy consists of naively transforming P(Match | ~Source) into P(~Source | Match).

The first instance of this transposition occurs at page 47, when the Petros quote from a letter intended to persuade a county prosecutor of Clarence Elkins' innocence (and of the guilt of a different inmate in the same cellblock):

We had a very convincing match. In a letter [the Ohio Innocence Project] and Elkin's attorneys ... informed the Summit County prosecutor that newly conducted DNA testing "conclusively exonerates Elkins and implicates Earl Mann in the murder and rapes in which Elkins was convicted." The letter explained that the full profile of the DNA from the girl's panties and Mrs. Judy Johnson's vaginal swab were "consistent with Earl Mann's DNA for full 12-point match."

How convincing was this match? This "full 12-point match," did not involve a random match probability P(Match | ~Source) as small as those for normal STR matches. It came from Y-STR testing. Ordinary forensic STR testing uses loci scattered across different pairs of autosomal chromosomes. For those STRs, estimating the probability of a 12-locus match would involve 24 multiplications of smallish fractions and give rise to tiny match probabilities for any given profile. Not so for forensic Y-STR testing. Y-STRs all lie on a single Y chromosome and are inherited father to son, as one package. Multiplying the population frequencies for individual Y-STRs would not make sense. 3/ Instead of multiplying,

As in all Y-STR DNA analysis, the the odds of finding a match are calculated on how many times that specific configuration of markers has been seen in a particular database. In this case, Earl Mann's DNA, in a database of 4,000 samples, matched the crime scene DNA. The letter explained, "Thus far, it ... is a unique Y-STR profile, and there is less than a 1 in 4,000 chance that it is not Earl Mann who left his DNA at the crime scene in the most highly probative areas."

Presumably, "less than ... 1 in 4,000" refers to the fraction 1/4001 that expresses how often the profile has been seen -- only once -- compared to how many Y-STR profiles have been recorded -- 4000 previous profiles plus Mann's. 4/ A 1/4001 "chance that it is not Earl Mann" given that the trace DNA and Mann's have the Y-STR profile is P(~Source | Match). In contrast, 1/4001 is the probability of randomly picking Mann's profile from a population in which 1/4001 profiles are just like Mann's. It is P(Match | ~Source). Elkin's lawyers have transposed. To build their case against Mann, they have committed the prosecutor's fallacy.

Now, Earl Mann was almost certainly guilty -- but not just because he had a matching profile. According to the 2000 Census, Summit County was home to approximately 140,000 men between the ages of 20 and 59. At a rate of 1 man per 4001, we would expect to find 140000/4001 = 35 of them with matching DNA. Looking at just the Y-STR match, it no longer sounds as if the chance that Mann was not the source is only 1/4001.

In fact, one could argue the chance that Mann was not the source is P(~Source | Match) = 34/35! After all, there were some 35 men in the right age range and locale for whom one could say "the full profile of the DNA from the girl's panties and Mrs. Judy Johnson's vaginal swab were 'consistent with ... for full 12-point match.'" Mann is just one of them. As such, for him, P(Source | Match) = 1/35; hence, P(~Source | Match) = 34/35.  5/

The passage quoted above is not the only instance of transposition in False Justice. It occurs just about every time the Petros quote a random match probability. Most of these probabilities are so small that the resulting likelihood ratio would swamp any reasonable prior probability, making the fallacy for particular transpositions somewhat academic. Still, False Justice does not get its description of the meaning of small match probabilities quite right.

Notes

1. Jim Petro & Nancy Petro, 2015. False Justice: Eight Myths that Convict the Innocent. Routledge: New York, NY (rev. ed.).
2. William C. Thompson & Edward L. Shumann, (1987). Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor's Fallacy and the Defense Attorney's Fallacy. Law and Human Behavior, 2(3): 167-187 (introducing the phrase).
3. See, e.g., David H. Kaye, 2010. The Double Helix and the Law of Evidence. Harvard Univ. Press: Cambridge, MA.
4. Another way to estimate the Y-STR profile frequency is more commonly used, but that is tangential to the issue of transposition.
5. A better way to arrive at P(~Source | Match) is to apply Bayes' rule. That formula yields 34/35 if one assumes that Mann and every other man in Summit County in the age range mentioned has the same prior probability of being the source of the trace DNA and that everyone else in the world has a source probability of zero. 

Hair Evidence in the “Clearly Not Exonerated” Exoneration of Mark Reid

On November 3, the fictional forensic scientist in the world’s most watched television drama was aghast that she had once performed microscopic hair comparisons. Having learned that such comparisons are entirely discredited, NCIS’s Abby Sciuto is horrified: “Can you imagine if I messed up, what that really means? It means that innocent people went to jail because of me, because of my mistakes.” As a writer for Entertainment Weekly wrote, “Abby’s spinning out of control, locked in her lab and reexamining every single case she’s ever touched.” 1/

The “16 Years” episode is fiction, but real people have gone to jail for longer than that — and some could have been executed — because of mistakes by examiners. One disturbing hair-comparison case is State v. Reid. 2/ I have cited the Connecticut Supreme Court's opinion in two publications that survey different ways to testify about the implications of similarities between trace evidence and samples from known sources (such as the defendant). 3/ In doing so, I was not expressing the slightest agreement with the supreme court’s reasoning or arguing that the court was correct to hold that the trial judge properly admitted the testimony. But the case does illustrate how a careful criminalist called upon to testify could proceed in the face of gaping scientific uncertainty about the significance of similarities in the trace material and the known samples. It also illustrates the different sorts of errors than can occur in ascertaining hair morphology and drawing inferences from it as well as the different types of exonerations that can occur with the benefit of DNA testing.

I. The Criminalist Gives “Features Only” Testimony in Reid

It seems hard to deny that various physical features of hair display at least some variation within a population.  But without extensive population data that might permit at least rough estimates of the relative frequencies of the features, and without employing methods that have demonstrated reliability in measuring the features of interest, it is not clear how, or even if, this information should be used in trials.

One possibility is to limit the testimony to a presentation of the observed features (and perhaps a characterization of the samples' features as similar or different, as the case may be). According to the Connecticut Supreme Court, the analyst in Reid pursued this “features only” approach:

[He] displayed an enlarged photograph of one of the defendant's hairs and one of the hairs recovered from the victim's clothing as they appeared side-by-side under the comparison microscope. [He] explained to the jurors how the hairs were similar and what particular features of the hairs were visible. He also drew a diagram of a hair on a courtroom blackboard for the jurors. The jurors were free to make their own determinations as to the weight they would accord the expert's testimony in the light of the photograph and their own powers of observation and comparison.

The trial court had held a pretrial hearing to decide whether this testimony satisfied the preliminary showing of scientific validity normally required of all suitably challenged scientific evidence. The court found that it did, but the supreme court did not rely on or discuss either the scientific validity or the general scientific acceptance of visual hair comparisons. It avoided the issue by holding that the testimony did not have to satisfy such standards — because it was not “scientific evidence” at all. Rather, the expert “testified about a subject that simply required the jurors to use their own powers of observation and comparison.”

Three years later, in 2003, the Superior Court granted a petition for a new trial. 4/ Its opinion casts doubt on the no-science theory. Elaborating on the supreme court’s description of the testimony, this court observed that at the trial, the expert “indicated that hair comparison analysis ... is generally accepted as reliable within the field of forensic science” and “that he could state, ‘to a reasonable degree of scientific certainty,’ that the pubic hairs found on the victim's clothing were microscopically similar to those pubic hair samples taken from Mark Reid.” On such a record, the supreme court’s conclusion that the usual standards for scientific evidence are beside the point is hard to swallow.

II. The Superior Court Orders a New Trial While Insisting that the DNA Evidence Does Not Exonerate Reid

In any event, at Reid’s trial

Mr. Settachatgul testified that the three hairs recovered from the victim's clothing were pubic hairs. These hairs were rootless, indicating that they were shed, not plucked; one was found on the victim's jeans, another on a sock, and another on her lower undergarment (panty). ... Based on the microscopic analysis, Mr. Settachatgul's conclusion was that the three rootless hairs recovered from M.'s clothing were Negroid pubic hairs which had similar characteristics to the pubic hairs supplied by petitioner.

As the State’s Attorney explained in her summation, “the hairs ... appeared to be the same color, both had an abundance of fuci, and both exhibited the shadow of twisting, indicative of pubic hair.”  However, she acknowledged that “statistics are not done in the comparison field, [and] the only conclusion that can be drawn is similar or dissimilar characteristics, not the percentage of the population which shares those hair characteristics.” She told the jury “this is not conclusive evidence. I agree with the defense to that degree. It is not conclusive. But it is supportive of the victim's I.D.”

This support collapsed when postconviction mitochondrial DNA testing established that the three public hairs came from the same individual or from individuals in the same maternal lineage. Critically, Reid was not in that maternal line, while the victim, a white woman, was. This left two major possibilities: either the unknown rapist was the source of the three hairs or the victim was. The former scenario totally exculpates Reid; the latter renders the “supportive” expert testimony inconclusive.

Reid argued that the mtDNA test proved his actual innocence — that the criminalist’s determination of race had to be true, that the victim was white, and therefore the true rapist must have been some other black man who deposited three hairs on the victim’s clothes. The court did not buy this argument. And for good reason. Because the mtDNA testing showed that the victim’s mtDNA sequences matched those of the three hairs, the most plausible conclusion is that the criminalist erred in finding “Negroid pubic hairs.” The hairs probably were the victim’s rather than any rapist’s.

Thus, nothing was left of to connect Reid to the rape except the victim’s identification of him. The Superior Court concluded:

This is a close, difficult case. The new mtDNA evidence merely excludes petitioner as the depositor of the unknown hairs; it clearly does not exonerate him. And, as stated, the victim, M., was certain and steadfast in her identification of Mark Reid, and the circumstances surrounding that identification support its reliability, at least when viewed absent the newly discovered mtDNA evidence.

There are reasons to question this rosy picture of the eyewitness testimony, but whatever one thinks of that identification, the Superior Court found that Reid was entitled to a new trial at which he could use the mtDNA evidence to devastate a major part of the state’s case — the hair testimony.

III. The End of the Story Leaves Questions Hanging

In the end, there was no retrial. According the University of Michigan Law School’s National Registry of Exonerations, the state dismissed the charges “after the victim declined to participate ... . Reid, who had other felony convictions, was deported to his native Jamaica. In 2004, Reid filed a lawsuit seeking $2 million in damages from East Hartford.” The Registry does not report the outcome of that action. 5/

Whether or not all microscopic hair testimony is scientifically invalid because the comparisons have not been shown to be scientifically reliable, there is agreement that criminalists frequently have erred by using the similarity between hairs to make strong or quantified statements about the source of the trace hairs. 6/ This type of overclaiming apparently did not occur in Reid. In granting a new trial, the Superior Court emphasized that “Mr. Settachatgul was testifying only to the very ‘narrow opinion’ that the three pubic hairs recovered from the victim's clothing were similar to the samples obtained from the defendant, and, that he, Settachatgul could not say that the questioned specimens were the pubic hairs of petitioner/defendant.”

This testimony would not be judged as scientifically invalid under the FBI’s guidelines for reviewing microscopic-hair-comparison testimony. 7/ Likewise, the prosecutor’s summation was not infected with the sort of egregious overstatements, such as “There is one chance, perhaps for all we know, in 10 million that it could [be] someone else’s hair,” heard in other cases. 8/

The hair analyst may have erred in concluding that the features were similar. But mtDNA testing cannot tell us that. The small DNA molecules in the mitochondria do not relate to hair morphology. They provide a complementary — and more specific — test for identity. Sequence differences can exclude suspects when the inherently less discriminating visible features cannot. Standing alone, this limitation does not make microscopic hair analysis scientifically invalid, and it does not mean that the analyst misjudged the visual features here. But it does underscore the need to estimate the likelihoods or conditional error rates for microscopic hair comparisons. Without this information, how can anyone know what weight to give to the criminalist’s findings of similar hairs?

Finally, given the mitochondrial results for the victim and the pubic hairs in Reid, the hair analyst probably erred in concluding that the three pubic hairs were of “Negroid origin.” As noted earlier, the woman who was attacked in Reid was white, and the simplest conclusion is that the hairs on her clothes were hers, as the mtDNA sequences suggest. Furthermore, the unqualified assurance as to the racial origin of the hairs was unjustified — even if the classification turned out to be correct. Skimming a few forensic science textbooks, I can find no reference to publications in the scientific literature to support the position that hair analysts can make firm determinations of biogeographic ancestry. Caution is usually advised. 9/

Notes

1. Sara Netzley, “16 Years,” Entertainment, http://www.ew.com/recap/ncis-season-13-episode-7/2
2. 757 A.2d 482 (Conn. 2000). The case is noted in J.M. Taupin, Forensic Hair Morphology Comparison— A Dying Art or Junk Science?, 44 Sci. & Justice 95 (2004).
3. David H. Kaye, David E. Bernstein & Jennifer L. Mnookin, The New Wigmore: A Treatise on Evidence: Expert Evidence (2d ed. 2011); David H. Kaye, Presenting Forensic Identification Findings: The Current Situation, in Communicating the Results of Forensic Science Examinations 12–30 (C. Neumann et al. eds. 2015) (Final Technical Report for NIST Award 70NANB12H014).
4. Reid v. State, No. CV020818851, 2003 WL 21235422 (Ct. Super. Ct. May 14, 2003).
5. Maurice Possley, Mark Reid, The National Registry of Exonerations.
6. David H. Kaye, Ultracrepidarianism in Forensic Science: The Hair Evidence Debacle, 72 Wash. & Lee L. Rev. Online 227 (2015)
7. Id.
8. Spencer S. Hsu, Santae Tribble Cleared in 1978 Murder Based on DNA Hair Test, Dec.14, 2012 (quoting from federal prosecutor David Stanley’s closing argument).
9. Max M. Houck & Jay A. Siegel, Fundamentals of Forensic Science 303 (2015) (“Estimating the ethnicity or ancestry of an individual from his or her hairs is just that: an estimate.”); Richard Saferstein, Forensic Science: An Introduction 419 (2d ed. 2011) (“all of these observations are general, with many possible exceptions. The criminalist must approach the determination of race from hair with caution and a good deal of experience.”).

Acknowledgment: Thanks to Chris Fabricant for thoughts on State v. Reid and for pointing me to the full history of the case.

What Proves that "the expert and his methods couldn’t possibly be reliable"?

I am developing an allergic reaction to the following kind of argument: "A forensic-science expert testified that a trace at the crime scene or on the victim was associated (to some degree of certainty) with the defendant. DNA later evidence exonerated the defendant. Therefore, the expert’s methods couldn’t possibly be reliable."

This reasoning is not very different from saying that a pitcher who does not strike out every batter couldn’t possibly be a reliable pitcher; that a polling firm that fails to correctly predict every election must be using methods that couldn’t possibly be reliable; or that a test for heart disease that sometimes errs couldn’t possibly be reliable. Without considering the success as well as the failure rate of the method, it is impossible to say that it is unreliable—or, by the same token, that it is reliable (in the sense of being worth relying on). 1/

Yet, the argument from cases of exonerations—lacking any comparison group—are legion in discourse on the use of trace evidence for identification. The latest example I encountered comes from a Washington Post blog site. Two days ago, Radley Balko wrote that

[A] defendant was convicted due to the testimony of a forensic expert who claimed that his “science” showed the defendant, and only the defendant, could have committed the crime. That conviction was later upheld by an appeals court in an opinion that explained in detail why the expert and his methods were legitimate and reliable. The defendant was later exonerated by DNA testing, thus demonstrating that the expert and his methods couldn’t possibly be reliable.

Mr. Balko went on to write that “not only did the courts continue to allow bite mark matching into evidence, every single time a defendant challenged its validity, that defendant lost.” 2/

Both the reasoning and the description of legal history are not quite right. To begin with, even absolute proof of innocence only shows that the test has a nonzero false-positive error rate (no surprise there) and that the witness should not have claimed to a certainty that no one else could have left the mark in question.

To be sure, some evidence that has found breathing space in the courtroom should be squeezed out entirely. But let’s face it—no scientific test meets the standard of perfection. If every case in which evidence that has produced false convictions meant “that the expert and his methods couldn’t possibly be reliable,” there could be no evidence. Overselling has occurred with every type of forensic evidence—from bitemarks to toolmarks to fingerprints to DNA. Courts, scientists, and criminalists should do their best to prevent this. Thus, whether through rules of evidence or through education and monitoring of analysts, testimony must be calibrated to the power of the scientific technique. The testimony should fairly express the known probative value of evidence from a validated method.

An example of testimony that violates this precept comes from Ege v. Yukins. 3/ In that case, a discredited dental expert testified as follows:

Q: Now, Doctor, with regard to your testimony, you indicated that it's highly consistent with the dentition of Defendant Carol Ege; is that correct?
A: Yes.
Q: Okay. With regard to—let me ask you a question. Let's say you have the Detroit Metropolitan Area, three, three and a half million people. Would anybody else within that kind of number match like she did?
A: No, in my expert opinion, nobody else would match up. 4/

The eventual outcome in the case contradicts the assertion that no challenge to the admission of bitemark evidence has succeeded. The state trial judge in Ege realized that the testimony was improper and only “denied [postconviction] relief because of the lack of a contemporaneous objection and a view that the showing of prejudice was insufficient.” 5/

A federal district court also concluded that “expert testimony identifying the petitioner as the only possible perpetrator of the alleged bite mark in the Detroit metropolitan area was improperly admitted.” 6/

The U.S. Court of Appeals for the Sixth Circuit agreed “with the district court that ‘Dr. Warnick's opinion that the petitioner was the only person in the entire Detroit metropolitan area who could have made the mark on the corpse carried an aura of mathematical precision pointing overwhelmingly to the statistical probability of guilt, when the evidence deserved no such credence.’” 7/ It affirmed the order for a new trial.

In short, the argument that a method of forensic identification that has been proved to be fallible is, for that reason alone, inadmissible proves too much. Likewise, the claim that no challenge to bitemark evidence has ever prevailed is exaggerated (although not by much). 8/

Please do not misunderstand me. The series of articles on bitemark evidence from which the remarks I have quoted were taken is impressive and useful. In offering these corrections to two small parts that seem a bit extreme, I am not arguing that bitemark analysis, which has little claim to validity, is either reliable (in the statistical sense that repeated analyses of the same marks give the same answers) or valid (in the sense that the answers are more often correct when marks from the same source are analyzed than when marks from different sources are compared). From the writing I have seen, bitemark analysis does not cut it.

I also believe that cases of false convictions should be studied and that the existence of a given type of scientific evidence in these cases should not be ignored. Finding a large number of false convictions with such evidence present is a warning signal. The evidence may come from a method that has a large false-positive rate, 9/ and that possibility must be investigated to decide whether the evidence should be excluded across the board or whether juries should receive the information -- together with an honest and clear explanation of the uncertainty in the results.

NOTES

1. See infra note 9.
2. Radley Balko, A High-ranking Obama Official Just Called for the “Eradication” of Bite Mark Evidence, The Watch, Wash. Post, July 22, 2015.
3. 485 F.3d 364 (6th Cir. 2007).
4. Ege v. Yukins, 380 F.Supp. 2d 852, 871 (E.D. Mich. 2005), affirmed in part, reversed in part, 485 F.3d 364 (6th Cir. 2007).
5. Id. at 857–58.
6. Id. at 858
7. 485 F.3d at 376.
8. The federal courts in Ege treated the answer to the 3.5 million people as "probability testimony" without questioning Michigan's general rule that bitemark identifications are admissible. A true (and wrongly decided) case of bitemark probability evidence is State v. Garrison, 585 P.2d 563 (Ariz. 1978).
9. The false-positive probability is P(+|O), where + is a positive statement ("the defendant left the mark") and O is the fact that some other person left the mark. Even if this probability is small, a disturbing number of false convictions could involve this evidence. Suppose that P(+|O) = 0.02, that 1,000 tests are performed in a set of cases with marks, and guilty defendants left the marks in 60% of these cases. The expected number of false positives is (0.02)(400) = 8. Assume that the probability of a true positive is P(+|S) = 0.96, where S means that the defendant is the source of the mark. Then the expected number of true positives is (600)(0.96) = 576. If defendants are convicted in all these cases, 8 convictions will be false (assuming that the culprit left the mark), and the many true positives will not be seen in the cases of exonerations of the innocent defendants. As indicated at the outset of these remarks, other data than exonerations are required to judge whether the test is reliable and valid.

Looking Backwards: How Safe Are Fingerprint Identifications?

Yesterday, I explained why the frequency with which factors like confessions are found in cases of wrongful convictions does not measure the general prevalence of those factors. I questioned one published claim that false confessions occur in at least 25% of all cases. My argument was not that this conclusion is wrong, but rather that the studies of false convictions do not provide data that are directly applicable to estimating prevalence.

My analysis was not confined to confessions. It is based on the fact that the wrongful-conviction studies are retrospective. We take the outcome—a false conviction—and ask what evidence misled the judge or jury. This backwards look reveals the frequency of the type of evidence e given false convictions. The statistic P(e|FC) is equal to the prevalence of e in all cases if there is no association between false convictions and e. 1/ In general, such independence is most unlikely.

The flip side of invoking wrongful conviction statistics to conclude that false confessions are common is calling on them to show that fingerprint misidentifications are extremely rare. In United States v. Herrera, 704 F.3d 480 (7th Cir. 2013), Judge Richard Posner wrote that

Of the first 194 prisoners in the United States exonerated by DNA evidence, none had been convicted on the basis of erroneous fingerprint matches, whereas 75 percent had been convicted on the basis of mistaken eyewitness identification. 2/

For this remark, he received some flak. Northwestern University law professor Jay Koehler chastized Judge Posner for ignoring a clear case of misidentification. Koehler wrote that the court’s “claim is inaccurate. Stephan Cowans, who was the 141st person exonerated by postconviction DNA evidence, was famously convicted on the strength of an erroneous fingerprint match.” 3/ However, whether a 0 or instead a 1 belongs in the numerator is not so clear.

Judge Posner cited Greg Hampikian et al., The Genetics of Innocence: Analysis of 194 U.S. DNA Exonerations, 12 Annual Rev. of Genomics and Human Genetics 97, 106 (2011), for the view that there were no erroneous fingerprint matches. Interestingly,  this paper gives a larger figure than either 0 or 1. It claims that three “cases ... involving fingerprint testimony were found to be invalid or improper.” 4/ However, none of the “invalid or improper” fingerprinting results misidentified anyone. Rather, “[i]n the 3 cases that were found to be problematic, the analyst in 1 case reported that the fingerprint was unidentifiable when in fact there was a clear print (later discovered and analyzed); in the 2 other cases, police officers who testified did not disclose the fact that there were fingerprints that excluded the exonerees.” 5/ Taking these words at face value, the court could well conclude that none of the exonerations involved false positives from fingerprint comparisons.

However, the fingerprint evidence in the Cowans case involved both concealment and an outright false identification. As Professor Koehler noted, one of the foremost scholars of false convictions, Professor Brandon Garrett of the University of Virginia School of Law, reported the Cowans case as a false positive. Garrett clearly stated that although the Boston Police fingerprint examiner “realized at some point prior to trial that Cowans was excluded , ... he concealed the fact and instead told the jury that the print matched Cowan’s.” 6/ Likewise, along with the national Innocence Project’s co-founder and co-director Peter Neufeld, Professor Garrett explained in an earlier law review article that the trial transcript showed that “Officer LeBlanc misrepresented to the jury that the latent print matched Cowans’s.” 7/ Thus, the Innocence Project serves up 1.7% as the figure for “improper” fingerprint evidence in the first 300 exonerations.

This may seem like much ado about almost nothing. One problem case in a small sample is not significantly different from none. But there is a legal issue lurking here. To ascertain the more appropriate figure we need to specify the purpose of the inquiry. Do we want to estimate the prevalence of all kinds of improper behavior—including perjury (or at least knowing falsehoods uttered or implied) by fingerprint examiners? If so, the Hampikian or Koehler numbers are the candidates for further analysis.

But Judge Posner was responding to a Daubert challenge to fingerprinting. The question before the Herrara court was whether latent fingerprint examiners can provide valid, seemingly scientific, testimony—not whether they can lie or conceal evidence.  The rate of unintentional misidentifications therefore is the relevant one, and that rate seems closer to zero (in the exonerations to date) than to 1.7%. 8/

So Judge Posner is not clearly wrong in speaking of zero errors. But what can we legitimately conclude from his observation that "[o]f the first 194 prisoners in the United States exonerated by DNA evidence, none had been convicted on the basis of erroneous fingerprint matches, whereas 75 percent had been convicted on the basis of mistaken eyewitness identification"? Does this comparison prove that latent print examiners are more accurate than eyewitnesses?

Not necessarily. In rape cases, where DNA exonerations are concentrated (because DNA for postconviction testing is more likely to be available), there are more instances of eyewitness identifications than of fingerprint identifications. Even if the probability of a false positive identification is the same for fingerprint examiners as for eyewitnesses, there are fewer opportunities for latent print misidentifications to occur. Consequently, the set of false rape convictions will be disproportionately populated with eyewitness errors. The upshot of this base rate effect is that the relative frequency of the errors with different types of evidence in a sample of wrongful convictions may not reflect the relative accuracy of each type of evidence.

Nonetheless, we still have to ask why it is that no (or almost no) cases of unintentional false positives have emerged in the wrongful-conviction cases. Does not this absence of evidence of error prove that errors are absent? Koehler’s answer is that

The fact that few of the DNA exonerations cases overturned verdicts based on erroneous fingerprint matches says virtually nothing about the accuracy of fingerprint analysis precisely because cases involving fingerprint matches are rarely selected for postconviction DNA analyses. By this flawed logic, one might also infer that polygraph errors are “very rare” because none of the DNA exoneration cases overturned erroneous polygraph testimony. 9/

But gathering latent prints is more common than polygraphing defendants, and Koehler does not document his assertion that cases with fingerprint matches are much more rarely the subject of postconviction DNA testing than are cases with other kinds of evidence. Traditionally, it may have been harder to obtain DNA testing when a reported fingerprint match indicated guilt, but postconviction DNA testing has become more widely available. Indeed, Virginia has pursued a test-them-all approach in convictions (with available DNA) for sexual assaults, homicides, and cases of non-negligent manslaughter from 1973 to 1987. 10/ Nevertheless, a selection effect that creates a bias against the inclusion of reported fingerprint matches in the sample of known false verdicts cannot be dismissed out of hand. Certainly, Virginia’s comprehensive testing is exceptional.

Even so, pointing to a likely selection effect is not the same as assessing its impact. Selecting against fingerprinting cases reduces the value of P(FV|FL), the proportion of detected false verdicts given false latent print matches. At the same time, a reported latent print match is highly persuasive evidence. This boosts the value of P(FV|FL). If Koehler’s selection effect is dominant, we might try out a value such as P(FV|FL) = 0.04. That is, we assume that only 4% of all cases with false latent print matches culminate in detected false convictions. How large a fraction of false matches (out of all declared matches) could be reconciled with the observation that no more than 1% or so of the false convictions established by DNA testing involved an arguable latent fingerprint false positive error?

As explained yesterday, this will depend on other variables, some of which are interrelated. Consider 1,000 cases in which police recover and examine latent prints suitable for comparison in 100 (10%) of them. Suppose that 15 of these examinations (15%) produce false matches, and that (as proposed above) only 4% of these false-confessions cases terminate in convictions later upended by DNA evidence. The result is about 1 false conviction. Now consider the other 900 cases with no fingerprint evidence. If, say, 80% of these cases end in convictions of which 10% are false, 72 other false convictions will accrue. Upon examining the 73 false-conviction cases, one would find confessions present in 1/73 (about 1%) of them. Yet, a full 15% of all the fingerprint matches were (by hypothesis) false positives.

Now, I am not contending that any of these hypothetical numbers is realistic. But they do show how a high rate of false fingerprint identification can occur in general casework along with a low rate in the known DNA-based exonerations. Better evidence of the general validity of latent fingerprint analysis than the figures from exonerations should be—and is—available.

Notes

1. By definition, P(e|FC) = P(e & FC) / P(FC). If e and FC are independent, then P(e & FC) = P(e) P(FC) / P(PC) = P(e).
2. Id. at 487. 
3. Jonathan J. Koehler, Forensic Fallacies and a Famous Judge, 54 Jurimetrics J. 211, 217 (2014) (note omitted).
4. Greg Hampikian et al., The Genetics of Innocence: Analysis of 194 U.S. DNA Exonerations, 12 Annual Rev. of Genomics and Human Genetics 97, 106 (2011)
5. Id.
6. Brandon L. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong 107 (2011).
7. Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 74 (2009).
8. I say “closer” because it appears that Office LeBlanc first reported that Cowans’ prints were on a mug at the site of the murder for which he was convicted. According to the Innocence Project, “Cowans' fingerprints were actually compared to themselves and not to the fingerprint on the evidence.” Innocence Project, Wrongful Convictions Involving Unvalidated or Improper Forensic Science that Were Later Overturned through DNA Testing. Independent auditors concluded that LeBlanc knew of his mistake before trial but tried to conceal it. Garrett & Neufeld, supra note 7, at 73–74. Perhaps we should score an initially mistaken analysis that an examiner knows is mistaken (and that would not produce false testimony from an honest analyst and that would be caught by the simple safeguard of blind verification) as half a misidentification?
9. Koehler, supra note 3, at 217. 
10. John Roman et al., Urban Institute, Post-conviction DNA Testing and Wrongful Conviction 11–12 (2012).

Related Post: Looking Backwards: How Prevalent Are False Confessions?, July 27, 2014

Looking Backwards: How Prevalent Are False Confessions?

Studies of false convictions are tremendously important. They can rebut complacent assumptions that such things never happen, and they can shine a light on procedures that should be corrected. But the statistics that emerge from these studies are sometimes misunderstood. What should one make of findings that the evidence that convicted innocent defendants often involved confessions (20%), "invalid" forensic science (60%), or eyewitness identifications (70%)? Does this mean that false confessions occur in 20% of all criminal cases, for example? Or that eyewitnesses are wrong 70% of the time?

Years ago, Professor Roger Park at the University of California's Hastings College of the Law asked this question about eyewitnesses. As he pointed out, something had to go wrong in these cases. That something could be common or rare. In more technical jargon, how can you infer the prevalence of a factor from a retrospective study?

Another law professor, David Harris of the University of Pittsburgh, tried to do just this. In a recent and generally penetrating book on flaws in the criminal justice system, he argued as follows:

[T]he basic data are available for all to see. Of the more than 250 exonerations now on record, 25% involved "innocent defendants who made incriminating statements, delivered outright confessions, or plead guilty." ... Recall that DNA evidence--the basis for nearly all the exonerations to date--is available only for a fraction of all criminal cases; experts estimate that police recover testable biological evidence in only 5 to 10 percent of all cases. [I]n these other 90 to 95% of the cases, we have no reason to think that interrogation tactics work any differently, or any better, than in cases in which police recover DNA evidence. Thus, if false statements by suspects occur in 25 percent of the DNA-testable cases, we should expect a similar percentage in the other 90 to 95 percent of the cases. Put another way, if there is no reason to think that the DNA-based exoneration cases differ from others in the system, they provide us with a window into the whole criminal justice system. And that means that the problem we see--the 25 percent of the DNA cases in which false statements occur--represents the tip of the proverbial iceberg, and rational, conservative assumptions would lead us to believe that we should expect to see false convictions and statements by defendants in 25 percent of all cases.

David A. Harris, Failed Forensics: Why Law Enforcement Resists Science 76-77 (2012) (emphasis in original).

But surely there is a mistake here. By this reasoning, if every exoneration were a case in which an eyewitness identified the defendant, "rational, conservative assumptions would lead us to believe that we should expect to see false convictions and [eyewitness identifications] in 100% of all cases." That hardly seems rational.

What has gone wrong? First, the assumption that cases in which DNA evidence can be recovered are comparable to the large remainder of criminal cases overlooks the fact that convictions are not clearly comparable to nonconvictions (acquittals or dismissals). For the professed equality to hold, defendants who are not convicted would have to be just as likely to confess and to do so falsely as are those defendants who are falsely convicted and whose cases can be studied. This necessary premise is implausible. Because confessions cause convictions, the incidence of confessing should be less in the nonconvicted group. In addition, if the case against a suspect is weak, the police may resort to more coercive tactics to obtain a confession. For such reasons, we would not expect to find that the proportion of false confessions in cases of false convictions in DNA (or any other batch of) exoneration cases equals the proportion in all criminal cases.

Bayes' rule also has a role here. Let's stipulate that 25% of all false convictions—not just those of DNA exonerations—involved confessions. This statistic is compatible with many possible ratios of false confessions to true ones in all cases. To prove this, I'll run through some algebra, then give a numerical example, but you can skip the box of algebra if you want.

Let P(FC) be the unknown prevalence of false confessions in all cases. Let P(TC) be the prevalence of true confessions, and P(NC) be the prevalence of the remainder of cases in which defendant does not confess. Under each of these conditions, there is some probability that a verdict of guilty (V) will be attained. (For simplicity, I am going to restrict the analysis to cases without guilty pleas. Discovering that some innocent defendants plead guilty would not be much of a surprise.) For example, the probability of a (false) conviction (FV) given a false confession is P(FV|FC). Bayes rule then states that P(FC|FV) = P(FC)P(FV|FC) / [P(FC)P(FV|FC) + P(TC)P(FV|TC) + P(NC)P(FV|NC)]. Because a guilty verdict cannot follow a true confession, P(FV|TC) = 0, and we have P(FC|FV) = P(FC)P(FV|FC) / [P(FC)P(FV|FC) + P(NC)P(FV|NC)]. We observe P(FC|FV) = 0.25 and want to infer P(FC). Solving for P(FC) yields P(FC) = kzw/(1-k)y, where k = P(FC|FV), y = P(FV|FC), z = P(NC), and w = P(FV|NC). Thus the prevalence of false convictions depends not just on the observed value k = P(FC|FV) = 0.25. It varies with the other conditional probabilities (w and y) and the prevalence of cases without confessions (z).

For a numerical example, consider a set of 1000 cases in which 230 defendants (23%) confess. Suppose that 1 in 10, or 23, of these confessions are false, and that 90% of these false-confessions cases terminate in convictions. The result is about 21 false convictions. Now consider the other 770 cases with no confessions. If, say, 80% of these cases end in convictions of which 10% are false, that will add another 62 false confessions. Upon examining the 83 false-conviction cases, one would find confessions present in 21/83 = 25% of them. Yet, only 2.3% of all the cases (10% of all the confessions) were false confessions.

The lesson of this exercise is simply that the proportion of confessions within a set of cases of false convictions cannot produce a meaningful estimate of their prevalence. Of course, this lesson is not confined to confessions. It applies to any factor that appears (or does not appear) among the known cases of false convictions. If there were no fingerprint identifications in the DNA exoneration cases, would that demonstrate that latent fingerprint identification is highly accurate?

Judge Richard Posner thought so. In United States v. Herrera, 704 F.3d 480 (7th Cir. 2013), he wrote that "[o]f the first 194 prisoners in the United States exonerated by DNA evidence, none had been convicted on the basis of erroneous fingerprint matches, whereas 75 percent had been convicted on the basis of mistaken eyewitness identification." Id. at 487. I will discuss this assertion and its implications in a later posting.

Related Postings

Looking Backwards: How Safe Are Fingerprint Identifications?, July 28, 2014

False Confessions, True Confessions, and the Q Factor, July 2, 2014

False Confessions, True Confessions, and the Q Factor

EXHIBIT A Steven A. Drizin and Richard A. Leo, The Problem of False Confessions in the Post-DNA World, 82 N.C. L. Rev. 891, 930-31 (2004) In this Article, we analyze 125 recent cases of proven interrogation-induced false confessions (i.e., cases in which indisputably innocent individuals confessed to crimes they did not commit) and how these cases were treated by officials in the criminal justice system. ... This is ... the largest cohort of interrogation-induced false confession cases ever identified and studied in the research literature. ... [T]here is no existing universe or database of police interrogation cases from which researchers would be able to draw a random sample. Moreover, even if such a database existed, it would make as much sense for someone studying the causes and consequences of interrogation-induced false confessions to randomly sample police interrogations as it would for someone studying the causes and consequences of lung cancer to randomly sample healthy individuals. The appropriate universe of cases for this study is interrogation-induced false confessions, not all police interrogations.

EXHIBIT B Neil A. Accortt et al., Chronic Disease Mortality in a Cohort of Smokeless Tobacco Users, 156 Am. J. Epidemiology 730 (2002) Our aims are to assess the potential increased mortality risks associated with smokeless tobacco use in a representative sample of the US population and to evaluate the effects of the combined use of smokeless tobacco and cigarette smoking on mortality from chronic diseases. ... We used data from ... a national probability sample of the noninstitutionalized US population ... . After adjustment for age, race, gender, and poverty status, ... [f]emale smokeless tobacco users experienced an increase in mortality from all cancers ... of borderline significance. ... Male exclusive smokeless tobacco users did not experience significant increases in mortality for any type of cancer considered. ... [T]he lung cancer mortality among combined users was nearly twice that of exclusive smokers. ... .

The study of false confessions described in Exhibit A is important and influential. There is much to be said for detailed, historical studies of miscarriages of justice. Analyzing what went wrong in a series of cases can suggest possible reforms to prevent or abate these outcomes in the future.

But by design, the study is not capable of determining the causes of any of the confessions of the 125 DNA-exonerated men. It is comparable to a series of medical case reports rather than to an epidemiological study, such as the one in Exhibit B, which is designed to detect a possible risk factor or cause of cancer. Physicians may report on unusual and interesting cases in which a disease follows an exposure to some factor, such as lung cancer after years of smoking. But without data on the incidence of lung cancer among nonsmokers (or relevant experiments on the effects of smoking), how can one conclude from the study that smoking causes (or even is risk factor for) cancer? A case series offers almost nothing in the way of proof of causation. The persuasive evidence that smoking causes cancer came from large case-control and cohort studies in several populations—not from the earlier case series.

Confronted with defendants' efforts to introduce the testimony of psychological experts on the factors found to be present in DNA exonerations, prosecutors have harped on this point. They have argued that although factors such as physical isolation and extended interrogation sessions certainly can contribute to false confessions, they may also contribute to true confessions, and there is no proof that they are more likely to contribute to a false confession than a true one. In other words, a factor may be relevant to voluntariness—whether the defendant’s will was overborne by unfair pressure—but not be evidence of falsity.

Several courts have agreed that the proffered risk factors are not adequately linked to false as opposed to true confessions.  One federal district court rebuffed defendant’s risk-factor testimony as “nothing more than guesswork: coercive interrogation techniques may lead to false confessions but also produce true confessions, and such techniques were used in this case ... .” United States v. Deuman, 892 F.Supp.2d 881, 888 (W.D. Mich. 2012). The Michigan Supreme Court ruled that the research was methodologically flawed for “failing to compare true and false confessions and identify factors that contribute to false confessions but not true confessions.” People v. Kowalski, 821 N.W.2d 14, 22 (Mich. 2012). Likewise, the Supreme Judicial Court of Massachusetts found it telling that the expert “acknowledged that no studies have been conducted comparing the prevalence of these factors among false confessions to either ‘true’ confessions or all confessions” and “that the studies based on proven false confessions that have been used to identify these relevant factors were based on a sample size of approximately 150 to 200 proven false confessions.” Commonwealth v. Hoose, 5 N.E.3d 843, 861 (Mass. 2014).

These criticisms, I believe, are valid—but only as applied to a possible effort to define a false-confession “profile” or to conclude that when the putative risk factors are present, a confession is false. Defense experts in these cases do not pretend that they can diagnose whether a confession is true or false.  They simply want to present information that they are convinced will help the jury make this judgment. And their beliefs about causes and risk factors for false confessions are based on a richer set of studies than just the case reports.

Consequently, the opinions summarily dismissing all false-confession research as scientifically flawed because of the lack of a comparison group in studies like the one in Exhibit A are too hasty. Furthermore, the testimony could serve a function other than telling the jury which factors are relevant to concluding that a confession is false (for which data on causation or correlation is essential). Suppose that research reveals a factor—I will call it factor Q—that makes virtually every innocent suspect confess. Moreover, although there are no data directly on point, it is highly plausible that Q would have the same effect on guilty suspects. Because the probability of a confession conditional on Q is very large for both innocent and guilty suspects, the relative risk is close to 1. Q is not indicative of whether a given confession is true or false.

Nonetheless, informing the jury of the factor and its impact on innocent defendants could be helpful (if the jury would not otherwise recognize the presence and significance of Q). The point of this testimony would not be to prove that Q has probative value in distinguishing between true and false confessions. It would be show that the confession in the case lacks the probative value that the untutored jury might attribute to it.

Rather than issuing sweeping denunciations of the entire body of psychological research into false confessions as methodologically sterile, courts should focus on whether the expert testimony exceeds the bounds of valid generalizations from the research and how much the particular testimony really can assist the jury. Are there powerful factors like Q that the jury would not recognize? I cannot say that I know the answer, but it is a fair question.

Acknowledgments: Thanks to Debbie Davis and Tom Workman for correspondence on the logic of "risk factors" for false confessions. A longer discussion of the admissibility of expert testimony on false confessions will appear in the 2015 Cumulative Supplement to The New Wigmore: A Treatise on Evidence.

Flawed Journalism on Flawed Forensics in Slate Magazine

Yesterday, Slate magazine published an article by Mark Joseph Stern announcing that “Forensic Science Isn’t Science.” 1/ The writer’s objective—to urge that forensic science be conducted rigorously and fairly—is laudable. But just as shabby science should not be tolerated in the courtroom or the police station, journalism that pays little heed to the facts should not be acceptable in serious publications.

I’ll give one example, chosen because it pervades the publication. The article begins with the claim that “[f]orensic analysis of semen introduced at trial had convinced the jury that [Earl] Washington [Jr.] ... had brutally raped and murdered a young woman in 1982.” It asks, “[h]ow could forensic evidence, widely seen as factual and unbiased, nearly send [this] innocent person to his death?” It ends with the plaintive thought that “[o]ur national experiment in untested forensics may soon be coming to a close. But it hasn’t ended in time to prevent a few more people like Earl Washington from being sacrificed on the altar of pseudoscience.”

The conviction and exoneration of Earl Washington have much to teach us about criminal justice. But it would be hard to find a worse example of “an innocent man being sacrificed on the altar of pseudoscience.”  There was no forensic evidence—scientific or pseudoscientific—introduced in the trial. Had there been, the outcome might have been different. This is the conclusion that follows from the description of the case in an important book, Convicting the Innocent, by Professor Brandon Garrett.

Garrett's research reveals that the police made every effort to keep science away when they built their case around a classic false confession from a “borderline mentally retarded farmhand” 2/ with convincing detail fed to him by police. One officer was found in a later civil rights action to have “fabricated the confession.” 3/ The alleged confession included the revelation (known to the police) of the killer’s blood-stained shirt with a torn-off patch left in the victim’s dresser drawer. Although forensic analysts had excluded five other suspects as possible sources of hairs found in the shirt pocket, “police instructed the state crime laboratory not to compare [Washington’s] hairs.” 4/

Even more telling—but untold—was the serological evidence in the case. According to Mr. Stern, it was “semen introduced at trial” that “convinced the jury.” But no semen was introduced at trial. No “semen analysis,” as Mr. Stern calls it, was offered into evidence. If only it had been!

“The semen-stained blanket from the victim’s bed was blood-typed, and that rudimentary technique had ruled out Washington.” 5/ The prosecutor would hardly want to introduce this evidence. (Indeed, it is hard to see how he ethically could go to trial without having proof that the blood-typing was incorrect.) As for the inexperienced defense counsel, 6/ “[t]he lawyer later said that while he saw the forensic reports, he ‘was not familiar with the significance of the analysis.’” 7/ Worse still, “the state concealed crucial evidence of innocence, including forensic evidence, from the defense.” 8/

In short, presenting the conviction and near-execution of Earl Washington, Jr., as the example of “a decades-long experiment in which undertrained lab workers jettison the scientific method in favor of speedy results that fit prosecutors’ hunches” disguises the real lessons of the case. The Washington case is an awful illustration of (1) evidence of a false confession that could have been prevented by proper interviewing techniques (including recording the confession); (2) willful blindness on the part of the police and the prosecution to the warnings signs in the confession; (3) suppression of and failure to pursue contradictory scientific evidence; and (4) ignorance of the scientific evidence that gave the lie to the alleged confession.

Are there real examples of “flawed forensics” contributing mightily to false convictions? Of course. Do we know how many? Not really, but whatever the precise number may be, there are too many such cases. An article making this now well known point easily could have started with a more a propos example.

Were this the only defect in the article, one might chalk it up to a combination of the expectancy effect and poor research. Perhaps the writer picked the Washington case without worrying too much about the actual facts because he already knew what to expect. (Dare I say that Mr. Stern was not writing on a blank Slate?) Unfortunately, however, there are other inaccuracies in the article. I comment on them in the next posting.

Notes

1. Mark Joseph Stern, Forensic Science Isn’t Science: Why juries hear—and trust—so much biased, unreliable, inaccurate evidence, Slate, June 11, 2014.
2. Brandon L. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong 145 (2011).
3. Id. at 30.
4. Id. at 35.
5. Id. at 147.
6. Id. at 147-48.
7. Rather than present a vigorous defense—“[t]he entire defense case lasted only 40 minutes,” id. at 146, Washington’s lawyer—who had never tried a capital case before— “simply asked for the mercy of the jury.” Id. at 154. He did not even point out “the glaring inconsistencies” between the compliant confession and some of the facts in the case—including the race of the white woman who was murdered in front of her two children. Id. at 147. When asked whether she was white or black, Washington chose “black.” Id.
8. Id. at 148 (note omitted). The “forensic evidence” in question seems to be the following:

   An analyst working for the Virginia Bureau of Forensic Science had tested stains on a central piece of evidence, a blue blanket found on the murdered victim’s bed, and found Transferrin CD, a fairly uncommon plasma protein that is most found in African-Americans. The analyst even ran a second test to double-check the result. The next year, when Earl Washington, Jr., was arrested, they tested his blood and found he did not possess the unusual Transferrin CD. The state did not give the defense the report indicating Washington was excluded by that characteristic. Instead, the state gave the defense an “amended” report. Without having done any new tests, the altered report stated that the results of the Transferrin CD testing “were inconclusive.” The original lab report came to light decades later when Washington filed a civil rights lawsuit after his exoneration.

   Id. at 108 (notes omitted). Inasmuch as the “inconclusive” serum protein test would not have much significance for the defense, I assume that the “rudimentary” blood-typing results that excluded Washington, which the defense saw but overlooked, would have been even more damaging to the prosecution than this amended test for Transferrin CD.

Maryland v. King: “Quite a Worldview”

Supreme Court watchers took note of an article by an astute reporter on “an irony” in the fact that Justice Kennedy’s opinion for the Court in Maryland v. King cited Actual Innocence, an important book about DNA exonerations. See A Digression on Ellipses, Actual Innocence, and Dr. Mengele, June 13, 2013.

But one of the book's authors, Peter Neufeld, was “feeling less than honored” by this nod from the Court:

Part of the problem was what he called an irony. [¶] In 2009, Justice Kennedy joined the majority opinion in a 5-to-4 decision that said prisoners had no constitutional right to DNA testing that might prove their innocence. Mr. Neufeld, who founded the Innocence Project with Barry Scheck, represented the prisoner on the losing end of that case, District Attorney’s Office v. Osborne.

But last week, Mr. Neufeld said, Justice Kennedy concluded that “it’s O.K. for the state to take DNA, without a warrant, from mere arrestees, who may ultimately have their charges dismissed.” [¶] The combination of the two decisions baffled Mr. Neufeld. “That is quite a worldview,” he said of a jurisprudence that allows nonconsensual testing of people presumed innocent but denies voluntary testing to people who insist that they really are innocent.

Adam Liptak, Cited by a Justice, But Feeling Less Than Honored, N.Y. Times, June 11, 2013, at A15.

This juxtaposition of King and Osborne is “quite a worldview,” but it is not an accurate description of the Court’s jurisprudence on DNA evidence. King establishes that the constitution permits—but does not require—a state routinely to acquire DNA identification profiles from individuals charged with violent crimes or burglary and to test these against databases of profiles of DNA found at crime-scenes and on victims. Osborne establishes that the constitution permits—but does not always require—the state to undertake or permit postconviction DNA testing that apparently could exonerate a convicted offender alleging actual innocence.

To be sure, the Osborne Court did not delineate the extent to which due process mandates postconviction DNA testing. In the view of the majority, it did not need to because the state did not refuse William Osborne a fair opportunity for DNA testing. But even if the Osborne Court was mistaken in treating the procedures in Alaska a basically fair, that Court's view of the reach of the constitution is consistent with its view in King. Sure, Alaska’s dogged refusal to allow Osborne to test, at his own expense, the condom left in the snow where (the jury found) he and another man assaulted, battered, and shot a woman, seems unconscionable. But as Justice Kennedy viewed the case:

Respondent was convicted for a brutal sexual assault. At trial, the defense declined to have DNA testing done on a semen sample found at the scene of the crime. Defense counsel explained that this decision was made based on fear that the testing would provide further evidence of respondent's guilt. After conviction, in an unsuccessful attempt to obtain parole, respondent confessed in detail to the crime. Now, respondent claims that he has a federal constitutional right to test the sample and that he can go directly to federal court to obtain this relief without giving the Alaska courts a full opportunity to consider his claim.

District Attorney's Office for the Third Judicial District v. Osborne, 128 S.Ct. 2308 (2009) (Alito, Kennedy, and Thomas, JJ, concurring). Excerpts from the majority opinion, which Justice Kennedy also joined, are included at the end of this comment.

The reasoning is not a statement that everyone who claims innocence after a conviction should be barred from DNA testing. See also Innocence Project, Press Release, U.S. Supreme Court Decision on DNA Testing Is Disappointing But Will Have Limited Impact, June 18, 2009. Instead, Osborne identifies one situation in which federally ordered postconviction testing is not part of the due process of law essential to criminal punishment. Cf. Innocence Project, Model Law.

In other words, Osborne demonstrates that, a state may (but need not) take and use a DNA sample from every convicted person who volunteers one for postconviction testing. Likewise, King confirms that a state may (but need not) compel every individual who is arrested and charged with a serious crime to give a DNA sample for a suspicionless database trawl. The wisdom of these two decisions certainly is subject to reasonable dispute, but their juxtaposition forms an internally coherent constitutional jurisprudence.

Appendix
Some of the Osborne Court’s reasoning

Alaska provides a substantive right to be released on a sufficiently compelling showing of new evidence that establishes innocence. It exempts such claims from otherwise applicable time limits. The State provides for discovery in postconviction proceedings, and ... this discovery procedure is available to those seeking access to DNA evidence. ... The evidence must indeed be newly available to qualify under Alaska's statute, must have been diligently pursued, and must also be sufficiently material. These procedures are similar to those provided for DNA evidence by federal law and the law of other States ... . [¶] And ... the Alaska Court of Appeals has suggested that ... in an appropriate case the State Constitution may provide a failsafe even for those who cannot satisfy the statutory requirements under general postconviction procedures. To the degree there is some uncertainty in the details of Alaska's newly developing procedures ... , we can hardly fault the State for that. Osborne has brought this §1983 action without ever using these procedures in filing a state or federal habeas claim relying on actual innocence. ... When Osborne did request DNA testing in state court, he sought RFLP testing that had been available at trial, not the STR testing he now seeks, and the state court relied on that fact in denying him testing under Alaska law. His attempt to sidestep state process through a new federal lawsuit puts Osborne in a very awkward position. If he simply seeks the DNA through the State's discovery procedures, he might well get it. If he does not, it may be for a perfectly adequate reason, just as the federal statute and all state statutes impose conditions and limits on access to DNA evidence. It is difficult to criticize the State's procedures when Osborne has not invoked them. ... These procedures are adequate on their face, and without trying them, Osborne can hardly complain that they do not work in practice.

Maryland v. King: A Digression on Ellipses, Actual Innocence, and Dr. Mengele

New York Times Supreme Court correspondent Adam Liptak recently tweaked the noses of the justices who upheld the constitutionality of routinely taking DNA from individuals arrested of violent crimes and burglary. [1]

The Court’s opinion, penned by Justice Kennedy, enumerated the arguable benefits of routine DNA collection before conviction, concluding with with the observation that “[f]inally, in the interests of justice, the identification of an arrestee as the perpetrator of some heinous crime may have the salutary effect of freeing a person wrongfully imprisoned for the same offense.” To support this conclusion, Justice Kennedy added a quotation from the important and gripping book Actual Innocence: Five Days to Execution and Other Dispatches from the Wrongly Convicted [2]. As quoted in the justice's opinion, the authors of Actual Innocence -- Barry Scheck, Peter Neufeld, and Jim Dwyer -- had written that “[P]rompt [DNA] testing . . . would speed up apprehension of criminals before they commit additional crimes, and prevent the grotesque detention of . . . innocent people.”

I. Toward More Punctility

Mr. Liptak noted that this quotation was not “especially punctilious.” One might think that the problem is that the quotation speaks to preventing erroneous convictions, whereas the sentence it is supposed to support is about "freeing a person wrongfully imprisoned." (Emphasis added.) But Mr. Liptak's point, which turns out to be related, was that “[t]hose first three dots covered a lot of ground. They took the place of more than six sentences and suggested a different point than the one the authors were making.”

What exactly was that little ellipsis hiding? To be particularly punctilious, let’s look at the preceding paragraph, all the omitted words, and a few that follow. This material reads as follows (with the quoted part highlighted):

All wrongfully convicted people take the lash of punishment for someone else's crime; that is the very definition of their predicament. Far too often, they are surrogates for serial criminals and killers, as in California, where Kevin Green carried the weight for a crime by Gerald Parker, who for twenty years stood unprosecuted for five murders. In Oklahoma, Robert Miller was condemned to die for murdering and raping two elderly women before DNA testing put a man named Ronnie Lott in their houses, as well as in the homes of several other women who survived his rapes. In Chicago, Rolando Cruz and Alejandro Hernandez were sentenced to death for killing Jeanine Nicarico, although it turned out that Brian Dugan admitted to murdering Jeanine and five others, including children, during sexual assaults. In North Carolina, Ronald Cotton was cleared of two rapes committed by a man tied to eight others. In Virginia, David Vasquez, a borderline mentally retarded man, pleaded guilty to raping a woman and hanging her from a pipe. Vasquez said he was innocent but copped a plea to avoid the death penalty. The time he served belonged to Timothy Spencer, who raped and murdered not only the victim in the Vasquez case, but three other women, all of them hanged. And because Spencer ultimately was sent to death row for the murders, he was not prosecuted for eight other rapes in which he was the prime suspect.

Each year, the technology for linking and solving these kinds of crimes gets faster and cheaper. Computers can sort through data banks of DNA samples with dazzling speed and connect identical profiles among far-flung atrocities. If a genetic profile of a criminal is already lodged in a data bank, identifying him is a no-brainer. Even if a pattern of crimes can't be tagged to a known criminal, prompt testing will prevent the lengthy detention of innocent suspects and immediately put the police back to the task of finding the real perpetrator. The failure to take full advantage of this technology, both for solving crimes and freeing the innocent, is a national scandal. Current investigative approaches must change.

Typically, DNA testing of evidence is done only after a suspect has been apprehended—and then takes two or three months. Hundreds of thousands of rape kits from unsolved cases are thrown out or sit in dead storage for years with no effort made by the authorities to run DNA tests, squandering opportunities to identify serial offenders and clear the wrongly convicted, such as Kevin Green. If crime scene materials were tested and catalogued immediately, one case could be linked quickly to others. Since a forensic DNA test can be completed in two or three days, crime laboratories could be given the resources to finish a case within a week. This would speed up apprehension of criminals before they commit additional crimes, and prevent the grotesque detention of thousands of innocent people.

And that goes not only for rapes and homicides but for burglaries, robberies, assaults—any kind of incident in which a criminal may have left a trace of skin, saliva, hair, or any biological evidence.

The second paragraph on "prompt testing" is a little confusing. How will "prompt testing" of rape kits "prevent the lengthy detention of innocent suspects and immediately put the police back to the task of finding the real perpetrator" when there is no match to a convicted offender in the database? Sticking to cold hits, there will be more of them if arrestee DNA profiles are placed in the database along with the profiles of convicted offenders and are periodically searched against the database of profiles from unsolved crimes. Statements from the matching individuals might exonerate some convicted offenders (for whom post-conviction DNA testing has been unavailable or ambiguous under the circumstances of the cases). Furthermore, these cold hits might prevent the arrests and subsequent convictions of some innocent suspects (who do not receive prompt DNA testing or whose exclusion as the source of the crime-scene DNA is not sufficient, under the circumstances of the case, to demonstrate their innocence).

The next paragraph, however, makes it clear that the "prompt testing" demanded by the the authors of Actual Innocence is only the prompt testing of crime-scene and victim samples. They take no position (at least not here) on the value of "prompt testing" of arrestees, and one of them, Peter Neufeld, told Mr. Liptak that all three "never endorsed arrestee databases.” [1]

Given that Actual Innocence did not explicitly endorse arrestee sampling, how should the opinion have used some of the book's prose? How about a parenthetical phrase -- the darling of the ever punctilious Bluebook (the legal style manual so dear to student law review editors). The paragraph should have read:

Finally, in the interests of justice, the identification of an arrestee as the perpetrator of some heinous crime may have the salutary effect of freeing a person wrongfully imprisoned for the same offense. "[P]rompt [DNA] testing . . . would speed up apprehension of criminals before they commit additional crimes, and prevent the grotesque detention of . . . innocent people." J. Dwyer, P. Neufeld, & B. Scheck, Actual Innocence 245 (2000) (referring to the importance of prompt testing of crime-scene samples).

II. Arrestee DNA and Resolving Unsolved Cases

So I have to agree with Mr. Liptak. The citation lacks (parenthetical) punctility. But is it not at least consistent with the logic of the paragraphs from which it quotes? To achieve maximum exonerations and to avoid apprehension of the wrong people for unsolved crimes, should not DNA profiles from arrestees go into the databases for subsequent searches against crime-scene samples along with those of convicted offenders?

Well, not necessarily. It could be argued that arrestees are just gravy -- that if there is not enough money to fund lavishly three sources of samples, crime-scene samples and convicted-offender samples should come first. Interestingly, a brief of "14 Scholars of Forensic Evidence" (that is, law professors) filed on behalf of Mr. King claimed that arrestee sampling exacerbates backlogs and is not necessary to exonerate the innocent. Should a court therefore hold a DNA database law unconstitutional on the ground that a state should be spending more money on one part of the system than another? What if the state then turns around and spends more money on the parts the court, guided by this scholarship, thought more important? Should the court function as a kind of equitable master periodically reviewing funding decisions, average processing times for crime-scene samples, and the number and training of the investigators collecting DNA from crime-scenes? These things, among others, bear on how well the system works in identifying true offenders, but do they have the necessary clarity and susceptibility to judicial supervision to be dividing lines of constitutional magnitude?

I doubt it. As long the legislature pursues a reasonable combination of the essentials required of a database for investigating offenses in which DNA evidence can be recovered, courts should not vitiate the legislative judgment on the ground that the system could be realigned to be still more effective. Courts faithfully interpret the mandate of the Fourth Amendment when they uphold measures that are not unreasonably invasive of personal security. That these measures could be made more efficacious is important in formulating and implementing public policy, but it cannot be the basis for a ruling of unconstitutionality.

III. Dr. Mengele Comes to the Databanks

The Court's reliance on Actual Innocence prompts a final thought (for now).  Immediately after the paragraphs that I quoted in full, the authors add that

No matter whose privacy is at stake, data banks full of genetic information can be dangerous, particularly when the samples disproportionately come from black and brown men. Modern-day Mengeles, in search of evidence for master-race theories, would find data bases irresistible in the quest for a genetic link to criminality. A simple reform could slam the door on such eugenic inquiries. Any leftover blood, tissue, or DNA should be discarded once a profile has been stored.

This passage is not a model of clarity. I do not say this because I disagree with the conclusion. On the contrary, I have written that the case for indefinite sample retention is weak. [3] My concern is with the poorly woven strands of the argument.

Let us remember who Josef Mengele was and what he did. As an SS physician, assigned to the Auschwitz concentration camp, "with full license to maim or kill his subjects, Mengele performed a broad range of agonizing and often lethal experiments with Jewish and Roma ('Gypsy') twins, most of them children. ... Mengele firmly endorsed the doctrine of National Socialist racial theory and engaged in a wide spectrum of experiments which aimed to illustrate the lack of resistance among Jews or Roma to various diseases. He also attempted to demonstrate the 'degeneration' of Jewish and 'Gypsy' blood through the documentation of physical oddities and the collection and harvesting of tissue samples and body parts. Many of his 'test subjects' died as a result of the experimentation or were murdered in order to facilitate post-mortem examination." [4]

What would Mengele do with the DNA samples in a law enforcement repository (were he given access to one, in violation of the laws governing many of them)? Look for some association between the genes of all the individuals in the database and a comparison group of presumably law-abiding individuals? Suppose, contrary to previous research, he found some plausible relationship between a genotype and criminality. Might he then discover that this genotype exists in a larger proportion of the "black and brown men" convicted of crimes than in the men of his favored "master race"? How does the disproportionate representation of the former groups in the criminal databanks make this outcome more likely -- if that is what "particularly dangerous" refers to?

This is not to deny that genetic arguments for racism -- no matter how contrived -- will appeal to some people. There are, by way of comparison, Holocaust deniers and plenty of people willing to dismiss evolution as merely "a theory." But the people who would believe the resurrected Mengele of Actual Innocence hardly need more pseudo-science to convince them that "black and brown men" are genetically inferior. That DNA samples should be destroyed because a Mengele could use them to demonstrate the racial inferiority of some groups seems like one of weakest arguments for sample destruction.

In any event, there is a real connection between Mengele and DNA evidence. Mengele eluded capture for 34 years, from 1945 until his death in 1979. He was buried as "Wolfgang Gerhard" near Sao Pãolo. In 1985, the corpse was exhumed and the remains identified as Mengele. In 1992, DNA evidence confirmed this conclusion. [4]

References

1. Adam Liptak, Cited by a Justice, But Feeling Less Than Honored, N.Y. Times, June 11, 2013, at A15.

2. Barry Scheck, Peter Neufeld, & Jim Dwyer, Actual Innocence: Five Days to Execution, and Other Dispatches From the Wrongly Convicted (2000).

3. David H. Kaye, Behavioral Genetics Research and Criminal DNA Databanks, 69 Law & Contemp. Probs. 259 (2006).

4. United States Holocaust Memorial Museum, Holocaust Encyclopedia: Josef Mengele http://www.ushmm.org/wlc/en/?ModuleId=10005143. Accessed June 13, 2013.

Hot Off the Presses: Chimeric Criminals

Nearly two years ago, I raised a question about the depth of the documentation and analysis in the book Genetic Justice by Sheldon Krimsky and Tania Simoncelli. A discussion of chimerism and the threat it supposedly poses to DNA exonerations prompted the following debunking essay: Chimeric Criminals, Minnesota Journal of Law, Science, and Technology, Vol. 14, No. 1, Winter 2013, pp. 1-9. It is available from SSRN or on the review's website.

Abstract: According to some commentators, an obscure genetic condition known as chimerism “could undermine the very basis of the forensic DNA system” and force a reconsideration of “the entire project of forensic DNA.” This conclusion is as unfounded as it is unnerving. Chimerism is a consideration in, but not a real obstacle to DNA identification. This essay explains why.