Likelihoods, Paternity Probabilities, and the Presumption of Innocence in People v. Gonis

In People v. Gonis, 1/ Illinois prosecuted Kenneth Gonis for sexual penetration with his daughter, T.G., when she was 16 years old. T.G. had two children. The first, J.G., was born when she was 17 years old; A.G. was born two years later. To investigate the sexual assault charge, the Illinois State Police Joliet laboratory conducted DNA tests of Gonis, T.G, and the two children. The lab sent the results to the Northeastern Illinois Regional Crime Laboratory for interpretation. That laboratory’s DNA technical leader, Kenneth Pfoser, “entered the DNA profiles into a computer containing a statistical calculator.” He learned that

• “at least 99.9999% of the North American Caucasian/White men would be excluded as being the biological father of [J.G. and A.G.]”;
• the “paternity index” with respect to J.G. was about 195,000,000 and with respect to A.G., it was 26,000,000; and
• “the probability that defendant was the biological father of J.G. and A.G. was 99.9999%.”

In a bench trial before Judge Lance Peterson, the court admitted these findings and convicted Gonis. On appeal, Gonis argued that the trial court erred in denying a pretrial motion to exclude the DNA test results. In an opinion written by Justice Daniel Schmidt, Illinois’ intermediate court of appeals described the motion as asserting only that

[T]he tests were inconsistent with the presumption of innocence because a statistical formula used in the testing assumed a prior probability of paternity. Specifically, the motion alleged:

Assuming that the Northeastern Illinois Regional Crime Laboratory tested the DNA sample using widely accepted practices in the scientific community, said testing was conducted using a statistical mathematical formula. These formulae, as their basis, include a component to determine paternity which by its nature ‘assumes’ that sexual intercourse has in fact taken place.

In other words,

The motion alleged that to allow such paternity test results would violate the presumption of innocence because “the state would be allowed to introduce statistical evidence presuming sexual intercourse, in order to prove an act of sexual intercourse.”

The argument is fallacious for three reasons. First, the probability pertains to the chance that the child was conceived by the mother and the accused man. Conception—the fertilization of an ovum—can occur without penetration.  In the hearing on the motion to exclude, the technical leader referred to artificial insemination, but as insemination and hence pregnancy can occur without penetration by natural mechanisms as well.

Second, even if conception were not merely improbable, but impossible without penetration, it would not follow that a probability of paternity presumes penetration. After all, a probability is not a certainty. To say that an electron has a probability Ψ*ΨdV of being located in a small volume dV is not to presume that the electron is actually located there. To say that the probability of an extended trade war between the U.S. and China is 0.5 (or some other number less than 1) does not presume that this event will occur. That the paternity probability for the defendant is 0.5 (or 0.99999, or any other other number less than 1) also does not presume that the defendant truly is the source of the fertilizing spermatazoon.

Finally, the evidentiary aspect of the presumption of innocence merely directs the judge or jury not to use the fact of the indictment as evidence of guilt. The probabilities in question do not change depending on whether or not a man is indicted.

The opinion in Gonis seems to rely on the activity-level possibility of artificial insemination to reject the defendant's presumption-of-innocence objection. It also comes close to recognizing the second rejoinder, for it states that "Logically, since Bayes's Theorem allowed for the possibility that defendant may not be the father of T.G.'s children, it did not assume that defendant necessarily had sexual intercourse with T.G."

But the court thought that the details of Bayes' Theorem rather than the very definition of probability made the computation compatible with the presumption of innocence. The opinion states that

Pfoser testified that Bayes's Theorem was a likelihood ratio based on two competing hypotheses: (1) defendant was the father, or (2) a random, unrelated individual was the father. Pfoser stated that Bayes's Theorem took “the assumed probability that the person in question is the father of the child” and divided it “by the probability that some unrelated person within the same race group in the general population is the father of the child.” Thus, Pfoser's testimony indicated that Bayes's Theorem posited that either defendant or an individual other than defendant could have been the father of T.G.'s children. Logically, since Bayes's Theorem allowed for the possibility that defendant may not be the father of T.G.'s children, it did not assume that defendant necessarily had sexual intercourse with T.G.

Although a likelihood ratio appears in Bayes' rule, that is not all there is to it, and the description of how the rule works is garbled. The probability that the defendant is the father is not obtained by dividing a probability that he is the father by the probability that an unrelated man is the father. If the expert knew the probability that an unrelated man is the father (and no other alternatives to the defendant's paternity were worthy of consideration), Bayes' rule would be surperfluous. The probability not assigned to the random man is the defendant's probability, so if we have the random-man probability, all we need to do is to subtract it from 1. What remains is the defendant's probability.

The technical leader used Bayes' rule because he did not know the probability that a random man was the father. Let’s look at his explanation of the computation, as presented by the appellate court. The court starts by recounting that

Pfoser testified that DNA paternity testing had three components. The first component of the test involved an exclusion analysis where Pfoser entered the DNA profiles into a computer containing a statistical calculator. If there were any inconsistencies between the alleged father and the child, the computer would give a result of “0” for paternity index.

Apparently, each child shared at least one allele per locus with the defendant, so the computer program did not report an approximate probability of zero, 2/ and the opinion continued:

The next stage involved the calculation of the paternity index, which was a formula used to determine “the likelihood that the assumed alleged father in question is in fact a father as opposed to a random individual that's unrelated in the general population.”

If "likelihood" has the technical meaning of statistical "support" for a hypothesis, this statement could be literally true. But if "likelihood" means probability, as the court evidently and understandably thought, then the explanation is either meaningless or misleading. There is no probability that the defendant is the father "as opposed to a random individual that's unrelated." There is a probability that the defendant is the father (as opposed to everyone else in the population, given all the evidence in the case). And, the paternity index is not even a probability, let alone that one. It is a ratio of two different probabilities. As the court wrote, "the paternity index is the ratio of 'the probability of the alleged father transmitting the alleles and the probability of selecting these alleles at random from the gene pool.' ... (quoting Ivey v. Commonwealth, 486 S.W.3d 846, 851 (Ky. 2016) (quoting D.H. Kaye, The Probability of an Ultimate Issue: The Strange Cases of Paternity Testing, 75 Iowa L. Rev 75, 89 (1989))."

The important aspect of the paternity index is that it is a likelihood ratio that expresses the support that the reported DNA profiles of the mother-child-defendant trio provide (if correctly determined) for the hypothesis that the defendant is the biological father relative to the hypothesis that an unrelated man is the father. The idea is that if the profiles are some number L times more probable under one hypothesis than the other, then they support that hypothesis L times more than they support the alternative. This ratio does not assume that one hypothesis is true and the other false. Rather, it treats both hypotheses as equally worthy of consideration and addresses the probability of the evidence when each one is considered. Thus, the use of the ratio to describe the strength of the evidence for the better supported hypothesis does not conflict with the presumption of innocence. Had the expert simply given the paternity index and spoken of relative support, the defendant's objection would have had even less traction that it did.

But the technical leader did not describe the paternity index in this “likelihoodist” way. Instead, to quote from the opinion,

Pfoser testified that the third component of DNA paternity testing converted the paternity index into a probability of paternity percentage using a statistical, mathematical formula called “Bayes' Theorem.” Pfoser explained:

“Bayes' Theorem is essentially a basis for a likelihood ratio. Like I kind of described before, you're basing it on two conflicting hypotheses or two conflicting assumptions. One is that the individual in question is in fact the father as opposed to a completely random unrelated individual could be the father.”

Pfoser further explained:

“[S]o you're taking two, essentially two, calculations, one calculation is * * * the prior probability or the assumed probability that the person in question is the father of the child and that is divided by the probability that some unrelated person within the same race group in the general population is the father of the child.”

And     "Pfonis testified that the prior probability of paternity was set at 50%.”

As the earlier remarks on Bayes' rule indicate, this explanation of Bayes' rule cries out for corrections at every turn, but now I will just focus on the last sentence because the concept of a prior probability is what triggers worries about the presumption of innocence. The idea of a prior probability is intuitive but not easily mapped onto the legal setting. If I want to infer whether a furry animal that I glimpsed running outside my window is a squirrel (as opposed to a groundhog, a rabbit, a chipmunk, a possum, a cat, a skunk, a bear, or any other furry critter in this neck of the woods), I can start by asking how often various creatures go by. Based on my past observations, I would order the possibilities as squirrel, chipmunk, rabbit, groundhog, cat, and so on. If squirrels account for half of the past sightings, I might select 50% for the probability of a squirrel. This is my prior probability.

Now I think about the details of what I saw in the periphery of my vision. How small was it? What color? Did it seem to have short legs? Was it scurrying or hopping? To the extent that the set of characteristics I was able to discern are more probable for squirrels than for other creatures, I should adjust my probability upwards to arrive at my posterior probability.

Bayes’ rule is a prescription for making the adjustment. It instructs us to multiply the prior odds by the likelihood ratio. Then, voilà, the posterior odds emerge. Suppose my likelihood ratio is 3. I think the characteristics I perceived are 3 times as probable when a squirrel zips by than when the average non-squirrel does. 3/ If the prior probability is ½, the prior odds are 1 to 1, and the posterior odds are 3 × 1:1 = 3:1. Odds of 3:1 correspond to a posterior probability of 3/(3+1) = 3/4. Following Bayes’ rule, I moved from a prior probability of ½ to a posterior of 3/4.

The expert in Gonis arrived at his posterior probability of paternity by making up a set of prior odds — he chose 1:1 — for defendant’s paternity and multiplying them by the paternity index. This looks like a Bayesian calculation. 4/ But in the squirrel-sighting case, there was an empirical basis for the prior odds. I know something about the animals in my neighborhood. The DNA technical leader apparently offered no such justification for his choice of the same number. And how could he? His expertise does not extend to the sexual and criminal conduct of the defendant and everyone else in the male population. The judge or jury, not the DNA profiling expert, is supposed to consider the nongenetic evidence in the case and to rely on its general background information in processing the totality of the evidence in the case to reach its best verdict.

In Gonis, trial judge, who was the factfinder in the case, was explicit about why he found the prior probability of ½ to be acceptable:

The court noted that the cases cited by the State explained why “the .5 number presumption that they start off with is actually just a truly neutral number. It assumes the same likelihood that the defendant was not the father of the child as it does that he would be the father of the child.

This rationale is specious. For a Bayesian, starting with a probability of ½ amounts to believing, before learning about the DNA profiles, that the defendant owns half the probability and that the other half is distributed across every else in the population. Maybe the other evidence in the case would justify that belief, but it hardly seems “neutral” toward the defendant. It treats him very differently from every other man in the population. The more “neutral” position might be to assign the same per capita probability to everyone, including the defendant, and then make adjustments according to the specifics of the case.

The appellate court took no stand on whether the trial court’s conception of neutrality was scientifically or legally tenable. Construing the defendant’s objection narrowly, the court did "not reach the issue of whether a 50% prior probability is a neutral number."

A bona fide Bayesian procedure would be to display the posterior probability for many values of the prior probability. This “variable prior odds approach” avoids the need for the expert to tell the judge or jury which prior probability is correct. 5/

That said, the uncontested likelihood ratios in Gonis, as Justice Schmidt observed, would swamp most prior probabilities. Even if we regarded all the men in the Chicago metropolitan area as equally likely, a priori, to have fathered the two children, the posterior odds of paternity still would be substantial. There are fewer than five million men (of all ages) living in the metropolitan area. So the per capita prior odds are 1:5 million. For the likelihood ratios of 195 million and 26 million, the posterior odds would be more than 39:1 for the paternity of J.G. and 5:1 for the paternity of A.G.

NOTES

1. 2018 IL App (3d) 160166, No. 3-16-0166, 2018 WL 6582850 (Ill. App. Ct. Dec. 13, 2018).
2. Particularly at a single locus, an exclusion does not mean that the probability of paternity is strictly zero. Mutations at some of the STR loci are known to occur at nonzero rates.
3. The phrasing about an "average non-squirrel" is imprecise. There are n+1 mutually exclusive hypotheses H₀, H₁, H₂, ..., H_n, about the animal. Each H_j has a prior probability Pr(H_j) and a likelihood Pr(E|H_j). Let H₀ be the squirrel hypothesis. The appropriate factor for the multiplication of the prior odds is the squirrel likelihood Pr(E|H₀) divided by a weighted average of the other likelihoods. The weight for each non-squirrel hypothesis H_j (j = 1, .., n) is my prior probability on that hypothesis renormalized to reflect that it is conditional on ~H₀. In other words, the Bayes factor is Pr(E|H₀) × [1−Pr(H₀)] divided by Pr(H₁) × Pr(E|H₁) + ... + Pr(H_n) × Pr(E|H_n).
4. By limiting attention to an unrelated man as the only possible alternative, the technical leader was ignoring the terms in the denominator of the Bayes factor for possible related men. See supra note 3. As a result, the Bayesian interpretation he provided was not strictly correct.
5. For discussions of such proposals and their reception in court and in the scholarly literature, see David H. Kaye, David E. Bernstein & Jennifer Mnookin, The New Wigmore on Evidence: Expert Evidence ch. 15 (2d ed. 2011) (updated annually).

Last updated: 16 May 2019, 1:20 PM

State v. Sharpe: What If Other Forensic Science Methods Were Given the Same Scrutiny as Polygraph Evidence?

Earlier this year, the Alaska Supreme Court adopted the majority rule excluding polygraph evidence. That outcome is not surprising, but how the court reached this result merits attention. The court's careful opinion varies from the insightful to the misconceived. If some of the same reasoning were applied to other parts of forensic science, judicial opinions would improve. But one part of the court's analysis of "error rates" cannot be reconciled with Daubert and reproduces an error exposed in the legal and statistical literature over thirty years ago.

Chief Justice Craig Stowers' analysis for a unanimous court begins with the somewhat technical legal issue of the standard of review on appeal. Does the appellate court have to defer to the trial judge's determination of whether the evidence constitutes "scientific knowledge" within the meaning of Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993), unless that determination is an "abuse of discretion"? Or does the appellate court review the record and literature for itself in a "de novo" review? Before the several cases decided along with State v. Sharpe, 435 P.3d 887 (Alas. 2019), Alaska, like the federal courts, used the former standard.

In Sharpe, however, the court overruled State v. Coon, 974 P.2d 386 (Alaska 1999), to adopt the minority rule of de novo review. I think that is the right result. For one thing (that the court does not discuss), the demeanor of the expert witnesses in a pretrial  hearing on the state of the science is less important than the expert's articulated reasoning and the pertinent studies. The latter can be assessed almost as well on a cold record as they can be after listening the witnesses.

Testing the Technique

Applying the de novo standard, the Sharpe opinion moves through the usual Daubert factors. To begin with, it concludes that the testing of "the psychological hypotheses that serve as the underlying premise of polygraph testing" is insufficient and that some of them "may not be readily testable." The problem here seems to be that it is hard to know from low-stakes experiments whether "a truthful person will respond more strongly to the comparison questions [and] a deceptive person will have a stronger reaction to the relevant questions," while "field studies have difficulties establishing the 'ground truth' of whether an examined person was actually lying." Hence, "this factor weighs decidedly against admitting polygraph testimony as scientific evidence."

The court did not apply so exacting an analysis in Coon. There, it upheld a determination that voice spectrographic identification of speakers was scientifically valid without discussing if or how the physiological assumptions of that technique had been tested. In Sharpe, the court observed that "a 2003 review of the scientific evidence on polygraphy by the National Research Council concluded that '[p]olygraph research has not developed and tested theories of the underlying factors that produce the observed responses.'" In Coon, it ignored a 1979 NRC report that stated that spectrographic voice identification "lacks a solid theoretical basis" and that its most crucial assumption had not been adequately tested. In Sharpe, the court agonized over the limited ability of laboratory studies to replicate real-world conditions. In Coon, it paid no attention to the difficulties in simulating factors of ambient noise, other sounds, transmission channels, and mismatched recording conditions.

Peer Review and Publication

The Sharpe court gave "little weight" to the existence of a substantial body of peer-reviewed publications on polygraphy. Considering the tendency of some proponents of criminalistics methods to provide long lists of publications as if the sheer number and age of the writings prove scientific validity, this part of the opinion is refreshing. The court explained that "the mere fact of publication in a peer-reviewed journal is not itself probative of a technique’s validity." "Most of the studies cited by Dr. Raskin in support of the technique are from the 1980s and 1990s, with some dated as far back as the late 1970s." "Thus, although studies regarding CQT polygraphy have been published in peer-reviewed journals, it does not appear that this has resulted in the kind of refinement and development that makes publication and peer review relevant to a Daubert analysis."

Error Rates

The court's analysis of error rates is less perceptive. It begins as follows:

[T]he studies cited by Dr. Raskin showed an accuracy rate of 89% to 98%, while those cited by Dr. Iacono had accuracy rates from 51% to 98%, with an average of 71%. Dr. Raskin estimated that the overall accuracy rate of CQT polygraph testing was around 90%. 

Dr. David Raskin, a professor emeritus of psychology at the University of Utah, who testified in support of the validity of polygraph procedure, is well aware that it takes two probabilities or statistics--sensitivity and specificity--to define the accuracy of a test with a yes-or-no outcome. Dr. William Iocono, a psychology professor at the University of Minnesota, who testified for the state, also knows this. Sensitivity is the probability of a positive result (here, a finding that the subject is consciously lying) given that the condition (conscious deception) is really present. It can be abbreviated as P(+ | D). Specificity is the probability of a negative result (here, a finding that the subject is not consciously lying) given that the condition is not present: P(– | ~D). A highly accurate test is both very sensitive and very specific. When confronted with conscious deception, the examiner almost always detects it (high sensitivity); when confronted with truthful responses, the examiner rarely diagnoses deception (high specificity). High sensitivity corresponds to a small false-negative error probability (because P(– | D) + P(+ | D) = 1); high specificity corresponds to a low false-positive probability (because P(+ | ~D) + P(– | ~D) = 1).

I am not sure what "the overall accuracy rate" means here, but to try to unpack the court's reasoning, I am going to assume that the best studies established the figure of "around 90%" for both sensitivity and specificity. It follows that both the false-negative and the false-positive error rate are around 10%. Are those error probabilities so high that they counsel against admission under Daubert? I would argue that they are sufficient for "evidentiary reliability" as defined in Daubert -- if the evidence can be presented so that they jury gives the polygraph findings the limited weight they deserve. Some lawyers and scientists would disagree and say that higher accuracy than "about 90%" is necessary. Statistically, the best way to express the lawyer's concept of probative value of a binary test finding is with the likelihood ratio L = P(+ | D) / P(– | D) for a positive finding or L = P(– | ~D) / P(+ | ~D) for a negative finding. In Sharpe and its companion cases, the findings were negative -- no deception -- with L = 90% / 10% = 9. In other words, the report of no deception was nine times more probable when the subject is truthful than when the subject is lying. A diagnosing physician might want to order a test for cancer that is this discerning, even though it would be far from conclusive.

Rather than conclude that 90% accuracy is mildly supportive of validity, the Sharpe court took a different tack. First, it pointed to Dr. Iocono's criticisms that the laboratory experiments lacked realism and that the field studies suffered from selection bias and inadequate knowledge of "ground truth." Those are important points. If the studies do not apply to criminal cases or do no prove what they are supposed, then who cares about the numbers they generate? To that extent, the court is again saying that the method has not been adequately tested and is difficult to test.

However, the court's discussion of error rates did not stop here. The opinion muddied the waters by bringing up "base rates" as a necessary component of probative value. The opinion reads:

[T]he empirical basis for polygraph examinations suffers from another fault: the lack of a reliable “base rate.” In the three cases currently before this court, each defendant was said to have passed his polygraph test; the relevant question for the factfinder is whether, given this fact, the defendant was likely truthful or whether the test was a false negative. To determine this likelihood, more information is required; specifically, information about the base rate of deceptive and truthful subjects.
The lack of a reliable base rate estimate was the underlying reason for the Connecticut Supreme Court upholding its traditional per se ban on admitting polygraph evidence in State v. Porter. Noting “wide disagreement” about the accuracy rates for “a well run polygraph exam,” the court decided that, even if the estimates of polygraph proponents were accepted, the technique would still be “of questionable validity.” ... The court ... reasoned that, even if a test is accurate, its probative value as scientific evidence depends on its “predictive value”—the likelihood “that a person really is lying given that the polygraph labels the subject as deceptive” and the likelihood “that a subject really is truthful given that the polygraph labels the subject as not deceptive.” This predictive value, the court explained, depends not only on the accuracy of the test but also “on the ‘base rate’ of deceptiveness among the people tested by the polygraph.” Because the Porter court found a “complete absence of reliable data on base rates,” it concluded that it had no possible way of assessing the test’s probative value. With that in mind, the court concluded that even if polygraph evidence satisfies the Daubert standard, which it assumed without deciding, the probative value of such evidence is very low and substantially outweighed by its prejudicial effects.

As in Porter, the record before us is devoid of reliable data about the base rate of deceptiveness among polygraph examinees outside of lab tests; we also have not found such data in academic literature. Absent some reliable estimate of this base rate there is no way to estimate the reliability of polygraph results, and thus no way to determine whether any particular accuracy rate is acceptable. We conclude that the superior court clearly erred in finding the error rate of CQT polygraph testing to be “sufficiently reliable.” Accordingly, this factor weighs against admitting polygraph evidence.

If the error-rate factor of Daubert "weighs against admitting ... evidence" unless there is a “reliable estimate of the base rate,” then back in Coons, the Alaska Supreme Court was wrong to rely on claims of small error rates to uphold the admission of voice spectrographic identification. There was no testimony, let alone scientific knowledge, of the "base rate" of matching spectrographs in the relevant suspect population. That also was true of the case the Supreme Court cited when it invoked "error rates" as a factor in Daubert. United States v. Smith, 869 F. 2d 348, 353-354 (7the Cir. 1989), listed studies such as one in which "the error rate for false identifications was 2.4% and the error rate for false eliminations was about 6%." It did not mention "base rates" or "predictive value" -- terms that are defined the box below:

Terminology for Accuracy and Probative Value of Tests that Classify Things into Two Categories

---

Operating Characteristics (How accurate is the test itself?)
● Sensitivity P(+ | D), probability of a positive finding (e.g., "the suspect is lying") given that the condition (e.g., conscious deception) is present
● False negative probability P(– | D) = 1 - P(+ | D) = 1 - sensitivity
● Specificity P(– | ~D), probability of a negative finding (e.g., "the subject is not lying") given that the condition is not present
● False positive probability P(+ | ~D) = 1 – P(– | ~D) = 1 – specificity

---

Efficacy (How dispositive are the test findings?)
● Prevalence or base rate F(D), relative frequency of the condition in the group being tested
● Prior odds Odds(D), odds of the condition in an individual being tested
● Positive predictive value PPV = P(D | +), probability of the condition given the positive test finding
● Negative predictive value: NPV = P(~D | –), probability of the absence of the condition given the negative test finding
● Posterior odds Odds(D | +) or Odds(D| –), odds of the condition given the test finding

---

Probative value (How much relative support does the result provide?)
Likelihood ratio L (How many times more probable is the test result for the different possibilities?)
● For a positive finding, L_pos = P(+ | D) / P(– | D)
● For a negative finding, L_neg = P(– | ~D) / P(+ | ~D)

---

Bayes rule (How much does the test finding change the prior odds?)
● For a positive finding, Odds(D | +) = L_pos × Odds(D)
● For a negative finding, Odds(~D | -) = L_neg × Odds(~D)

The opinion in Sharpe has confused probative value -- the extent to which evidence tends to prove the proposition that it is offered to prove -- with the probability that the proposition is true. The latter is surely what the jury wants to know, but it gets to that probability by considering all the evidence that supports or undermines the proposition in question. The likelihood ratio (rather than the "predictive value") for an item of evidence expresses its probative value. The error-rate factor in Daubert requires courts to ask whether false-positive and false-negative error probabilities are so large that the test has too little probative value to justify its admission as scientific evidence.

Scientific evidence need not be conclusive to be valid and admissible. If only 1 out of 91 polygraphed people would lie -- that is the base rate -- and if no other evidence that the defendant would lie to the polygrapher were available, then the prior odds that the defendant was truthful arguably would be 1 to 90. The posterior odds then would still be low—namely, 9 to 90, for a “predictive value” or posterior probability of only 9/99 = 1/11. On the other hand, if the base rate and the prior odds were higher, say 1/2 and 1 to 1, respectively, then the predictive value and posterior probability would be 9/10. But in both cases, the finding is probative and worth knowing.

In sum, whether the base rate of lying among criminal suspects in general is high or low does not alter the extent to which the evidence tends to prove that the suspect in a particular case is or is not lying. The odds of lying change in the same ratio. A test that produces results that are strongly indicative of the presence or absence of a condition—compared to what was known beforehand—is a valid classifier regardless of the base rate for the condition in some population (or the prior probability in the case at bar).

Controlling Standards

Daubert spoke of “the existence and maintenance of standards controlling the technique’s operation.” Courts tend to cite any kind of a standard (such as one prescribing the educational qualifications of a practitioner) as if it controls how the test is to be performed. The Sharpe court noted that "many states ... have statutes governing polygraph test administration, examinees’ privacy rights, and licensing of examiners," but it also pointed out that "the formulation and ordering of questions, the conducting of the pretest interview, the choice of scoring system, and the evaluation of the examinee’s demeanor leave much to the examiner’s discretion." Consequently, it concluded that "the lack of clear controlling standards for CQT administration weighs against its admissibility."

General Acceptance

Among other things, the court wrote that in light of "the apparently lackluster support for the technique outside the community of practicing polygraph examiners, we conclude that this factor also weighs against admitting polygraph evidence." In contrast, when "outside" bodies review identification methods in forensic science, practitioners invariably complain (if the reviews are unflattering) that they were not adequately represented in the process.

Financial Interest

The factors enumerated in Daubert are not exhaustive. Going one step beyond them, the Sharpe court expressed concern over “the danger of a hidden litigation motive” behind research. It cautioned that "[m]any of the studies cited as approving polygraph testing as scientifically valid were performed by ... practicing examiners, and a number of the studies were published in polygraph industry publications." This, too, has implications for much of the research in other areas of forensic science.

FURTHER READING

• David H. Kaye, The Validity of Tests: Caveant Omnes, 27 Jurimetrics J. 349 (1987)
• David H. Kaye, The Polygraph and the PVP, 2 Statistical Sci. 223 (1987)
• David H. Kaye, Apples and Oranges: Confidence Coefficients and the Burden of Persuasion, 73 Cornell L. Rev. 54 (1987)
• David H. Kaye, Review-essay, Digging into the Foundations of Evidence Law, 116 Mich. L. Rev. 915-34 (2017)

The Latest Guidance on Expert Reports from England's Forensic Science Regulator

This week, the Forensic Science Regulator for England and Wales issued a third version of guidance on the content of expert reports for criminal cases. Forensic Science Regulator, Guidance on the Content of Reports Issued by Expert Witnesses in the Criminal Justice System in England and Wales, Apr. 17, 2019.

 These reports are required "when the witness will, either in the report or in testimony at court, provide evidence of opinion." Some items are rather mundane -- for instance, a report must be signed by its author. But several parts of the guidance are noteworthy:

• The author must certify that the report is truthful and, if introduced into evidence, wilful falsehoods would be criminal. (If included in a report in the United States, such declarations could be significant on the question of whether the report is "testimonial" hearsay that would trigger Con frontation Clause protections.)
• The report must state that "the witness will inform all parties and where appropriate the court in the event that his/her opinion changes on any material issues."
• "The witness must state any limitations to their experience and whether any particular issue, which is discussed in the statement (or on which they had been requested to discuss), falls outside their expertise."
• Consideration should be given to stating "[w]hether the witness tends to work for the prosecution, defence or both" amd "[w]hether the witness has been the subject of criticism and is aware of that criticism."
• Qualifications must be impartially listed, and "[t]he membership of an organisation which is obtained solely by applying for membership (and perhaps, payment of a fee) is not a qualification," and presenting such memberships as proof of expertise "is, at the least, irrelevant and may be misleading."
• "Where the witness is aware that reputable experts in the field would hold a range of opinions on the issue under consideration, the report must set out what that range is and justify why the witness’s opinion falls at a particular point within that range."
• The report should delineate "[a]ny results obtained which would undermine the opinion expressed"; [a]ny work, or other information, the witness is aware of which would undermine the opinion"; "[a]ny limitations to the analytical approach adopted"; and "[t]he uncertainty of measurement associated with the methods employed and the manner in which this has been addressed."

The document also advises experts "to avoid the term 'consistent'" because it "has been criticised by the Courts. It can sound significant but, in reality, mean little." On the other hand, "'not consistent' is usually clear and does not give rise to the same issues."

The guidance on "statistics" asks the expert to keep "detailed consideration" of "complex statistical analysis" out of the report but to make it available some other way. Moreover, it seems to place all "Bayesian inference" into this category! In full, the guidance is that

The Courts have, on a number of occasions, expressed concern about the evidence of complex statistical analysis (e.g. Bayesian inference) being discussed before the jury.
It is therefore advisable to avoid incorporating detailed consideration of such analysis in reports. The statement should set out the model employed and the conclusions drawn.
The details of the analysis should be available for disclosure.

This is a matter that the Regulator should reconsider in the next revision of the guidance -- along with the suggestion that "reference to statistical significance" is an "appropriate term" for explaining "how safe or unsafe" is "an inference from any findings." The phrase "statistical significance" has not fared well in recent policy pronouncements from statisticians.

Questions on Kentucky's Rapid DNA Program for Sex Crime Investigations

An Associated Press report being picked up in newspapers such as the Washington Post and USA Today gives the impression that Kentucky is on the verge of replacing conventional DNA analysis of rape kits with a "rapid DNA" instrument. (1) The article states that

The equipment, known as the ANDE Rapid DNA system, can generate DNA identification from forensic samples in less than two hours, Kentucky law enforcement officials said. ...
“If you are a sexual predator in ... Kentucky, we’re going to come after you,” Kentucky State Police Commissioner Richard Sanders said at a press conference. “And we now have new equipment to come after you quicker. We now have another way to identify who you are.” ...
A 2017 federal law authorized the FBI director to issue standards and procedures for rapid DNA analysis, described as a fully automated process, according to an FBI website.The ANDE system received FBI approval last year for use in accredited laboratories, the company’s website says. A cheek swab is inserted into an ANDE device roughly the size of an office photocopier and results appear within hours. By comparison, DNA samples sent to conventional crime labs can take months to analyze.

The FBI has indeed approved the ANDE 6C Rapid DNA System for use in accredited laboratories -- but only with "[k]nown reference buccal DNA sample[s]." (2) In other words, an STR profile from a cheek swab from a known individual can be added to or checked against a database that is part of the Combined DNA Index System (CODIS). The profile from the male fraction of a sample in sexual assault case ascertained via a rapid DNA machine cannot be.

"The analysis of forensic samples by a Rapid DNA system is not ... permitted to be uploaded [or] searched in CODIS at this time." -- FBI (2)

So what does Kentucky propose to do with the rapid profiles? In a press release, Governor Matt Bevin enthused that they "can help us to identify an assailant in a matter of hours – allowing us to focus the investigations of sexual crimes more quickly than ever before." (3) Of course, it does not take months for a "conventional crime lab" to perform the steps that generate an electropherogram in the ordinary way and to interpret the data, but rapid DNA analysis is less labor intensive. That is a good thing, but how will Kentucky "identify an assailant in a matter of hours" from the rapid profile without using any CODIS database?

Even without a database, rapid DNA can help if there already is a suspect. It might further implicate the suspect, or it might exonerate him, propelling the investigation in a new direction. But that seems to be an afterthought in Kentucky's description of "how rapid DNA will be used." The press release (3) states that

By matching the DNA from the crime to an I.D. in a criminal database, this could give immediate data that focuses the investigation and get [sic] a rapist off the streets. By collecting an additional sample from the victim – just one more swab – a perpetrator’s identity can be matched within two hours."
How Rapid DNA will be used for sexual assault investigations:
● If a victim presents at a hospital or police station, and agrees to having DNA specimens taken for testing, a DNA ID can be generated to inform the investigation. This is a separate process from the Sexual Assault Kit but can be taken during the sexual assault exam.
● Once the male DNA is separated from the female DNA, an operator can test the male DNA for a definitive DNA Identification pattern. This process takes less than two hours.
● If a DNA ID is generated, it can be compared to criminal databases. If a match is found, this will provide important information to law enforcement about the attacker. It may exonerate an innocent suspect. It may connect other crimes to this attack.

The release does not explain how this "separate process" can "match[] the DNA from the crime to an I.D. in a criminal database" when the FBI has yet to approve rapid profiles for CODIS database searches. Another Kentucky State Police document (4) indicates that Kentucky somehow has been able to perform database searches with rapid DNA profiles:

The state of Kentucky has been running a pilot program for several months, using the ANDE Rapid DNA system to test samples from rape cases. It has proved invaluable in several cases where there is a match to DNA in a criminal database and to others in which a specific suspect was under consideration. This has convinced Kentucky officials to more broadly implement Rapid DNA testing.

To be clear, I have no objection to the use of a more efficient technology, but I am left puzzled. Has the FBI given Kentucky a special dispensation to use rapid profiles with CODIS databases? Is Kentucky using some database outside of that system?

And what is the basis for Kentucky's claim that inasmuch as "Rapid DNA uses Short Tandem Repeats for DNA ID, which has no coding information,.... there is no genealogy or health information gathered."? (3) The STRs certainly convey information on (close) genetic relationships. (5, 6) As for health-related information, they lie somewhere between none (like a passport number) and not much (like an ABO blood type). (7)

REFERENCES

1. Bruce Schreiner  (AP), Kentucky To Use Rapid DNA Tests for Sex Assault Cases, Wash. Post, Apr. 10, 2019, https://www.washingtonpost.com/business/technology/kentucky-to-use-rapid-dna-tests-for-sex-assault-cases/2019/04/10/a28d484e-5bd4-11e9-98d4-844088d135f2_story.html
2. Rapid DNA, www.fbi.gov/services/laboratory/biometric-analysis/codis/rapid-dna (viewed Apr. 13, 2019)
3. Kentucky State Police, Kentucky Rape Reduction with ANDE Rapid DNA News Conference Fact Sheet, available via link at https://www.ande.com/kentucky-case-study/
4. Kentucky State Police, The Use of Rapid DNA to End the Sexual Assault Epidemic, available via link at https://www.ande.com/kentucky-case-study/
5. David H. Kaye, The Genealogy Detectives: A Constitutional Analysis of “Familial Searching”, 51 Am. Crim. L. Rev. 109 (2013), preprint available at ssrn.com/abstract=2043091
6. Henry T. Greely & David H. Kaye, A Brief of Genetics, Genomics and Forensic Science Researchers in Maryland v. King, 53 Jurimetrics J. 43 (2013), available at ssrn.com/abstract=2403063
7. David H. Kaye, Mopping Up After Coming Clean About "Junk DNA", Nov. 23, 2007, available at http://ssrn.com/abstract=1032094.

OMICS Journals and Conferences Slapped with 20-year Injunction and $50 Million Fine

A few days ago, a U.S. District Court held Srinubabu Gedela and his companies, the OMICS Group, Inc., iMedPub, and Conference Series, liable for over $50 million. It also issued a permanent injunction against many deceptive practices on the part of this nest of companies and the various other entities through which they operate (such as Allied Academies, Meetings International, and Pulsus). Excerpts from the opinion that describe the deceptive practices and other information are posted on the Flaky Academic Journals blog. 1/

The opinion is significant for editors of OMICS-related journals and authors who place their papers in them. The court was persuaded by the Federal Trade Commission's  "evidence indicating that Defendants’ peer review practices are a 'sham.'" The journals affected by the order include

• Journal of Forensic Research
• Journal of Forensic Biomechanics
• Journal of Forensic Medicine
• Journal of Forensic Pathology
• Journal of Forensic Psychology
• Journal of Forensic Toxicology and Pharmacology
• Journal of Medical Toxicology and Clinical Forensic Medicine
• Global Journal of Nursing and Forensic Studies

The first journal lists Sheila Willis (Forensic Science Laboratory, Ireland) and Jian Tie (Nihon University School of Medicine) as editors-in-chief, and Peter Gill (University of Oslo), Harvey Ho (Alabama State University), and other individuals inside and outside of academia as editors. Special issue editors from the US are Jianye Ge (University of North Texas), Oliver Grundmann (University of Florida), Thomas J Holt (Michigan State University), Harvey Ho, and Jawahar L Mehta (University of Alabama Little Rock Medical School).

The editor-in-chief of the last journal is, again, Jian Tie. Linda Howe (University of Central Florida) is an editor. I have not looked at the editorial boards of the other journals recently, but earlier postings on the Flaky Academic Journals blog list some of the editors of the Journal of Forensic Medicine and the Journal of Forensic Psychology. The Flaky Academic Conferences blog tracks OMICS conferences on forensic science and technology and some pf the speakers there..The conferences are often marketed under other brand names.

These lists likely include people who would be surprised to learn of their alleged roles or who have tried to have OMICS remove their names. 2/ One of the deceptive practices is misrepresenting the membership of editorial boards.

NOTES

1. The posting also notes an interview before the order in which Dr. Gedela suggested that any lapses in peer review and quality control were the responsibility of the unpaid editors and authors rather than the publisher. An Ars Technica report characterized OMICS' conduct as "egregious enough that a judge doesn't even wait for a trial." If "egregious" meant "uncontroverted," that would have been correct, but courts do not grant summary judgment in lieu of a trial because the alleged conduct is flagrant. The court granted summary judgment because the defendants failed to point to acceptable evidence contradicting the FTC's showing of persistent and egregious misconduct.
2. See OMICS Journal of Forensic Research, Flaky Academic Journals, Mar. 18, 2017.