Monday, July 18, 2016

NCFS to Ask NIST to Institute an Additional "Technical Merit" Review Mechanism for Some Types of OSAC Forensic Science Standards?

A subcommittee of the National Commission on Forensic Science (NCFS) has approved a draft Recommendation to the Attorney General on Technical Merit Evaluation of Forensic Science Methods and Practices. It is available for public comment until August 15, 2016.

It asks the Attorney General (to whom NCFS reports) to ask the National Institute of Standards and Technology (NIST) to establish a committee of "measurement scientists and statisticians" who would review the scientific literature and do whatever else is needed to establish the technical merit of test methods and practices used in forensic science disciplines. The document also asks OSAC (the Organization of Scientific Area Committees for Forensic Science that NIST established a few years ago) not to place standards for forensic science test methods and practices on its Registry of Approved Standards before their technical merit has been reviewed by such an "independent scientific body."

In full, the draft recommendations are as follows:
1. NIST should establish an in-house entity with the capacity to conduct independent scientific evaluations of the technical merit of test methods and practices used in forensic science disciplines.

2. The results of the evaluations will be issued by NIST as publicly available resource documents. NIST’s evaluation may include but is not limited to: a) research performed by other agencies and laboratories, b) its own intramural research program, or c) research studies documented in already published scientific literature. NIST should initially begin its work by piloting three resource documents to establish their design and requirements. The release of these documents should be broadly disseminated in the scientific and criminal justice communities and accompanied by judicial trainings.

3. The Organization of Scientific Area Committees for Forensic Science (OSAC) leadership, the Forensic Science Standards Board (FSSB), should commit to placing consensus documentary standards on the OSAC Registry of Approved Standards for only those forensic science test methods and practices where technical merit has been established by NIST, or in the interim, established by an independent scientific body. An example of an interim independent scientific body could be an OSAC created Technical Merit Resource Committee composed of measurement scientists and statisticians appointed by NIST and tasked with the evaluation of technical merit.
The surrounding explanation can be found at https://www.justice.gov/ncfs/work-products . Presumably, the Commission will discuss the recommendation at its September 12-13 meeting at, appropriately enough, NIST's campus in Gaithersburg, Maryland.

Sunday, July 10, 2016

If the Glass Fits, Declare It: The Justice Department's ULTR for Glass that "Physically Fits Together"

The Department of Justice’s “Proposed Uniform Language for Testimony and Reports for the Forensic Glass Discipline” allows a broad range of possible conclusions and statements of probability. The strongest conclusion of association would be that “the glass fragments were once part of the same broken object.” The draft ULTR permits a criminalist to “state or imply” this source conclusion—but only “when two or more pieces of broken glass physically fit together.”

The notion that fragments that fit together must have come from the same larger piece of  glass sounds like common sense. Almost all of us have broken glass or ceramic materials at one time or another and seen that the fragments (at least the major ones) can be reassembled. Apparently, criminalists have had the same experience. The supporting documentation for the ULTR notes that “[i]t has long been reported by forensic glass examiners that two glass objects that physically fit together were once part of the same broken object.” 1/

But what type of scientific inquiry would establish the truth of the common sense idea? Let’s do a series of thought experiments. I have ten drinking glasses in my cupboard. (Actually, six are in the dishwasher, but I’ll pretend all are clean and back in the cupboard.) They sure look alike to me, being the same size, shape, and color.
  • Experiment 1. I take one glass out and strike it with a hammer. Then I pick up the pieces and discover that all pairs of fragments that were adjacent in the unbroken glass fit together.

  • Experiment 2. I take out a second glass and strike it with a hammer. Then I put all the pieces in a bag with those from the first glass and mix them up. Will I find any pairs of fragments that seem to fit together that are Glass1-Glass2 (heteroginous, if I can coin a word that means “of different origin”), or will all the pairs that fit together be Glass1-Glass1 or Glass2-Glass2 (homoginous, a neologism for "of the same origin")? Let’s suppose that the subset of all the possible pairs that I find to match are indeed homoginous.

  • Experiment 3. You guessed it. I take a third glass, strike it with the hammer, and examine all possible pairs from all the fragments from the three glasses. This is getting very time-consuming, for the number of pairs to consider is growing exponentially, but I am persistent. Can we conclude, on the basis of common sense, that each and every physically matching pair will be homoginous?
You can see where I am going with this. Common sense plus some cogitation suggests that homoginality — a given pair of matching fragments coming from the same original piece of glass — depends on how many fragments are in the relevant population and the complexity of the match (that is, the extent of the irregularities in the edges of the fragments being fitted together). It is not so obvious that every physically matching pairs of glass fragments that can possibly exist are homoginous. That glass examiners have said that all matching pairs must be homoginous does not make it so. Neither does labeling mechanical fit as an individual as opposed to a class characteristic (to use the popular terminology in forensic science). The labeling exercise begs the question.

Nonetheless, it is quite reasonable to conjecture that only a tiny fraction (which could include 0) of apparently matching fragments that have complex edge patterns are heteroginous, and thus that finding a physical fit is powerful evidence of homoginality. 2/ Of course, the reasonableness of the conjecture depends on the criteria for a fit and the method for declaring one—neither of which are mentioned in the DOJ materials—but let’s assume the existence and maintenance of rigorous criteria and a reliable method. Can criminalists currently estimate how often the conclusion of homoginality is correct when the fragments (unbeknownst to the analyst) actually come from different pieces of glass?

Even if the answer to this question is in the negative, there are ways to present the physical match without overclaiming. Why must examiners “state or imply that the glass fragments were once part of the same broken object”? Examiners who believe that there is a physical fit can contribute a great deal simply by documenting and exhibiting the fit itself. If, additionally, they have some special expertise in moving from the fit to the conclusion of homoginality, then doing also could assist the fact finder. But if their inference is simply that of common sense, the expert aspect of the probative value of that part of the testimony is questionable. In these circumstances, the criminalist might be better advised to show how well the fragments fit together and leave it to the judge or jury to assess the weight of this demonstrable fact.

Another alternative to testimony “to the exclusion of all other sources” (see note 1) is a statement that the observed match is more probable when fragments really come from the same broken object than when they come from different broken objects. I do not know whether specific studies have been performed to justify this likelihood-ratio-type statement, but it should not be outside the capacity of forensic science to generate data that would support this rather modest statement. Indeed, the claim probably is too weak. My intuition is that the likelihood ratio is much greater than 1—it is a heck of a lot more probable to generate a matching pair of fragments by breaking the same object than by breaking different ones. A lawyer can appeal to give the evidence substantial weight on the basis of similar beliefs. If criminalists have experimental or observational studies to quantify how much more probable the physical match is for homoginous fragments, then they can provide that likelihood ratio.

Based on the material the DoJ has supplied, however, the more ambitious criminalists who want to “render an opinion of positive identity” and asservate “to the exclusion of all other sources” (see note 1), as the proposed ULTR apparently allows, should be required to express the limitation that no matter how positive they are in their opinion, the core of that opinion is a common sense judgment of what it means for pieces of glass to fit together rather than the product of elaborate scientific experimentation.

Note
  1. An accompanying footnote cites four sources:
    • Kirk, P., Density and Refractive Index: Their Application in Criminal Identification, Charles C. Thomas, Springfield, IL, 1951, p. 4-5.
    • “Don’t Overlook Evidentiary Value of Glass Fragments”, Law Enforcement Bulletin, Vol. 33, No. 10, October 1964, p. 19.
    • Bottrell, M.C., “Forensic Glass Comparison: Background Information Used in Data Interpretation”, Forensic Science Communications [Online], (April, 2009). Available: http://www.fbi.gov/about-us/lab/forensic-science-communications/fsc/april2009/review/2009_04_review01.htm. accessed on August 15, 2013.
    • Koons, R. D., Buscaglia, J., Bottrell, M., and Miller, E. T. Forensic glass comparisons. In: Forensic Science Handbook. vol. I, 2nd ed. Richard Saferstein, Ed., Prentice Hall, Upper Saddle River, New Jersey, 2002, p. 161–213.

    I do not have Paul Kirk’s 1951 book at hand. The Law Enforcement Bulletin is not generally regarded as a substantial scientific publication. The 2009 publication is a generally informative and comprehensive review article from the FBI. However, it contains no reports of examiners' experiences or experiments with physically fitting fragments of glass back together. In fact, only two sentences even refer to physically fitting fragments—and neither of them explains the basis for the conclusion that the ULTR endorses. (The first sentence is this: “Only physically matching two or more broken glass fragments allows for their association with each other to the exclusion of all other sources (Scientific Working Group for Materials Analysis [SWGMAT] 2005c).” The second is equally devoid of information on the foundation for the conclusion. It reads: “The ... Trace Evidence Unit Quality Assurance Manual (2006) states [that a] glass association is defined as two or more glass samples that can be fracture fitted together, or that exhibit indistinguishable observable properties and/or range overlap in all measured properties.”)

    Finally, the FBI chapter in Richard Saferstein’s handbook discusses “types of fractures” (pp. 173-77) and later, “the mechanical fit.” The authors write that
    The mechanical fit is one of the most desirable of forensic glass examinations because the examiner can render an opinion of positive identity—that two or more pieces of glass were once a portion of, and were broken from, the same pane or object. Glass is particularly suitable for this type of examination. It is amorphous and brittle, is neither stretched nor distorted by breakage, and can be reassembled to its original configuration. Because it is amorphous, no two glass objects will break in exactly the same way. (P. 179).
    As has been discussed extensively for other types of matching evidence, the claim of uniqueness at some possibly unattainable level of precision of measurement does not imply the truth of an unqualifiedly positive opinion of identity.

  2. See generally David H. Kaye, Beyond Uniqueness: The Birthday Paradox, Source Attribution, and Individualization in Forensic Science Testimony, 12 Law, Probability & Risk 3 (2013); David H. Kaye, Probability, Individualization, and Uniqueness in Forensic Science Evidence: Listening to the Academies, 75 Brooklyn L. Rev. 1163 (2010); David H. Kaye, Identification, Individuality, and Uniqueness: What's the Difference?, 8 Law, Probability & Risk 85 (2009).

Tuesday, July 5, 2016

Conference to explode DNA forensic analysis through alchemy

Following a review by its scientific program committee, BIT Group Global Ltd. has slotted the following paper for presentation in a session on Forensic Biology and DNA Analysis at its 3rd International Congress on Forensics and Police Expo:
K-DNA, A Nonoptical Isomer with Strongly Chimerical Properties

Abstract: DNA is known to have many possible double helical structures, including the three biologically active forms A-, B-, and Z-DNA. This paper announces the discovery, through the technique of alchemy, of a new form of DNA exhibiting biological hyperactivity. Using swish-and-push precipitation (SAP), K-DNA has the potential to explode DNA forensic analysis. Subjecting K-DNA to CRISPR/R2d2 confabulation can deconvolute complex DNA mixtures into components from each contributor that can be read via any commercially available STR analyzer. Other potential applications also are discussed.
BIT Global brands itself as "Your Think Tank." According to the tentative scientific program, other speakers that day include
  • Dr. Joye Carter, Owner, Biblical Dogs, Inc., USA
  • Dr. Gary Collins, Director, New York University, USA
  • Dr. Jim Fraser, Professor, University of Strathclyde, UK
  • Dr. Michael F. Rieders, Chairman, Board of Directors Chief Scientific Officer, NMS Labs, Inc, USA
  • Dr. Robert H. Shaw, President, California State Coroner’s Association, USA
(Please note that Wikipedia reports that BIT "congress websites ... falsely include speakers [who] have not confirmed their attendance.")

Related postings
Appendix: Correspondence with BIT Global
From: amy
Sent: Monday, June 13, 2016 3:00 AM
To: David H. Kaye
Subject: Confirmation for Oral Presentation of the BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016

Dear Dr. David H. Kaye,
How are you doing? I am Cherry, the Program Coordinator of BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016. I am writing the letter to make sure if you have received my letter previously, it is about the conference: WCF-2016 which will be held during October 27-29, 2016 at International Conference Center, Dalian, China. Maybe there some problems with my mailbox and I haven't received your kindly reply. I am writing again to cordially welcome you to give a speech. Now we have invited some distinguished experts including Dr. Henry Lee and Dr. John Zheng Wang and many others to talk about the impact of recent forensic scientific and technological breakthroughs around the world, you can find the renowned speakers online at: http://www.bitcongress.com/WCF2016/. In the past two congresses, we have invited more than 300 worldwide leading experts, famous police officers, and law practitioners updating the most recent advances in forensic sciences & Technology. ... WCF-2016 will consists of 4 Pathes in the Thematic Program ... We cordially welcome you to join us for this upgrading event and enjoy the beautiful scenery of Dalian. Would you kindly inform if it is available to you? ...
Best regards,
Ms. Cherry Dong
Organizing Committee of the Forensics & Police Tech Expo 2016
To: besty@wcf-2016.com
Date: 2016-6-13
Subject: RE: Confirmation for Oral Presentation of the BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016

Dear Ms. Cherry Dong,
It will be difficult to get away given my class schedule, but I would consider it if you can confirm that you will provide travel, lodging, and an honorarium. I could speak on my latest research on "Resolving Complex DNA Mixtures with K-DNA, A Nonoptical Isomer with Strongly Chimerical Properties."
I look forward to your reply,
David Kaye
From: Cherry
Sent: Wednesday, June 15, 2016 12:01 AM
To: Kaye, David
Subject: Re: Fw: RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics & PoliceTech Expo 2016

Dear David,
Thank you for your letter. We are interested in your speech topic, and welcome you to make an oral presentation in the conference. With regards for the expenses supports, I would like to apply to support part of your registration fee, but sorry I'm afraid it is difficult for us to support your travel fee due to our limited budgets, as you know, we need to pay more rent fee, operation fee and the travel fee to organize such conference, and we don’t have sponsor this time. Hope you can understand. Would you please kindly let me know if it is accepted to you in this situation? Looking forward to hearing from you as soon as possible.
Best regards,
Cherry
From: Cherry
Sent: Monday, June 27, 2016 11:33 PM
To: Kaye, David
Subject: Re:RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics & PoliceTech Expo 2016

Dear David,
Hope everything goes well. How about your consideration to attend our conference? If you decide to join us, would you like to send us a potential speech title first? Thank you for your time. Looking forward to hearing from you as soon as possible.
Best regards,
Cherry
From: Kaye, David
Sent: Monday, July 4, 2016 11:02 AM
To: Cherry
Subject: RE: Re:RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics & PoliceTech Expo 2016

Dear Cherry,
Thank you for your kind response. As I wrote before, the title is "K-DNA, A Nonoptical Isomer with Strongly Chimerical Properties." The abstract follows: “DNA is known to have many possible double helical structures, including the three biologically active forms A-, B-, and Z-DNA. This paper announces the discovery, through the technique of alchemy, of a new form of DNA exhibiting biological hyperactivity. Using swish-and-push precipitation (SAP), K-DNA has the potential to explode DNA forensic analysis. Subjecting K-DNA to CRISPR/R2d2 confabulation can deconvolute complex DNA mixtures into components from each contributor that can be read via any commercially available STR analyzer. Other potential applications also are discussed.” Please let me if you need other information to add this talk to to the programme.
With regards,
David Kaye
From: Cherry
Sent: Monday, July 4, 2016 10:09 PM
To: Kaye, David
Subject: Re:RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics_&_PoliceTech_Expo_2016

Dear David,
Thank you for your speech title and abstract, it is suited to Path A2: Forensic Biology and DNA Analysis, now it has been accepted by the congress committee. Now if everything is OK, would you like to register online at: http://www.bitcongress.com/WCF2016/Register.asp to confirm your speech position? Looking forward to hearing from you again, and please feel free to contact me should you have any problem. Best regards, Cherry
Ms. Cherry Dong
Organizing Commission of Forensics & Police Tech Expo 2016
East Area, F11, Building 1, Dalian Ascendas IT Park,
1 Hui Xian Yuan, Dalian Hi-tech Industrial Zone,
LN 116025, China
Tel: 0086-411-84799609-822
Two days later ...
From: Amy
Sent: Wednesday, July 6, 2016 1:32 AM
To: David H. Kaye
Subject: New Progress of BIT’s3rdInternationalCongressof Forensics & Police Tech Expo 2016

Dear Dr. David H. Kaye,
How are you doing? I am Cherry, the Program Coordinator of BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016. I am writing the letter to make sure if you have received my letter previously, it is about the conference: WCF-2016 which will be held during October 27-29, 2016 at International Conference Center, Dalian, China. ... We cordially welcome you to join us for this upgrading event and enjoy the beautiful scenery of Dalian. Would you kindly inform if it is available to you? Looking forward to hearing from your prompt reply.
Best regards,
Ms. Cherry Dong
Organizing Committee of the Forensics & Police Tech Expo 2016
Twenty minutes later ...
From: Amy
Sent: Wednesday, July 6, 2016 1:52 AM
To: Kaye, David
Subject: New Progress of BIT’s3rdInternationalCongressof Forensics & Police Tech Expo 2016

Dear Dr. Kaye, David,
How are you doing? I am Cherry, the Program Coordinator of BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016. I am writing the letter to make sure if you have received my letter previously, it is about the conference: WCF-2016 which will be held during October 27-29, 2016 at International Conference Center, Dalian, China. ... We cordially welcome you to join us for this upgrading event and enjoy the beautiful scenery of Dalian. Would you kindly inform if it is available to you? Looking forward to hearing from your prompt reply.
Best regards,
Ms. Cherry Dong
Organizing Committee of the Forensics & Police Tech Expo 2016
Six days later
From: Cherry
Sent: Tue 7/12/2016 3:02 AM
To: Kaye, David
Subject: Re: Fw: RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics & PoliceTech Expo 2016

Dear David,
Hope everything goes well. We have updated new conference program on the website, please check it at: http://www.bitcongress.com/WCF2016/ScientificProgramme-4-1.asp Now the conference developed well, the congress committee have accepted your speech title and abstract, if it is OK, would you like to register online at: http://www.bitcongress.com/WCF2016/Register.asp to confirm your speech position? Looking forward to hearing from you as soon as possible.
Best regards,
Cherry
From: Kaye, David
Sent: Wednesday, July 13, 2016 10:47 AM
To: Cherry
Subject: RE: Re:RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics & PoliceTech Expo 2016

Dear Cherry, I am running into some difficulty replicating the preliminary research that encouraged me to write the abstract. The SAP precipitation now is giving inconsistent results. Unless I can resolve this soon, I don't think I can present this paper. It would be best if you removed it from the program for now or listed it as tentative and considered offering the slot to another speaker. With apologies for this development,
David
From: Cherry
Sent: Wednesday, July 13, 2016 10:39 PM
To: Kaye, David
Subject: Re: Fw: RE: Confirmation for Oral Presentation of theBIT’s 3rd International Congress of Forensics & PoliceTech Expo 2016

Dear David,
It is regretful, hope you can join our future events. It will be appreciated if you can recommend us to your friends, your colleagues and whom you think might be interested in the conference. Thank you for your time.
Best regards,
Cherry
How soon we forget. About a week later, Cherry reverted to her second form letter. On August 1, 2016, another three form emails about "speech posiitions" and "some renowned speakers [who] have confirmed their participation as followings" arrived.

Saturday, July 2, 2016

State Supreme Court Approves of Variable-Prior-Odds Presentation of the Bayesian Probability of Paternity

In 2002, the Supreme Court of Kentucky wrote a disappointing opinion on the admissibility of the "probability of paternity" in a case of rape and sexual abuse of a minor. In 2016, the court corrected its mistakes in a similar case. The more recent opinion is a welcome addition to the jurisprudence on probability evidence in criminal cases.

Butcher

In the first case, Butcher v. Commonwealth, 96 S.W.3d 3 (Ky. 2002), Larry Butcher lived with a woman and her young daughter in Johnson County. After the woman had twin girls fathered by Larry, he "began a pattern of sexual abuse with H.B., who was then ten years old. . . . H.B. became pregnant in April 1987, at age fourteen. H.B. gave birth to a baby girl on January 19, 1988." Larry was convicted of eleven counts of first-degree rape and other offenses.

The evidence against him included "a DNA paternity test indicating a 99.74 percent likelihood that [he] was the father of H.B.'s child." The court explained that a "Mr. DeGuglielmo testified that tests performed on the eight genetic markers previously discussed yielded a paternity index of 388/1, meaning Appellant was 388 times more likely to be the father of the child than a randomly selected male of the same race."

The meaning the court ascribed to the paternity index is clearly wrong. The paternity index of 388 meant that the genotypes were 388 times more probable if Larry were the father than if a randomly selected man was. The probability that Larry was the father instead of a different man is something else. To mechanically equate the two is to transpose conditional probabilities. As is now widely recognized, the "probability of paternity" is a posterior probability that depends on both the likelihood ratio and the prior probability of paternity. If and only if the prior probability is one-half are the prior and the posteriors exactly equal.

Having initially conflated the paternity index with the posterior probability, the Butcher court went on to explain that the "final part of the paternity test translates the paternity index into a percentage that is more understandable . . . using Bayes' Theorem, a formula that takes into account actual events and circumstances, as opposed to random sequences of events." It thought that "[i]n paternity tests generally, this formula combines the paternity index and another value representing the prior probability that an event occurred, including such factors as access to the mother, fertility, and date of conception."

Of course, paternity test reports in both civil and criminal cases do not generally consider these factors. Rather, they "typically insert a standard prior probability of .5 regardless of any other factors, which indicates a fifty percent chance that the alleged father actually had sexual intercourse with the mother." (This is not quite right either, since not all intercourse leads to children.)

In any event, Butcher concluded that "there was a 99.74 percent likelihood that Appellant was the father of H.B.'s child. Although Mr. DeGuglielmo did not testify that he had used a prior probability value of .5 in reaching his expert conclusions, the mathematical results of the test suggest that he did."

In response to a garbled argument from Butcher that the use of a prior probability of one-half offended the presumption of innocence in a criminal case, the court relied on a 1998 Texas Court of Appeals opinion’s strange conclusion that "[t]he use of a prior probability of .5 is a neutral assumption. The statistic merely reflects the application of a scientifically accepted mathematical theorem which in turn is an expression of the expert's opinion testimony."

This is strange because any value from 0 to 1 for the prior probability would be part of "the application of a scientifically accepted mathematical theorem"—namely, Bayes’ rule. The expert supposedly has no opinion about the prior probability and is using the midpoint of 0.5 to express ignorance about what the value actually is. Yet, to assign a probability of 0.5 to the defendant is to credit the fact that he—as opposed to all other men in the population—has been charged with a crime. Letting an accusation of criminality create a substantial probability of guilt does seem to offend the presumption of innocence.

Ivey

This year’s case, Ivey v. Commonwealth, No. 2014-SC-000345-MR, 2016 WL 671153 (Ky. Feb. 18, 2016), is a major leap forward. Alfred Ivey, Jr., lived in Hardin County with his half-sister and her daughter from a previous relationship. The daughter became pregnant at the age of 13. Seven years later, she revealed that Ivey was the father and had repeatedly raped her before then. A paternity test confirmed that Ivey was the father, and a jury convicted him of two rapes of a minor.

Ivey objected that the probability of paternity—a whopping 99.9999%—was inadmissible under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). At a pretrial hearing on his motion to exclude,
Ivey proffered an academic article, Ernest P. Chiodo et al., An Error in Statistical Logic in the Application of Genetic Paternity Testing, J. Mod. Applied Stat. Methods, Winter 2002, Vol. 1, Issue 1, at 126, but did not offer any additional proof, such as expert testimony. His argument at that hearing was based solely on the article, which criticizes the use of the 50% prior probability to calculate the probability of paternity in some circumstances because it "highlights a serious error in statistical methodology known as the ‘principle of indifference.’"
The trial court admitted testimony about the parentage test anyway, and the state supreme court revisited its opinion in Butcher. This time around, it recognized that
The paternity index "does not measure the chance of defendant's paternity compared to that of a randomly selected man." D.H. Kaye, Plemel as a Primer on Proving Paternity, 24 Willamette L.Rev. 867, 877 (1988). . . . Technically speaking, the paternity index is the ratio of "the probability of the alleged father transmitting the alleles and the probability of selecting these alleles at random from the gene pool." D.H. Kaye, The Probability of an Ultimate Issue: The Strange Cases of Paternity Testing, 75 Iowa L.Rev. 75, 89 (1989–1990). It has been described as a "likelihood ratio," that is, "a ratio of two conditional probabilities," but "is not itself a probability." 1 McCormick on Evidence § 211 n. 17 (7th ed. 2013) "Unlike probabilities, which are bounded by 0 and 1, the likelihood ratio can exceed one. The larger the ratio, the more probative the evidence."
As such, the court acknowledged that its understanding of the paternity index in Butcher was "not, strictly speaking, completely accurate." Likewise, Ivey is more careful in describing the nature of a prior probability.

Nevertheless, the court seemed uncertain whether "the use of the 50% prior probability is a logical error." It wrote that "the ‘error,’ to the extent it is one, appears to matter only under unrealistic conditions." It noted that the likelihood ratio of 11,900,000 in the case swamped any plausible prior probability:
[I]n this case, even with very low assumed prior probabilities, the probability of paternity exceeded 99.99%. The Commonwealth's expert explained generally to the jury how the prior-probability notion affected the probability of paternity, albeit without getting into the details of Bayes' theorem, and postulated several variations less than the assumed standard 50%. For example, the expert explained that if the prior probability were only 1%, or .01, then the probability of paternity dropped only to 99.9991%. The extreme example of a low prior probability, .1%, or .001, used in the Chiodo article also results in very high probability of paternity for Ivey. Though the expert did not testify to this number, it can easily be calculated with the formula laid out elsewhere in this opinion, with the resulting probability of paternity being 99.9916%.
After seemingly sidestepping the question of whether the 0.5 prior probability is "a logical error," however, the court agreed with the commentary correcting the claims of "neutrality" for this number: "Those critical of using a 50% prior probability have a point. [T]hat prior probability is fairly high (almost satisfying the civil burden of proof by itself). Despite our assertions in the past, it is not a truly neutral number. It, at best, gives us a ‘useful working hypothesis.’" That said, "a 50% prior probability is far from neutral."

Yet, the court was unfazed by the fact that the prosecution’s expert testified that "she tried to be neutral, though ‘ideally’ the probability of paternity would take into account all the other evidence. . . . She testified that 50% was the only neutral number that would not weigh the results one way or the other."

What, then, can be done to enable a laboratory to explain the import of its genetic findings? Not being in a position to evaluate the nongenetic evidence in a case, the laboratory has no basis for estimating a case-specific prior probability. Neither does it have a usable statistic for the prevalence of paternity on the part of defendants in criminal conduct that results in pregnancies. Consequently, it cannot apply Bayes’ rule to give a single posterior probability to the jury.

One solution would be to forget Bayes’ rule—to eschew talk of the posterior of paternity and restrict testimony to the likelihood ratio and its components. Citing publications of the Royal Statistical Society, 1/ a concurring justice urged this approach. The paternity tester might testify along these lines: X% of all children would be expected to inherit the genetic characteristics that the child has if the defendant (or an identical twin) were the father. Only Y% would be expected to have these characteristics if an unrelated man were the father. Only Z% of children with (an unknown) brother to the defendant would be expected to inherit the child’s characteristics. Thus, the DNA types in this case strongly support the conclusion that the defendant is the father. (Obviously, this conclusion only follows if X is much greater than Y and Z, as was the case in Ivey.)

Ivey considered, but did not settle on another approach—sometimes called "variable prior odds" 2/ —which involves presenting jurors with a spectrum of prior and posterior probabilities to show them how the posterior probability changes as a function of the prior probability.

The Ivey court observed that
Though this proposed approach has drawn some criticism, see Meyerson & Meyerson, supra 3/, at 789 (stating that “[t]his proposal, though well-meaning, is hopelessly misguided”), it is not without its charm. It avoids the problem of the expert using a 50% prior probability without disclosing what that figure means, and allows the jury to use “Bayes's rule merely ... as a heuristic device, displaying the force of the evidence across a wide range of prior probabilities.” Kaye, DNA Evidence, supra, at 167. And this approach has been used in some courts, especially before DNA testing became as refined as it is today. See State v. Spann, 130 N.J. 484, 617 A.2d 247, 264–65 (1993); Plemel v. Walter, 303 Or. 262, 735 P.2d 1209, 1219 (1987).
But the court was not sufficiently charmed to insist on the use of variable prior odds:
We need not decide today whether such an approach is required. We note it only as one possible solution to the problem of the 50% prior probability. Whether presenting a spectrum is definitively a better method than simply assuming a 50% prior probability is a question better left, in the first instance, to a trial court presented with evidence to evaluate whether it better fits under Daubert and KRE 702. And it is likely that such an examination will be unnecessary, as even in Butcher, we noted that the use of a 50% prior probability could be disregarded by the jury and that it could “be weakened on cross and in argument.” 96 S.W.3d at 8 . . . . And defendants are free to offer their own expert witness critical of the 50% prior probability, as happened in this case.
Inviting a debate before a jury over Laplace's principle of insufficient reason and assertions of "neutrality," however, is less than ideal. Experts should avoid these dubious rationales in favor of
what was used in this case. [T]he Commonwealth's expert testified as to the effect of several lesser prior probabilities, including 1%, on the resulting probability of paternity. In each instance, the probability of paternity remained above 99.99%. Thus, the jury could see the effect of different prior probabilities on the resulting probability of paternity. ... [¶] As Professor Kaye noted in 1989, as genetic testing becomes "more refined ... the controversy over [probability of paternity] will wither." . . . This prophesy has not completely born out, as questions are still being raised more than 25 years later. But as testing improves, we are convinced those questions are becoming of little consequence.
Notes
  1. Colin Aitken, Paul Roberts, & Graham Jackson, Communicating and Interpreting Statistical Evidence in the Administration of Criminal Justice: Practitioner Guide No. 1, Fundamentals of Probability and Statistical Evidence in Criminal Proceedings 35–36 (Royal Statistical Society 2010); Roberto Puch–Solis, Paul Robert, Susan Pope & Colin Aitken, Practitioner Guide No. 2, Assessing the Probative Value of DNA Evidence (2012); Graham Jackson, Colin Aitken, & Paul Roberts, Communicating and Interpreting Statistical Evidence in the Administration of Criminal Justice: Practitioner Guide No. 4, Case Assessment and Interpretation of Expert Evidence 37 (2015).
  2. David H. Kaye, David E. Bernstein & Jennifer L. Mnookin. The New Wigmore: A Treatise on Evidence: Expert Evidence §14.3.2(c) (2d ed. 2011).
  3. Michael I. Meyerson & William Meyerson, Significant Statistics: The Unwitting Policy Making of Mathematically Ignorant Judges, 37 Pepp. L.Rev. 771 (2010).

District of Columbia Court of Appeals Delivers Coup de GrĂ¢ce to "Unqualified" Pattern-matching Testimony

Earlier this year, the highest court for the District of Columbia indicated that it was not disposed to testimony that “that when a bullet is fired from a particular gun, the gun leaves ‘unique’ identifying marks, 'similar to a fingerprint, basically.'" See Traditional Forensic Pattern-matching Testimony in Williams v. United States, June 18, 2016. Forensic Sci., Stat. & L., http://for-sci-law.blogspot.com/2016/06/traditional-forensic-pattern-matching.html (discussing Williams v. United States, 130 A.3d 343 (D.C. 2016)).

Last month, in Gardner v. United States, Nos. 11–CF–557, 14–CO–832, 2016 WL 3474642 (D.C. June 23, 2016), the court declared that "in this jurisdiction a firearms and toolmark expert may not give an unqualified opinion, or testify with absolute or 100% certainty, that based on ballistics pattern comparison matching a fatal shot was fired from one firearm, to the exclusion of all other firearms."

The defendant, Eric Gardner, was on trial (for a second time) for shooting Andrew Kamara, a taxicab driver. Part of the case against him consisted of expert testimony about bullet cartridges associated with the shooting. Defense counsel wanted the trial court to rule that "what is appropriate is for the expert to testify specifically in this case that the bullet that was recovered from the decedent is consistent with ... one of the pistols that he was given to examine but not state that it was ... [with] any scientific certainty." The trial court replied that he would let the expert "state his conclusions—his reasonable conclusions and you can impeach him up and down, if you want."

And so the government's expert, Lyndon Watkins, "stated that the silver gun was the murder weapon." Furthermore, he
confirmed his unqualified opinion on cross-examination and redirect examination. From his analysis, Mr. Watkins testified unequivocally that in his opinion, the silver gun fired the killing shot, reiterating that conclusion on cross examination when defense counsel stated, "I believe your ... expert opinion was the bullet recovered from Mr. Kamara . . . was consistent with having been fired from the silver pistol; is that correct?" Mr. Watkins responded, "It was fired from the pistol, yes sir." On redirect examination, . . . "It was identified as having been fired from Government Exhibit 71."
The Court of Appeals noted that despite the long history of judicial acceptance of bullet-matching testimony, "[b]eginning around 2008, however, questions about pattern matching generally, and bullet pattern matching specifically, surfaced in the scientific community." Although it cited only law review articles for this assessment, it then pointed to reports of the National Research Council and the fact that "some jurisdictions began to limit the scope of a ballistics expert's testimony."

After noting that the issue had not been resolved in Williams and an earlier case on the subject, the court announced that
[W]e now hold that the trial court erred by allowing Mr. Watkins to give an unqualified opinion about the source of the bullet that killed Mr. Kamara. We further hold that in this jurisdiction a firearms and toolmark expert may not give an unqualified opinion, or testify with absolute or 100% certainty, that based on ballistics pattern comparison matching a fatal shot was fired from one firearm, to the exclusion of all other firearms.
Apparently, the government did not directly defend its prosecutor’s use of "that’s the gun" testimony, but contended that "any possible error in the admission of Watkins' testimony without 'qualification' was harmless."

Where this leaves the law of the District of Columbia is unclear. Apparently, error can arise short of the tag phrase "to the exclusion of all other guns in the world," but what "qualification" of the testimony that "This is the gun!" is required? Defense counsel maintained "that the error in admitting Mr. Watkins' unqualified opinion was not harmless because . . . ‘the difference between whether the silver gun was definitely the murder weapon or whether it was simply believed by the ballistician to be the murder weapon may well have affected the jury's deliberations.’" But it hardly seems satisfactory to draw the line between the statement of an expert toolmark examiner that (1) "This is the gun that fired the bullet" and (2) “"his is the gun that, in my opinion as an expert trained and experienced in these matters, fired the bullet."

In earlier cases, the government represented that its policy was to permit testimony that to "a reasonable scientific certainty," this is the gun. That is a "qualification," I suppose, but it is no better than the "unqualified" one in resolving "questions about pattern matching . . . in the scientific community."

Of course, the beauty of the common law is that courts need not (and cannot) definitively resolve the myriad of factual patterns not yet presented, but the absence of any real explanation in Gardner of why a positive identification of a gun from expended bullets is unacceptable means that the opinion leaves a wide range of "qualified" testimony up in the air.