Saturday, November 30, 2019

An Opinion on Historical Cell Site Location Evidence (with an Peculiar Explanation of Rule 702)

In United States v. Medley, 312 F.Supp.3d 493 (D. Md. 2018), an FBI agent prepared charts "showing the geographic layout of the various cell towers to which [Jovon] Medley's phone supposedly connected at the time and date of [a] carjacking." United States District Judge Paul W. Grimm overruled Medley's objection to these charts and testimony about them, but he also directed the expert to limit his testimony so as to avoid "reliability" concerns. This limiting technique is becoming increasingly popular for forensic science testimony that clearly has some probative value but is not as "scientific" as one might hope. 1/

Citing an opinion from the Court of Appeals for the Seventh Circuit, Judge Grimm noted that "[c]uriously, despite the frequent admission of historical cell site location evidence by trial courts in criminal cases, '[n]o federal court of appeals has yet said authoritatively that historical cell-site analysis is admissible to prove the location of a cell phone user.'" He added that "the Circuit Courts that have considered admissibility of this evidence have split in their assessment of its reliability, and consequent admissibility." This split, as described in Medley, seems to emanate from an unpublished opinion of the Sixth Circuit in United States v. Reynolds, 626 Fed.Appx. 610, 614–15 (6th Cir. 2015),

The Reynolds court did not exclude testimony linking a cellphone to a location, for the government only used the cell tower records to rule out the presence of the phone in certain areas. But the Reynolds court was unimpressed with the argument that "historical cell site location analysis was reliable because it had been tested and accepted by the law enforcement community" and "that the methodology had been tested successfully over a thousand times in locating suspects with historical cell site tracking data." The court of appeals disdainfully wrote that:
This claim appears to be precisely the sort of “ipse dixit of the expert” testimony that should raise a gatekeeper's suspicion. While being successfully employed “1000 times” may sound impressive, the claim is not subject to independent peer review and fails to establish an error rate with which to assess reliability because there was no information on how many times the technique was employed unsuccessfully.
After a digression on whether or when cell site location evidence could be presented by a lay witness (Judge Grimm acknowledged the "impressive credentials and qualifications" of the witness to testify as an expert), the opinion equated Rule 702's "reliability" requirements to Rule 401's definition of relevance:
[T]he essence of Rule 702 is to avoid misleading the jury with unreliable evidence (which, by its very nature, is irrelevant, because unreliable evidence has no tendency to make a fact that is of consequence to the litigation more probable than it would be without the evidence, and irrelevant evidence fails to be helpful to a jury). Opinions that are solidly grounded in the facts of a particular case, employ reliable methodology or principles, have passed scrutiny by other experts in the same field, and which were reached by strict adherence to established standards and procedures are helpful. Those that are not amount to nothing more than scientific or technical guesses.
But reliability falls on a continuum. "Helpful" as used in Rule 702 is by not a simple corollary of the definition of relevance as the tendency to make a fact more probable. Expertise encompasses more than just two states--the very secure and the useless. Probative but imperfect information fills a middle range, and the relevance requirement (in Rules 401 and 402) does not bar evidence in this zone. "Guesses" by someone who is good at guessing are relevant--they alter the subjective probability of the contested fact. But even shrewd guesses may be excluded under Rule 702 if and when they are dressed up as scientific facts or impeccable wisdom. Moreover, Rule 403, which requires that prejudicial effect not substantially outweigh probative value, also supports such exclusion.

The end result in Medley can best be  understood in this light. The inferences from the cell tower data, Judge Grimm concluded, must be limited to "general location" and must include "a candid explanation of the accuracy of the particular location method used, which must, by necessity, include its limits as well as strengths." At that point, it becomes probative and helpful enough to satisfy rules 403 and 702.

Of course, how to explain "accuracy" when a "mathematical error rate has not been calculated" remains something of a puzzle, but Judge Grimm announced that
While I will allow the Government to elicit the opinion from SA [Special Agent] Fennern that the location of the defendant's cell phone was “consistent with” the location of the crime scene at the time of the carjacking, he may not do so until after he has fully explained during direct examination the inherent limitations of the accuracy of his location evidence—namely, the phone can only be placed in the general area of the cell tower sector that it connected to near the time of the carjacking, and that it cannot be placed any more specifically within that sector.
This approach reflects the judgment that more definite inferences from cell tower records would not be sufficiently reliable or that such testimony would not be worth the time it might take for a battle over the more powerful assertions. At the same time, the court deemed "general location" testimony with  disclaimers up front as sufficiently probative even if it provokes an extensive cross-examination. Indeed, the court slipped in the suggestion that the defense could use "learned treatises" in its cross-examination.

NOTE

1. In an unreported ruling from the bench in Medley, Judge Grimm also followed a middle path for firearms-toolmark testimony, limiting the examiner to "consistent with" testimony. See United States v. Johnson, No. (S5) 16 Cr. 281 (PGG), 2019 WL 1130258, at *20 (S.D.N.Y. Mar. 11, 2019) (declining to do the same; discussed at http://for-sci-law.blogspot.com/2019/03/another-us-district-court-finds.html).

Sunday, November 24, 2019

Commonwealth v. Ross: Is Bitemark Identification Evidence Becoming Toothless?

In a case that has received national attention, a Pennsylvania Superior Court (the state's intermediate appellate court) finally decided that the trial judge had to at least hold a hearing on the general scientific acceptance of bitemark identification evidence. The trial court had ruled that no such "Frye hearing" was required before admitting proposed testimony from two forensic odontologists that the defendant cannot be excluded as the source of human bite marks on the rape-murder victim whereas several other potential suspects can be.

Mysteriously, the appellate court chose to keep its lengthy opinion unreported and of "no precedential value." (It is reproduced at the end of this posting and available as a pdf file.) Many appellate courts do not publish opinions that simply apply settled law and would be of interest only to the parties, but that is not the situation here. The trial judge's refusal to grant the defense's request for such a "Frye hearing" has received national condemnation, 1/ and the new opinion is one of a very few to take seriously widespread misgivings about bitemark evidence.

As described in an NBC report complaining that "the justice system ignores science," 2/
A little after noon on June 24, 2004, a fisherman on Canoe Creek Lake in Blair County, Pennsylvania, found the bound, beaten and sexually abused body of Tina Miller near a boat launch. Suspicion focused on [Paul Aaron] Ross, who’d been with Miller nearby the night before.

There were no witnesses to the killing, leaving authorities to build a case on circumstantial evidence: a man who said he had dropped Miller and Ross off near the boat launch, former girlfriends who said he’d abused them, a mark found on one of Miller’s breasts, and items collected from the scene ─ including a shoe print and a beer bottle. None of it definitively linked Ross to the murder, and Ross denied killing Miller, asserting that another man had picked her up in a truck. But prosecutors argued that, taken collectively, the evidence proved his guilt.

He went to trial in 2005, and a bite-mark examiner helped seal his conviction.

The examiner, Dr. Dennis Asen, was a dentist with 25 years of experience identifying bodies by their teeth and examining bite marks. On the stand, he recalled viewing Miller’s body at the morgue and concluding that her breast bore patterns of a bite mark. He took a photograph of it and made an “overlay tracing” that he uploaded to a computer and compared with molds of Ross’ teeth and the teeth of four other suspects, including three who’d been in contact with Miller in the hours before her death. Asen said he found the mark “very highly consistent” only with Ross’ teeth. Asen acknowledged, however, that he couldn’t say with 100 percent certainty that Ross had made the mark. The defense hired its own bite-mark examiner, who said he was unable to determine whether Ross left the mark.

Ross was convicted of first-degree murder and sentenced to life in prison. But the conviction was overturned by an appeals court that ruled the trial court had “abused its discretion” in refusing to give Ross’ lawyer more time to prepare, and allowing the former girlfriends’ testimony.

Prosecutors plan to try Ross again, but the process has stalled amid a battle over the bite-mark evidence.

Defense lawyers, including members of the New York-based Innocence Project, have cited government reports and academic studies, including from the National Academy of Sciences, the President's Council of Advisors on Science and Technology and the Texas Forensic Science Commission.

The reports have shown that no one has sufficiently proven that human bite marks are unique, or that skin is a reliable material to record them ─ or that forensic dentists can reliably identify a human bite mark in the first place.
What was the Commonwealth's rejoinder? The Blair County district attorney is "The Honorable Richard A. Consiglio." (That is his title on the county's website, which emphasizes his success in dealth penalty cases along with his receipt of "an award ... for standing up to the Blair County Courts on behalf of the unborn." "Rich," as the website also refers to him, told now retired Blair County Judge Jolene Grubb Kopriva in 2016 that
the authors of scientific reports lack “real world” experience. He ... pointed out that no court in any state has ruled against admitting it. “This is evidence, your honor, that has been accepted for 40 or 50 years, not quite as old as I am, but 40 or 50 years, maybe longer, accepted by courts everywhere.” 3/
Acknowledging that "the use of bite-mark evidence is beginning to face challenges," Judge Kopriva ruled the evidence admissible on the basis of the lawyers' arguments. She wrote in Match 2017 that "it would be premature for this court to order that the methodology is no longer generally accepted in the relevant scientific community.” 4/

In a blog post (since deleted), "the Innocence Project described the case as an example of "'willful scientific illiteracy among the judiciary.' ... Judge Kopriva imposed a gag order preventing either side from talking publicly about the case outside of court." 5/

But she also certified an interlocutory appeal of her order denying the motion for a Frye hearing. (Ordinarily, such evidentiary rulings are only reviewed as part of an appeal from the verdict at the end of the trial.) The Superior Court refused to accept the interlocutory appeal. But then the Pennsylvania Supreme Court vacated the Superior Court's order and directed it to decide whether there had to be an evidentiary pretrial hearing on the admissibility of bitemark evidence

This week, on November 21, the Superior Court released its opinion. The court held that the trial court abused its discretion in agreeing to admit bitemark opinion testimony without first conducting a Frye hearing. The opinion, written by Judge Mary Murray, discusses Frye v. United States, 293 F. 1013 (D.C. Cir. 1923), the Superior Court's understanding of the general-acceptance requirement as it has been construed in Pennsylvania, and how this requirement applies to a request for an evidentiary hearing on bitemark evidence.

According to the Ross opinion, "Frye contemplated a judicial inquiry, informed by experts, into the general acceptance of the scientific methods used." Yet, Frye contains no mention of "a judicial inquiry" in the form of a pretrial evidentiary hearing on the status of a method among scientists. Without any such hearing to draw on, the Court of Appeals in Frye determined that "the systolic blood pressure deception test has not yet gained such standing and scientific recognition among physiological and psychological authorities as would justify the courts in admitting expert testimony deduced from the discovery, development, and experiments thus far made." 6/ The court simply announced its conclusion in an enigmatic eight-paragraph opinion. The task of devising suitable substitutes for judicial fiat fell to later generations.

Today, the use of hearings with expert testimony on scientific acceptance is well developed. In Pennsylvania, however, the touchstone for deciding when such hearings are essential often is said to be "novelty." Thus, Judge Kopriva relied on the lack of novelty in denying the defense's requested hearing, and the Superior Court focuses on the supposed requirement as well.

But Frye itself imposed no "novelty" requirement. All that Frye said in this regard that was the blood pressure test had yet to cross "the line between the experimental and demonstrable stages." 7/ The opinion did not consider whether a procedure could cross the line, but then fall back once more into the darker side of "this twilight zone." 8/ Bitemark evidence once was thought to have been valid (by its practitioners at least), but, as the defense in Ross argued, today the larger scientific community (and a growing number of dental experts) do not recognize its scientific validity.

Fortunately, the Superior Court, after flailing about in an effort to decide what needed to be novel, also addressed the more appropriate question of whether the relevant scientific community still believes (if it ever did) that the kind of testimony that Judge Kopriva was going to admit is generally accepted. It rejected District Attorney Consiglio's idea that acceptance within the forensic odontology community is sufficient, and it recognized that recent literature indicated significant dissension even within that community and outright rejection by a broader scientific community. Consequently, it held that the trial court could not admit the prosecution's evidence without a meaningful hearing on general acceptance in the broader community.

Of course, this holding does not dictate the outcome of the hearing. Indeed, the trial court could avoid an evidentiary hearing entirely, by reviewing the readily available scientific literature (with briefing by the parties) and concluding that it does not support the general acceptance of the comparative analysis that the prosecution proposes to present. After all, the proponent of the scientific evidence bears the burden of demonstrating its general acceptance, and that will not easy for the state to do in this case.

Nonetheless, that is too great an about-face to expect from the trial court. More likely, the court will proceed with the full-blown hearing. That hearing should focus on the specific testimony that the two dentists slated to testify might give. Is there sufficient "scientific recognition" 9/ that the forensic odontologists can validly determine that the bite marks on the breast of the victim's partially submerged body are human (if that is what they propose to say)? That (whether or not the marks are from a human source) they do not come from the four other potential suspects? That they were "very highly consistent" with Ross's dentition (as Dr. Asen testified at the original trial)? If the testimony were kept to a bare minimum, such as "unable to exclude as a possible source of the marks," with no explanation of the implications of such an inclusion (because scientific knowledge is too thin), 10/ would it be sufficiently helpful to be admissible? 11/ Or would it be inadmissible as being inscrutable or too easily overvalued?

The time when these issues could be avoided by arguments about "novelty" and "real world experience" has passed.

NOTES
  1. John Shuppe, 'We Are Going Backward': How the Justice System Ignores Science in the Pursuit of Convictions, NBS News, Jan. 23, 2019, https://www.nbcnews.com/news/us-news/we-are-going-backward-how-justice-system-ignores-science-pursuit-n961256.
  2. Id.
  3. Id. There are exceptions to the District Attorney's bald assertion that "courts everywhere" deem bitemark testimony admissible. See infra note 11. Mr. Consiglio's timeline also was incorrect. The earliest use of bitemarks as identification in a U.S. criminal case may have been some 150 years ago, in 1870. See Barry E. Lipton et al., History of Odontology, in Manual of Forensic Odontology 1, 27 (David R. Senn & Richard A. Weems eds., 2013). The first reported case was a 1954 murder trial in Texas. Id. at 28. But see Michael J. Saks et al., Forensic Bitemark Identification: Weak Foundations, Exaggerated Claims, 3 J. L. & Biosci. 538 (2016), available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5570687/ (asserting that before 1974 a some years thereafter, it was "the rule among forensic dentists that crime scene bite marks could not be trusted to yield accurate source identifications.).
  4. Id.
  5. Id.
  6. Frye v. United States, 293 F. 1013, 1014 ( D.C.. Cir 1923).
  7. Id.
  8. Id. 
  9. Id.
  10. This seems to be the only type of inclusionary testimony the American Board of Forensic Odontology accepts from its members. See ABFO, Standards and Guidelines for Evaluating Bitemarks § 3 (2018) ("bitemark linkage conclusions should  only a) exclude  or b) not exclude  (include) a  dentition.The  specific  terms  found  in 2.c.are: a)  for  exclusion, Excluded  as Having  Made  the  Bitemark,  and  b)  for  inclusion, Not  Excluded  as  Having Made  the  Bitemark.Stronger  terms of  attribution are  not condoned ... .").
  11. Cf. Ege v. Yukins, 485 F.3d 364 (6th Cir. 2007) (the more useful testimony of exclusions and "highly consistent" matching followed by "my expert opinion" that "nobody else" among the "three and a half million people" in the Detroit area "would match up" was without foundation and violated due process). The American Board of Forensic Odontology no longer condones such testimony. See supra note 10..

Commonwealth v. Ross
No. 1738 WDA 2018 (Pa. Super. Ct. Nov. 21, 2019)
(not reported, pdf file available at
www.pacourts.us/assets/opinions/Superior/out/J-S55007-19m%20-%2010423159687925902.pdf

BEFORE: MURRAY, J., McLAUGHLIN, J., and COLINS, J.
MEMORANDUM BY MURRAY, J.:

Paul Aaron Ross (Appellant) appeals from the order denying his request for a Frye hearing. 1/ After careful consideration, we vacate the trial court’s order denying Appellant’s request for a Frye hearing and remand to the trial court for proceedings consistent with this decision.

This appeal arises from the June 27, 2004 murder of Tina Miller at Canoe Creek Lake in Canoe Creek State Park. Appellant was arrested and charged with Ms. Miller’s murder. On November 23, 2005, a jury found Appellant guilty of first-degree murder, aggravated assault, involuntary deviate sexual intercourse, unlawful restraint, simple assault, false imprisonment, and indecent assault. 2/
A prior panel of this Court summarized the post-trial procedural history:
At the sentencing phase of the trial, the jury rejected imposition of the death penalty. The trial court then proceeded to sentence [Appellant] to life in prison plus 24 to 48 years. [Appellant] filed post-trial motions, which the trial court denied on January 30, 2006. On February 10, 2006, [Appellant] filed a timely notice of appeal, but [Appellant’s] counsel failed to file an appellate brief and the appeal was consequently dismissed. On September 26, 2008, [Appellant] filed a PCRA petition seeking the reinstatement nunc pro tunc of his direct appeal rights, which the trial court granted on August 14, 2009.
Commonwealth v. Ross, 57 A.3d 85, 90 (Pa. Super. 2012) (en banc).

On October 12, 2012, an en banc panel of this Court vacated Appellant’s judgment of sentence and remanded for a new trial. Id. at 105. On November 17, 2015, the Pennsylvania Supreme Court denied the Commonwealth’s petition for allowance of appeal.

On March 1, 2016, Appellant filed pre-trial motions in which he sought, inter alia, the exclusion of any expert evidence relating to bite mark identification and a Frye hearing. At trial, the Commonwealth intends to introduce the testimony of Dr. Dennis Asen (Dr. Asen) and Dr. Lawrence Dobrin (Dr. Dobrin). Dr. Asen and Dr. Dobrin are both dentists and practice in the field of forensic odontology (the study of the structure of teeth). Dr. Asen and Dr. Dobrin intend to testify that the mark on Ms. Miller’s left breast was caused by a human bite, and when they compared five sets of teeth molds, including one from Appellant, Dr. Asen and Dr. Dobrin could exclude four of the molds from having made the bite mark, but not Appellant’s.

On December 2, 2016, following the filing of several supplemental motions by Appellant and objections by the Commonwealth, the trial court heard oral argument on Appellant’s request for a Frye hearing. On March 8, 2017, after the parties submitted additional briefs on Appellant’s request for a Frye hearing, the trial court entered an order concluding that bite mark identification evidence is not novel and therefore a Frye hearing was not warranted. The court further provided that the Commonwealth’s experts were to adhere to the guidelines set forth by the American Board of Forensic Odontologists (ABFO).

On April 5, 2017, Appellant filed a motion to amend the March 8, 2017 order to include language relating to Pennsylvania Rule of Evidence 702(c), so that the trial court could address whether the expert methodology is generally accepted in the relevant field. Appellant also requested that the court certify for immediate appeal its decision not to hold a Frye hearing on the bite mark identification evidence.

On November 6, 2017, the trial court entered an amended order once again denying Appellant’s request for a Frye hearing. The trial court also included in the order language addressing Rule 702(c) and granting Appellant’s request for certification of immediate appeal. On December 5, 2017, Appellant filed a petition for permission to file an interlocutory appeal with this Court, which we denied by per curiam order on May 7, 2018. On June 1, 2018, Appellant filed a petition for allowance of appeal to the Pennsylvania Supreme Court. On November 20, 2018, our Supreme Court granted Appellant’s petition for allowance of appeal, vacated this Court’s order denying Appellant’s petition for permission to file an interlocutory appeal, and remanded the case to this Court for disposition.

On appeal, Appellant presents the following issues for review:
1) WHETHER THE SUBSTANTIAL EVIDENCE [APPELLANT] PROPOSED TO PRESENT AT A MOVED-FOR FRYE HEARING DISCREDITING BITE MARK ANALYSIS COMES FROM AND IS PART OF THE RELEVANT SCIENTIFIC COMMUNITY FOR FRYE PURPOSES THUS ENTITLING HIM TO A FRYE HEARING; OR WHETHER PENNSYLVANIA COURTS MUST LIMIT THEIR RELIABILITY INQUIRY TO THE VIEWS OF CURRENT PRACTICTIONERS OF THE PARTICULAR TECHNIQUE AT ISSUE?
2) WHETHER THE SUBSTANTIAL EVIDENCE PROFFERED BY [APPELLANT] AT THE MOVED-FOR FRYE HEARING PRESENTED A LEGITIMATE DISPUTE REGARDING THE RELIABILITY OF BITE MARK ANALYSIS THUS ENTITLING HIM TO A FRYE HEARING; AND SUCH THAT THE TRIAL COURT’S ADMISSION OF THE SAME WITHOUT A FRYE HEARING WAS IN ERROR?
Appellant’s Brief at 5.

Both of Appellant’s issues related. Therefore, we address them together. We begin with our standard of review:
As a general rule, this Court’s standard of review of a trial court’s evidentiary ruling, including a ruling whether expert scientific evidence is admissible against a Frye challenge, is limited to determining whether the trial court abused its discretion. Grady v. Frito–Lay, Inc., 839 A.2d 1038, 1046 (Pa. 2003); Zieber v. Bogert, 773 A.2d 758, 760 n.3 (Pa. 2001) (citing Commonwealth v. Minerd, 753 A.2d 225 (Pa. 2000)). “An abuse of discretion may not be found merely because an appellate court might have reached a different conclusion, but requires a result of manifest unreasonableness, or partiality, prejudice, bias, or ill-will, or such lack of support so as to be clearly erroneous.” Grady, 839 A.2d at 1046 (citing Paden v. Baker Concrete Constr., Inc., 658 A.2d 341, 343 (Pa. 1995)).
Commonwealth v. Dengler, 890 A.2d 372, 379 (Pa. 2005) (citations modified).

Appellant argues that the trial court abused its discretion in denying his request for a Frye hearing on the admissibility of the Commonwealth’s bite mark identification evidence. This Court recently articulated the legal standards implicated by a Frye analysis:
The Frye standard originally was intended to prevent the situation in which a party would seek to introduce scientific evidence that was so new that it would be impossible to “produce rebuttal experts, equally conversant with the mechanics and methods of a particular technique.” [U.S.] v. Addison, 498 F.2d 741, 744 (D.C. App. 1974). Frye contemplated a judicial inquiry, informed by experts, into the general acceptance of the scientific methods used. The standard required that “the thing from which the [expert’s] deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.” Frye, supra at 1014. At issue in Frye was admissibility of the systolic blood pressure deception test, commonly known as the lie detector test. The trial court excluded the evidence, and the court affirmed that ruling on appeal, explaining:
Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.
Frye, supra at 1014. Pennsylvania adopted the Frye standard in Commonwealth v. Topa, 369 A.2d 1277 (Pa. 1977), a case involving the propriety of the trial court’s admission of voice print identification evidence through an expert, Lieutenant Nash, of the Michigan State Police. Our High Court, applying Frye, reasoned that
[t]he requirement of general acceptance in the scientific community assures that those most qualified to assess the general validity of a scientific method will have the determinative voice. Additionally, the Frye test protects prosecution and defense alike by assuring that a minimal reserve of experts exists who can critically examine the validity of a scientific determination in a particular case. Since scientific proof may in some instances assume a posture of mystic infallibility in the eyes of a jury of laymen, the ability to produce rebuttal experts, equally conversant with the mechanics and methods of a particular technique, may prove to be essential.
Topa, supra at 1282 (quoting Addison, supra at 744). The Topa Court went on to conclude that the testimony of one expert could not satisfy this standard, citing commentaries questioning the reliability of sound spectrographs and voiceprints and demonstrating that it was not generally accepted within the field of acoustical science.
Thus, the Frye standard originally was intended to prevent a party from introducing scientific evidence that was so new that it would be impossible to “produce rebuttal experts, equally conversant with the mechanics and methods of a particular technique.” Addison, supra at 744. Frye contemplated a judicial inquiry, informed by experts, into the general acceptance of the scientific methods used.

In the years since the adoption of the Frye standard, this Court has clarified that “Frye only applies to determine if the relevant scientific community has generally accepted the principles and methodology the scientist employs, not the conclusions the scientist reaches.” Trach v. Fellin, 817 A.2d 1102, 1112 (Pa.Super. 2003) (en banc).
Walsh v. BASF Corp., 191 A.3d 838, 842-43 (Pa. Super. 2018), appeal granted, 203 A.3d 976 (Pa. 2019). 3/

The above principles have been incorporated into Pennsylvania Rule of Evidence 702, which states:
A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if: (a) The expert’s scientific, technical, or other specialized knowledge is beyond that possessed by the average layperson; (b) The expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; (c) The expert’s methodology is generally accepted in the relevant field. Pa.R.E. 702 (emphasis added).
Appellant argues that the trial court erred in determining a Frye hearing was unnecessary in this case. Appellant asserts that the court wrongly concluded that the methodology employed by the Commonwealth’s experts in conducting their bite mark identification analysis was generally accepted in the scientific community of forensic odontology. Additionally, Appellant contends that the trial court should have examined whether the expert’s methodology for bite mark identification analysis was generally accepted in the broader scientific community, as opposed to limiting its examination to the field of forensic odontology. Appellant maintains that bite mark identification analysis has applications in several different scientific communities, and several of these communities have determined that bite mark identification analysis is not reliable and has little or no evidentiary value.

In support of his argument, Appellant relies on our Supreme Court’s decision in Betz v. Pneumo Abex, LLC, 44 A.3d 27 (Pa. 2012). At issue in Betz was the admissibility of expert opinion evidence relating to the “any-exposure” theory of legal causation, i.e., “that each and every exposure to asbestos – no matter how small – contributes substantially to the development of asbestos-related diseases.” Id. at 30. As part of its analysis, the Supreme Court had to first address, as a threshold issue, whether the trial court was correct in concluding that a Frye hearing was necessary to determine the admissibility of the “any-exposure” evidence. Id. at 52-55.

In concluding that the trial court’s decision to conduct a Frye hearing was correct, the Supreme Court explained:
There is inherent tension among the various measures for admissibility of expert testimony. The threshold common law test requires merely some reasonable pretension to specialized knowledge. See, e.g., Miller v. Brass Rail Tavern, Inc., 664 A.2d 525, 528 (Pa. 1995). Our evidentiary rules, on the other hand, suggest trial courts may take a greater role in assessing whether the testimony will assist the trier of fact to understand the evidence or determine a fact in issue, see Pa.R.E. 702, and in screening evidence to avoid unfair prejudice, confusion of the issues, or misleading of the jury, see Pa.R.E. 403. For better or for worse, however, in the context of the more conventional realms of science, the Pennsylvania decisions tend to downplay the courts’ screening function. See, e.g., Commonwealth v. Nazarovitch, 436 A.2d 170, 172 (Pa. 1981) (“[C]ourts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery[.]” (quoting Frye, 293 F. at 1014)). A manifestation of this trend is that challenges generally are vetted through the Frye litmus, which winnows the field of the attacks by application of the threshold requirement of novelty. See Grady, 839 A.2d at 1043-44.

Various reasons underlie the preference to limit the courts’ involvement in determining the admissibility of scientific evidence. There is the concern that liberality in allowing challenges would substantially increase the number of challenges (and cases in which lengthy pre-trial proceedings would ensue). The competency of trial judges to accept or reject scientific theories remains a legitimate subject of controversy. Additionally, a claim or defense in many cases may rise or fall based upon expert testimony and, therefore, there is some reluctance on the part of courts to deprive litigants of their day in court.

On the other hand, this Court has recognized the influential nature of expert testimony on complex subjects, and the potential that distortions have to mislead laypersons. See id. at 1045; Topa, 369 A.2d at 1281-82. It would be naïve, in this regard, to assume that the possibility for distortion is limited to the very newest realms of science. Cf. Grady, 839 A.2d at 1045 (explaining that Frye applies not only to novel science, but also where scientific methods are utilized in a novel way).

We therefore agree with Appellants that a reasonably broad meaning should be ascribed to the term “novel.” Furthermore, we conclude that a Frye hearing is warranted when a trial judge has articulable grounds to believe that an expert witness has not applied accepted scientific methodology in a conventional fashion in reaching his or her conclusions. Accord id. We believe a narrower approach would unduly constrain trial courts in the appropriate exercise of their discretion in determining the admissibility of evidence. See id. at 1046.
Id. at 52-53 (emphasis added, footnote omitted, citations modified).

Additionally, in Betz, the Supreme Court rejected the notion that the defendants could not challenge the methodology of the plaintiff’s expert pathologist with the testimony of risk assessors, toxicologists, and epidemiologists. Id. at 54. The Court reasoned that the expert pathologist’s “any-exposure” opinion “was not couched in terms of a methodology or standard peculiar to the field of pathology[,]” and “was plainly grounded on risk assessment.” Id. at 54-55. Thus, the Supreme Court determined that the subjects at issue “are not within the particular expertise of a pathologist, but, rather, are interdisciplinary in character.” Id. at 55.

In this case, the trial court reached two conclusions in denying Appellant’s request for a Frye hearing. First, the trial court determined that “[b]ite mark evidence is currently generally accepted in the relevant scientific community of forensic odontologists.” Trial Court Opinion, 11/6/17, at 5. The court explained:
The American Board of Forensic Odonotologists (ABFO) has promulgated guidelines regarding the approved methodology concerning bite mark evidence. This [c]ourt notes that the aforementioned methodology rejects the notion that human dentitions are unique to the individual. Any evidence proffered by the Commonwealth to suggest that an individual can be identified by a bite mark, rather than merely excluded or not excluded as having made the mark, would be inadmissible.
Id.

Second, the trial court rejected Appellant’s reliance on Betz, finding that case distinguishable:
It is true that the Pennsylvania Supreme Court allowed the Betz defendants to address the methodology of a pathologist through the testimony of risk assessors, toxicologists, and epidemiologists. However, Betz can be distinguished from the instant case. The Betz Court found that the pathologist’s opinion was not “couched in terms of a methodology or standard peculiar to the field of pathology.” [Betz, 44 A.3d at 54]. Rather, the pathologist’s opinion was “plainly grounded on risk assessment” and was “interdisciplinary in character.” Id. at 55. The Betz Court based its decision not upon the role of pathologists generally, but upon the specific methodology employed by the individual pathologist in question. Betz does not make a broad assertion that Pennsylvania law “mandates a broader definition of the relevant scientific community” where the methodology of a pathologist is at issue. Here, the Commonwealth does not seek to offer the kind of broad-scale scientific testimony that was at issue in Betz.
Id. at 6.

After careful consideration, we find support for Appellant’s position that a Frye hearing was warranted in this case. First, with respect to the trial court’s conclusion that bite mark identification analysis has general acceptance in the field of forensic odontology, Appellant offered evidence indicating that there is a lack of consensus among forensic odontologists on whether bite mark identification analysis is reliable and valid. While there is no dispute that the ABFO has established standards and a methodology for conducting bite mark identification analysis, see Commonwealth’s Third Brief in Opposition to Appellant’s Request for a Frye Hearing, 2/21/17, Exhibit B, Appellant presented numerous reports to the trial court indicating that practitioners within the community of forensic odontology question whether this methodology reliably enables forensic odontologists to identify an injury as a human bite mark.

For example, Appellant presented evidence from a presentation by Dr. David Senn, DDS, Vice-President of the American Board of Forensic Odontology, to the National Academies: Committee on Identifying the Needs of the Scientific Community. See Defendant’s Post-Argument Supplement to “Defendant’s Motion in Limine: Frye Test – Bite Mark Evidence” (hereinafter Defendant’s Post-Argument Supplement), 1/17/17, Exhibit 4 (Presentation to the National Academies: Committee on Identifying the Needs of the Forensic Science Community – Forensic Odontology Bite Marks (hereinafter Senn Presentation), 4/23/07, at 31-34). While Dr. Senn opined that bite mark identification analysis was important to the investigation and adjudication of certain crimes, id. at 45, Dr. Senn identified several “major problems” with bite mark identification analyses, including: “[t]he uniqueness of the human dentition has not been scientifically established”; “[t]he ability of the dentition, if unique, to transfer a unique pattern to human skin and maintain that uniqueness has not been scientifically established”; “[a] clear statement of the type, quality, and number of class and individual characteristics or other features required to indicate that a bite mark has reached a threshold of evidentiary value has not been established”; and “Forensic Odontology certifying organizations have not created or administered bite mark analysis proficiency tests for their board certified members.” Id. at 31-34.

Likewise, Appellant presented a report by the President’s Council of Advisors on Science and Technology, which revealed the following:
Empirical research suggest that forensic odontologists do not consistently agree even on whether an injury is a human bitemark at all. A study of the American Board of Forensic Odontology (ABFO) involved showing photos of 100 patterned injuries to ABFO board-certified bitemark analysts, and asking them to answer three basic questions concerning (1) whether there was sufficient evidence to render an opinion as to whether the patterned injury is a human bitemark; (2) whether the mark is a human bitemark, suggestive of a human bitemark, or not a human bitemark; and (3) whether distinct features (arches and toothmarks) were identifiable. Among the 38 examiners who completed the study, it was reported that there was unanimous agreement on the first question in only 4 of the 100 cases and agreement of at least 90 percent in only 20 of the 100 cases. Across all three questions, there was agreement of at least 90 percent in only 8 of the 100 cases.
Defendant’s Post-Argument Supplement, 1/17/17, Exhibit 5 (PRESIDENT’S COUNCIL OF ADVISORS ON SCIENCE AND TECHNOLOGY, REPORT TO THE PRESIDENT: FORENSIC SCIENCE IN CRIMINAL COURTS: ENSURING SCIENTIFIC VALIDITY OF FEATURE-COMPARISON METHODS (hereinafter PCAST Report) 84-85 (2016)) (emphasis added).

The studies proffered by Appellant challenge the trial court’s conclusion that the ABFO’s methodology for using bite marks to eliminate persons as suspects is generally accepted in the field of forensic odontology. These reports reflect that individuals within the forensic odontology community question not only whether the ABFO’s methodology can reliably aid experts in using bite marks to validly identify or exclude individuals as criminal actors, but also whether the methodology enables experts to identify a wound as a human bite mark. Therefore, Appellant provided the trial court with articulable grounds that the Commonwealth’s expert witnesses on bite mark identification analysis have not applied accepted scientific methodology in reaching their conclusions. See Betz, 44 A.3d at 53. Accordingly, we conclude that the trial court abused its discretion in denying Appellant’s request for a Frye hearing.

We are likewise persuaded by Appellant’s argument that the trial court should not have limited consideration of the general acceptance of the experts’ methodology in this case to the field of forensic odontology. As several of the reports Appellant cites reflect, bite mark identification analysis implicates numerous scientific fields. See generally Defendant’s Post-Argument Supplement, 1/17/17, Exhibit 2 (NATIONAL ACADEMY OF SCIENCES, COMMITTEE ON IDENTIFYING THE NEEDS OF THE FORENSIC SCIENCES COMMUNITY, STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES: A PATH FORWARD (2009), Exhibit 5 (PCAST Report), Exhibit 11 (Michael J. Saks, et al., Forensic bitemark identification: weak foundations, exaggerated claims, 3(3) J. LAW BIOSCI. 1 (2016)). As these studies indicate, bite mark identification analysis not only involves concepts relating to forensic science generally, but also pathology, biology, statistics, and metrology. See Defendant’s Post-Argument Supplement, 1/17/17, Exhibit 11 (Michael J. Saks, et al., Forensic bitemark identification: weak foundations, exaggerated claims, 3(3) J. LAW BIOSCI. 1 (2016)) (noting that the forensic identification process is “fundamentally probabilistic” and that it involves the interplay of different scientific disciplines including blood (pathology), skin (biology and dermatology), and measurements (metrology)). Because the act of biting a human involves not only the biter’s teeth, but also the skin, muscle, tissue, and blood with which the teeth make contact, the notion that bite mark identification analysis involves scientific disciplines beyond forensic odontology is reasonable.

Finally, we emphasize that our decision in no way represents a determination as to the general acceptance of the methodology underlying bite mark identification analysis utilized by the Commonwealth’s experts in this case. We make no judgment as to the admissibility of the bite mark identification evidence at issue. Rather, we simply conclude that Appellant provided the trial court with articulable grounds to believe that the Commonwealth’s expert witnesses on bite mark identification analysis may not have applied generally accepted scientific methodology in reaching their conclusions, and consequently, the trial court erred in concluding that a Frye hearing was not necessary. While Appellant’s evidence expresses negative opinions on bite mark identification analysis, we cite it only to support our conclusion that a Frye hearing is proper for the resolution of these discrepancies, and to afford both parties the opportunity to present evidence in support of their positions. Therefore, we vacate the order denying Appellant’s request for a Frye hearing and remand this matter to the trial court for a hearing in accordance with Frye.

Order vacated. Case remanded. Jurisdiction relinquished. Judgment Entered.

Footnotes

* Retired Senior Judge assigned to the Superior Court.
1. A Frye hearing, named after the decision in Frye v. United States, 293 F. 1013 (D.C. Cir. 1923), “is a hearing held for the trial court to determine whether the general scientific community has reached a general acceptance of the principles and methodology used by the expert witness.” Commonwealth v. Walker, 92 A.3d 766, 769 n.1 (Pa. 2014).
2. 18 Pa.C.S.A. §§ 2501(a), 2702(a)(1), 3123(a)(1), 2902(a)(1), 2701(a)(1), 2903(a), 3126(a)(2).
3. This case was argued before our Supreme Court on October 15, 2019, and is awaiting disposition. See Walsh v. BASF Corp., 203 A.3d 976 (Pa. 2019). The issue before our Supreme Court is whether, in conducting a Frye analysis, “trial courts are not permitted to act as ‘gatekeepers’ to ensure the relevance and reliability of scientific studies offered by experts to support their opinions by scrutinizing whether those studies actually support their opinions.” Id. at 978. While the Supreme Court may overturn our Walsh decision, their disposition will have no bearing on the outcome of this case or the general standards triggering the necessity of a Frye hearing that were thoughtfully set forth in the Walsh decision.

Monday, November 11, 2019

More on Rule 704 and Source Attribution in Forensic Science

On July 17, I observed that
At a webinar on probabilistic genotyping organized by the FBI, the Department of Justice’s Senior Advisor on Forensic Science, Ted Hunt, summarized the rules of evidence that are most pertinent to scientific and expert testimony. In the course of a masterful survey, he suggested that Federal Rule of Evidence 704 somehow conflicts with the evidence-centric approach to evaluating laboratory results recommended by a subcommittee of the National Commission on Forensic Science, by the American Statistical Association, and by European forensic-science service providers.
I proceeded to analyze Rule 704 and its application to the conclusion-centric testimony used in US courts. In a comment posted on July 21 (and reproduced below), Mr. Hunt generously took the time to respond. Taken together, his webinar presentation, my posting on it, and his comment on this blog discuss three issues:
(1) What does Rule 704 reveal about the proper or permissible role of an expert witness?
(2) Are certain publications on the desirability of the evidence-centric approach deficient or misleading for failing to mention Rule 704?
(3) Does Rule 704 pertain to every source-attribution opinion?
As explained below, the short answers are "not much," "no," and "no," respectively. In additiion, an essay on the first and third issues will appear in the Winter 2020 issue of Jurimetrics Journal.

I

Mr. Hunt commented that “I don’t disagree that Rule 704 ‘does not counsel in favor’ of admitting expert opinion on an ultimate issue. I never said or implied anything or the sort.” At the same time, he wrote that “the Federal Rules (and most state rules of evidence) affirmatively allow an expert to offer an opinion about a proposition.”

Listening to the talk, I thought I was hearing the germ of a legal argument that Rule 704 not only allows, but also affirmatively supports the admission of source-attribution opinions such as “I am sure that the defendant is the source of the latent print.” The recording of the webinar is on the Forensic Technology Center of Excellence website, I transcribed the two small parts that pertain to Rule 704. I think I have captured all the words on Rule 704 and source opinions; they are in Appendix A.

The talk begins with a reference to “lots of statements” on the role of the expert witness “that have been pushed out in the literature” and therefore merit “a deeper dive and look at the American judicial system and the role of the expert in it.” After reviewing the most important rules affecting expert and scientific testimony generally, Mr. Hunt quoted several statements from organizations to the effect that experts ought to explain the degree to which the scientific data support the relevant hypotheses (“propositions” about the source of the data) rather than opine on the truth of the hypotheses themselves. Various forensic scientists, psychologists, law professors, and others have been advocating this approach for decades.

Mr. Hunt cautioned the audience as follows:
However, in the US system there is Rule 704, which is a rule that you don't hear much about, and I think there are some people who would like to pretend that it doesn't exist, but it actually goes against that school of thought and allows an expert to opine on an ultimate contested proposition before the court, not only on the strength of the evidence for or against the proposition. Specifically an opinion is not objectionable just because it embraces an ultimate issue, and you can add proposition in there as well.
He concluded his exposition of Rule 704 with the observation "So in the DNA context this would allow a source-attribution statement" and then described a few cases that accepted DNA source conclusions.

To my ear, the suggestion that people are pretending that Rule 704 does not exist because "it actually goes against that school of thought" sounded like an argument that Rule 704 is an affirmative reason for a court to allow traditional source-attribution testimony. My criticism of this argument (whether or not it was Mr. Hunt's) had two components. First, sometimes source opinions are not opinions on "ultimate" issues or facts—things like whether the killer was insane or whether the deceased committed suicide. In those cases, the rule does not even come into play.

Second, to say that Rule 704(a) "allows" source attributions is merely to say that a previous and ill-defined common-law rule on opinion testimony is no longer a reason to exclude the testimony. Rule 704(a) just cancels out a categorical rule of exclusion, leaving the court with a big zero. The rule does not militate either for or against the idea that forensic scientists should avoid proffering source opinions. It merely means that the objecting party had better have some other reason for the court to exclude the opinion.

This much is true whether the issue or fact is called ultimate or intermediate. Parties opposing source-attribution opinions could argue for exclusion under the helpfulness requirement of Rule 702 or the balancing mandated by Rule 403. The documents cited by Mr. Hunt do not invoke these rules, just as they do not discuss Rule 704, because they are not making a legal argument.

In emphasizing that Rule 704 is merely a rule of non-exclusion, I hasten to add that I am not suggesting that Mr. Hunt misrepresented the rule as mandating admission of ultimate-opinion testimony. He did nothing of the kind. The undisputed legal principle is that Rule 704 is agnostic toward ultimate-opinions. As Mr. Hunt agreed, it does not counsel in favor of their admission. Once that much is understood, readers can decide for themselves whether the rule “actually goes against that school of thought” that asks criminalists to avoid expressing opinions on the truth of hypotheses.

II

To emphasize that he was not suggesting "that Federal Rule of Evidence 704 somehow conflicts with the evidence-centric approach to evaluating laboratory results," as I originally put it, Mr. Hunt explained that he only was trying to correct a possible misunderstanding of what he called the "role assignment" documents:
My comments about FRE 704 were directed to the claim of ‘role assignment’ by the authors of the cited sources. The documents I cited (ENFSI, Evett, NIFS, AFSP, NCFS, ASA) make statements such as ‘the role of the expert is to comment on the probability of their findings, given these propositions and not on the propositions themselves.’ (NIFS).

I simply noted that this purported assignment of ‘roles’ is more limited than that which is permitted by the Federal Rules of Evidence. None of these documents concede (or even acknowledge) that the Federal Rules (and most state rules of evidence) affirmatively allow an expert to offer an opinion about a proposition, rather than limiting that opinion to the relative strength of the evidence in support of competing propositions.

Omitting this information may mislead some to believe that the ‘role’ assigned by these documents to an expert is instead assigned by legal rules, rather than a ‘best practices’ world view.
***
In conclusion, to the extent there is any ‘tension’ or ‘inconsistency’ between Rule 704 and the noted references, it was created by the referenced documents’ omission of a discipline-specific basis for their claim that ‘the role of the expert is to comment on the probability of their findings, given these propositions and not on the propositions themselves.’ Each of the cited sources fail to make it clear that this limited ‘role’ is self-imposed, not legal in nature. That was the point of my observation.
Informing people that the law allows well founded source-attribution opinions is entirely reasonable. Only a few courts depart from that generalization. But the position papers and guides to evaluative reporting with which I am familiar surely are not deficient, deceptive, or misleading about this.

The document from which the quoted words, "role of the expert," come is the Introductory Guide to Evaluative Reporting prepared by a special committee created by the National Institute of Forensic Science. NIFS is a directorate within the Australian and New Zealand Policing Advisory Agency (ANZPAA). The "role assignment" language occurs within a box contrasting "traditional reporting" and "evaluation using a logical framework." I don't care for the phrase "logical framework" (to the extent it intimates that nothing else is logical), but the words "role of the expert" are appropriately used only to refer to the role of the expert within the stated alternative to the traditional approach. Let’s put the words back into the full sentence in which they occur (with the language that Mr. Hunt questioned in bold):
The fundamental principles of evaluative reporting or interpretation are that (i) the crime must be considered to have occurred within a framework of circumstances, (ii) that the findings must be considered in light of at least two competing propositions that will be guided by the case circumstances and (iii) that the role of the expert is to comment on the probability of their findings, given these propositions and not on the propositions themselves. (P. 6).
The first page of the NIFS Guide explains that the purpose is to introduce readers to an alternative to traditional and continuing practice and to “provide[] managers with advice on implementing evaluative reporting where they consider it appropriate.” (P. 1, emphasis added). A "discipline-specific basis for their claim" that reporting likelihoods has cross-disciplinary advantages over elimination-versus-identification opinions would make little sense in a document designed to enable "many traditional comparative forensic disciplines" to produce reports and testimony in "ways that can be standardised between analysts within, and between, organisations and forensic disciplines." (P. 6).

Little would be gained by reviewing the details of the other documents Mr. Hunt feels should have "conceded" or "acknowledged" the existence of Rule 704. I cannot see how many readers of these documents could construe them as describing the current rules of U.S. evidence law (or that of other countries), 1/ but it is always good to to distinguish between legal and statistical or scientific principles.

Adhering to this distinction, there is a legal argument for the role that the documents recommend for the expert. Proponents of this “likelihood-ratio” or “Bayes-factor” mode of presentation believe that it nicely captures the role of a scientist supplying the legal factfinder with information based strictly on the physical or chemical features of the specimens that are compared. DNA experts do not normally testify that the defendant's DNA is present in the crime-scene DNA. Instead, they testify to probability of the measured quantities of identifying DNA fragments (or to the probability of the alleles inferred from these measurements) if the defendant's DNA is not present in the sample. They also may testify to the probability if it is, or to the ratio of these probabilities.

To reach the further conclusion that the defendant's DNA actually is in the questioned sample would require information outside of the fragment lengths measured in the laboratory. In Bayesian terms, any conclusion about the true source "necessarily rests, in part, on an implicit assessment or assumption about the prior odds that the suspect is the source," and “[m]aking those assessments or assumptions takes forensic scientists beyond their scientific expertise in ways that arguably usurp the role of legal fact-finders." 2/ The core of the argument is that experts exceed their legally assigned role of assessing the physical evidence when they venture beyond the scientific data to which the likelihood ratio pertains. "Experts are rarely in a good position to evaluate the prior odds that the items they are comparing have a common source, and arguably have no business doing so." 3/

That criminalists dealing with other types of evidence were implicitly opining on prior probabilities beyond their expertise was not obvious in the days when it was thought that every specimen—I'll use fingerprints as an example—is unique. Uniqueness of prints makes it sound as if the criminalist does not have to think about the population from which a latent print came in order to conclude that the print had to be the defendant's. But uniqueness of friction ridge skin is not a sufficient basis for a source attribution. It overlooks the fact that latent print examiners do not deal with three-dimensional skin. They compare impressions of a complete and clear set of prints to incomplete and often low quality impressions. The two-dimensional representations vary from one impression to the next. The only question the analyst can answer strictly on the basis of the features (and knowledge of their joint distribution in a relevant population) is how the probability of a latent print being as similar as it is to the exemplar print—when it is, in fact, from the finger that produced the exemplar print—compares to the probability of the latent print being so similar when it is from someone else's finger.

This "role-assignment" argument is not conclusive. What if experts legitimately can say that in every relevant population of plausible size, the probability is very high that only one finger could have produced the observed features? Or that the likelihood ratio is so huge that it swamps the prior probability for an individual randomly drawn from that population? Then the expert who indulges in source attribution (without specifying a prior probability for the judge or jury to consider) arguably would not exceed the role of the expert defined in Federal Rule of Evidence 702.

III

Moving back to the content of Rule 704, it seems that we have a sharp disagreement about the reach of the "ultimate issue" rule that Rule 704 (building on earlier case law and academic criticism) decisively rejected. The rule applies to any opinion from any witness—expert or lay—that "embraces an ultimate issue." But what does that phrase mean? I maintained that Rule 704 does not apply to all source attributions because "the identity of the source of a trace is not necessarily an ultimate issue." To make this point, I wrote as follows:
To use the example of latent-print identification given in the webinar, the traditional opinion that a named individual is the source of a print is not an opinion on an ultimate issue. Courts have long allowed examiners to testify that the print lifted from a gun comes from a specific finger. But this conclusion is not an opinion on whether the murder defendant is the one who pulled the trigger. The examiner’s source attribution bears on the ultimate issue of causing the death of a human being, but the examiner who reports that the prints were defendant's is not opining that the defendant not only touched the gun (or had prints planted on it) but also pulled the trigger. Indeed, the latent print examiner would have no scientific basis for such an opinion on an element of the crime of murder.
Mr. Hunt's comment insists that every source attribution is an ultimate-issue opinion to which Rule 704 applies. Here is his argument:
[Y]our blog states that ‘the identity of the source of a trace is not necessarily an ultimate issue.’ As evidence for that assertion, you used my latent print example, claiming that ‘the traditional opinion that a named individual is the source of a print is not an opinion on an ultimate issue.’

Your claim is directly contrary to the contents of the judicial opinion I cited in the questioned slide, which states in full: ‘[Defendant] . . . contends that the district court erred in allowing the expert to specifically testify that he had no doubt that the prints on the parking stub were [Defendant’s] because the expert was testifying to the "ultimate issue’ of whether it was Sherwood's print. This argument fails in light of Fed. R. Evid. 704, which permits expert testimony on the ultimate issue in a case.” United States v. Sherwood, 98 F.3d 402, 408 (9th Cir. 1996).

In Sherwood, the ultimate issue was not described as a connection between the latent print and activity-level behavior; rather, it was the expert’s testimony about the source of that print. Your description of an ultimate issue as limited to one in which (using your example), ‘the murder defendant is the one who pulled the trigger’ is a narrower description of the types of factual assertions entailed by Rule 704 than those described throughout case law. As such, your explanation of the Rule is not entirely correct.

For other examples of courts describing identification testimony as ‘ultimate’ in nature, see U.S. v. Rose, 731 F.2d 1337, 1346-47 (print match to shoe) United States v. Bice-Bey, 701 F.2d 1086 (4th Cir. 1883) (voice identification); United States v. Sellers, 566 F.2d 884 (4th Cir. 1977) (identification of suspect in a picture); United States v. Couser, 15 Fed. R. Evid. Serv. (CBC) 837 (4th Cir. Apr. 5, 1984) (comparison of suspect’s features to photograph). These cases clearly demonstrate that source identification testimony constitutes an opinion that embraces an ‘ultimate’ fact.
It is true that the Ninth Circuit in Sherwood referred to the source attribution as an "ultimate" opinion. I assume every other case he lists did the same. However, this use of authority is an instance of looking for law in all the wrong places. Every time a party objects, "Your honor, my identity as the source of the trace is an ultimate issue!", the court can summarily reject the objection by citing Rule 704's text. Why bother to try to trace the boundaries of the "the so-called 'ultimate issue' rule" (Fed. R. Evid. 704 Advisory Committee's Note) when the murky rule "is specifically abolished" (id.)? Regardless of the reach of Rule 704, the outcome is the same. If the expert opinion is outside the old rule, then it is "not objectionable" just because it embraces an ultimate issue. If it is covered by the old rule, it is "not objectionable" because the old rule is no longer a barrier.

So the fact that modern courts often assume—without analysis—that a source opinion is ultimate-opinion testimony tells us next to nothing about how the opinion would have fared under the defunct common-law exclusionary rule. To answer that arcane question, we must look to the old cases applying that rule to exclude opinion testimony.

Where should we begin? McCormick on Evidence § 12, at 80 (7th ed. 2013), documents the "general doctrine that witnesses may never give their opinions or conclusions on an ultimate fact in issue" with two cases: United States v. Spaulding, 293 U.S. 498 (1935), and State v. Carr, 196 N.C. 129, 144 S.E. 698 (1928). In Spaulding the "ultimate issue" was whether the plaintiff was totally disabled while an insurance policy was in force. He produced "medical opinions" that he "became totally and permanently disabled before his policy lapsed." The Supreme Court wrote "that question is not to be resolved by opinion evidence. It was the ultimate issue ... . The experts ought not to have been asked or allowed to state their conclusions on the whole case." 293 U.S. at 506.

Carr is a homicide case. The defense was that a man who died in bed of a gunshot wound had committed suicide. The state produced a medical doctor who testified that "I don't think it is possible for the deceased to have fired the gun and made the wound that I saw." 144 S.E. at 700. The North Carolina Supreme Court held that the trial court erred in admitting the opinion. It explained that
[T]he opinion or inference of the witness must not be an answer to the exact issue which the jury is to determine. When the witness testified that he did not think it possible for the deceased to have fired the gun and to have made the wound he necessarily testified in effect that in his opinion the deceased did not kill himself. True, the "exact issue" was whether the defendants are guilty, but if the deceased killed himself the conclusion that the defendants did not kill him would necessarily follow.
Id. These cases reveal that the essence of the ultimate-issue rule is that the witness may not opine on a fact that determines whether an element of the offense or complaint, or of the defense, is satisfied.

Other facts are intermediate. They are links in a chain of reasoning that terminates in the answer to one of these ultimate issues. In an action for a declaration of paternity, for example, an expert opinion that (based on DNA tests) the defendant is the father, falls under the former ultimate-issue rule. But suppose the government introduces an expert opinion that a murder defendant is the father of a fetus whose remains were found along with the mother's merely to show that he had a motive to kill the mother, who refused to have an abortion as he had demanded. Now the source conclusion is not an opinion on an ultimate issue. The jury can accept the source opinion but conclude that defendant, despite the motive, was not the killer. In contrast to Spaulding, the expert opinion does not decide "the whole case." Unlike Carr, the ultimate conclusion does not "necessarily follow."

On which side of the ultimate-intermediate line the examiner's opinion in Sherwood falls I cannot say. The opinion is too sparse. But the question is academic. One purpose of Rule 704 is to spare the courts from having to pin these labels on opinion testimony. (See Advisory Committee's Note, observing that the old rule was "difficult of application"). With the rule in place, they can just say "so what—the argument fails in light of Rule 704." But that does not mean that a source opinion is always an opinion on "an ultimate issue." That depends on how the opinion fits into the case.When the source opinion is near enough to the "ultimate" endpoint, Rule 704 does not say, "OK, let it in." In conjunction with Rule 702, it says "Well, yes, but there is no mechanical rule excluding it on that basis alone—just address how helpful it is for the jury to hear it."

When courts address the question of helpfulness, they just might conclude that instead of old-style source attributions, statements about the probabilities of the findings under different hypotheses would fulfill the role of helping the factfinder find the facts better that traditional expressions of an examiner's belief that the defendant is the source of the trace. Or they might not. Either way, no one is trying to deny that Rule 704 exists. Why would they when it sheds so little light on the choice between traditional conclusion-centric opinons (source attributions that, to a Bayesian, reflect an expert's personal prior probabilities and utilities) and evidence-centric ones (such as Bayes factors that enable experts to express the probative value of the findings without venturing into that nonscientific territory)?


NOTES
  1. Australia's uniform Evidence Acts § 80(a), adopted by many of its jurisdictions, abolished the common-law rule against ultimate opinions ("[e]vidence of an opinion is not inadmissible only because it is about—a fact in issue or an ultimate issue."). New Zealand did the same in its 2006 Evidence Act ("(2) An opinion by an expert is not inadmissible simply because it is about—(a) an ultimate issue to be determined in a proceeding; or (b) a matter of common knowledge."). 
  2. William C. Thompson, How Should Forensic Scientists Present Source Conclusions?, 48 Seton Hall L. Rev. 773, 791 (2018).
  3. Id.; see also David H. Kaye et al., The New Wigmore, A Treatise on Evidence: Expert Evidence § 2.2.2, at 44 (2d ed. 2011) (notes omitted):
    But this form of opinion leaves the jury with an inappropriate choice. If the jury is to accept the opinion on a matter within the witness’s expertise, it must also accept the expert’s (presumably unarticulated) assessment of evidence that has nothing to do with the expertise and that falls within the jury’s ability to asses. Or, it must accept an unvalidated and implausible theory of individualization.

APPENDIX A

Ted R. Hunt, The Expert’s Role in the American Judicial System (Probabilistic Genotyping of Evidentiary DNA Typing Results – An Online Workshop Series, Module 8: Probabilistic Genotyping Summation and Special Topics, July 17th, 2019, available at https://forensiccoe.org/ (excerpt of all remarks on Rule 704 and source conclusion testimony)

*** My talk is entitled "the role of the expert" because there aren't special rules that just apply to DNA analysts and no one else, and the reason that I want to talk about it today is, in the forensic literature in the last number of years, lots of statements have been made about the role of the courts and the role of the experts, and those statements aren't specific to any judicial system. They're pretty much general statements that have been pushed out in the literature so I thought we'd take a deeper dive and look at the American judicial system and the role of the expert in it.
***

Now I mentioned before at the beginning of this talk there's a lot of literature on the role of the courts and the role of the expert in the forensic literature. I have some examples here. The European Network has its guidelines for evaluative reporting in forensic science. Number 3 here in this excerpt talks about the role of the forensic practitioner to consider the probability of the findings considering the propositions and not the probability of those propositions. Similarly, other documents in the community from Australia and New Zealand, from Dr. Evett, from the Association of Forensic Science Service Providers. All basically say the same thing, talking about the role of the expert to comment on the probability of the findings given those propositions and not the prepositions themselves.

This has gotten into the US forensic community to a certain extent. On the left is the National Commission [on Forensic Science] Views Document. I just want to be clear this document did not get voted to approval through the Commission, but nevertheless there is a final stand-alone version to that document that basically says the same thing: Forensic practitioners should not state that a specific individual or object is the source of forensic evidence and refers to the formal method of evaluative reporting as well. And basically the same people who wrote that document wrote the American Statistical Association document which, for the most part, says the same thing, discouraging the statements to the effect that a specific person or object is the source of evidence.

However, in the US system there is Rule 704, which is a rule that you don't hear much about, and I think there are some people who would like to pretend that it doesn't exist, but it actually goes against that school of thought and allows an expert to opine on an ultimate contested proposition before the court, not only on the strength of the evidence for or against the proposition. Specifically an opinion is not objectionable just because it embraces an ultimate issue, and you can add proposition in there as well.

I need to add that 704 does not lower the bar to admit otherwise inadmissible evidence. It will work in conjunction with the other rules of evidence. The only exception in the American system or the US federal system is that the expert may not opine on the defendant's mental state —an ultimate legal proposition before the court—or witness credibility. However, they can opine on an ultimate issue.

Just as an example, this is a latent fingerprint case from the 9th circuit in 1996 [United States v. Sherwood, 98 F.3d 402 (9th Cir. 1996)] and a defendant objected that an expert was specifically testifying that he had no doubt that the prints on the piece of evidence were the defendant’s. The defendant objected because the expert was testifying to an ultimate issue, and the court basically said, so what—the argument fails in light of 704, which permits an expert to opine on ultimate issues.

Like I said before, I’m trying to be very descriptive. I’m not being normative, and I’m not weighing in on the logic or the advisability of this. I'm simply trying to correctly state what I understand the law to be.

So in the DNA context this would allow a source attribution statement. ***


APPENDIX B

Comment posted to this blog, July 21, 2019

Professor Kaye,
     My comments about FRE 704 were directed to the claim of ‘role assignment’ by the authors of the cited sources. The documents I cited (ENFSI, Evett, NIFS, AFSP, NCFS, ASA) make statements such as ‘the role of the expert is to comment on the probability of their findings, given these propositions and not on the propositions themselves.’ (NIFS).
     I simply noted that this purported assignment of ‘roles’ is more limited than that which is permitted by the Federal Rules of Evidence. None of these documents concede (or even acknowledge) that the Federal Rules (and most state rules of evidence) affirmatively allow an expert to offer an opinion about a proposition, rather than limiting that opinion to the relative strength of the evidence in support of competing propositions.
     Omitting this information may mislead some to believe that the ‘role’ assigned by these documents to an expert is instead assigned by legal rules, rather than a ‘best practices’ world view. I never said or implied that Rule 704 ‘stands in the way’ of an expert’s provision of a more limited opinion about the strength of the evidence relative to competing propositions.
     In addition, your blog states that ‘the identity of the source of a trace is not necessarily an ultimate issue.’ As evidence for that assertion, you used my latent print example, claiming that ‘the traditional opinion that a named individual is the source of a print is not an opinion on an ultimate issue.’
     Your claim is directly contrary to the contents of the judicial opinion I cited in the questioned slide, which states in full: ‘[Defendant] . . . contends that the district court erred in allowing the expert to specifically testify that he had no doubt that the prints on the parking stub were [Defendant’s] because the expert was testifying to the "ultimate issue’ of whether it was Sherwood's print. This argument fails in light of Fed. R. Evid. 704, which permits expert testimony on the ultimate issue in a case.” United States v. Sherwood, 98 F.3d 402, 408 (9th Cir. 1996).
      In Sherwood, the ultimate issue was not described as a connection between the latent print and activity-level behavior; rather, it was the expert’s testimony about the source of that print. Your description of an ultimate issue as limited to one in which (using your example), ‘the murder defendant is the one who pulled the trigger’ is a narrower description of the types of factual assertions entailed by Rule 704 than those described throughout case law. As such, your explanation of the Rule is not entirely correct.
     For other examples of courts describing identification testimony as ‘ultimate’ in nature, see U.S. v. Rose, 731 F.2d 1337, 1346-47 (print match to shoe) United States v. Bice-Bey, 701 F.2d 1086 (4th Cir. 1883) (voice identification); United States v. Sellers, 566 F.2d 884 (4th Cir. 1977) (identification of suspect in a picture); United States v. Couser, 15 Fed. R. Evid. Serv. (CBC) 837 (4th Cir. Apr. 5, 1984) (comparison of suspect’s features to photograph). These cases clearly demonstrate that source identification testimony constitutes an opinion that embraces an ‘ultimate’ fact.
     I don’t disagree that Rule 704 ‘does not counsel in favor’ of admitting expert opinion on an ultimate issue. I never said or implied anything or the sort.
    In conclusion, to the extent there is any ‘tension’ or ‘inconsistency’ between Rule 704 and the noted references, it was created by the referenced documents’ omission of a discipline-specific basis for their claim that ‘the role of the expert is to comment on the probability of their findings, given these propositions and not on the propositions themselves.’ Each of the cited sources fail to make it clear that this limited ‘role’ is self-imposed, not legal in nature. That was the point of my observation.
     Thanks for your participation in the webinar series and for your interest in my talk.

Ted R. Hunt
Senior Advisor on Forensic Science
U.S. Department of Justice

Saturday, November 9, 2019

State v. Simmer: The First High Court Opinion on "Probabilistic Genotyping"

Last week, the Nebrasks Supreme Court affirmed the admission of testimony about what has been dubbed "probabilistic genotyping." Twelve years ago, in 2007, Joy Blanchard was discovered on the floor of her home. Two knives protruded from her neck. A spindle broken from the banister lay nearby. The cause of death was blunt force trauma and stab wounds to the head and neck. Investigators swabbed the spindle, the handles on both knives, and the interior front door knob. Nine years passed before DNA testing indicated the presence of DNA from Joy's nephew, Charles Simmer, on one of the knife handles and the door knob.

Charged with first-degree murder Simmer filed a pretrial motion to preclude “any and all testimony concerning DNA testing and the results of said testing,” including “identification” and “comparison.” After an evidentiary hearing, the district court denied the motion. At the jury trial, the state presented both "probabilistic genotyping" from Dr. Mark Perlin, the inventor of a complex software program named TrueAllele, that "the match between the knife handle and Simmer was 3.71 thousand times more probable than a coincidental match to an unrelated person, while the match between the doorknob and Simmer was 4.22 quintillion times more probable than a coincidental match to an unrelated person." Simmer was convicted and sentenced to life imprisonment.

What made the "genotyping" "probabilistic" is not that genes were typed. The STRs used in the identification are not genes. They are repeated, noncoding sequences of DNA base pairs (such as TAGATAGATAGATAGATAGATAGATAGA) that are sprinkled within and between genes. Neither Is the typing "probabilistic" because probabilities were quoted. Ordinary DNA typing results are accompanied by "random match probabilities."

Rather, unlike ordinary DNA profiling of a single-source samples, these probabilities were not derived from any one individual's features as inferred by a human examiner. The samples were mixtures of DNA from (at least) two men, and no one could say which two individual profiles were in the mixture. The computer program, however, could estimate the probability of the mixed-up collection of DNA features displaying the measured "peak heights" when coming from DNA from several individuals including the defendant as opposed to the probability of the same data coming from DNA from individuals not including the defendant.

Dr. Perlin testified to the ratio of those probabilities (more technically, of those probability densities) using the confusing phrase "match probability" when in fact there was no simple match. He framed the question as "How much more does the suspect match the evidence at this location than a random person?" But any random profile either would match -- in the sense of having the same alleles as those giving rise tom the observed peaks -- or it would not. Because of the ambiguity as to the true profiles in the mixture, the common-sense notion of a match -- of inclusions to and exclusions from -- the class of possible contributors to the mixture is not part of the logic of "probabilistic geneotyping."

Be that as it may, the issue on appeal in State v. Simmer, 304 Neb. 369 (2019), was not how to present the output of the software. It was whether that output could be the subject of any testimony at all. The Nebraska Supreme Court ran through the usual "Daubert factors." Despite arguments about the type of validation studies, the court was impressed with the sheer number of studies and the length of time that the system has been used.

The defendant's expert was Nathaniel Adams, who "has a bachelor’s degree in computer science and is employed by a company that 'consult[s] with lawyers.'" Mr. Adams testified (correctly) that one cannot directly validate the accuracy of particular likelihood ratios and that the software had not been developed and tested from the standpoint of software engineering, which, he suggested, requires independent study of source code over long periods of time. The Supreme Court was not impressed. It held that validation "from a software engineering perspective" was not essential and that the publication of research on the performance of the software (as best as that could be measured for known mixtures and via simulations) was sufficient.

The defense also maintained that "the validation studies must be discounted because Perlin is a coauthor of some of the publications and, as the owner of the company that owns. TrueAllele, has a financial interest in seeing it found reliable" To this, the Supreme Court responded that there were plenty of studies in which Perlin was not an author, but it also suggested that the source of validation studies is irrelevant because that fact pertains only to credibility:
In other words, Simmer is contending that there are reasons to question Perlin’s credibility. Even assuming that is true, an attack on the credibility of an expert witness is not aimed at the expert’s reasoning or methodology and is thus not the proper basis for a Daubert/Schafersman challenge. See Smith v. Colorado Organ Recovery Sys., 269 Neb. 578, 694 N.W.2d 610 (2005) (stating that challenge to expert witnesses' credibility is not Daubert/Schafersman claim).
This cannot be correct. Although the credibility of a particular witness certainly is fair game at trial, Independent replication of experiments is an important factor in scientific practice. The absence of validation from independent researchers should count against admissibility, although it would not necessarily be dispositive under Daubert.

Another questionable feature of the opinion lies in its discussion of the PCAST Report. In 2016 and 2017, the President's Council of Advisors on Science and Technology evinced dissatisfaction with the extent to which probabilisitc genotyping as applied in some situations had been validated. The software developers disputed the report -- but later responded with additional studies. The peculiarity in Simmer is not that the court determined that the output of TrueAllele was admissible despite PCAST's reservations. It is that the Nebraska Supreme Court refused to read the PCAST Report because it "is not in [the] record [on appeal]." Instead, it looked only at an obscure and incomplete "Washington trial court order containing a description of the report." On that basis, it wrote that "with no scientific evidence in the record indicating that TrueAllele could not generate reliable results under the circumstances here, we cannot say that a conclusion that TrueAllele could reliably be applied to the facts of this case would amount to an abuse of discretion."

But why was appellant was "unable to cite directly to the report"? It is a duly published report of a prestigious advisory body. Appellate lawyers and courts frequently cite books, articles, and reports about scientific research and policy. National Academy reports and Office of Technology Assessment reports have long been cited and discussed in opinions on the admissibility of DNA evidence. They address "legislative facts." Legislative facts apply across cases." In contrast, "adjudicative facts" are not general truth; they are the how, what, where, and when of specific cases.

Under the Nebraska Rule of Evidence (see, e.g., Neb. R. Evid. 201(a)), only adjudicative facts need to be proved at trial. Both appellate and trial courts have the power to consult published scientific opinions on the limitations of a scientific method. Here, the alleged limitation is that the software was not shown to work well with mixtures that contain very small fractions of a suspect's DNA. Courts should be more open to receiving information in the scientific and related literature on such issues.

Saturday, September 21, 2019

Police Genetic Genealogy at GEDmatch: Is Opt-in the Best Policy?

With over a million DNA scans in its database, GEDmatch has been instrumental in the success of genetic genealogy searches in criminal investigations. The genealogy website
provides applications for comparing your DNA test results with other people. There are also applications for estimating your ancestry. Some applications are free. More advanced applications require membership in the GEDmatch Tier1 program at $10 per month.
By uploading the text file of genetic data from a direct-to-consumer genetic testing company such as 23andMe or Ancestry.com, to a publically accessible database such as GEDmatch, genealogy enthusiasts can discover whether they have large blocks of DNA in common with other individuals who have uploaded their data. Extensive haploblock matching might reflect membership in the same family tree.

Police have used this service to identify possible relatives of the unknown individuals whose DNA was recovered from a crime-scene. Ordinary genealogy research into various records may lead to the common ancestor and then back down to the descendants -- including the one who left the crime-scene DNA.

This process of genetic genealogy for criminal investigations attracted extensive publicity, leading to a series of changes in GEDmatch's policies. At first, the DNA data from all participants (who chose to make their data available for searches by others) was open to law enforcement trawls. Later, the website's users could designate their data as not available for criminal investigations but still open for searches by the general public. In other words, they could opt-out of this one use. Then, the policy became opt-in only. Unless a participant affirmatively chooses to make his or her data available for criminal investigations, it is not included in trawls for police who identify themselves as such.

If police respect these policies, GEDmatch may be useless to them. In the first three weeks since GEDmatch moved to the restrictive policy, only 50,000 people opted in. CeCe Moore, the genetic genealogist with Parabon NanoLabs, which has scored the most successes for police, reported that the crime-scene uploads in that period stopped producing usable matches. "It’s basically useless now," said Moore. "Our work on any new cases is significantly stalled."
Does the opt-in policy go too far? GEDmatch encourages people to use an alias. Users can be contacted, but the individual inquiring won’t find them unless they choose to respond, and anyone can take his or her information off GEDmatch at any time. However, when there is a potential lead to a crime-scene sample, it would be easy for police or prosecutors to issue a subpoena or secure a search warrant to find the email address if not the name behind the alias.

By the way, the owner of GEDmatch said that he is considering charging police a fee to use the website. Unless the database accessible to police returns to its earlier searchable size of a million or so, he may not have many such customers.
C.J. Guerrini et al., Should Police Have Access to Genetic Genealogy Databases?, PLOS Biology, Oct. 2, 2018, 16(10): e2006906.

We conducted the survey online using Amazon Mechanical Turk (MTurk) ... . We restricted participation to individuals who were 18 years of age or older and located in the United States and paid them US$0.25 for taking the survey. ... [T]he majority supported police searches of genetic websites that identify genetic relatives (79%) and disclosure of DTC genetic testing customer information to police (62%), as well as the creation of fake profiles of individuals by police on genealogy websites (65%) (Fig 1). However, respondents were significantly more supportive of these activities (all p < 0.05) when the purpose is to identify perpetrators of violent crimes (80%), perpetrators of crimes against children (78%), or missing persons (77%) than when the purpose is to identify perpetrators of nonviolent crimes (39%).

My note: MTurk respondents may not be representative of users of GEDmatch.

SOURCES