Showing posts with label bite marks. Show all posts
Showing posts with label bite marks. Show all posts

Sunday, June 11, 2023

Is "Bitemark Analysis" Better than "Bitemark Comparisons"?

In October 2022, NIST released a draft report entitled "Bitemark Analysis: A NIST Scientific Foundation Review." A press release announced "Forensic Bitemark Analysis Not Supported by Sufficient Data, NIST Draft Review Finds." In March 2023, the final version reaching the same conclusions was released. Soon afterward, the NIST-supported Organization of Scientific Area Committees for Forensic Science (OSAC) revised the scope of the work that its forensic odontology subcommittee can undertake. The description now specifies, in italics no less, that "The Forensic Odontology Subcommittee does not develop standards on bitemark recognition, comparison, and identification."

Yet, some medical examiners believe that "analysis" of marks on the skin "frequently yields valuable information that forensic odontologists testify to in courts of law, just as forensic pathologists do with respect to their objective findings and their interpretations of those findings based on experience, training and the circumstances of the event." Richard Souviron & Leslie Haller, Bite Mark Evidence: Bite Mark Analysis Is Not the Same as Bite Mark Comparison or Matching or Identification, 4 J. L. & Biosci. 617, 618 (2017). They distinguish between "analysis" and "comparison," recognizing that the latter is not scientifically well founded, and seeking to preserve the former as a legitimate expert endeavor. They propose that

The analysis process involves answering basic, crucial, questions such as whether or not the pattern injury is a human bite mark. This question can be the most difficult part of the entire process. After establishing whether a patterned injury is, indeed, a bite mark, other questions must be asked. Is it a human bite mark? Was it made by an adult or a child? Was it swabbed for DNA? Was it made through clothing? If so, was the clothing swabbed for DNA? Where is it located on the victim and in what position was the victim when it happened? Could it have been self-inflicted? What was the position of the biter? Was it offensive or defensive? Was it affectionate or does it demonstrate violence? Will it produce a permanent injury? If so, simple battery may become aggravated battery. When was the bite inflicted in relation to the time of death? Is it fresh, a scar or somewhere in between? Was the person bitten alive or dead at the time? Are there any unique dental characteristics that could be used to exclude possible suspects? In cases of multiple bites, did the same biter make them all? Were they all made at the same time or do they establish a pattern of long-term abuse?

These questions, and more, are the essential core of the analysis of every bite mark, and produce a large amount of information that can be of considerable value to an investigation before any suspects are identified or charged.

Id. So where are the experiments or other studies to show that most of these "essential" parts of bitemark analysis can be done validly and reliably? Can medical examiners correctly classify "pattern injuries" as bitemarks? As human bitemarks? As the mark of a child or an adult? As affectionate? As unique? As coming from the same biter?

Bite (and other) marks will be encountered in autopsies. They need to be photographed and examined along with other injuries or characteristics. But odontologists and medical examiners should think hard before they claim the ability to do all these things "and more" as part of "analysis."

Sunday, November 24, 2019

Commonwealth v. Ross: Is Bitemark Identification Evidence Becoming Toothless?

In a case that has received national attention, a Pennsylvania Superior Court (the state's intermediate appellate court) finally decided that the trial judge had to at least hold a hearing on the general scientific acceptance of bitemark identification evidence. The trial court had ruled that no such "Frye hearing" was required before admitting proposed testimony from two forensic odontologists that the defendant cannot be excluded as the source of human bite marks on the rape-murder victim whereas several other potential suspects can be.

Mysteriously, the appellate court chose to keep its lengthy opinion unreported and of "no precedential value." (It is reproduced at the end of this posting and available as a pdf file.) Many appellate courts do not publish opinions that simply apply settled law and would be of interest only to the parties, but that is not the situation here. The trial judge's refusal to grant the defense's request for such a "Frye hearing" has received national condemnation, 1/ and the new opinion is one of a very few to take seriously widespread misgivings about bitemark evidence.

As described in an NBC report complaining that "the justice system ignores science," 2/
A little after noon on June 24, 2004, a fisherman on Canoe Creek Lake in Blair County, Pennsylvania, found the bound, beaten and sexually abused body of Tina Miller near a boat launch. Suspicion focused on [Paul Aaron] Ross, who’d been with Miller nearby the night before.

There were no witnesses to the killing, leaving authorities to build a case on circumstantial evidence: a man who said he had dropped Miller and Ross off near the boat launch, former girlfriends who said he’d abused them, a mark found on one of Miller’s breasts, and items collected from the scene ─ including a shoe print and a beer bottle. None of it definitively linked Ross to the murder, and Ross denied killing Miller, asserting that another man had picked her up in a truck. But prosecutors argued that, taken collectively, the evidence proved his guilt.

He went to trial in 2005, and a bite-mark examiner helped seal his conviction.

The examiner, Dr. Dennis Asen, was a dentist with 25 years of experience identifying bodies by their teeth and examining bite marks. On the stand, he recalled viewing Miller’s body at the morgue and concluding that her breast bore patterns of a bite mark. He took a photograph of it and made an “overlay tracing” that he uploaded to a computer and compared with molds of Ross’ teeth and the teeth of four other suspects, including three who’d been in contact with Miller in the hours before her death. Asen said he found the mark “very highly consistent” only with Ross’ teeth. Asen acknowledged, however, that he couldn’t say with 100 percent certainty that Ross had made the mark. The defense hired its own bite-mark examiner, who said he was unable to determine whether Ross left the mark.

Ross was convicted of first-degree murder and sentenced to life in prison. But the conviction was overturned by an appeals court that ruled the trial court had “abused its discretion” in refusing to give Ross’ lawyer more time to prepare, and allowing the former girlfriends’ testimony.

Prosecutors plan to try Ross again, but the process has stalled amid a battle over the bite-mark evidence.

Defense lawyers, including members of the New York-based Innocence Project, have cited government reports and academic studies, including from the National Academy of Sciences, the President's Council of Advisors on Science and Technology and the Texas Forensic Science Commission.

The reports have shown that no one has sufficiently proven that human bite marks are unique, or that skin is a reliable material to record them ─ or that forensic dentists can reliably identify a human bite mark in the first place.
What was the Commonwealth's rejoinder? The Blair County district attorney is "The Honorable Richard A. Consiglio." (That is his title on the county's website, which emphasizes his success in dealth penalty cases along with his receipt of "an award ... for standing up to the Blair County Courts on behalf of the unborn." "Rich," as the website also refers to him, told now retired Blair County Judge Jolene Grubb Kopriva in 2016 that
the authors of scientific reports lack “real world” experience. He ... pointed out that no court in any state has ruled against admitting it. “This is evidence, your honor, that has been accepted for 40 or 50 years, not quite as old as I am, but 40 or 50 years, maybe longer, accepted by courts everywhere.” 3/
Acknowledging that "the use of bite-mark evidence is beginning to face challenges," Judge Kopriva ruled the evidence admissible on the basis of the lawyers' arguments. She wrote in Match 2017 that "it would be premature for this court to order that the methodology is no longer generally accepted in the relevant scientific community.” 4/

In a blog post (since deleted), "the Innocence Project described the case as an example of "'willful scientific illiteracy among the judiciary.' ... Judge Kopriva imposed a gag order preventing either side from talking publicly about the case outside of court." 5/

But she also certified an interlocutory appeal of her order denying the motion for a Frye hearing. (Ordinarily, such evidentiary rulings are only reviewed as part of an appeal from the verdict at the end of the trial.) The Superior Court refused to accept the interlocutory appeal. But then the Pennsylvania Supreme Court vacated the Superior Court's order and directed it to decide whether there had to be an evidentiary pretrial hearing on the admissibility of bitemark evidence

This week, on November 21, the Superior Court released its opinion. The court held that the trial court abused its discretion in agreeing to admit bitemark opinion testimony without first conducting a Frye hearing. The opinion, written by Judge Mary Murray, discusses Frye v. United States, 293 F. 1013 (D.C. Cir. 1923), the Superior Court's understanding of the general-acceptance requirement as it has been construed in Pennsylvania, and how this requirement applies to a request for an evidentiary hearing on bitemark evidence.

According to the Ross opinion, "Frye contemplated a judicial inquiry, informed by experts, into the general acceptance of the scientific methods used." Yet, Frye contains no mention of "a judicial inquiry" in the form of a pretrial evidentiary hearing on the status of a method among scientists. Without any such hearing to draw on, the Court of Appeals in Frye determined that "the systolic blood pressure deception test has not yet gained such standing and scientific recognition among physiological and psychological authorities as would justify the courts in admitting expert testimony deduced from the discovery, development, and experiments thus far made." 6/ The court simply announced its conclusion in an enigmatic eight-paragraph opinion. The task of devising suitable substitutes for judicial fiat fell to later generations.

Today, the use of hearings with expert testimony on scientific acceptance is well developed. In Pennsylvania, however, the touchstone for deciding when such hearings are essential often is said to be "novelty." Thus, Judge Kopriva relied on the lack of novelty in denying the defense's requested hearing, and the Superior Court focuses on the supposed requirement as well.

But Frye itself imposed no "novelty" requirement. All that Frye said in this regard that was the blood pressure test had yet to cross "the line between the experimental and demonstrable stages." 7/ The opinion did not consider whether a procedure could cross the line, but then fall back once more into the darker side of "this twilight zone." 8/ Bitemark evidence once was thought to have been valid (by its practitioners at least), but, as the defense in Ross argued, today the larger scientific community (and a growing number of dental experts) do not recognize its scientific validity.

Fortunately, the Superior Court, after flailing about in an effort to decide what needed to be novel, also addressed the more appropriate question of whether the relevant scientific community still believes (if it ever did) that the kind of testimony that Judge Kopriva was going to admit is generally accepted. It rejected District Attorney Consiglio's idea that acceptance within the forensic odontology community is sufficient, and it recognized that recent literature indicated significant dissension even within that community and outright rejection by a broader scientific community. Consequently, it held that the trial court could not admit the prosecution's evidence without a meaningful hearing on general acceptance in the broader community.

Of course, this holding does not dictate the outcome of the hearing. Indeed, the trial court could avoid an evidentiary hearing entirely, by reviewing the readily available scientific literature (with briefing by the parties) and concluding that it does not support the general acceptance of the comparative analysis that the prosecution proposes to present. After all, the proponent of the scientific evidence bears the burden of demonstrating its general acceptance, and that will not easy for the state to do in this case.

Nonetheless, that is too great an about-face to expect from the trial court. More likely, the court will proceed with the full-blown hearing. That hearing should focus on the specific testimony that the two dentists slated to testify might give. Is there sufficient "scientific recognition" 9/ that the forensic odontologists can validly determine that the bite marks on the breast of the victim's partially submerged body are human (if that is what they propose to say)? That (whether or not the marks are from a human source) they do not come from the four other potential suspects? That they were "very highly consistent" with Ross's dentition (as Dr. Asen testified at the original trial)? If the testimony were kept to a bare minimum, such as "unable to exclude as a possible source of the marks," with no explanation of the implications of such an inclusion (because scientific knowledge is too thin), 10/ would it be sufficiently helpful to be admissible? 11/ Or would it be inadmissible as being inscrutable or too easily overvalued?

The time when these issues could be avoided by arguments about "novelty" and "real world experience" has passed.

NOTES
  1. John Shuppe, 'We Are Going Backward': How the Justice System Ignores Science in the Pursuit of Convictions, NBS News, Jan. 23, 2019, https://www.nbcnews.com/news/us-news/we-are-going-backward-how-justice-system-ignores-science-pursuit-n961256.
  2. Id.
  3. Id. There are exceptions to the District Attorney's bald assertion that "courts everywhere" deem bitemark testimony admissible. See infra note 11. Mr. Consiglio's timeline also was incorrect. The earliest use of bitemarks as identification in a U.S. criminal case may have been some 150 years ago, in 1870. See Barry E. Lipton et al., History of Odontology, in Manual of Forensic Odontology 1, 27 (David R. Senn & Richard A. Weems eds., 2013). The first reported case was a 1954 murder trial in Texas. Id. at 28. But see Michael J. Saks et al., Forensic Bitemark Identification: Weak Foundations, Exaggerated Claims, 3 J. L. & Biosci. 538 (2016), available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5570687/ (asserting that before 1974 a some years thereafter, it was "the rule among forensic dentists that crime scene bite marks could not be trusted to yield accurate source identifications.).
  4. Id.
  5. Id.
  6. Frye v. United States, 293 F. 1013, 1014 ( D.C.. Cir 1923).
  7. Id.
  8. Id. 
  9. Id.
  10. This seems to be the only type of inclusionary testimony the American Board of Forensic Odontology accepts from its members. See ABFO, Standards and Guidelines for Evaluating Bitemarks § 3 (2018) ("bitemark linkage conclusions should  only a) exclude  or b) not exclude  (include) a  dentition.The  specific  terms  found  in 2.c.are: a)  for  exclusion, Excluded  as Having  Made  the  Bitemark,  and  b)  for  inclusion, Not  Excluded  as  Having Made  the  Bitemark.Stronger  terms of  attribution are  not condoned ... .").
  11. Cf. Ege v. Yukins, 485 F.3d 364 (6th Cir. 2007) (the more useful testimony of exclusions and "highly consistent" matching followed by "my expert opinion" that "nobody else" among the "three and a half million people" in the Detroit area "would match up" was without foundation and violated due process). The American Board of Forensic Odontology no longer condones such testimony. See supra note 10..

Commonwealth v. Ross
No. 1738 WDA 2018 (Pa. Super. Ct. Nov. 21, 2019)
(not reported, pdf file available at
www.pacourts.us/assets/opinions/Superior/out/J-S55007-19m%20-%2010423159687925902.pdf

BEFORE: MURRAY, J., McLAUGHLIN, J., and COLINS, J.
MEMORANDUM BY MURRAY, J.:

Paul Aaron Ross (Appellant) appeals from the order denying his request for a Frye hearing. 1/ After careful consideration, we vacate the trial court’s order denying Appellant’s request for a Frye hearing and remand to the trial court for proceedings consistent with this decision.

This appeal arises from the June 27, 2004 murder of Tina Miller at Canoe Creek Lake in Canoe Creek State Park. Appellant was arrested and charged with Ms. Miller’s murder. On November 23, 2005, a jury found Appellant guilty of first-degree murder, aggravated assault, involuntary deviate sexual intercourse, unlawful restraint, simple assault, false imprisonment, and indecent assault. 2/
A prior panel of this Court summarized the post-trial procedural history:
At the sentencing phase of the trial, the jury rejected imposition of the death penalty. The trial court then proceeded to sentence [Appellant] to life in prison plus 24 to 48 years. [Appellant] filed post-trial motions, which the trial court denied on January 30, 2006. On February 10, 2006, [Appellant] filed a timely notice of appeal, but [Appellant’s] counsel failed to file an appellate brief and the appeal was consequently dismissed. On September 26, 2008, [Appellant] filed a PCRA petition seeking the reinstatement nunc pro tunc of his direct appeal rights, which the trial court granted on August 14, 2009.
Commonwealth v. Ross, 57 A.3d 85, 90 (Pa. Super. 2012) (en banc).

On October 12, 2012, an en banc panel of this Court vacated Appellant’s judgment of sentence and remanded for a new trial. Id. at 105. On November 17, 2015, the Pennsylvania Supreme Court denied the Commonwealth’s petition for allowance of appeal.

On March 1, 2016, Appellant filed pre-trial motions in which he sought, inter alia, the exclusion of any expert evidence relating to bite mark identification and a Frye hearing. At trial, the Commonwealth intends to introduce the testimony of Dr. Dennis Asen (Dr. Asen) and Dr. Lawrence Dobrin (Dr. Dobrin). Dr. Asen and Dr. Dobrin are both dentists and practice in the field of forensic odontology (the study of the structure of teeth). Dr. Asen and Dr. Dobrin intend to testify that the mark on Ms. Miller’s left breast was caused by a human bite, and when they compared five sets of teeth molds, including one from Appellant, Dr. Asen and Dr. Dobrin could exclude four of the molds from having made the bite mark, but not Appellant’s.

On December 2, 2016, following the filing of several supplemental motions by Appellant and objections by the Commonwealth, the trial court heard oral argument on Appellant’s request for a Frye hearing. On March 8, 2017, after the parties submitted additional briefs on Appellant’s request for a Frye hearing, the trial court entered an order concluding that bite mark identification evidence is not novel and therefore a Frye hearing was not warranted. The court further provided that the Commonwealth’s experts were to adhere to the guidelines set forth by the American Board of Forensic Odontologists (ABFO).

On April 5, 2017, Appellant filed a motion to amend the March 8, 2017 order to include language relating to Pennsylvania Rule of Evidence 702(c), so that the trial court could address whether the expert methodology is generally accepted in the relevant field. Appellant also requested that the court certify for immediate appeal its decision not to hold a Frye hearing on the bite mark identification evidence.

On November 6, 2017, the trial court entered an amended order once again denying Appellant’s request for a Frye hearing. The trial court also included in the order language addressing Rule 702(c) and granting Appellant’s request for certification of immediate appeal. On December 5, 2017, Appellant filed a petition for permission to file an interlocutory appeal with this Court, which we denied by per curiam order on May 7, 2018. On June 1, 2018, Appellant filed a petition for allowance of appeal to the Pennsylvania Supreme Court. On November 20, 2018, our Supreme Court granted Appellant’s petition for allowance of appeal, vacated this Court’s order denying Appellant’s petition for permission to file an interlocutory appeal, and remanded the case to this Court for disposition.

On appeal, Appellant presents the following issues for review:
1) WHETHER THE SUBSTANTIAL EVIDENCE [APPELLANT] PROPOSED TO PRESENT AT A MOVED-FOR FRYE HEARING DISCREDITING BITE MARK ANALYSIS COMES FROM AND IS PART OF THE RELEVANT SCIENTIFIC COMMUNITY FOR FRYE PURPOSES THUS ENTITLING HIM TO A FRYE HEARING; OR WHETHER PENNSYLVANIA COURTS MUST LIMIT THEIR RELIABILITY INQUIRY TO THE VIEWS OF CURRENT PRACTICTIONERS OF THE PARTICULAR TECHNIQUE AT ISSUE?
2) WHETHER THE SUBSTANTIAL EVIDENCE PROFFERED BY [APPELLANT] AT THE MOVED-FOR FRYE HEARING PRESENTED A LEGITIMATE DISPUTE REGARDING THE RELIABILITY OF BITE MARK ANALYSIS THUS ENTITLING HIM TO A FRYE HEARING; AND SUCH THAT THE TRIAL COURT’S ADMISSION OF THE SAME WITHOUT A FRYE HEARING WAS IN ERROR?
Appellant’s Brief at 5.

Both of Appellant’s issues related. Therefore, we address them together. We begin with our standard of review:
As a general rule, this Court’s standard of review of a trial court’s evidentiary ruling, including a ruling whether expert scientific evidence is admissible against a Frye challenge, is limited to determining whether the trial court abused its discretion. Grady v. Frito–Lay, Inc., 839 A.2d 1038, 1046 (Pa. 2003); Zieber v. Bogert, 773 A.2d 758, 760 n.3 (Pa. 2001) (citing Commonwealth v. Minerd, 753 A.2d 225 (Pa. 2000)). “An abuse of discretion may not be found merely because an appellate court might have reached a different conclusion, but requires a result of manifest unreasonableness, or partiality, prejudice, bias, or ill-will, or such lack of support so as to be clearly erroneous.” Grady, 839 A.2d at 1046 (citing Paden v. Baker Concrete Constr., Inc., 658 A.2d 341, 343 (Pa. 1995)).
Commonwealth v. Dengler, 890 A.2d 372, 379 (Pa. 2005) (citations modified).

Appellant argues that the trial court abused its discretion in denying his request for a Frye hearing on the admissibility of the Commonwealth’s bite mark identification evidence. This Court recently articulated the legal standards implicated by a Frye analysis:
The Frye standard originally was intended to prevent the situation in which a party would seek to introduce scientific evidence that was so new that it would be impossible to “produce rebuttal experts, equally conversant with the mechanics and methods of a particular technique.” [U.S.] v. Addison, 498 F.2d 741, 744 (D.C. App. 1974). Frye contemplated a judicial inquiry, informed by experts, into the general acceptance of the scientific methods used. The standard required that “the thing from which the [expert’s] deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.” Frye, supra at 1014. At issue in Frye was admissibility of the systolic blood pressure deception test, commonly known as the lie detector test. The trial court excluded the evidence, and the court affirmed that ruling on appeal, explaining:
Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.
Frye, supra at 1014. Pennsylvania adopted the Frye standard in Commonwealth v. Topa, 369 A.2d 1277 (Pa. 1977), a case involving the propriety of the trial court’s admission of voice print identification evidence through an expert, Lieutenant Nash, of the Michigan State Police. Our High Court, applying Frye, reasoned that
[t]he requirement of general acceptance in the scientific community assures that those most qualified to assess the general validity of a scientific method will have the determinative voice. Additionally, the Frye test protects prosecution and defense alike by assuring that a minimal reserve of experts exists who can critically examine the validity of a scientific determination in a particular case. Since scientific proof may in some instances assume a posture of mystic infallibility in the eyes of a jury of laymen, the ability to produce rebuttal experts, equally conversant with the mechanics and methods of a particular technique, may prove to be essential.
Topa, supra at 1282 (quoting Addison, supra at 744). The Topa Court went on to conclude that the testimony of one expert could not satisfy this standard, citing commentaries questioning the reliability of sound spectrographs and voiceprints and demonstrating that it was not generally accepted within the field of acoustical science.
Thus, the Frye standard originally was intended to prevent a party from introducing scientific evidence that was so new that it would be impossible to “produce rebuttal experts, equally conversant with the mechanics and methods of a particular technique.” Addison, supra at 744. Frye contemplated a judicial inquiry, informed by experts, into the general acceptance of the scientific methods used.

In the years since the adoption of the Frye standard, this Court has clarified that “Frye only applies to determine if the relevant scientific community has generally accepted the principles and methodology the scientist employs, not the conclusions the scientist reaches.” Trach v. Fellin, 817 A.2d 1102, 1112 (Pa.Super. 2003) (en banc).
Walsh v. BASF Corp., 191 A.3d 838, 842-43 (Pa. Super. 2018), appeal granted, 203 A.3d 976 (Pa. 2019). 3/

The above principles have been incorporated into Pennsylvania Rule of Evidence 702, which states:
A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if: (a) The expert’s scientific, technical, or other specialized knowledge is beyond that possessed by the average layperson; (b) The expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; (c) The expert’s methodology is generally accepted in the relevant field. Pa.R.E. 702 (emphasis added).
Appellant argues that the trial court erred in determining a Frye hearing was unnecessary in this case. Appellant asserts that the court wrongly concluded that the methodology employed by the Commonwealth’s experts in conducting their bite mark identification analysis was generally accepted in the scientific community of forensic odontology. Additionally, Appellant contends that the trial court should have examined whether the expert’s methodology for bite mark identification analysis was generally accepted in the broader scientific community, as opposed to limiting its examination to the field of forensic odontology. Appellant maintains that bite mark identification analysis has applications in several different scientific communities, and several of these communities have determined that bite mark identification analysis is not reliable and has little or no evidentiary value.

In support of his argument, Appellant relies on our Supreme Court’s decision in Betz v. Pneumo Abex, LLC, 44 A.3d 27 (Pa. 2012). At issue in Betz was the admissibility of expert opinion evidence relating to the “any-exposure” theory of legal causation, i.e., “that each and every exposure to asbestos – no matter how small – contributes substantially to the development of asbestos-related diseases.” Id. at 30. As part of its analysis, the Supreme Court had to first address, as a threshold issue, whether the trial court was correct in concluding that a Frye hearing was necessary to determine the admissibility of the “any-exposure” evidence. Id. at 52-55.

In concluding that the trial court’s decision to conduct a Frye hearing was correct, the Supreme Court explained:
There is inherent tension among the various measures for admissibility of expert testimony. The threshold common law test requires merely some reasonable pretension to specialized knowledge. See, e.g., Miller v. Brass Rail Tavern, Inc., 664 A.2d 525, 528 (Pa. 1995). Our evidentiary rules, on the other hand, suggest trial courts may take a greater role in assessing whether the testimony will assist the trier of fact to understand the evidence or determine a fact in issue, see Pa.R.E. 702, and in screening evidence to avoid unfair prejudice, confusion of the issues, or misleading of the jury, see Pa.R.E. 403. For better or for worse, however, in the context of the more conventional realms of science, the Pennsylvania decisions tend to downplay the courts’ screening function. See, e.g., Commonwealth v. Nazarovitch, 436 A.2d 170, 172 (Pa. 1981) (“[C]ourts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery[.]” (quoting Frye, 293 F. at 1014)). A manifestation of this trend is that challenges generally are vetted through the Frye litmus, which winnows the field of the attacks by application of the threshold requirement of novelty. See Grady, 839 A.2d at 1043-44.

Various reasons underlie the preference to limit the courts’ involvement in determining the admissibility of scientific evidence. There is the concern that liberality in allowing challenges would substantially increase the number of challenges (and cases in which lengthy pre-trial proceedings would ensue). The competency of trial judges to accept or reject scientific theories remains a legitimate subject of controversy. Additionally, a claim or defense in many cases may rise or fall based upon expert testimony and, therefore, there is some reluctance on the part of courts to deprive litigants of their day in court.

On the other hand, this Court has recognized the influential nature of expert testimony on complex subjects, and the potential that distortions have to mislead laypersons. See id. at 1045; Topa, 369 A.2d at 1281-82. It would be naïve, in this regard, to assume that the possibility for distortion is limited to the very newest realms of science. Cf. Grady, 839 A.2d at 1045 (explaining that Frye applies not only to novel science, but also where scientific methods are utilized in a novel way).

We therefore agree with Appellants that a reasonably broad meaning should be ascribed to the term “novel.” Furthermore, we conclude that a Frye hearing is warranted when a trial judge has articulable grounds to believe that an expert witness has not applied accepted scientific methodology in a conventional fashion in reaching his or her conclusions. Accord id. We believe a narrower approach would unduly constrain trial courts in the appropriate exercise of their discretion in determining the admissibility of evidence. See id. at 1046.
Id. at 52-53 (emphasis added, footnote omitted, citations modified).

Additionally, in Betz, the Supreme Court rejected the notion that the defendants could not challenge the methodology of the plaintiff’s expert pathologist with the testimony of risk assessors, toxicologists, and epidemiologists. Id. at 54. The Court reasoned that the expert pathologist’s “any-exposure” opinion “was not couched in terms of a methodology or standard peculiar to the field of pathology[,]” and “was plainly grounded on risk assessment.” Id. at 54-55. Thus, the Supreme Court determined that the subjects at issue “are not within the particular expertise of a pathologist, but, rather, are interdisciplinary in character.” Id. at 55.

In this case, the trial court reached two conclusions in denying Appellant’s request for a Frye hearing. First, the trial court determined that “[b]ite mark evidence is currently generally accepted in the relevant scientific community of forensic odontologists.” Trial Court Opinion, 11/6/17, at 5. The court explained:
The American Board of Forensic Odonotologists (ABFO) has promulgated guidelines regarding the approved methodology concerning bite mark evidence. This [c]ourt notes that the aforementioned methodology rejects the notion that human dentitions are unique to the individual. Any evidence proffered by the Commonwealth to suggest that an individual can be identified by a bite mark, rather than merely excluded or not excluded as having made the mark, would be inadmissible.
Id.

Second, the trial court rejected Appellant’s reliance on Betz, finding that case distinguishable:
It is true that the Pennsylvania Supreme Court allowed the Betz defendants to address the methodology of a pathologist through the testimony of risk assessors, toxicologists, and epidemiologists. However, Betz can be distinguished from the instant case. The Betz Court found that the pathologist’s opinion was not “couched in terms of a methodology or standard peculiar to the field of pathology.” [Betz, 44 A.3d at 54]. Rather, the pathologist’s opinion was “plainly grounded on risk assessment” and was “interdisciplinary in character.” Id. at 55. The Betz Court based its decision not upon the role of pathologists generally, but upon the specific methodology employed by the individual pathologist in question. Betz does not make a broad assertion that Pennsylvania law “mandates a broader definition of the relevant scientific community” where the methodology of a pathologist is at issue. Here, the Commonwealth does not seek to offer the kind of broad-scale scientific testimony that was at issue in Betz.
Id. at 6.

After careful consideration, we find support for Appellant’s position that a Frye hearing was warranted in this case. First, with respect to the trial court’s conclusion that bite mark identification analysis has general acceptance in the field of forensic odontology, Appellant offered evidence indicating that there is a lack of consensus among forensic odontologists on whether bite mark identification analysis is reliable and valid. While there is no dispute that the ABFO has established standards and a methodology for conducting bite mark identification analysis, see Commonwealth’s Third Brief in Opposition to Appellant’s Request for a Frye Hearing, 2/21/17, Exhibit B, Appellant presented numerous reports to the trial court indicating that practitioners within the community of forensic odontology question whether this methodology reliably enables forensic odontologists to identify an injury as a human bite mark.

For example, Appellant presented evidence from a presentation by Dr. David Senn, DDS, Vice-President of the American Board of Forensic Odontology, to the National Academies: Committee on Identifying the Needs of the Scientific Community. See Defendant’s Post-Argument Supplement to “Defendant’s Motion in Limine: Frye Test – Bite Mark Evidence” (hereinafter Defendant’s Post-Argument Supplement), 1/17/17, Exhibit 4 (Presentation to the National Academies: Committee on Identifying the Needs of the Forensic Science Community – Forensic Odontology Bite Marks (hereinafter Senn Presentation), 4/23/07, at 31-34). While Dr. Senn opined that bite mark identification analysis was important to the investigation and adjudication of certain crimes, id. at 45, Dr. Senn identified several “major problems” with bite mark identification analyses, including: “[t]he uniqueness of the human dentition has not been scientifically established”; “[t]he ability of the dentition, if unique, to transfer a unique pattern to human skin and maintain that uniqueness has not been scientifically established”; “[a] clear statement of the type, quality, and number of class and individual characteristics or other features required to indicate that a bite mark has reached a threshold of evidentiary value has not been established”; and “Forensic Odontology certifying organizations have not created or administered bite mark analysis proficiency tests for their board certified members.” Id. at 31-34.

Likewise, Appellant presented a report by the President’s Council of Advisors on Science and Technology, which revealed the following:
Empirical research suggest that forensic odontologists do not consistently agree even on whether an injury is a human bitemark at all. A study of the American Board of Forensic Odontology (ABFO) involved showing photos of 100 patterned injuries to ABFO board-certified bitemark analysts, and asking them to answer three basic questions concerning (1) whether there was sufficient evidence to render an opinion as to whether the patterned injury is a human bitemark; (2) whether the mark is a human bitemark, suggestive of a human bitemark, or not a human bitemark; and (3) whether distinct features (arches and toothmarks) were identifiable. Among the 38 examiners who completed the study, it was reported that there was unanimous agreement on the first question in only 4 of the 100 cases and agreement of at least 90 percent in only 20 of the 100 cases. Across all three questions, there was agreement of at least 90 percent in only 8 of the 100 cases.
Defendant’s Post-Argument Supplement, 1/17/17, Exhibit 5 (PRESIDENT’S COUNCIL OF ADVISORS ON SCIENCE AND TECHNOLOGY, REPORT TO THE PRESIDENT: FORENSIC SCIENCE IN CRIMINAL COURTS: ENSURING SCIENTIFIC VALIDITY OF FEATURE-COMPARISON METHODS (hereinafter PCAST Report) 84-85 (2016)) (emphasis added).

The studies proffered by Appellant challenge the trial court’s conclusion that the ABFO’s methodology for using bite marks to eliminate persons as suspects is generally accepted in the field of forensic odontology. These reports reflect that individuals within the forensic odontology community question not only whether the ABFO’s methodology can reliably aid experts in using bite marks to validly identify or exclude individuals as criminal actors, but also whether the methodology enables experts to identify a wound as a human bite mark. Therefore, Appellant provided the trial court with articulable grounds that the Commonwealth’s expert witnesses on bite mark identification analysis have not applied accepted scientific methodology in reaching their conclusions. See Betz, 44 A.3d at 53. Accordingly, we conclude that the trial court abused its discretion in denying Appellant’s request for a Frye hearing.

We are likewise persuaded by Appellant’s argument that the trial court should not have limited consideration of the general acceptance of the experts’ methodology in this case to the field of forensic odontology. As several of the reports Appellant cites reflect, bite mark identification analysis implicates numerous scientific fields. See generally Defendant’s Post-Argument Supplement, 1/17/17, Exhibit 2 (NATIONAL ACADEMY OF SCIENCES, COMMITTEE ON IDENTIFYING THE NEEDS OF THE FORENSIC SCIENCES COMMUNITY, STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES: A PATH FORWARD (2009), Exhibit 5 (PCAST Report), Exhibit 11 (Michael J. Saks, et al., Forensic bitemark identification: weak foundations, exaggerated claims, 3(3) J. LAW BIOSCI. 1 (2016)). As these studies indicate, bite mark identification analysis not only involves concepts relating to forensic science generally, but also pathology, biology, statistics, and metrology. See Defendant’s Post-Argument Supplement, 1/17/17, Exhibit 11 (Michael J. Saks, et al., Forensic bitemark identification: weak foundations, exaggerated claims, 3(3) J. LAW BIOSCI. 1 (2016)) (noting that the forensic identification process is “fundamentally probabilistic” and that it involves the interplay of different scientific disciplines including blood (pathology), skin (biology and dermatology), and measurements (metrology)). Because the act of biting a human involves not only the biter’s teeth, but also the skin, muscle, tissue, and blood with which the teeth make contact, the notion that bite mark identification analysis involves scientific disciplines beyond forensic odontology is reasonable.

Finally, we emphasize that our decision in no way represents a determination as to the general acceptance of the methodology underlying bite mark identification analysis utilized by the Commonwealth’s experts in this case. We make no judgment as to the admissibility of the bite mark identification evidence at issue. Rather, we simply conclude that Appellant provided the trial court with articulable grounds to believe that the Commonwealth’s expert witnesses on bite mark identification analysis may not have applied generally accepted scientific methodology in reaching their conclusions, and consequently, the trial court erred in concluding that a Frye hearing was not necessary. While Appellant’s evidence expresses negative opinions on bite mark identification analysis, we cite it only to support our conclusion that a Frye hearing is proper for the resolution of these discrepancies, and to afford both parties the opportunity to present evidence in support of their positions. Therefore, we vacate the order denying Appellant’s request for a Frye hearing and remand this matter to the trial court for a hearing in accordance with Frye.

Order vacated. Case remanded. Jurisdiction relinquished. Judgment Entered.

Footnotes

* Retired Senior Judge assigned to the Superior Court.
1. A Frye hearing, named after the decision in Frye v. United States, 293 F. 1013 (D.C. Cir. 1923), “is a hearing held for the trial court to determine whether the general scientific community has reached a general acceptance of the principles and methodology used by the expert witness.” Commonwealth v. Walker, 92 A.3d 766, 769 n.1 (Pa. 2014).
2. 18 Pa.C.S.A. §§ 2501(a), 2702(a)(1), 3123(a)(1), 2902(a)(1), 2701(a)(1), 2903(a), 3126(a)(2).
3. This case was argued before our Supreme Court on October 15, 2019, and is awaiting disposition. See Walsh v. BASF Corp., 203 A.3d 976 (Pa. 2019). The issue before our Supreme Court is whether, in conducting a Frye analysis, “trial courts are not permitted to act as ‘gatekeepers’ to ensure the relevance and reliability of scientific studies offered by experts to support their opinions by scrutinizing whether those studies actually support their opinions.” Id. at 978. While the Supreme Court may overturn our Walsh decision, their disposition will have no bearing on the outcome of this case or the general standards triggering the necessity of a Frye hearing that were thoughtfully set forth in the Walsh decision.

Friday, December 21, 2018

"But most of all, we have the bite mark"

Two days ago, Texas's highest court for criminal appeals granted habeas corpus relief and issued opinions in Ex parte Chaney, No. WR-84,091-01 (Tex. Crim. App. Dec. 10, 2018). The case was easy to decide, as the state conceded that the trial involved egregious misconduct and misinformation. The opinions address the meaning and standards for findings of "actual innocence," violations of the prosecution's duty to disclose exculpatory evidence (scientific and otherwise), the use of false evidence, and "new scientific evidence [that] contradicts bitemark-comparison evidence relied on by the State at trial." Excerpts from Judge Barbara Hervey's opinion for the court about the bitemark evidence follow. (Footnotes and most citations are omitted.)

On June 20, 1987, ... the bodies of John and Sally Sweek [were found] in their apartment. Their throats were slashed, and they had been stabbed multiple times. Police also found what they believed to be a human bitemark on John's left forearm. There were no eyewitnesses to the offense. ...

The final piece of the State's case was testimony from two forensic odontologists that a mark found on John's left forearm was a human bitemark made by Chaney at the time of the murders. ... Doctor James Hales said that there was only a "[o]ne to a million" chance that someone other than Chaney bit John because the mark was a "perfect match" with "no discrepancies" and "no inconsistencies." He claimed that the "one to a million" statistic was found in "the literature." He also testified that the injury was inflicted at the time of the murders. ... Doctor Homer Campbell, testified that the mark was actually at least four separate human bitemarks and that, after comparing dentition models and examining photographs, he was certain to a "reasonable degree of dental certainty" that Chaney was the one who bit John. The bitemark evidence was the State's strongest evidence according to its own closing arguments. ...

The defense called two witnesses to testify about the mark on John's left forearm. ... Linda Norton, testified that the mark was a human bitemark but that it was "virtually unsuitable for making a good dental comparison . . . ." because "almost anyone who has relatively even top and bottom teeth is going to be capable of leaving this bite mark." According to her, she would not have submitted the mark for comparison. ... Doctor John McDowell, a forensic odontologist, agreed with the State's experts that the mark on John's left forearm was a human bitemark, and he agreed that the photographs from Weiner's office were of good quality, but his comparisons were nonetheless inconclusive. ...

The rest of the defense's closing argument focused on discrediting the State's bitemark evidence. The defense argued that the bitemark should have been better preserved and that better equipment should have been used to examine the mark. It also asserted that bitemark comparisons are merely "interpretative," pointing to the conflicting testimony about whether the injury was even a bitemark. ...

The State spent almost all its second summation discussing the bitemark evidence. The prosecutor emphasized Hales's testimony that "only one in a million could have possibly made that bite mark" before asking the jury "[w]hat more do you need?" He then cited Campbell's testimony that he was sure, to a "reasonable degree of dental certainty," that Chaney bit John. The State also tried to discredit Norton, one of the defense experts, as a charlatan somewhere between "Quincy and Matt Dillon" and painted McDowell's testimony ... as helpful to the State even though he testified that his comparisons were inconclusive. The prosecutor concluded by arguing that the bitemark evidence was "better than eyewitness testimony. [Eyewitnesses] can make mistakes, as [defense counsel] said" and that,
But, most of all, we have the bite mark. I wouldn't ask you to convict just based on the testimony of the tennis shoes, of the statements Chaney made to Westphalen, or the statements he made to Curtis Hilton. But, by golly, I'm going to ask you to convict on that dental testimony.
... According to Chaney, "while much of th[e] [trial] testimony appeared to be in accord with the state of scientific knowledge in 1987 about what could and could not be concluded from a bite mark, in the intervening decades since [his] conviction, the ground on which Drs. Hales and Campbell based their assertions [about bitemark comparisons] has given way entirely." He contends that the "[s]cientific understanding about whether it is possible to 'match' a particular person to a bite mark in skin and whether random match probabilities can be given for a bite mark has now reversed course." He also argues that he is entitled to relief because Hales has changed his trial opinion that the bitemark was inflicted at the time of John's death, which was an opinion upon which the State heavily relied. Hales now believes that the wound was two to three days old when John and Sally were killed. ...

In 2013, the legislature enacted Article 11.073 of the Texas Code of Criminal Procedure, which allows a defendant to obtain post-conviction relief based on a change in science relied on by the State at trial. ...

In its agreed findings of facts and conclusions of law, the habeas court found that
no scientific evidence has been produced to support the basis of individualization of a bite mark to the exclusion of all other potential sources in an open population. [T]he reference manual published by the American Board of Forensic Odontology (ABFO) [in March 2015] ... prohibits ABFO Diplomates from testifying to individualization of bite marks in an open population, i.e., where the universe of potential suspects, or "biters," is unknown. ... Dr. Hales's use of the terms ["]match["] and ["]biter["] as it related to [Chaney] was appropriate under the ABFO guidelines and scientific field of forensic odontology at the time of trial. ... Dr. Hales's and Dr. Homer Campbell's testimony that it was their opinion, to a reasonable degree of dental certainty, that [Chaney] made the bite mark on John Sweek's arm was appropriate under the ABFO guidelines and scientific field of forensic odontology at that time. However, ... such testimony would not be justified, admissible, or accurate under today's guidelines because the scientific community and the ABFO guidelines have invalidated individualization of bite marks in an open population, as we have in this case. ... [T]he changes in science and the evolution of the field of forensic odontology as it relates to bite mark comparisons constitutes relevant scientific evidence that was not available to be offered by [Chaney] at the time of trial in 1987. As such, ... the current relevant scientific evidence related to bite marks was not available at the time of [Chaney]'s trial because the evidence was not ascertainable through the exercise of reasonable diligence by [Chaney] before the date of or during trial. ...  [H]ad the bite mark evidence been presented at trial under current scientific standards, on the preponderance of the evidence [Chaney] would not have been convicted. ... [T]he [ABFO] Manual was updated again in [March 2016]. The current Manual prohibits individualization testimony entirely, regardless of whether the population at issue is open or closed. Under the current Manual, the only permissible conclusions for ABFO Diplomates are: "Excluded as Having Made the Bitemark"; "Not Excluded as Having Made the Bitemark"; or "Inconclusive." ...
The record reasonably supports the findings of the habeas court, so we adopt those findings. [N]ot only has the body of scientific knowledge underlying the field of bitemark comparisons evolved in a way that discredits almost all the probabilistic bitemark evidence at trial, but also ... Hales's new opinion [is] that the bitemark was inflicted days before the murders based on his new scientific knowledge that was not available at Chaney's trial. ...

To support his "change in the body of the science" arguments, Chaney cites (1) excerpts from the 2009 National Academy of Science: "Strengthening Forensic Science in the United States: A Path Forward" (NAS Report), (2) an affidavit from Drs. Mary Bush, DDS, and Peter Bush; (3) an affidavit from Hales, who testified at trial; (4) a supplemental affidavit from Hales; (5) an odontology report written by Dr. Alastair Pretty; (6) a supplemental odontology report written by Pretty; (7) an affidavit from Pretty, (8) an affidavit from Drs. Cynthia Brzozowski, James Wood, and Anthony Cardoza (Brzozowski et al.); (9) a supplemental affidavit from Brzozowski, et al.; (10) an affidavit of Dr. Michael Baden, M.D.; and (11) an amicus curiae brief filed in the California Supreme Court, which was authored by 38 "scientists, statisticians, and law-and-science scholars and practitioners."

In response to Chaney's writ application, the State "acknowledges and concedes that the science behind forensic odontology, as it relates to bite mark comparison, has considerably evolved since the time of trial in 1987" and that "[u]nder today's scientific standards, Dr. Hales relayed that he 'would not, and could not' testify as he did at trial, nor could he testify that there was a 'one to a million' chance that anyone other than [Chaney] was the source of the bite mark." The State succinctly summarizes its position, when it states that "the bitemark evidence, which once appeared proof positive of . . . Chaney's guilt, no longer proves anything."

The dual principles underlying Hales's and Campbell's opinions were that a human dentition, like a fingerprint, is unique and that human skin is a medium capable of recording a person's biting surface with sufficient fidelity that a particular individual can be identified as the source of a particular bitemark. If either of those premises are invalid, then the comparisons by Hales and Campbell claiming that Chaney was a "match" have no probative value because they are based on principles now known to be unsupported by science. According to Chaney (and his experts), although those two assumptions were accepted by the scientific community at the time of Chaney's trial, that community now rejects them. He argues that experts in the field have developed a new body of science, mainly in response to the NAS Report. That report also asserted that those principles were unproven and unreliable. That report concluded that:
(1) The uniqueness of the human dentition has not been scientifically established.
(2) The ability of the dentition, if unique, to transfer a unique pattern to human skin and the ability of the skin to maintain that uniqueness has not been scientifically established.
i. The ability to analyze and interpret the scope of extent of distortion of bite mark patterns on human skin has not been demonstrated.
ii. The effect of distortion of different comparison techniques is not fully understood and therefore had not been quantified.
(3) A standard for the type, quality, and number of individual characteristics required to indicate that a bite mark has reached a threshold evidentiary value has not been established.
NAS Report at 175-76. It also stated that "bite marks on the skin will change over time and can be distorted by the elasticity of the skin, the unevenness of the surface bite, and swelling and healing. These features may severely limit the validity of forensic odontology." Id. at 174.
...
The Bushes undertook a number of peer-reviewed studies to test the assumptions underlying Hales's and Campbell's testimony. The first group of studies tried to replicate the Rawson Study's conclusion—the literature relied on by Hales and Campbell at trial—that each human dentition is unique. The Rawson Study "examined tooth positions within dentitions and concluded that the very large number of possible positions meant that the human dentition is unique 'beyond any reasonable doubt.'" However, that study was based on two unproven assumptions, according to the Bushes. The first was that there was "no correlation of tooth position (i.e., that the position of one tooth did not affect the position of any other)," and second was that "there was a uniform or equal distribution over all possible tooth positions (i.e., that tooth locations did not gather into common patterns)." Using Rawson's methods, the Bushes plotted "landmark points on two sets of dentitions, resulting in x, y, and angle coordinates for each tooth." They then looked for matches one, two, three, four, five, and six teeth at a time. They ran two thousand simulated tests to verify their results and to determine whether the Rawson Study's results would remain accurate "if its assumptions about the lack of correlation and non-uniformity of dental arrangement were ignored." Their results were contrary to those of the Rawson Study—the Bushes observed "significant correlations and non-uniform distributions of tooth positions in [their] data sets." In other words, they found that the human dentition is not unique.

In a second series of peer-reviewed studies, the Bushes devised another way to test the unique-dentition theory. They studied "dental shape in large populations using geometric morphometric analysis and mathematical modeling methods common in other scientific disciplines." They found that dental shape matches occurred in the populations that they studied, which was consistent with the results of their earlier studies and indicated that the human dentition is not unique.

The Bushes also tried to replicate the Rawson Study's conclusion that human skin can record the characteristics of a bitemark with sufficient resolution to trace the source of the bitemark to the "biter." The Bushes "began with a series of studies that used the same dentition impressed into cadavers to explore how skin might distort any marks." For example, they "examined how anisotropy might create distortion by examining bitemarks made both parallel and perpendicular to skin's tension lines (also known as Langer lines)." They also looked at the effect of tissue movement and found that "the same dentition did not produce identical marks across these conditions." Id. They found that some marks made by the same dentition were "dramatically distorted from others," and that "bitemarks created by the same dentition on the same individual appeared substantially different depending on the angle and movement of the body and whether the mark was made parallel or perpendicular to tension or Langer lines." A number of experts (and the NAS Report) agree that the human dentition is not unique and that, even if it was, skin is an inadequate medium to "match" a bitemark to a "biter."

In addition to those studies, Chaney also directs us to Hales's affidavits about his own testimony and the evolving standards of the ABFO. In his first affidavit, Hales explains that his testimony about "biters" and "matches" was acceptable at the time of trial under ABFO guidelines; however, the scientific body of knowledge about bitemark comparisons has changed since trial and that, under current guidelines, he would not, and could not, give the same opinions that he did at Chaney's 1987 trial. The habeas court reached the same conclusion, noting that the 2016 ABFO Manual has completely invalidated any population statistics, regardless of whether the population is open or closed, and that the Manual no longer allows examiners to give opinions to a "reasonable degree of dental certainty."

[W]e agree with Chaney. The body of scientific knowledge underlying the field of bitemark comparisons has evolved since his trial in a way that contradicts the scientific evidence relied on by the State at trial. New peer-reviewed studies discredit nearly all the testimony given by Hales and Campbell about the mark on John's left forearm and Chaney being a "match." The revised ABFO standards and affidavits attached to Chaney's writ application support that conclusion. ...

In his next complaint, Chaney argues that Hales's testimony that there was only a "one to a million" chance that someone other than Chaney was the source of the injury was false according to the literature at the time. He also argues that Weiner's and Hales's testimony that the bitemark was inflicted at the time of the murders was false and misleading and that the described testimony was material to his conviction. The habeas court agreed with Chaney, and we adopt the findings of fact and conclusions of law of the habeas court because they are supported by the record. ...

Even though the scientific principles at the time of Chaney's trial supported some level of individualization (although those principles are no longer credible), Hales confesses that he knew at the time of trial that the body of science did not support his "one to a million" testimony. Other record evidence supports Hales's assertions, including newly discovered notes from Hales's trial file, where he initially wrote that the odds were "thousands to one," with a nearby notation of "100,000 to 1," the Bushes's peer-reviewed studies disproving the tenets and conclusions of the Rawson Study dealing with population statistics, and the 2016 ABFO Manual forbidding the use of all population statistics. ...

Acknowledgments: Thanks to Ed Imwinkelried for calling the opinions to my attention.

Friday, October 27, 2017

Dodging Daubert to Admit Bite Mark Evidence

At a symposium for the Advisory Committee on the Federal Rules of Evidence, Chris Fabricant juxtaposed two judicial opinions about bite-mark identification. To begin with, in Coronado v. State, 384 S.W.3d 919 (Tex. App. 2012), the Texas Court of Appeals deemed bite mark comparisons to be a “soft science” because it is “based primarily on experience or training.” It then applied a less rigorous standard of admissibility than that for a “hard science.”

The state’s expert dentist, Robert Williams, “acknowledged that there is a lack of scientific studies testing the reliability of bite marks on human skin, likely due to the fact that few people are willing to submit to such a study. However, he did point out there was one study on skin analysis conducted by Dr. Gerald Reynolds using pig skin, ‘the next best thing to human skin.’” The court did not state what the pig skin study showed, but it must have been apparent to the court that direct studies of the ability of dentists to distinguish among potential sources of bite marks were all but nonexistent.

That dentists have a way to exclude and include suspects as possible biters with rates of accuracy that are known or well estimated is not apparent. Yet, the Texas appellate court upheld the admission of the "soft science" testimony without discussing whether it was presented as hard science, as "soft science," or as nonscientific expert testimony.

A trial court in Hillsborough County, Florida, went a step further. Judge Kimberly K. Fernandez wrote that
During the evidentiary hearing, the testimony revealed that there are limited studies regarding the accuracy or error rate of bite mark identification, 3/ and there are no statistical databases regarding uniqueness or frequency in dentition. Despite these factors, the Court finds that this is a comparison-based science and that the lack of such studies or databases is not an accurate indicator of its reliability. See Coronado v. State, 384 S.W. 3d 919 (Tex. App. 2012) ("[B]ecause bite mark analysis is based partly on experience and training, the hard science methods of validation such as assessing the potential rate of error, are not always appropriate for testing its reliability.")
The footnote added that "One study in 1989 reflected that there was a 63% error rate.” This is a remarkable addition. Assuming "the error rate" is a false-positive rate for a task comparable to the one in the case, it is at least relevant to the validity of bite-mark evidence. In Coronado, the Texas court found the absence of validation research not preclusive of admissibility.  That was questionable enough. But in O'Connell, the court found that the presence of research that contradicted any claim of validity “inappropriate” to consider! That turns Daubert on its head.

Tuesday, September 20, 2016

The PCAST Report on Forensic Science: "A Roadmap for Defense Lawyers"

Yesterday, U.S. Court of Appeals judge Alex Kozinski announced in the Wall Street Journal that after examining "the scientific validity of forensic-evidence techniques—DNA, fingerprint, bitemark, firearm, footwear and hair analysis," the President’s Council of Advisors on Science and Technology (PCAST) "concludes that virtually all of these methods are flawed, some irredeemably so." The report, he predicted in this op-ed on "Rejecting Voodoo Science," will "immediately influence ongoing criminal cases, as it provides a road map for defense lawyers to challenge prosecution experts."

Here I will merely point to some large landmarks on this map. Later, I hope to critically explore the report's seemingly tendentious use of phrases like "validity," "reliability," and "scientifically meaningless." In haec verba, the "scientific findings ... concerning foundational validity of six forensic feature comparison methods" are as follows:
  • DNA analysis of single-source samples or simple mixtures of two individuals, such as from many rape kits, is an objective method that has been established to be foundationally valid (P. 147).
  • DNA analysis of complex mixtures based on CPI [Combined Probability of Inclusion]-based approaches has been an inadequately specified, subjective method that has the potential to lead to erroneous results. As such, it is not foundationally valid (P. 148).
  • Objective analysis of complex DNA mixtures with probabilistic genotyping software is relatively new and promising approach. ... At present, published evidence supports the foundational validity of analysis, with some programs, of DNA mixtures of 3 individuals in which the minor contributor constitutes at least 20 percent of the intact DNA in the mixture and in which the DNA amount exceeds the minimum required level for the method (P. 148).
  • [B]itemark analysis does not meet the scientific standards for foundational validity, and is far from meeting such standards (P. 148).
  • [L]atent fingerprint analysis is a foundationally valid subjective methodology—albeit with a false positive rate that is substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis. [¶] Conclusions of a proposed identification may be scientifically valid, provided that they are accompanied by accurate information about limitations on the reliability of the conclusion—specifically, that (1) only two properly designed studies of the foundational validity and accuracy of latent fingerprint analysis have been conducted, (2) these studies found false positive rates that could be as high as 1 error in 306 cases in one study and 1 error in 18 cases in the other, and (3) because the examiners were aware they were being tested, the actual false positive rate in casework may be higher (P. 149).
  • [F]irearms analysis currently falls short of the criteria for foundational validity, because there is only a single appropriately designed study to measure validity and estimate reliability (P. 150).
  • [T]here are no appropriate empirical studies to support the foundational validity of footwear analysis to associate shoeprints with particular shoes based on specific identifying marks (sometimes called “randomly acquired characteristics). Such conclusions are ... not scientifically valid. (P. 150).
It seems safe to predict that the phrase "foundational validity" and the basis for the report's conclusions will be the subject of heated debate. More on that later.

Friday, July 24, 2015

What Proves that "the expert and his methods couldn’t possibly be reliable"?

I am developing an allergic reaction to the following kind of argument: "A forensic-science expert testified that a trace at the crime scene or on the victim was associated (to some degree of certainty) with the defendant. DNA later evidence exonerated the defendant. Therefore, the expert’s methods couldn’t possibly be reliable."

This reasoning is not very different from saying that a pitcher who does not strike out every batter couldn’t possibly be a reliable pitcher; that a polling firm that fails to correctly predict every election must be using methods that couldn’t possibly be reliable; or that a test for heart disease that sometimes errs couldn’t possibly be reliable. Without considering the success as well as the failure rate of the method, it is impossible to say that it is unreliable—or, by the same token, that it is reliable (in the sense of being worth relying on). 1/

Yet, the argument from cases of exonerations—lacking any comparison group—are legion in discourse on the use of trace evidence for identification. The latest example I encountered comes from a Washington Post blog site. Two days ago, Radley Balko wrote that
[A] defendant was convicted due to the testimony of a forensic expert who claimed that his “science” showed the defendant, and only the defendant, could have committed the crime. That conviction was later upheld by an appeals court in an opinion that explained in detail why the expert and his methods were legitimate and reliable. The defendant was later exonerated by DNA testing, thus demonstrating that the expert and his methods couldn’t possibly be reliable.
Mr. Balko went on to write that “not only did the courts continue to allow bite mark matching into evidence, every single time a defendant challenged its validity, that defendant lost.” 2/

Both the reasoning and the description of legal history are not quite right. To begin with, even absolute proof of innocence only shows that the test has a nonzero false-positive error rate (no surprise there) and that the witness should not have claimed to a certainty that no one else could have left the mark in question.

To be sure, some evidence that has found breathing space in the courtroom should be squeezed out entirely. But let’s face it—no scientific test meets the standard of perfection. If every case in which evidence that has produced false convictions meant “that the expert and his methods couldn’t possibly be reliable,” there could be no evidence. Overselling has occurred with every type of forensic evidence—from bitemarks to toolmarks to fingerprints to DNA. Courts, scientists, and criminalists should do their best to prevent this. Thus, whether through rules of evidence or through education and monitoring of analysts, testimony must be calibrated to the power of the scientific technique. The testimony should fairly express the known probative value of evidence from a validated method.

An example of testimony that violates this precept comes from Ege v. Yukins. 3/ In that case, a discredited dental expert testified as follows:
Q: Now, Doctor, with regard to your testimony, you indicated that it's highly consistent with the dentition of Defendant Carol Ege; is that correct?
A: Yes.
Q: Okay. With regard to—let me ask you a question. Let's say you have the Detroit Metropolitan Area, three, three and a half million people. Would anybody else within that kind of number match like she did?
A: No, in my expert opinion, nobody else would match up. 4/
The eventual outcome in the case contradicts the assertion that no challenge to the admission of bitemark evidence has succeeded. The state trial judge in Ege realized that the testimony was improper and only “denied [postconviction] relief because of the lack of a contemporaneous objection and a view that the showing of prejudice was insufficient.” 5/

A federal district court also concluded that “expert testimony identifying the petitioner as the only possible perpetrator of the alleged bite mark in the Detroit metropolitan area was improperly admitted.” 6/

The U.S. Court of Appeals for the Sixth Circuit agreed “with the district court that ‘Dr. Warnick's opinion that the petitioner was the only person in the entire Detroit metropolitan area who could have made the mark on the corpse carried an aura of mathematical precision pointing overwhelmingly to the statistical probability of guilt, when the evidence deserved no such credence.’” 7/ It affirmed the order for a new trial.

In short, the argument that a method of forensic identification that has been proved to be fallible is, for that reason alone, inadmissible proves too much. Likewise, the claim that no challenge to bitemark evidence has ever prevailed is exaggerated (although not by much). 8/

Please do not misunderstand me. The series of articles on bitemark evidence from which the remarks I have quoted were taken is impressive and useful. In offering these corrections to two small parts that seem a bit extreme, I am not arguing that bitemark analysis, which has little claim to validity, is either reliable (in the statistical sense that repeated analyses of the same marks give the same answers) or valid (in the sense that the answers are more often correct when marks from the same source are analyzed than when marks from different sources are compared). From the writing I have seen, bitemark analysis does not cut it.

I also believe that cases of false convictions should be studied and that the existence of a given type of scientific evidence in these cases should not be ignored. Finding a large number of false convictions with such evidence present is a warning signal. The evidence may come from a method that has a large false-positive rate, 9/ and that possibility must be investigated to decide whether the evidence should be excluded across the board or whether juries should receive the information -- together with an honest and clear explanation of the uncertainty in the results.

NOTES
  1. See infra note 9.
  2. Radley Balko, A High-ranking Obama Official Just Called for the “Eradication” of Bite Mark Evidence, The Watch, Wash. Post, July 22, 2015.
  3. 485 F.3d 364 (6th Cir. 2007).
  4. Ege v. Yukins, 380 F.Supp. 2d 852, 871 (E.D. Mich. 2005), affirmed in part, reversed in part, 485 F.3d 364 (6th Cir. 2007).
  5. Id. at 857–58.
  6. Id. at 858
  7. 485 F.3d at 376.
  8. The federal courts in Ege treated the answer to the 3.5 million people as "probability testimony" without questioning Michigan's general rule that bitemark identifications are admissible. A true (and wrongly decided) case of bitemark probability evidence is State v. Garrison, 585 P.2d 563 (Ariz. 1978).
  9. The false-positive probability is P(+|O), where + is a positive statement ("the defendant left the mark") and O is the fact that some other person left the mark. Even if this probability is small, a disturbing number of false convictions could involve this evidence. Suppose that P(+|O) = 0.02, that 1,000 tests are performed in a set of cases with marks, and guilty defendants left the marks in 60% of these cases. The expected number of false positives is (0.02)(400) = 8. Assume that the probability of a true positive is P(+|S) = 0.96, where S means that the defendant is the source of the mark. Then the expected number of true positives is (600)(0.96) = 576. If defendants are convicted in all these cases, 8 convictions will be false (assuming that the culprit left the mark), and the many true positives will not be seen in the cases of exonerations of the innocent defendants. As indicated at the outset of these remarks, other data than exonerations are required to judge whether the test is reliable and valid.

Monday, May 14, 2012

ASCLD Meets Frontline

On April 17, PBS aired a Frontline documentary on “The Real CSI.” Within the week, the president of the American Society of Crime Lab Directors (ASCLD), Jill Spriggs, gave the following assessment to her fellow crime lab directors:
In the last week, published reports dusted off the same old forensic cases from the past and regurgitated negative information from years ago in order to provide doubt in the minds of the public and the court. I don’t know about you but some of my neighbors watched the Frontline documentary and had many questions. “Were latent prints really not a validated science?” “Is latent print analysis not what I thought it is?” “Were bite marks really a product of bad forensic science?” “How could anyone rely on bite marks?” “And, how many employees use certificate mills to obtain a forensic science certification?” My answer—You can be confident about latent print evidence. Latent prints are a validated science. Of all of the millions of fingerprint samples in the databases throughout the world, no two people have ever matched the same fingerprint. Bite marks are not an accredited crime laboratory discipline and “no” we don’t use forensic science certification mills to certify our analysts.
Watching the documentary, I shared her sense of boring familiarity. What’s new here? Same faces, same criticisms. But Ms. Spriggs’ rejoinder likewise is outdated in the logic it uses to shrug off the criticisms. Frontline summarized real problems with the way the American criminal justice system produces and consumes forensic science evidence. Although ASCLD is not responsible for these problems, let’s look more carefully at the three matters that Ms. Spriggs mentions.

1. Credible Credentials

In case after case, witnesses bolster their credentials as forensic experts with credentials from the American College of Forensic Examiners Institute—the organization depicted as a diploma mill on the Frontline show (and elsewhere). Courts often take the bait. For example, in People v. McWhorter, 212 P.3d 692 (Cal. 2009), the California Supreme Court juxtaposed a defense expert’s seemingly strong credentials—he “was certified by an organization known as the American Board of Recorded Evidence; and was a member of the American College of Forensic Examiners” with his puzzling inability to “identify the computer program he used to enhance or ‘electronically emboss’ the image in question [and to] satisfactorily explain the full nature of the process he used to create it.” Id. at 726.

It is appropriate to note that “we [all government crime labs?] don’t use forensic science certification mills to certify our analysts.” But what about the rest of the forensic science community—and the courts? Some prosecutors seem to have no compunction about presenting their witnesses as qualified at least in part because they are “a life fellow for The American College of Forensic Examiners” or some such thing. Chavarria v. State, 307 S.W.3d 386, 387 (Tex. Ct. App. 2009).

Judges and lawyers need to learn which organizations have meaningful standards and which do not. If there were less demand for dubious credentials, expert witnesses—government employees and private consultants alike—would have less incentive to pad their CVs with such credentials.

2. “Bite marks are not an accredited crime laboratory discipline”

Disavowing bite-mark evidence because it comes from outside the public laboratory is not responsive to the question, “Were bite marks really a product of bad forensic science?” Does ASCLD believe that forensic science is limited to the reports of crime laboratories? That forensic odontologists are not accredited? The American Board of Forensic Odontology "was organized in 1976 under the auspices of the National Institute of Justice" and "is accredited by the Forensic Specialties Accreditation Board (FSAB) as a forensic specialty offering board certification to qualified forensic dentists." If the public cannot trust this field's accredited expertise, why should it trust "an accredited crime laboratory discipline"? (There are a number of possible answers, but Ms. Spriggs does not provide them.)

3. “You can be confident about latent print evidence. Latent prints are a validated science.”

Latent prints often contain valuable information for ascertaining the origin of the print. However, the Frontline interviewees pointed to the lack of objective criteria for deciding when prints do or do not come from the same source and the danger that unconscious bias could influence these judgments. These problems notwithstanding, there are scientifically sound studies suggesting that latent fingerprint examiners can get the correct answers most of the time. The Frontline show could be faulted for failing to acknowledge this research while publicizing the NIJ’s refusal to fund a rigorous, comprehensive audit of actual laboratory work. Such a study should be done to complement the recently acquired experimental data, but why would Frontline omit these findings from its televised and online materials?

Yet, Ms. Spriggs does the same thing. Rather than refer to the scientific research that is responsive to the calls for better validation, she repeats the same old story: We can have great confidence in the work of latent print analysts because “[o]f all of the millions of fingerprint samples in the databases throughout the world, no two people have ever matched the same fingerprint.”

What is wrong with this proof of validity? First, whether any individuals have the same fingerprint tells us virtually nothing about the ability of analysts to compare prints accurately. In the Frontline show, Jennifer Mnookin made this point when she explained that the task confronting a latent print analyst is to discern when a partial print comes from a particular person’s finger. That task is distinct from deciding whether pairs of full prints come from different fingers.

Second, just because there are millions of prints on file, it hardly follows that “the millions of fingerprint samples in the databases throughout the world” match. The FBI alone has over 71 million prints in its database. To verify that no pair of these exemplar prints match would require approximately 2.5 x 1015 (2.5 quadrillion) comparisons of pairs of 10 prints. It would take an examiner, working at the incredible pace of one comparison per digit per second, some 800 million years to complete this task. When the FBI’s contractor, Lockheed Martin, tried to prove the non-existence of matching prints in the database using an automated matcher, it limited itself to a mere 51,000 prints and failed to establish uniqueness.

Ms. Spriggs importuned her fellow crime lab directors to “speak out on these issues. . . . Don’t wait another day! Get started!” Indeed, there are important things to be said.