Thursday, January 19, 2023

"Double Blind Peer Review" of Research on Hair Fibers

Yesterday I heard about some new publications on forensic hair microscopy published in, of all places, a journal on pharmaceuticals. My first thought was that the journal might be a predatory one with deceptive advertising designed to con scholars into paying for publication in what appears to be a reputable scientific journal. But that was too cynical.

The papers are

The Journal of Advanced Pharmaceutical Technology & Research has a scientific society and a respectable publisher behind it. The former is the “Society of Pharmaceutical Education & Research (SPER),” which is one of the leading pharmaceutical association in the country [of India] ... with a member base of around 3,500, it spread [sic] across the country and have [sic] 13 state branches.”

The latter is Wolters Kluwer’s Medknow. Located in Mumbai, Medknow “provides publishing services for peer-reviewed, online and print-plus-online journals in medicine on behalf of learned societies and associations with a focus on emerging markets.” Wolters Kluwer insists that Medknow “journals employ a double-blind review process, in which the author identities are concealed from the reviewers, and vice versa, throughout the review process.”

Although the journal is not indexed in Medline, the research comes from the Saveetha Dental College, Chennai, Tamil Nadu, India—”one of the finest institutions in the world with a unique curriculum that is a spectacular fusion of the best practices of the east and west.”

So I read the papers. Unbelievable.

The abstract and the conclusion of "A Comparative Study of Male and Female Human Hair" announce that “[t]his study can be concluded that the structural comparison between male and female hair specimens can be used as evidence for forensic analysis at crime scenes.” How so? Well, for one thing, “[i]n this study, it is observed that the color of human male hair is completely black, while it is black on the proximal end and brown at the distal end of human female hair.”

Astonishingly, the sample of hairs is never described. How many men and women provided hair? Where did they come from? How many hairs were taken from each subject and compared? Were the examinations blind? Without this elementary information, no one can understand or assess the reported results.

The “Comparison of Human and Animal Hair – A Microscopical Analysis” is similarly devoid of any meaningful description of the research.

The “Comparative Study of Different Animal Hairs: A Microscopic Analysis” appears to be a description of four hairs – one each from a dog, a cat, a horse, and a rat. The researchers found some differences among them. This they found encouraging: "The present study might be used in forensic investigations."

So much for "double blind peer review."

Tuesday, August 2, 2022

Grand Jury Subpoenas for Newborn Screening Blood Spots

On July 10, the New Jersey Office of the Public Defender and the New Jersey Monitor sued the state department of health "to obtain redacted copies of [grand jury] subpoenas ... so that they can learn more about how the State Newborn Screening Laboratory has effectively turned into a warrantless DNA collection facility for State criminal prosecutions." \1/

New Jersey's neonatal screening program, like that in other states, uses a few drops of blood from the newborn’s heel to test "for certain genetic, endocrine, and metabolic disorders ... prior to discharge from a hospital or birthing center." \2/ The Department of Health explains that "[e]arly detection and treatment of the disorders on the newborn screening panel can prevent lifelong disabilities, including intellectual and developmental disabilities, and life threatening infections." \3/ Like many other states, New Jersey health officials retain a "Guthrie card" (named after Dr. Robert Guthrie, who in the 1960s, successfully championed mandatory screening laws for a metabolic disease that causes preventable intellectual disability). \4/

The complaint alleges that the Office of the Public Defender (OPD) "became alarmed" that State Police "are utilizing the residual blood spot samples" and that the health department rebuffed requests to provide information on subpoenas the department may have received from grand juries. The cause of the alarm is described as follows:

The State Police had re-opened an investigation into a “cold case” of sexual assault that had occurred in 1996 and had genetically narrowed the suspects to one of three brothers and their male offspring. ... [They] served a subpoena upon the Newborn Screening Laboratory in or about August 2021 to obtain residual dried blood spot samples that had been collected from a male newborn in or about June 2012.

To ascertain which family member was the suspect, the State Police sought the blood spot sample that was taken from an approximately nine-year-old child when he was a newborn to compare it to the DNA it had collected at the crime scene in 1996. The State Police successfully obtained the child’s blood spot sample, sequenced the DNA, and then ran further analysis utilizing a technique known as investigative genetic genealogy. The State Police alleges those results showed the newborn blood spot sample belonged to the genetic child of the suspect. From there, the State Police used those results to form the basis of an affidavit of probable cause to acquire a warrant to obtain a buccal swab from OPD’s client, who is the child’s father. OPD’s client was then criminally charged.

OPD further asserted "a significant interest in knowing how expansive this law enforcement practice is so that it may better represent its clients who may be subject to such warrantless searches." It did not explain how learning the number of subpoenas would improve its ability to defend any particular client.

The other plaintiff, the New Jersey Monitor, described itself as "the eyes and ears of the public [with] an interest in reporting to the public about this practice that violates basic concepts of genetic privacy."

The pleading claims that "law enforcement agencies are flouting search warrant requirements" and that "[b]ecause the Supreme Court of the United States and the New Jersey Supreme Court recognize that people have a right of privacy in their DNA and that the collection and analysis of that DNA is a search, a search warrant is generally required for such invasive actions."

I have not researched New Jersey jurisprudence, but I strongly doubt that the U.S. Supreme Court's opinions constitutionalize any free-floating "basic concepts of genetic privacy." \5/ The allegation of "subversion of the warrant requirement" of the Fourth Amendment presupposes that a warrant is required. That could be, but this question is not directly covered by Supreme Court precedent. It is the conclusion of what has to be a more complex legal argument. How might that argument go?

The Fourth Amendment declares that "[t]he right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause ... ." How do subpoenas for Guthrie cards come within this proscription? They are not quite seizures of any person or any person's papers or effects.

Are they searches of the person? Certainly, a physical intrusion into the body to extract blood would be, and the state has done that with a warrantless heel prick. But that search is constitutional because of an exception to the warrant-preference rule. The "special needs" exception allows the government to conduct searches and seizures to advance important government interests other than collecting information for criminal cases. Compulsory neonatal screening is an important public health program for providing early treatment or prevention of suffering and impairment. It predates DNA testing for identification (and DNA testing for disease, for that matter). New Jersey's legislation dates back to 1964. That grand jury subpoenas can be issued today to investigate a crime does not make the original search or seizure does not transform the original interference with bodily integrity into one that required probable cause. \6/

There is, however, a second search. The subpoena itself triggers Fourth Amendment protections -- but not to the extent of a physical entry to acquire information. The privacy and security interests are quite different, and the Supreme Court has held that the government may use an administrative subpoena to acquire documents so long as “the documents sought are relevant to the [investigation]” and the document request is “adequate, but not excessive,” for those purposes. \7/ Unlike the warrant process, a subpoena does not require probable cause.

At least, not normally. A Guthrie-card subpoena might be different. In Carpenter v. United States, \8/ the Supreme Court held that probable cause was required for the government to compel wireless carriers to produce time-stamped records of cell-site location information (CSLI) on a robbery suspect that had 12,898 location points cataloging his cell phone's movements over 127 days. Courts had issued orders for these business records in an FBI investigation into a series of robberies, under the Stored Communications Act, which merely requires "specific and articulable facts showing that there are reasonable grounds to believe that ... the records ... [sought] are relevant and material to an ongoing criminal investigation." \9/ Cause to believe that a record is relevant to an investigation is not probable cause to believe that the record is evidence of a suspect's criminal conduct. The majority opinion in Carpenter emphasized that CSLI records added up to (or will, in the near future, amount to) "a detailed chronicle of a person's physical presence compiled every day, every moment, over several years." \10/ As such, it held the relevance-based orders in question were unreasonable searches.

One can argue that the information that can be extracted from a DNA sample "implicates privacy concerns" at least as much as CSLI data. \11/ But the analogy requires attention to the kind of DNA information the government obtains (and the precautions it takes against other personal information being acquired from the DNA).

Until the blood is analyzed, no informational privacy is compromised. \12/ In the case mentioned in the complaint, the police "had genetically narrowed the suspects to one of three brothers and their male offspring." I would guess that they accomplished this by means of Y-STR typing combined with other leads. The police then obtained the Guthrie card for "an approximately nine-year-old child," "sequenced the DNA, and then ran further analysis utilizing a technique known as investigative genetic genealogy" to conclude that the child's "blood spot sample belonged to the genetic child of the suspect." 

It is difficult to discern what DNA testing was done. "Investigative genetic genealogy" normally involves comparisons of haploblocks from crime-scene DNA and DNA in genetic genealogy databases that are open to the public in order to pick possible relatives to the unknown person whose DNA was at the crime-scene. With those findings, ordinary genealogical research may produce a list of suspects. In the case mention in the complaint, police already had the list of suspects. Why perform the extensive haploblock analysis of "investigative genetic genealogy" if the three siblings and the child of one of them already are known? Would not comparing a number of autosomal STR loci not known to be medically informative have been able to show whether the child had a substantial probability of being the child of the man whose DNA was associated with the 1996 sexual assault that the police were investigating? That might be enough for probable cause for a court order compelling the implicated brother to provide a DNA sample for comparison to the one from the 1996 sexual assault. \13/

Of course, it can be argued that the particular loci the police actually used for the investigation hardly matter -- that the very fact that the sample contains medically relevant information that the police could acquire from the Guthrie card makes the case similar enough to the location tracking in Carpenter to require probable cause. In Carpenter, the FBI was only interested in associating the defendant's cell phone with towers near the robberies that were under investigation. Did they assemble detailed itineraries of Carpenter's movements at all other locations that he (or, more precisely, his phone) visited? Perhaps the mere fact that the many cell-site records were in their possession was enough. 

Yet, this argument resembles the one rejected in most cases on the constitutionality of forcing convicted offenders (or even arrestees) to surrender DNA for law-enforcement databases. Most judges, and the Supreme Court, rejected the argument that the potential to type all kinds of loci in itself required probable cause for collecting and profiling the DNA for identification only. \14/

None of this means that New Jersey's Guthrie-card subpoenas are clearly or even probably constitutional. I merely suggest that there could be more to the issue than the complaint alleges. Also, it seems worth noting that the exact connection between the the public records request and the constitutional issue is not entirely apparent. \15/

NOTES

     Thanks to Fred Bieber for news of the complaint.

  1. N.J. Office of the Public Defender v. N.J. Dep't of Health, Civ. No. ___ (Complaint, July 10, 2022), available at https://www.theverge.com/2022/7/29/23283837/nj-police-baby-dna-crimes-lawsuit-public-defender.
  2. Centers for Disease Control and Prevention, Newborn Screening Portal, Nov. 29, 2021, https://www.cdc.gov/newbornscreening/index.html.
  3. N.J. Dep't of Health, Newborn Screening and Genetic Services, Feb. 10, 2022, https://www.nj.gov/health/fhs/nbs/.
  4. Harvey L. Levy, Robert Guthrie and the Trials and Tribulations of Newborn Screening, 7(1) Int’l J. Neonatal Screening 5 (2021), available at https://doi.org/10.3390/ijns7010005.
  5. Cf. Dobbs v. Jackson Women's Health Organization, No. 19–1392 (U.S. June 24, 2022), available at https://www.supremecourt.gov/opinions/21pdf/19-1392_6j37.pdf.
  6. Cf. Ferguson v. Charleston, 532 U.S. 67 (2001), available at https://scholar.google.com/scholar_case?case=12447804856380641716. Another exception is consent. Although consent for Fourth Amendment purposes is far less onerous than medical informed consent, the only grounds for refusal in New Jersey are religious. 26 N.J. Stat. Ann. § 26:2-111. So the consent exception does not apply.
  7. Okla. Press Publ’g Co. v. Walling, 327 U.S. 186, 209 (1946) (upholding an FTC order for the production of a newspaper publishing corporation’s books and records as request was made pursuant to statute and was reasonably relevant). The Fifth Amendment privilege against self-incrimination offers protection when the act of production itself would be incriminating as an admission. E.g., United States v. Hubbell, 530 U.S. 27 (2000).
  8. 138 S.Ct. 2206 (2018), available at https://scholar.google.com/scholar_case?case=14655974745807704559.
  9. 18 U.S.C. § 2703(d).
  10. Id. at 2220.
  11. Id.
  12. Cf. id. at 2266-67 (Gorsuch, J., dissenting and asking "Why is the relevant fact the seven days of information the government asked for instead of the two days of information the government actually saw? ... And in what possible sense did the government 'search' five days' worth of location information it was never even sent?").
  13. See Maryland v. Pringle, 540 U.S. 366, 371-72 (2003) (finding probable cause for arresting three men in a car after finding $763 of rolled-up cash in the glove compartment and five plastic glassine baggies of cocaine were behind the back-seat armrest).
  14. See David H. Kaye, Why So Contrived? DNA Databases After Maryland v. King, 104 J. Crim. L. & Criminology 535 (2014); David H. Kaye, A Fourth Amendment Theory for Arrestee DNA and Other Biometric Databases, 15 U. Pa. J. Const. L. 1095 (2013).
  15. Whether accessing the Guthrie cards for criminal investigations is common or rare in New Jersey would not seem to affect the legality of the subpoenas. Of course, the extent of the access should be a matter of public concern, and widespread law enforcement use of the cards could prompt legislation to curtail the practice. But that is so whether or not the alleged invasions of "genetic privacy" are constitutional. Still, uncovering a widespread practice that is not only of general public interest, but also illegal, might add weight to the case for public disclosure under a balancing test for such disclosure. In that event, the allegations of unconstitutionality would not be superfluous to the complaint. Nonetheless, if the  opinions on the state and federal law of search and seizure are overly rhetorical, one might wonder whether they go beyond a simple "statement of the facts on which the claim is based." Rules Governing the Courts of the State of New Jersey, Rule 4:5-2, available at https://www.njcourts.gov/attorneys/assets/rules/r4-5.pdf.

Saturday, July 9, 2022

Preliminary Results from a Blind Quality Control Program

The Houston Forensic Science Center recently reported the results of realistic, blind tests of its firearms examiners. Realism comes from disguising materials to look like actual casework and injecting these "mock evidence items" into the regular flow of business. The judgments of the examiners for the mock cases can be evaluated with respect to the true state of affairs (ammunition components from the same firearm as opposed to components from different firearms). Eagerly, I looked for a report of how often the examiners declared an association for pairs of items that were not associated with one another (false "identifications") and how often they declared that there was no association for pairs that were in fact associated (false "eliminations").

These kinds of conditional "error rates" are by no means all there is to quality control and to improving examiner performance, which is the salutary objective of the Houston lab, but they are prominent in judicial opinions on the admissibility of firearms-toolmark evidence. So too, they (along with the cognate statistics of specificity and sensitivity) are established measures of the validity of tests for the presence or absence of a condition. Yet, I searched in vain for clear statements of these standard measures of examiner performance in the article by Maddisen Neuman, Callan Hundl, Aimee Grimaldi, Donna Eudaley, Darrell Stein and Peter Stout on "Blind Testing in Firearms: Preliminary Results from a Blind Quality Control Program," 67(3) J. Forensic Sci. 964-974 (2022).

Instead, tables use a definition of "ground truth" that includes materials being intentionally "insufficient" or "unsuitable" for analysis, and they focus on whether "[t]he reported results either matched the ground truth or resulted in an inconclusive decision." (Here, "inconclusive" is different from insufficient" and "unsuitable." For the sake of readers who are unfamiliar with firearms argot, Table 1 defines--or tries to--the terminology for describing the outcomes of the mock cases.)

TABLE 1. Statements for the Outcome of an Examination
(adapted from p. 966 tbl. 1)

Binary (Yes/No) Source Conclusions

Identification: A sufficient correspondence of individual characteristics will lead the examiner to the conclusion that both items (evidence and tests) originated from the same source.
Elimination: A disagreement of class characteristics will lead the examiner to the conclusion that the items did not originate from the same source. In some instances, it may be possible to support a finding of elimination even though the class characteristics are similar when there is marked disagreement of individual characteristics.
Statements of No Source Conclusion

Unsuitable: A lack of suitable microscopic characteristics will lead the examiner to the conclusion that the items are unsuitable for identification.
Insufficient: Examiners may render an opinion that markings on an item are insufficient when:
• an item has discernible class characteristics but no individual characteristics
• an item does not exhibit class characteristics and has few individual characteristics of such poor quality that precludes an examiner from rendering an opinion;
• the examiner cannot determine if markings on an item were made by a firearm during the firing process; or
• the examiner cannot determine if markings are individual or subclass.
Inconclusive: An insufficient correspondence of individual and/or class characteristics will lead the examiner to the conclusion that no identification or elimination could be made with respect to the items examined.
Note on "identification": The identification of cartridge case/bullet toolmarks is made to the practical, not absolute, exclusion of all other firearms. This is because it is not possible to examine all firearms in the world, a prerequisite for absolute certainty. The conclusion that sufficient agreement for identification exists between toolmarks means that the likelihood that another firearm could have made the questioned toolmarks is so remote as to be considered a practical impossibility.

There were 51 mock cases containing anywhere from 2 to 41 items (median = 9). In the course of the five-and-a-half year study, 460 items were examined for a total of 570 judgments by only 11 firearms examiners, with experience ranging from 5.5 to 23 years. The mock evidence varied greatly in its informativeness, and the article suggests that the lab sought to use a greater proportion of challenging cases than might be typical.

Whether or not the study is generalizable to other examiners, laboratories, and cases, the authors write that "no hard errors were observed; that is, no identifications were declared for true nonmatching pairs, and no eliminations were declared for true matching pairs." This sounds great, but how probative is the observation of "no hard errors"

Table 3 of the article states that there were 143 false pairs, of which 106 were designated inconclusive. It looks like the examiners were hesitant to make an elimination, even for a false pair. They made only 37 eliminations. Since there were no "hard errors," none of the false pairs were misclassified as identifications. Ignoring inconclusives, which are not presented as evidence for or against an association, the observed false-identification rate therefore was 0/37. Using the rule of three for a quick approximation, we can estimate the 95% confidence interval as going from 0 to 3/37. To use phrasing like that in the 2016 PCAST Report, the false-positive rate could be as large as 1 in 9.

Applying the same reasoning to the 386 true pairs, of which 119 were designated inconclusive, the observed false-elimination rate must have been 0/267. The 95% confidence interval for the false-elimination rate thus extends to about 3/267, or 1/89.

These confidence intervals should not be taken too seriously. The simple binomial probability model implicit in the calculations does not hold for dependent comparisons. To quote the authors (p. 968), "Because the data were examined at the comparison level, an item of evidence can appear in the data set in multiple comparisons and be represented by multiple comparison conclusions. For example, Item 1 may have been compared to Item 2 and Item 3 with comparison conclusions of elimination and identification, respectively." Moreover, I could be misconstruing the tables. Finally, even if the numbers are all on target, they should not taken as proof that error rates are as high as the upper confidence limits. The intervals are merely indications of the uncertainty in using particular numbers as estimates of long-term error rates.

In short, the "blind quality control" program is a valuable supplement to minimal-competency proficiency testing. The absence of false identifications and false eliminations is encouraging, but the power of this study to pin down the probability of errors at the Houston laboratory is limited.

Wednesday, July 6, 2022

Why Did the Proposed Amendment to Rule 702 Scuttle the "Preponderance of the Evidence"?

After posting a description of the changes to the proposed amendment to Federal Rule of Evidence 702, I received the following inquiry:

Which one is actually the proposal? "More likely than not" or "by a preponderance of the evidence"? The former seems to be a weakening, the latter (even if it is redundant for lawyers) puts forensic scientists on notice. Use of the word "evidence" in the latter is, however, potentially confusing. "Evidential reliability" is about the "reliability" [sic] of the "evidence", i.e., the "scientific validity" of the methods applied to arrive at the "opinion". The proposed change (if it is the proposed change) seems to refer to "evidence" about the "reliability" of the "evidence" (in which the first and second instance of the word "evidence" do not refer to the same thing).

The first iteration of the amendment used "preponderance." It read, "[a]n [expert] witness ... may testify ... if the proponent has demonstrated by a preponderance of the evidence that" the proposed evidence satisfies various requirements regarding what the Supreme Court in Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993), called "evidentiary reliability." Now the proposed text is, "An [expert] witness ... may testify ... if the proponent demonstrates to the court that it is more likely than not that" the proposed evidence satisfies these requirements.

Why the change? Partly because of the elliptical nature of the original formulation and partly because of the awkwardness of the construction "evidence that the evidence." As the rest of this posting explains, the new (green) version is better drafted, but the idea was never in doubt.

The governing principle comes from Federal Rule of Evidence 104(a) as interpreted in Bourjaily v. United States, 483 U.S. 171 (1987). The rule begins with a general observation that

The court must decide any preliminary question about whether a witness is qualified, a privilege exists, or evidence is admissible. In so deciding, the court is not bound by evidence rules, except those on privilege.

Fed. R. Evid. 104(a). So to decide whether proffered evidence is admissible at trial, the court can consider all pertinent, non-privileged information presented to it, whether or not the information about admissibility would be admissible in a trial.

But Rule 104 is silent on how confident the judge should be that the proposed evidence satisfies the requirements for admissibility. That is where Bourjaily comes in. In that case, the government wanted to introduce out-of-court statements of a coconspirator as evidence against the defendant. To avoid the rule against hearsay, it sought to persuade the court to apply the rule that certain statements of conspirators are admissible against everyone in the conspiracy. Defendant's membership in the conspiracy was thus a preliminary question for the court, and the Bourjaily Court explained that

We are ... guided by our prior decisions regarding admissibility determinations that hinge on preliminary factual questions. We have traditionally required that these matters be established by a preponderance of proof. Evidence is placed before the jury when it satisfies the technical requirements of the evidentiary Rules, which embody certain legal and policy determinations. The inquiry made by a court concerned with these matters is not whether the proponent of the evidence wins or loses his case on the merits, but whether the evidentiary Rules have been satisfied. Thus, the evidentiary standard is unrelated to the burden of proof on the substantive issues, be it a criminal case ... or a civil case. ... The preponderance standard ensures that, before admitting evidence, the court will have found it more likely than not that the technical issues and policy concerns addressed by the Federal Rules of Evidence have been afforded due consideration. ... Therefore, we hold that, when the preliminary facts relevant to Rule 801(d)(2)(E) are disputed, the offering party must prove them by a preponderance of the evidence.

483 U.S. at 175-76 (note omitted).

Applying Bourjaily to the preliminary questions in Rule 702, it is quite clear that the trial court has to find that "evidentiary reliability" under Rule 702 is more probable than not. To foreclose any debate about it, in Daubert itself, the Court pointed to the preponderance standard, writing that "[f]aced with a proffer of expert scientific testimony, then, the trial judge must determine at the outset, pursuant to Rule 104(a), whether the expert is proposing to testify to (1) scientific knowledge that (2) will assist the trier of fact to understand or determine a fact in issue." 509 U.S. at 592.

Yet, many public commenters did not see this. Some comments claimed that the word "evidence" in "preponderance of the evidence" would constrain the court to considering only such evidence as would be admissible at trial in deciding whether the proposed expert testimony is admissible. Other comments claimed that the phrase would keep previously admissible evidence from juries. Indeed, "almost all of the fire was directed toward the term 'preponderance of the evidence.'” Advisory Comm. on Evid. Rules, Report to the Standing Committee, May 15, 2022, at 7.

The Advisory Committee unabashedly rejected both these claims. In its report to the Standing Committee, it wrote that:

The Committee does not agree that the preponderance of the evidence standard would limit the court to considering only admissible evidence; the plain language of Rule 104(a) allows the court deciding admissibility to consider inadmissible evidence. Nor did the Committee believe that the use of the term preponderance of the evidence would shift the factfinding role from the jury to the judge, for the simple reason that, when it comes to making preliminary determinations about admissibility, the judge is and always has been a factfinder.

Id. Nevertheless,

[T]he Committee recognized that it would be possible to replace the term “preponderance of the evidence” with a term that would achieve the same purpose while not raising the concerns (valid or not) mentioned by many commentators. The Committee unanimously agreed to change the proposal as issued for public comment to provide that the proponent must establish that it is “more likely than not” that the reliability requirements are met. This standard is substantively identical to “preponderance of the evidence” but it avoids any reference to “evidence” and thus addresses the concern that the term “evidence” means only admissible evidence.

Id. Finally,

The Committee was also convinced by the suggestion in the public comment that the rule should clarify that it is the court and not the jury that must decide whether it is more likely than not that the reliability requirements of the rule have been met. Therefore, the Committee unanimously agreed with a change requiring that the proponent establish “to the court” that it is more likely than not that the reliability requirements have been met. The proposed Committee Note was amended to clarify that nothing in amended Rule 702 requires a court to make any findings about reliability in the absence of a proper objection.

Id. Overlooked in this debate over the niceties of the phrase "preponderance of the evidence" is a different drafting point. The proposed amendment makes it explicit that the standard pertains to the court's role in considering scientific validity, but it does not do the same for the other requirements of Rule 702--namely, that the witness be "qualified as an expert by knowledge, skill, experience, training, or education." That a witness is qualified to testify also must be established as more probable than not. For a rare case excluding testimony from a latent fingerprint examiner because she ran into problems in demonstrating proficiency, see United States v. Cloud, No. 1:19-cr-02032-SMJ-1, 2021 WL 7184484 (E.D. Wash. Dec. 17, 2021) (false exclusion in casework, a false exclusion on a proficiency test, and receiving help from her supervisor on a follow-up proficiency test).

Friday, July 1, 2022

Proposed Amendment to Federal Rule of Evidence 702 Clears More Hurdles

The following report appeared in the OSAC newsletter OSAC In Brief, June 2022, at 4-6 with the title "Proposed Amendment to Federal Rule of Evidence 702 Clears More Hurdles." It updates a report in the June 2022 issue (posted earlier today on this blog). Both reports are meant to be boringly factual. More opinionated remarks may appear later.

After five years of discussion, a proposed amendment to Federal Rule of Evidence 702 on testimony by expert witnesses has progressed to the Judicial Conference of the United States—the policy-making arm of the federal judiciary. If the Judicial Conference accepts the unanimous recommendations of both its Advisory Committee on Evidence Rules, which drafted the amendment, and its standing Committee on Rules of Practice and Procedure, which endorsed it this month, the amendment will be delivered to the Supreme Court for transmittal to Congress. Then, unless Congress intervenes, it will become effective by the end of next year.

But what effect would it have? According to the Advisory Committee chair, U.S. District Court Judge Patrick Schiltz, the amendment does not alter the meaning of the rule in the slightest. “It simply makes it clearer, makes it easier for people to understand, so that fewer mistakes will be made” (as reported June 7, in Bloomberg Law). Box 1 shows the proposed changes, which differ slightly from those discussed in the OSAC In Brief article of July 2021.

BOX 1. Proposed Changes to Federal Rule of Evidence 702
A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if the proponent demonstrates to the court that it is more likely than not that:
(a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;
(b) the testimony is based on sufficient facts or data;
(c) the testimony is the product of reliable principles and methods; and
(d) the expert has reliably applied expert's opinion reflects a reliable application of the principles and methods to the facts of the case.

On the face of it, the amendment does little, if anything, to alter the substance of the existing rule. It adds the words “if the proponent demonstrates to the court that it is more likely than not” in front of the criteria for admitting expert testimony, but the Supreme Court had already noted that in exercising a longstanding “gatekeeping” role, the district court needs to determine whether the conditions for admitting expert testimony are “established by a preponderance of proof.” (Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579, 592 n.10 (1993) (citing Fed. Evid. 104(a); as a result of public comments, the Advisory Committee substituted “more likely than not” for the “preponderance of evidence” to describe the proponent’s burden of persuasion on the issue of admissibility).

The other wording change concerns the well entrenched reliability-as-applied requirement (“the expert has reliably applied” in part (d)). The amendment uses an alternative phrase—“the expert's opinion reflects a reliable application.” Although one could argue that the specific reference to “opinion” limits the requirement to personal opinions, that is not the intent. An explanatory note that will accompany the revised rule (if and when it is adopted) makes it plain that it still must appear that the expert has applied a valid and reliable method proficiently and appropriately in making any and all findings and inferences. The only purpose of the change is “to emphasize that each expert opinion must stay within the bounds of what can be concluded from a reliable application” of a reliable method to the facts of the case. And, this Advisory Committee Note (ACN) adds that this directive is “is especially pertinent to the testimony of forensic experts,” for which “the judge should (where possible) receive an estimate of the known or potential rate of error of the methodology employed, based (where appropriate) on studies that reflect how often the method produces accurate results” rather than “assertions of absolute or one hundred percent certainty—or to a reasonable degree of scientific certainty ... .”

During the six-month comment period that ended in February, the draft received well over 500 comments. The Reporter to the Advisory Committee found the public reaction “somewhat surprising, because the proposed amendment essentially seeks only to clarify the application of Rule 702 as it was amended in 2000—and that amendment received [only] 179 comments.” Lawyers from the plaintiffs’ side of the civil bar opposed the latest amendment, while defendants’ lawyers supported it.

There were relatively few comments about the implications of the additional words and the accompanying note for the areas of forensic science covered by OSAC. These too were (predictably) divided. The National District Attorneys Association (NDAA) objected to the ACN’s singling out forensic-science testimony as a problem and saw the amendments as “a solution in search of a problem.” But the New York City Bar Association expressed “particular concern [with] criminal prosecutions” and “the scientific validity of many types of ‘feature-comparison’ methods of identification, such as those involving fingerprints, footwear and hair.” The New York State Crime Laboratory Advisory Committee (NYSCLAC) objected to “changes limiting forensic science testimony” but then maintained that its laboratories already complied with the guidance in the ACN. The Union of Concerned Scientists questioned parts of the NDAA and NYSCLAC statements and insisted that “forensic evidence should be required to present courts with estimates of error rates relevant to their methodologies.” The Innocence Project and other organizations and individuals submitted a joint statement praising the changes and pressing for more. They wanted the text of the rule to contain a requirement that testimony is not only “the product of reliable principles and methods” (the current wording), but also to specify that it “includes the limitations and uncertainty of those principles and methods.”

The conflicting comments regarding forensic science produced no modifications. If the amendment is adopted, it will implement, to some extent, the 2016 recommendation of the President’s Council of Advisors on Science and Technology that “the Judicial Conference of the United States ... should prepare ... an Advisory Committee note, providing guidance to Federal judges concerning the admissibility under Rule 702 of expert testimony based on forensic feature-comparison methods.”

Author’s disclaimer: This report presents the views of the author. Their publication in In Brief is not an endorsement by NIST or OSAC, and they are not intended to represent the views of any OSAC unit. No estimate of the known or potential rate of error is available.