Evidence-centric and Conclusion-centric: What's the Difference?

In recent months, various forensic practitioners have asked for a short and simple explanation of the difference between the "evidence-centric" and "conclusion-centric" forms of reporting and testifying. I figure that's what I get for introducing the phrases in a panel asked to talk about “Bringing Statistics into the Courtroom” at the Conference on Forensics, Statistics, and Law, at the University of Virginia in March 2018.^\1/ The particular phrases may have been original (at least I think they were), but the underlying ideas, which are what matter, have been around for a long time.

For concreteness, I will focus on a specific example--something that is the target of criticism du jour in US courts. I am thinking of firearms-toolmark examinations culminating in conclusions about the source of a bullet or shell casing. However, the dichotomy between statements of the evidence and its probative value, on the one hand, and statements of the inferences or conclusions that follow from the evidentiatary statements, on the other hand, is far more general. It applies to techniques for shedding light on the possible source of fingerprints, treadmarks, handwriting, images, glass fragments, paint chips, fibers, bones, bitemarks, or any other traces acquired at or around crime scenes or victims. Indeed, the distinction applies to all manner of empirical inquiry, as suggested by the subtitle of the late Frederick Schauer's book, The Proof: Uses of Evidence in Law, Politics, and Everything Else.^\2/ Moreover, nothing in the terminology reveals whether the evidence in question comes from a scientifically validated process, from (typically) less reliable modes of acquiring and assessing data, or from superstitions masquerading as knowledge.

A Recent Case

With these broad obervations in mind, let's turn to a recent appellate opinion comparing the state's arguments for introducing the conclusions of a firearms examiner to arguments that could be marshalled for believing in palmistry. In People v. Tidd, No. A167548, 2024 WL 3982134 (Cal. Ct. App., Aug. 29, 2024), someone in a white sport utility vehicle shot an enebriated pedestrian in the leg. Investigators located a nine-millimeter Luger cartridge case, manufactured by Speer, in the vicinity. Six days later, police arrrested Raymond Tidd as he was about to enter a white SUV carrying a loaded nine-millimeter Sig Sauer pistol. He pled guilty to some firearms possession and carrying charges but went to trial for attempted murder and other felonies for the shooting. Before trial, he objected to testimony from a criminalist with the San Francisco Police Department named Jacobus Swanepoel. The trial court held a hearing and ruled that the expert could testify that the cartridge case came from the defendant's gun as long as he refrained from asserting his opinion “to a scientific certainty.”

At trial, Swanepoel "flatly asserted the recovered cartridge and test cartridges fired from Tidd's gun 'were fired in the same firearm.'" The jury found Tibbs guilty of two firearms felonies for the shooting (but not of attempted murder). However, the court of appeals reversed because the prosecution never showed that the highly subjective judgmental procedure the expert described was "reasonably reliable."

The court's analysis of what it would take to achieve such reliability is open to debate, but that is a subject for another day. My target today is the statements that were made or could have been made about the cartridge cases and the gun or guns that fired them. What statements are "conclusion-centric"? Which are "evidence-centric"?

A Preliminary Point

"Evidence" exists in the context of some hypothesis about a matter of fact (a state of nature). Nothing can sensibly be deemed "evidence" without asking, "evidence of what?" That the sky is dark is evidence that it is nighttime. It is not conclusive evidence (nothing is conclusive evidence of an empirical fact) because darkness in the daytime can come from a total solar eclipse (or even less likely events). But darkness is pretty good evidence (E) for the hypothesis (H) that night has arrived. The hypotheses of most interest in cases like Tidd are source hypotheses like Swanepoel's assertion that the cartridges were "fired in the same gun" or "the cartridges were fired in different guns." These assertions are conclusions that the analyst reaches (or does not reach) on the basis of the evidence about the marks on the objects examined.

Conclusion-centric Statements

With respect to the source hypothesis, a scan of the opinion in Tibbs reveals the following conclusion-centered statements from the firearms examiner:

• The cartridges "were fired in the same gun." In other words, the same-gun hypothesis H is true (given the markings seen in the examination). It may not be "scientifically certain," but it is true nonetheless. In symbols, P(H|E) is practically (or exactly?) 1, where the expression "P(H|E)" stands for "the probability of the hypothesis H given the evidence E." Obviously, this is conclusion-centric, as it tells the jury what they should believe about H (or at least what the examiner believes about this conclusion).
• "[I]t was 'more likely than not that this [Sig Sauer pistol] fired the cartridge casing' he was analyzing." This sounds like a statement that, given the marks (E), H is probably true: P(H|E) > 1/2. This statement is also conclusion-centered. It is a statement about the (probable) truth of H. However, in this case,
• "He explained that by agreeing to 'more likely' he was 'saying it is this firearm,' and by 'not' he was expressing that the chance of finding another firearm 'with the same signature or the same fingerprint is remote or really small.'" Without getting hung up on the dissonance in this compound sentence, this is a statement that given the markings (E), the probability of not-H is "really small." Mathematically, P(not-H|E) implies that H is not merely probable--it is really probable: P(H|E) is almost 1. Another conclusion-centric statement.
• "[T]he Sig Sauer pistol could not be excluded from the class of firearms that could have fired the cartridge case submitted for analysis." Also conclusion-centered. "Could not be excluded" means "is included as" (or "is consistent with") a possible source, it asserts that P(H|E) > 0. That focuses on the (possible) truth of the conclusion. But classifying this phrasing as conclusion-centric is a close case. If "could not be excluded" were a more neutral statement of the pair of observations themselves (such as "has the same characteristics" or "matches") then it is a statement about the evidence E rather than what E proves.

Evidence-centric Statements

• "Investigators also located a nine-millimeter Luger cartridge case ... on the street near where the shooting took place." With respect to the source conclusion, the statement of the caliber is evidence-centric in the sense that it specifies a feature of ammunition that might have used in the shooting without drawing a conclusion (be it categorical or probabilistic) about the gun taken from Tidd as having had that ammunition fired in it. That Tidd was found with a gun of the same caliber makes the characteristic incriminating evidence--it raises the probability of H. In symbols, P(H|caliber match) > P(H), where P(H) is the probability of H prior to considering the caliber data.
• “[I]t would be rare to find another firearm that exhibits that same ... pattern ... .” Evidence-centric. We have an assertion that the frequency of the incriminating marks is unusual in the population of guns that might have been used in the shooting. Finding a rare pattern supports H because it would be surprising to see the same (or more persuasive) evidence E if a different gun had been used. This is an assertion about the value of the evidence as proof of H rather than assertion about the truth of H itself.^\3/

So far we have encountered (1) statements of conclusions (or their probabilities) about a source hypothesis and (2) statements of the evidence ("it's a 9-mm caliber bullet," "the bullets have matching patterns," and so on) with regard to the source hypothesis. But there is a further category of evidence-centric statements that are more interesting. These evidence-centric statements go beyond descriptions of the data and comparisons using the data, but they do not go so far as to express conclusions. Rather, they are statements of the degree to which the evidence supports one hypothesis relative to alternative hypotheses.

I cannot find these in Tidd, for the examiner there was following the conclusion-centric paradigm espoused by the Association of Firearm and Tool Mark Examiners (AFTE). But they not hard to envision. Their defining feature is that they concern the probability of the evidence under the different hypotheses rather than the conclusion-centric probability of the hypothesis given the evidence. For example:

• The patterns (considering all the similarities and differences) are much more probable for bullets fired from defendant's gun than for bullets fired from different guns.
• The patterns are much more compatible with the hypothesis that bullets were fired from the defendant's gun than the hypothesis that they were fired from different guns.
• The patterns provide strong support for the hypothesis that bullets were fired from the defendant's gun rather than the hypothesis that they were fired from different guns.

These statements all flow from assessing how probable the patterns are for pairs of hypotheses. They look to P(E|H) rather than to the examiner's judgments of P(H|E). The underlying principle is that patterns that are more probable under one hypothesis than another are stronger evidence for the former hypothesis and thus support it instead of supporting the competing hypothesis. The hope is that experts will better assist the judge or jury by telling them about the strength of the evidence in this way instead of telling them which hypothesis to believe (to some degree of certainty).

Notes

1. David H. Kaye, The Nikumaroro Bones: How Can Forensic Scientists Assist Factfinders?, 6 Va. J. Crim. L. 101 (2018), https://www.ssrn.com/abstract=3177752. See also David H. Kaye, Forensic Statistics in the Courtroom, in Handbook of Forensic Statistics 225 (David Banks et al. eds., 2021), https://www.ssrn.com/abstract=3561914.
2. Frederick Schauer, The Proof: Uses of Evidence in Law, Politics, and Everything Else (2022).
3. Thus, "really small" is a qualitative expression of p-value. In statistics, a p-value is the the probability of such extreme evidence arising when the "alternative hypothesis" (such as "different gun") is true. Everything else being equal, the smaller the p-value, the harder it is to believe the alternative hypothesis is true.