Fitting "Physical Fit" into the Courtroom

The logic of piecing together fragments of broken glass, torn tape, cut paper, and the like seems simple enough. \1/ If the pieces fit in all their details at the edges, and if all surface marks or impressions that would cross an edge also align nicely, one has circumstantial evidence that they were once part of the same object.

The strength of this evidence for a single source depends on the extent and detail of the concordance between the recovered pieces. A physical fit between two halves of a broken plank of wood is powerful evidence for the hypothesis that the two pieces resulted from breaking this one plank. But if the pieces are weathered and the splintered edges dulled, the physical fit will be less precise and less supportive of the claim that they came from the same original plank.

At the other extreme, if two pieces are plainly discordant, they might have come from different places on the same object, with the intermediate pieces being missing. Or they might have come from different objects entirely. Consider tearing off five pieces of duct tape from the same roll of tape and comparing the edges of the first and the last segments. The detailed structure of the edges should not be complementary. Likewise, tearing segments of tape from five different rolls should result in a mismatch between the first and the fifth segment.

Criminalists or materials experts can be extremely helpful in examining the recovered pieces of objects to determine the degree of physical fit -- that is, in elucidating how well the edges fit together and the extent to which a mark on the surface of one piece lines up with a mark on the other when the pieces are aligned. But how they should describe their findings seems to be muddled in forensic-science standards. This posting describes the current vocabulary and argues that it is articificial and a departure from the ordinary meaning of the term "fit." It then outlines better alternatives to reporting the results of an investigation into physical fit.

I. The Standard Approach

Let’s look at a couple of ASTM standards. E2225-19a (Standard Guide for Forensic Examination of Fabrics and Cordage) instructs that “[i]f a physical match is found, it should be reported in a manner that will demonstrate that the two or more pieces of material were at one time a continuous piece of fabric or cordage” (§ 7.2.2). This standard treats the “physical match” as an observable property of the specimens (concordant edges and surface marks) that is conclusive of the hypothesis of a single source (the inference from the data).

ASTM E3260−21 (Standard Guide for Forensic Examination and Comparison of Pressure Sensitive Tapes), on the other hand, characterizes “physical fit” not as a property of the materials, but as a “type of examination that can be performed” (§ 10.5.1). This “conclusive type of examination ... is a physical end match.” Id. It “involves the comparison of edges, fabric (if present), surface striae, and other surface irregularities between samples in which corresponding features provide distinct characteristics that indicate the samples were once joined at the respective separated edges.” Of course, "distinct characteristics that indicate the samples were once joined at the respective separated edges” are not necessarily "conclusive," making this definition of "physical fit" as a "type of examination" puzzling. The intent, it seems, is to define a physical fit examination (rather than a physical fit) as one that is capable of conclusively proving that the pieces were once joined together.

A Proposed New Standard Guide for the Collection, Analysis and Comparison of Forensic Glass Samples, ASTM WK72932, released for public comment late last year states that “broken objects can be reassembled to their original configuration ... called a ‘physical fit’ (§ 11.1). But a physical fit is the original configuration of a broken object only if the pieces come from that original object, and this origin story is not true just because a standard defines "physical fit" that way. The evidence from the examination may be that the separate pieces fit together extremely well. If so, the conclusion is that they were once together within or as a unitary object. This conclusion may well be true, but one cannot decide, by the fiat of a definition, that the pieces that are observed to fit together well have been realigned as they once were. Yet, a later section similarly asserts that “[a] glass physical fit is a determination that two or more pieces of glass were once part of the same broken glass object” (§ 11.2.8). This effort to define "physical fit" as inherently conclusive prompted eleven lawyers (including me) \2/ to caution ASTM that “[t]he hypothesis or conclusion that fragments come from the same object is not a physical fit. It is an inference drawn from the observations that produce the designation of a physical fit.”

Still more recently, an OSAC subcommittee released a Standard Guide for Forensic Physical Fit Examination (OSAC 2022-S-0015) for public comment before it is delivered to ASTM for consideration there. This proposed standard goes off in another direction. It equates a “physical fit” with the examiner’s state of mind about a hypothetical ensemble of experiments:

13.1 Physical Fit
13.1.1 The items that have been broken, torn, separated, or cut exhibit physical features that realign in a manner that is not expected to be replicated.
13.1.1.1 Physical Fit is the highest degree of association between items. It is the opinion that the observations provide the strongest support for the proposition that the items originated from the same source as opposed to the proposition they originated from different sources.

13.2 No Physical Fit
13.2.1 The items correspond in observed class characteristics, but exhibit physical features that do not realign, or they realign in a manner that could be replicated.
13.2.2 Alternatively, the items can exhibit physical features that partially realign, display simultaneous similarities and differences, show areas of discrepancy (e.g., warped areas, burned areas, missing pieces), or have insufficient individual characteristics that hinder the ability to determine the presence or absence of a physical fit.

Statisticians will notice the shift from (1) the incompletely expressed frequentist idea of an infinite sequence of trials in which different objects A and B are broken and the pieces from A never align with those from B to (2) the likelihoodist conception of support for the same-source hypothesis. But that implicit change in the theory of inference is hardly a cardinal sin in this context. If the probability of a fit at least as good as the one observed is practically zero for different sources, and if the probability of such a fit for the same source is much higher, then the support (the log-likelihood ratio) is very high.

Nevertheless, defining physical fit as a categorical opinion rather than a more variable degree of congruency that generates the opinion — and dumping everything short of a perceived fit into the category of ”no physical fit” — deviates from the common understanding that physical fit comes in degrees. There can be a remarkably great fit, a pretty good fit, and so on, down to a blatant misfit. The question the examiner must answer, at least intuitively, before the fit/no-fit classification can be made is just how well the pieces fit together. Fit is not a uniform degree of association that springs into existence exactly when a particular examiner is convinced that no other source could account for the complexity and extent of the fit. There is no such thing as “the strongest support.” One can always conceive of a situation with still stronger support (because a fracture or other separation of the pieces could generate an even richer set of irregularities in the edges).

The current approach of defining a physical fit as a single source for the pieces and calling everything else “no fit” does not create a vocabulary that judges or jurors will easily understand. A vocabulary in which physical congruency (fit) lies on a continuum — and that then addresses the inference that should be drawn from the observations — is more transparent.The definitions in the standards collapse the two steps of data acquisition and inference into one.

II. Inference: From Data to Conclusions

So how should examiners answer the question of how well the pieces fit together? An examination for fit yields multidimensional, spatial data. An examiner could present photographs of the aligned edges and surfaces and highlight the concordant and discordant features. Although the highlighting involves some interpretative thinking, I have called a courtroom presentation that stops at this point "features-only testimony." \3/ It is appropriate when examiners have no special expertise at interpreting how strongly their results support the same-source hypothesis. If they are no better than lay judges and jurors at discerning how improbable the features are in the hypothetical cases of repeatedly breaking the same object, it could be argued that these witnesses should not try to interpret the results any further. Such interpretation would not actually assist the trier of fact, as required by Federal Rule of Evidence 702.

For example, a few days ago, a forensic scientist told me of a case in which a criminalist was able to reassemble pieces of glass recovered at the site of a hit-and-run accident so that they fit neatly into the metal holder of a side rear mirror on the suspect’s car that was missing its glass. That’s good detective work, but did the criminalist have any special insights to offer into the obvious implications of this solution to the jigsaw puzzle? (The work was not presented in court because the crime laboratory’s management was concerned that there was no written protocol for pasting mirror fragments back in place. As the scientist observed, that's silly. The evidence practically speaks for itself, and its message is the same with or without a written protocol.)

Nevertheless, let’s assume that examiners do have specialized skill at interpreting the findings about the alignment of the features. The ASTM and OSAC-proposed standards ignore the possibility of a qualitative expression of relative support — for example, “It is far more likely to get the detailed alignment of the features I just showed you if the pieces were broken parts of the same objects than if they were from different objects.” Or, similarly, “The detailed alignment gives very strong support to the idea that the pieces broke off of the same object as opposed to two different objects.”

As Part I showed, the standards advocate a fit/no-fit classification in which “fit” is either a statement about the probability of the same-source hypothesis (that the pieces had to have come from the same object) or a statement of belief in the hypothesis (“my opinion is that they were together in the same object — that’s what makes it a physical fit). No-fit does not have a comparably sharp meaning. It could mean anything from no realistic possibility that the pieces were once contiguous parts of the same object to “partial fit features [that] increase the significance of the finding” (OSAC 2022-S-0015 § 13.2.4).

A more straightforward and comprehensible approach would be to have a three-tiered reporting scale for the support the data give to the same-source hypothesis. What is now called a physical fit would be designated a highly probative physical fit (that is, a physical fit that strongly supports the same-source hypothesis). “Partial fit features” would be described as a limited fit (that gives some support to the same-source hypothesis). Finally, an obvious mismatch could be called a misfit (which strongly supports the conclusion that the pieces were never adjacently located on the same object).

This tripartite classification is an imperfect way to express an underlying likelihood ratio formed from subjective probabilities. Whether better results would be achieved if analysts were forced to articulate their probabilities, either quantitatively or in the qualitative way mentioned earlier, is an interesting question. But the three-tiered reporting scale is closer to the current practice and seems feasible. \4/ It offers a framework for a better standard on reporting the results of a physical fit examination. Or so it seems to me — those who disagree are encouraged to hit the comment button.

NOTE

1. But see Forensic Science’s Latest Proof of Uniqueness, Dec. 22, 2013, http://for-sci-law.blogspot.com/2013/12/forensic-sciences-latest-proof-of.html.
2. The other commenters were Alyse Bertenthal, Amanda Black, Jennifer Friedman, Julia Leighton, Kate Philpott, Emily Prokesch, Matt Redle, Andrea Roth, Maneka Sinha, and Pate Skene.
3. David H. Kaye et al., The New Wigmore on Evidence" Expert Evidence (2d ed. 2011).
4. When there is a mismatch, testimony about a physical match has little value. Other features than the alignment of edges and surface markings will need to be studied if the expert is to shed light on whether the pieces came from a single object. The current and proposed standards are clear on this point.

If the Glass Fits, Declare It: The Justice Department's ULTR for Glass that "Physically Fits Together"

The Department of Justice’s “Proposed Uniform Language for Testimony and Reports for the Forensic Glass Discipline” allows a broad range of possible conclusions and statements of probability. The strongest conclusion of association would be that “the glass fragments were once part of the same broken object.” The draft ULTR permits a criminalist to “state or imply” this source conclusion—but only “when two or more pieces of broken glass physically fit together.”

The notion that fragments that fit together must have come from the same larger piece of  glass sounds like common sense. Almost all of us have broken glass or ceramic materials at one time or another and seen that the fragments (at least the major ones) can be reassembled. Apparently, criminalists have had the same experience. The supporting documentation for the ULTR notes that “[i]t has long been reported by forensic glass examiners that two glass objects that physically fit together were once part of the same broken object.” 1/

But what type of scientific inquiry would establish the truth of the common sense idea? Let’s do a series of thought experiments. I have ten drinking glasses in my cupboard. (Actually, six are in the dishwasher, but I’ll pretend all are clean and back in the cupboard.) They sure look alike to me, being the same size, shape, and color.

• Experiment 1. I take one glass out and strike it with a hammer. Then I pick up the pieces and discover that all pairs of fragments that were adjacent in the unbroken glass fit together.


• Experiment 2. I take out a second glass and strike it with a hammer. Then I put all the pieces in a bag with those from the first glass and mix them up. Will I find any pairs of fragments that seem to fit together that are Glass1-Glass2 (heteroginous, if I can coin a word that means “of different origin”), or will all the pairs that fit together be Glass1-Glass1 or Glass2-Glass2 (homoginous, a neologism for "of the same origin")? Let’s suppose that the subset of all the possible pairs that I find to match are indeed homoginous.


• Experiment 3. You guessed it. I take a third glass, strike it with the hammer, and examine all possible pairs from all the fragments from the three glasses. This is getting very time-consuming, for the number of pairs to consider is growing exponentially, but I am persistent. Can we conclude, on the basis of common sense, that each and every physically matching pair will be homoginous?

You can see where I am going with this. Common sense plus some cogitation suggests that homoginality — a given pair of matching fragments coming from the same original piece of glass — depends on how many fragments are in the relevant population and the complexity of the match (that is, the extent of the irregularities in the edges of the fragments being fitted together). It is not so obvious that every physically matching pairs of glass fragments that can possibly exist are homoginous. That glass examiners have said that all matching pairs must be homoginous does not make it so. Neither does labeling mechanical fit as an individual as opposed to a class characteristic (to use the popular terminology in forensic science). The labeling exercise begs the question.

Nonetheless, it is quite reasonable to conjecture that only a tiny fraction (which could include 0) of apparently matching fragments that have complex edge patterns are heteroginous, and thus that finding a physical fit is powerful evidence of homoginality. 2/ Of course, the reasonableness of the conjecture depends on the criteria for a fit and the method for declaring one—neither of which are mentioned in the DOJ materials—but let’s assume the existence and maintenance of rigorous criteria and a reliable method. Can criminalists currently estimate how often the conclusion of homoginality is correct when the fragments (unbeknownst to the analyst) actually come from different pieces of glass?

Even if the answer to this question is in the negative, there are ways to present the physical match without overclaiming. Why must examiners “state or imply that the glass fragments were once part of the same broken object”? Examiners who believe that there is a physical fit can contribute a great deal simply by documenting and exhibiting the fit itself. If, additionally, they have some special expertise in moving from the fit to the conclusion of homoginality, then doing also could assist the fact finder. But if their inference is simply that of common sense, the expert aspect of the probative value of that part of the testimony is questionable. In these circumstances, the criminalist might be better advised to show how well the fragments fit together and leave it to the judge or jury to assess the weight of this demonstrable fact.

Another alternative to testimony “to the exclusion of all other sources” (see note 1) is a statement that the observed match is more probable when fragments really come from the same broken object than when they come from different broken objects. I do not know whether specific studies have been performed to justify this likelihood-ratio-type statement, but it should not be outside the capacity of forensic science to generate data that would support this rather modest statement. Indeed, the claim probably is too weak. My intuition is that the likelihood ratio is much greater than 1—it is a heck of a lot more probable to generate a matching pair of fragments by breaking the same object than by breaking different ones. A lawyer can appeal to give the evidence substantial weight on the basis of similar beliefs. If criminalists have experimental or observational studies to quantify how much more probable the physical match is for homoginous fragments, then they can provide that likelihood ratio.

Based on the material the DoJ has supplied, however, the more ambitious criminalists who want to “render an opinion of positive identity” and asservate “to the exclusion of all other sources” (see note 1), as the proposed ULTR apparently allows, should be required to express the limitation that no matter how positive they are in their opinion, the core of that opinion is a common sense judgment of what it means for pieces of glass to fit together rather than the product of elaborate scientific experimentation.

Note

1. An accompanying footnote cites four sources:
   • Kirk, P., Density and Refractive Index: Their Application in Criminal Identification, Charles C. Thomas, Springfield, IL, 1951, p. 4-5.
   • “Don’t Overlook Evidentiary Value of Glass Fragments”, Law Enforcement Bulletin, Vol. 33, No. 10, October 1964, p. 19.
   • Bottrell, M.C., “Forensic Glass Comparison: Background Information Used in Data Interpretation”, Forensic Science Communications [Online], (April, 2009). Available: http://www.fbi.gov/about-us/lab/forensic-science-communications/fsc/april2009/review/2009_04_review01.htm. accessed on August 15, 2013.
   • Koons, R. D., Buscaglia, J., Bottrell, M., and Miller, E. T. Forensic glass comparisons. In: Forensic Science Handbook. vol. I, 2nd ed. Richard Saferstein, Ed., Prentice Hall, Upper Saddle River, New Jersey, 2002, p. 161–213.
   
   I do not have Paul Kirk’s 1951 book at hand. The Law Enforcement Bulletin is not generally regarded as a substantial scientific publication. The 2009 publication is a generally informative and comprehensive review article from the FBI. However, it contains no reports of examiners' experiences or experiments with physically fitting fragments of glass back together. In fact, only two sentences even refer to physically fitting fragments—and neither of them explains the basis for the conclusion that the ULTR endorses. (The first sentence is this: “Only physically matching two or more broken glass fragments allows for their association with each other to the exclusion of all other sources (Scientific Working Group for Materials Analysis [SWGMAT] 2005c).” The second is equally devoid of information on the foundation for the conclusion. It reads: “The ... Trace Evidence Unit Quality Assurance Manual (2006) states [that a] glass association is defined as two or more glass samples that can be fracture fitted together, or that exhibit indistinguishable observable properties and/or range overlap in all measured properties.”)
   
   Finally, the FBI chapter in Richard Saferstein’s handbook discusses “types of fractures” (pp. 173-77) and later, “the mechanical fit.” The authors write that

   The mechanical fit is one of the most desirable of forensic glass examinations because the examiner can render an opinion of positive identity—that two or more pieces of glass were once a portion of, and were broken from, the same pane or object. Glass is particularly suitable for this type of examination. It is amorphous and brittle, is neither stretched nor distorted by breakage, and can be reassembled to its original configuration. Because it is amorphous, no two glass objects will break in exactly the same way. (P. 179).

   As has been discussed extensively for other types of matching evidence, the claim of uniqueness at some possibly unattainable level of precision of measurement does not imply the truth of an unqualifiedly positive opinion of identity.


2. See generally David H. Kaye, Beyond Uniqueness: The Birthday Paradox, Source Attribution, and Individualization in Forensic Science Testimony, 12 Law, Probability & Risk 3 (2013); David H. Kaye, Probability, Individualization, and Uniqueness in Forensic Science Evidence: Listening to the Academies, 75 Brooklyn L. Rev. 1163 (2010); David H. Kaye, Identification, Individuality, and Uniqueness: What's the Difference?, 8 Law, Probability & Risk 85 (2009).

Forensic Science’s Latest Proof of Uniqueness

A federally funded study on the "Determination of Unique Fracture Patterns in Glass and Glassy Polymers" affirms that fracture-pattern matches are unique. The researchers believe their work permits experts to continue to provide their "usually conclusive" testimony about cracked glass and plastic.

"The purpose of the research" undertaken at the University of California at Davis's graduate program in forensic science was "to provide a first, objective scientific background that will illustrate that repetitive fractures, under controlled conditions on target materials such as glass window panes and glass bottles, are in fact different and unique. In this phase of our study, we fractured glass window panes, glass bottles (clear wine bottles), and polymer tail light lens covers. Each and every fracture was documented in detail for subsequent inter-comparison and to illustrate the uniqueness of the fracture pattern." (Tulleners et al. 2013, p. 7).

Not surprisingly, the researchers found that all their fractures were distinguishable. In all, they conducted 5,310 pairwise comparisons by examining the fracture patterns in all pairs formed within each of the three groups of 60 items. This finding, they concluded, "should aid the practitioner in any court testimony involving the significance of fracture matching of broken glass and polymers materials." (Ibid., p. 23).

What testimony might this be? "For the forensic community, the ability to piece together glass fragments in order to show a physical fit or a 'Physical Match' is the strongest evidentiary finding of an association." (Ibid., p. 6) "The usual statement is that 'the evidence glass fragment was physically matched to another glass establishing thus both share a common origin.'" (Ibid.) This testimony, the researchers suggest, is just fine: "we are substantiating the individuality of glass and polymer fractures under closely controlled conditions." (Ibid., p. 3, emphasis added). Thus, "[t]his research should enhance the capability of the analyst to testify in a court of law as to the uniqueness of a fracture." (Id., p. 61, emphasis added).

But why would the analyst want to claim universal uniqueness? Forensic science’s hoary division of its world into two parts -- "unique" feature sets and "class" characteristics -- is an article of faith. (E.g., Kaye 2009). The latest study certainly is of some use in confirming the intuition that fracture patterns are highly variable. The existence of varying patterns is one fact that makes "fractography" evidence, as it is called in the field, probative. But the study’s explanation of how it proves that every pattern is unique seems like a parody of scientific reasoning. The explanation is this:

In this research, it is hypothesized that every fracture forms a unique and nonreproducible fracture pattern. Alternately, it may be that some fracture patterns may be reproduced from time to time. If it is found that each fracture forms a unique and nonreproducible fracture pattern, then this finding will support the theory that coincidental duplication of fracture patterns cannot be attained. However, if duplicate fracture patterns are found, this would falsify the null hypothesis and show that some fracture patterns may be reproduced from time to time.

(Ibid., p. 27). Such is the power of the unique-vs-class thinking. This impoverished dichotomy collapses a spectrum of possible states of nature into two discrete states. Combined with a cartoon-like version of Sir Karl Popper’s criterion of falsification, it leads the researchers to believe that their failure to find a class characteristic proves the "null hypothesis" of uniqueness.

True, the failure to find "duplicate fracture patterns" in a small sample "support[s] the theory that coincidental duplication of fracture patterns cannot be attained." (Or it would in a study in which the analyst deciding on whether two patterns were the same did not already know that all of them came from different objects.)

But it also supports the alternative theory that coincidental duplication can be attained. Instead of taking no-duplication-is-possible as the “null hypothesis,” we could postulate that, on average, 1 in every 10,000 fractures of the items tested would produce indistinguishable fracture patterns. Or, we could hypothesize that the mean duplication rate is 1/100,000. Since we just spinning out hypotheses, we could pick still other rates.

A great many such hypotheses seem compatible with the finding of no duplicates among 180 fractures. Observing a unique set of patterns in the sample supports (to varying degrees) a wide range of hypotheses about the duplication probability. To indulge an overly simplistic model, if we were to assume that the probability of detecting a duplicated pattern in each of the 5,310 comparisons were some identical, albeit small, number, then the 95% confidence interval for this duplication probability would go from zero (uniqueness) all the way up to 1/1770. (See Eypasch et al. 1995). To testify that the experiment supports only “the theory that coincidental duplication of fracture patterns cannot be attained” would be foolish. A more accurate statement would be that it supports the theory that duplication occurs at an unknown, but not very large, rate.

To be sure, there is reason to believe that duplication is improbable, and the UC-Davis study adds to our knowledge of fracture patterns. However, fractographers should think twice (or more!) before they testify that the study demonstrates the utter uniqueness of all fractures. They gain little by embracing the claim of universal uniqueness (Cole 2009; Kaye et al. 2011), and this study does not deliver on the promise of "objective criteria to determine the uniqueness of a fit." (Tulleners et al., p. 7).

References

• Simon A. Cole, 2009. Forensics Without Uniqueness, Conclusions Without Individualization: The New Epistemology of Forensic Identification. Law, Probability and Risk 8:233-255
• Ernst Eypasch, Rolf Leferinga, C K Kuma, Hans Troid, 1995. Probability of Adverse Events That Have Not Yet Occurred: A Statistical Reminder. Brit. Med. J. 311:619, available at http://www.bmj.com/content/311/7005/619
• David H. Kaye, David E. Bernstein & Jennifer L. Mnookin, 2011. The New Wigmore, A Treatise on Evidence: Expert Evidence. New York: Aspen Pub. Co. (2d ed.)
• David H. Kaye, 2009. Identification, Individuality, and Uniqueness: What's the Difference? Law, Probability & Risk 8:85-89, http://ssrn.com/abstract=1261970 (abstract)
• Frederic A. Tulleners, John Thornton & Allison C. Baca, 2013. Determination of Unique Fracture Patterns in Glass and Glassy Polymers, available at https://www.ncjrs.gov/pdffiles1/nij/grants/241445.pdf