Statistics for Making Sense of Forensic Genetics

The European Forensic Genetics Network of Excellence (EUROFORGEN-NoE) is a group of “16 partners from 9 countries including leading groups in European forensic genetic research.” In 2016, it approached Sense About Science — “an independent charity that challenges misrepresentation of science and evidence in public life” — to prepare and disseminate a guide to DNA evidence. Within the year, the guide, entitled Making Sense of Forensic Genetics, emerged. The 40-page document has a long list of “contributors,” who, presumably, are its authors. According to EUROFORGEN-NoE, it is “designed to introduce professional and public audiences to the use of DNA in criminal investigations; to understand what DNA can and can’t tell us about a crime, and what the current and future uses of DNA analysis in the criminal justice system might be.”

By and large, it accomplishes this goal, offering well informed comments and cautions for the general public. Some of the remarks about probabilities and statistics, however, are not as well developed as they could be. The points worth noting have more to do with clarity of expression than with any outright errors.

Statistics do not arise in a vacuum. Proper interpretation requires some understanding of how they came to be produced. Thus, Making Sense correctly observes that:

DNA evidence has a number of limitations: it might be undetectable, overlooked, or found in such minute traces as to make interpretation difficult. Its analysis is subject to error and bias. Additionally, DNA profiles can be misinterpreted, and their importance exaggerated, as illustrated by the wrongful arrest of a British man, ... . Even if DNA is detected at a crime scene, this doesn’t establish guilt. Accordingly, DNA needs to be viewed within a framework of other evidence, rather than as a standalone answer to solving crimes.

With respect to the narrow question of whether two DNA samples originate from the same individual, Making Sense asks, “So what is the chance that your DNA will match that of someone else?” An ambiguity lurks in this question. Does it refer to probability of a matching profile somewhere in the population, or to the probability of a matching profile in  a single, randomly selected individual? Apparently, the authors have the latter question in mind, for Making Sense explains that

It depends on how many locations in the DNA (loci) you look at. If a forensic scientist looked at just one locus, the probability of this matching the same marker in another individual would be relatively high (between 1 in 20 and 1 in 100). ... Since European police forces today typically analyse STRs at 16 or more loci, the probability that two full DNA profiles match by chance is miniscule — in the region of 1 in 10 with 16 zeros after it (or 1 in 100 million billion). ... Although in the UK court, the statistics are always capped at 1 in a billion.

The 1-in-a-billion cap is not seen in the United States, where laboratories toss about estimates in the quintillionths, septillionths, and so on (and on). (Could this be an instance of “America First”?) The naive reader might be forgiven for thinking that when the probability of the same match to a randomly selected individual is far less than 1 in a billion, an analyst could conclude that the recovered DNA is either from the defendant or a close relative. But Making Sense rejects this thought, insisting that “DNA doesn’t give a simple ‘yes’ or ‘no’ answer.”

The explanation for its position is muddled. First, the report repeats that “with information available for all 16 markers, ... the risk of DNA retrieved from a crime scene matching someone unrelated to the true source is extremely low (less than 1 in a billion, and often many orders of magnitude lower than this).” So why is not this good enough for a “yes or no answer”? The hesitation, as expressed, is that

However, many of the DNA profiles retrieved from crime scenes aren’t full DNA profiles because they’re missing some genetic markers or there is a mixture of DNA from two or more people. So was it the suspect who left their DNA at the crime scene? The DNA evidence won’t give a ‘yes’ or ‘no’ answer: it can only ever be expressed in terms of probability.

But the conclusion that “it can only ever be expressed in terms of probability” is a non sequitur. The only thing that follows from the fact that not all crime-scene DNA samples lead to 16-locus profiles is that matches to the samples with less complete profiles are less convincing than matches to the samples with more complete profiles.

Of course, there is a sense in which all DNA evidence only gives rise to probabilities, and never to categorical conclusions. All empirical evidence only gives probable conclusions rather than certainties. Furthermore, it has been argued that forensic scientists should eschew source attributions because their expertise is limited to evaluating likelihoods — the probability of the match given that the sample came from a named individual and the probability given that it came from a different individual (or individuals). But that is not what Making Sense seems to be saying when declares yes-and-no answers impossible. The limits on all empirical knowledge and the role of an expert witness do not produce any line between 16-locus matches and less-than-16-locus matches.

Making Sense also points out that

[T]he match probability ... must not be confused (but often is) with how likely the person is to be innocent of the crime. For example, if a DNA profile from the crime scene matches the suspect’s DNA and the probability of such a match is 1 in 100 million if the DNA came from someone else, this does not mean that the chance of the suspect being innocent is 1 in 100 million. This serious misinterpretation is known as the prosecutor’s fallacy.

Conceptually, this transposition is a “serious misinterpretation,” but whether the correct inverse probability (one that is based on a prior probability and a Bayes factor on the order 100 million) gives a markedly different value is far from obvious. See David H. Kaye, The Interpretation of DNA Evidence: A Case Study in Probabilities, in Making Science-based Policy Decisions: Resources for the Education of Professional School Students, Nat'l Academies of Science, Engineering, and Medicine Committee on Preparing the Next Generation of Policy Makers for Science-Based Decisions ed., Washington, DC, 2016.

A reasonable approach is to have analysts present the two pertinent conditional probabilities mentioned above (the “likelihoods”) to explain how strongly the profiles support one hypothesis over the other. Making Sense refers to this approach in some detail, but it suggests that it is needed only “in more complex cases, such as mixtures of two or more individuals, or when there might be contamination by DNA in the environment.” Compared to the alternative ways to explain the implications of DNA and other trace evidence, however, the approach is more widely applicable.

If You Are Going To Do a “DNA Dragnet,” Cast the Net Widely

Police in Rockingham County, North Carolina, took a circuitous path to identify the killer of a couple who were shot to death in their home in Reidsville, NC. They utilized a “DNA dragnet,” kinship analysis, ancestry analysis, and DNA phenotyping to conclude that the killer was the brother-in-law of the daughter of slain couple. Had the initial DNA collection been slightly more complete, that effort alone would have sufficed.

The evidence that led to the man ultimately convicted of the double homicide were drops of the killer's blood:

Parabon Nanolabs, The French Homicides, Jan. 4, 2017 [hereinafter Parabon]

---

In the early hours of 4 Feb 2012, Troy and LaDonna French were gunned down in their home in Reidsville, NC. The couple awoke to screams from their 19-year old daughter, Whitley, who had detected the presence of a male intruder in her second floor room. As they rushed from their downstairs bedroom to aid their daughter, the intruder attempted to quiet the girl with threats at knifepoint. Failing this, he released Whitley and raced down the stairs. After swapping his knife for the handgun in his pocket, he opened fire on the couple as they approached the stairwell. During his escape, the perpetrator left a few drops of his blood on the handrail, apparently the result of mishandling his knife. ...

Seth Augenstein, Parabon’s DNA Phenotyping Had Crucial Role in North Carolina Double-Murder Arrest, Conviction, Forensic Mag., Jan. 5, 2017 [hereinafter Augenstein]

---

A couple were gunned down by an intruder in their North Carolina home in the early hours of Feb. 4, 2012. The teenaged daughter had seen the hooded gunman, when he had briefly held a knife to her throat, but she could apparently not describe him to cops. The attacker left several drops of blood on a handrail as he fled, apparently self-inflicted from his blade.

At a press conference, Sheriff Sam Page announced that "You can run, but you can’t hide from your DNA." Danielle Battaglia, Blood on the Stairs, News & Record Greensboro.com, Apr. 14, 2016 [hereinafter Battaglia]. But efforts to follow the DNA seemed to lead nowhere.

Parabon
Running short of leads, investigators began collecting DNA samples from anyone thought to have been in or around the French home. "We swabbed a lot of people," says Captain Tammi Howell of the RCSO. "Early on, if there was a remote chance someone could have been connected to the crime, we asked for a swab." In the first 12 months following the crime, over 50 subjects consented to provide a DNA sample. None of the samples matched the perpetrator.

Augenstein
"We swabbed a lot of people," said Capt. Tammi Howell, of the Rockingham County Sheriff’s Office, who led the investigation. "Early on, if there was a remote chance someone could have been connected to the crime, we asked for a swab." Those swabs produced no hits.

In particular, this screening of possible sources in the county eliminated "Whitley, her brother, and her boyfriend at the time, John Alvarez." Parabon. But police did not include Alvarez's father or his three brothers in their dragnet search, and when "[a]nalysts uploaded profiles of the blood drops and the skin fragments along with a sample from Whitley French into a database of known samples maintained by the FBI, [t]hey found no match." Battaglia. (According to Forensic Magazine, "the killer was not in any of the public databases," but law enforcement DNA databases are not public.)

There is some confusion in the accounts of what happened next.

Parabon
The first break in the case came when familial DNA testing, performed at the University of North Texas, revealed the possibility that the perpetrator might be related to John Alvarez, Whitley's boyfriend. Because traditional DNA testing is limited in its ability to detect all but the closest relationships (e.g., parent-child), this report alone did not provide actionable information. Subsequently, scientists at the University of North Texas performed Y-chromosome STR analysis, which tests whether two male DNA samples share a common paternal lineage. This analysis, however, showed that the perpetrator did not share a Y-STR lineage with John Alvarez, seemingly eliminating John's father and brother as possible suspects.

Augenstein
Further analysis then indicated that the daughter’s boyfriend, John Alvarez (who had given a swab), could be related to the killer. But it was only a possible relationship, since the STR did not definitively say whether the killer and the boyfriend shared ancestry.
The partial DNA matching led to a Y-STR analysis. The short-tandem repeat on the Y chromosome shows paternal links between fathers, sons and brothers, and has produced huge breakthroughs in cases like the Los Angeles serial killer Lonnie Franklin, Jr., infamously dubbed the “Grim Sleeper.” But in the Sleeper and other cases used “familial searching,” or “FS,” a painstaking and somewhat controversial process of combing large state and national databases like CODIS to find partial DNA matches eventually leading to a suspect. FS was not used in the Rockingham County case, where they had a limited pool of suspects.

Battaglia
Investigators then decided to send the DNA samples out of state for what the warrant called “familial DNA testing,” a type of analysis that allows scientists to match DNA samples to a parent, child or sibling. According to warrants, the samples were sent to the Center for Human Identification at the University of North Texas in Denton. But they do not appear to have gone to that lab. And Rockingham County District Attorney Craig Blitzer said that although a lab did the familial DNA test, it was not North Texas. He declined to say where it was done.

The term "familial searching" has no well-established scientific meaning. As explained in David H. Kaye, The Genealogy Detectives: A Constitutional Analysis of “Familial Searching”, 51 Am. Crim. L. Rev. 109 (2013), kinship testing of possible parents, children, and siblings can be done with the usual autosomal STR loci used for criminal forensic investigation. When this technique is applied to a database (local, state, or national), it sometimes reveals that the crime-scene DNA matches no one in the database but is a near miss to someone -- a near miss in such a way as to suggest the possibility that the source of the crime-scene sample is a brother, son, or parent of the nearly-matching individual represented in the database. In other words, "familial searching" is the process of trawling a database for possible matches to people outside of the database -- "outer-directed trawling," for short.

The Rockingham case evidently involved a conventional but fruitless database search ("inner-directed trawling") followed by testing -- in Texas or somewhere else -- to ascertain whether it was plausible that a close relative of the boyfriend was the source of the blood. Based on the autosomal STRs, this seemed to be the case. However, the laboratory threw a monkey wrench into the investigation when it reported that Y-STRs in the boyfriend's DNA did not match the blood DNA. Because Y-STRs are inherited (usually unchanged) from father to son, this additional finding seemed to exclude the untested father and brothers of the boyfriend.

But the social and familial understanding of a family tree does not always correspond to a biological family tree. It is not unheard of for genetic tests for parentage to reveal unexpected cases of illegitimate children. A man and child who believe that they are father and son may be mistaken. Genetic genealogists like to call the phenomenon of misattributed paternity a Non-Paternity Event, or NPE.

Thinking that the male members of the immediate Alvarez family had to be innocent, police were stymied. They turned to Parabon Nanolabs in Reston, Virginia.

Parabon
[For $3,500, the lab,] starting with 30 ng of DNA, ... genotype[d] over 850,000 SNPs from the sample, with an overall call rate of 98.9% [and advised the police that the blood probably came from a man with] fair or very fair skin, brown or hazel eyes, dark hair, and little evidence of freckling, ... a wide facial structure and non-protruding nose and chin, and ... admixed ancestry, a roughly 50-50 combination of European and Latino ancestry consistent with that observed in individuals with one European and one Latino parent. ... "The Snapshot ancestry analysis and phenotype predictions suggested we should not eliminate José as a suspect, despite the Y-STR results," said Detective Marshall. "The likeness of the Snapshot composite with his driver's license photograph is quite striking."

Augenstein
From approximately 30 nanograms of DNA, the software genotyped approximately 850,000 single-nucleotide polymorphisms, or SNPs, at a call rate of 98.9 percent. In this case, the blood showed the killer to be someone with mixed ancestry – apparently someone with one European and one Latino parent. ... "The Snapshot ancestry analysis and phenotype predictions suggested we should not eliminate Jose (Jr.) as a suspect, despite the Y-STR results," said Det. Marcus Marshall, the lead investigator on the case. "The likeness of the Snapshot composite with his driver’s license photograph is quite striking."

At this time, Parabon proudly juxtaposes the "Snapshot Composite Profile and a photo of José Alvarez, Jr., taken at the time of his arrest" on its website.(and shown below). One of the more intriguing (genetically associated?) similarities is the five o'clock shadow.


It also would be interesting to know how "confidence" in skin color and other phenotypes is computed. In any event, with this report, police finally obtained DNA samples by consent from the father, José Alvarez Sr., José Alvarez Jr., and Elaine Alvarez, the mother. Analysis indicated misattributed paternity -- and a conventional STR match of the DNA in the bloodstains. As a result,

Parabon
José Alvarez Jr. was arrested on 25 Aug 2015 on two counts of capital murder. He later pled guilty to both murders and on 8 Jul 2016 was sentenced to two consecutive life sentences without the possibility of parole.

Augenstein
Jose Alvarez, Jr., was ... arrested in August 2015 and charged with two counts of capital murder. He later pleaded guilty to killing the Frenches, and was sentenced to two consecutive life sentences without the possibility of parole in July 2016.

A final note on the twists and turns in the case is that John Alvarez's wedding to Whitley French "had been planned for months. Jose Alvarez Jr. served as a groomsman for his brother even as detectives were planning to arrest him on charges that he murdered his new sister-in-law’s parents." Battaglia.

Related posting

"We Can Predict Your Face" and Put It on a Billboard, Forensic Sci., Stat. & L., Nov. 28, 2016

Reflections on Glass Standards: Statistical Tests and Legal Hypotheses

Statistical Applicata (Italian Journal of Applied Statistics) recently published several issues (volume 27, nos. 2 & 3) devoted to statistics in forensic science and law. They include an invited article I prepared in 2016 on the statistical logic of declaring pieces of glass "indistinguishable" in their physical properties. 1/ The article contains some of the views expressed in postings on this blog (e.g., Broken Glass: What Do the Data Show?). However, the issue is much broader than glass evidence. The article notes the potential for confusion in reporting that any kind of trace-evidence samples match (or cannot be distinguished) without also describing data on the frequency of such matches in a relevant population. I am informed that NIST's Organization of Scientific Area Committees on Forensic Science (OSAC) is preparing guidelines or standards for explaining the probative value of results obtained from ASTM-approved test methods.

Abstract

---

The past 50 years have seen an abundance of statistical thinking on interpreting measurements of chemical and physical properties of glass fragments that might be associated with crime scenes. Yet, the most prominent standards for evaluating the degree of association between specimens of glass recovered from suspects and crime scenes have not benefitted from much of this work. Being confined to a binary match/no-match framework, they do not acknowledge the possibility of expressing the degree to which the data support competing hypotheses. And even within the limited match/no-match framework, they focus on the single step of deciding whether samples can be distinguished from one another and say little about the second stage of the matching paradigm–characterizing the probative value of a match. This article urges the extension of forensic-science standards to at least offer guidance for criminalists on the second stage of frequentist thinking. Toward that end, it clarifies some possible sources of confusion over statistical terminology such as “Type I” and “Type II” error in this area, and it argues that the legal requirement of proof beyond a reasonable doubt does not inform the significance level for tests of whether pairs of glass fragments have identical chemical or physical properties.

Note

1. The article is David H. Kaye, Reflections on Glass Standards: Statistical Tests and Legal Hypotheses, 27 Statistica Applicata -- Italian J. Applied Stat. 173 (2015). Despite the publication date assigned to the issue, the article, as stated above, was not written until 2016.