Tuesday, July 30, 2013

More on the Population-wide Database

Chris Halikes, posted a comment on yesterday's blog posting about expanding the DNA database to the entire population. He expressed the following concern:
The mathematics of coincidental matches, particularly with respect to the size of the database, is a thorny subject. However, the odds of a coincidental match seem to rise (the so-called birthday problem). Given the strength of the tunnel vision that sets in when DNA is involved (the Lukis Anderson/Raveesh Kumra case is a recent example), I would think long and hard about the wisdom of having an all-inclusive database.
The prospect of more false accusations definitely deserves thought.

In evaluating this risk, the birthday problem is not on point. In the birthday problem, the number of comparisons grows exponentially with the number of people of in the room because there is no single birthday of interest. For a group of size N, there are approximately N2/2 comparisons. For a fixed birthday, however, there are only N comparisons. 

In case work, no one examines all possible pairs of profiles. Like a fixed birthday, a single crime-scene profile is compared to the profiles in the database. Because the number of comparisons is N, the risk grows only linearly with the size of the database. The exponential growth in the Birthday Paradox does not occur.

CODIS is expanding the number of loci in the profile, so the chance that two people (other than monozygotic twins) share the same profile will be even smaller than it is now. When the crime-scene sample yields a good profile (say, 16 or more loci), I cannot see why increasing the database size to that of the entire population would produce many matches to people whose DNA was not at the crime-scene.

The record of an innocent monozygotic twin would pop up in every database trawl for a crime-scene DNA profile that actually belongs to the guilty twin. Strictly speaking, these are not these "coincidental," because there is a deterministic explanation. For such matches -- and for truly coincidental ones -- the population-wide database flags the problem for the police. They get matches to more than one individual! They will know that they must investigate further. Thus, universality alleviates the problem of a "coincidental match" by making every such match apparent.

The problem in the Kumra case, according to prosecutors, was secondary transfer to fingernails that (I assume) did not have DNA from the actual killers. Likewise, if the police or anyone else plants DNA from a target in an incriminating place where the perpetrator's DNA might be found -- and the perpetrator's DNA is not there -- the database trawl will identify the target instead of the perpetrator.

If a crime-scene sample becomes contaminated with extraneous DNA from individuals who were not present at the crime-scene  in an amount sufficient to yield a clear and complete profile, the result could be a false accusation and ensuing conviction for people living in the vicinity, being in the correct age range and physical capacity, and lacking a convincing alibi.

Another limitation of DNA databases is that, at best, they can only show that an individual was at a crime-scene at some point. If police and prosecutors unreflectively equate presence with guilt and a suspect has no persuasive explanation for his presence, injustice could follow. However, this is a problem today. Arguably, a universal database might diminish the problem: cases of innocent presence would arise more often, leading to greater sensitivity to this limitation.

Less worrisome are errors such as mislabeling or mistyping the samples in the database. If the profile recorded for me is not my profile but was present at the crime-scene, this mistake will become apparent when I am retested after the cold hit, as is standard procedure. The confirmatory test will exclude me.

A population-wide has other advantages than those I mentioned today and yesterday -- but it also has its share of disadvantages.

References

Related Postings

Posted by DH Kaye at 11:58 AM 0 comments
Labels: , ,

Monday, July 29, 2013

On the Hypothetical Population-wide DNA Database

The August ABA Journal landed in my mailbox. Usually, I ask reporters to check with me on the wording before quoting. Alas, I neglected to do so some weeks ago, when Mark Walsh asked me about DNA databases in the aftermath of the Supreme Court's opinion in Maryland v. King. Mr. Walsh's article quotes me as follows: "Is the point of arrest the sensible place to draw the line? I can imagine a system in which you take a sample from everyone. Newborns already have a heel prick taken for certain genetic testing. At the same time you could take a DNA sample. Not that you expect a newborn to commit a crime, but 20 years later the sample is there in the database."

Oops! I said that? I meant to say this: "Is the point of arrest the sensible place to draw the line? I can imagine a system in which you take a sample from everyone. Newborns already have a heel prick taken for certain genetic testing. Along with these genetic tests, you could obtain a DNA profile. Not that you expect a newborn to commit a crime, but 20 years later the profile is there in the database." The critical difference: "profile," not "sample."

Here is the way that Michael Smith, Ed Imwinkelried and I explained the idea (which was stimulated by remarks of Phil Reilly to the legal issues working group of the National Commission on the Future of DNA Evidence) in another ABA publication back in 2001:
Creating a national identification database all at once would be prohibitively expensive today, even if we had the laboratory capacity to do it. But DNA typing technology is advancing at a pace reminiscent of the exponential growth in computer microprocessing power that has made the “personal computer” a fixture on every desk. Soon it will be feasible to create a DNA identification record for everyone, at least prospectively. For example, it would be easy to extract identification profiles as an adjunct to the existing public health programs that for many years have screened DNA samples from almost all newborns, to identify infants with treatable genetic diseases. The identification profiles could be transmitted to a single, secure, national database. The genetic locations (“loci” is the technical term) used for those identification profiles would be strictly limited to sequences that have no implications for health or other significant physical or mental traits. Furthermore, access to the database would be limited to law enforcement personnel investigating specific crimes in which DNA trace evidence already has been found. Law enforcement agencies would not need—and should not be permitted—to handle, much less retain, the samples.

... Not only would a comprehensive database be valuable to the criminal justice system, but it also would be useful in identifying remains after natural disasters, mass accidents, and terrorist attacks. Such a database is, we  believe, socially advantageous. But we would be the first to acknowledge that this belief is surely debatable, and a panoply of questions must be considered.
...
In the database system we envision, the information that the government is allowed to have is very different from the types that have sparked debates over medical privacy. The preponderance of the human genome consists of sequences that have no medical importance or social significance. Much of the genome is “noncoding” — these sequences are not translated into the proteins that are the machinery of cells — and most of them are not genetically “linked” to any coding sequences. Even in the coding regions, many DNA sequences merely code for traits such as gross features of fingerprints or the pattern of hair follicles in the skin that have no stigmatizing potential. Consistent with current practice, we would limit the database loci to such regions. Consequently, the genetic information included in the database would be no more invasive of privacy than an image of the ridges and whorls in a fingerprint or of the blood vessels in the retina of the eye.
...
[T]he system we envision keeps almost all samples out of the hands of law enforcement officials. Recall that the initial typing would be done by health workers, not police, as part of neonatal screening. No samples would be sent to law enforcement agencies — they would receive only the biometric genotypes that have no use except for identification. To the extent that additional sampling, say, of immigrants or citizens born abroad, would be necessary to cover as much of the population as possible, the sample could be destroyed as soon as the typing is complete. In fact, an instrument could be built that would extract an identifying profile and destroy the sample at the same time. Proper procedures for sampling the DNA, extracting the identifying profile, and immediately destroying the sample would protect everyone’s genetic privacy — to the extent we have any when private hospitals and HMOs keep samples of our blood and other tissue together with information far more sensitive than the random bits of DNA that identify us. The government officials maintaining the database could neither invade privacy nor enable insurers or employers to do so.
Looking back at these words 12 years later, I would change some of them as well. That the identification loci are not protein-coding is not sufficient to prove that they have no clinical predictive or diagnostic value. Moreover, the loci certainly are more informative than fingerprints or retinal patterns with respect to ascertaining parentage or siblingship. A population-wide database of profiles with the current CODIS loci would make it possible for the government to do "legitimacy testing" -- that is, to check families for children born out of wedlock. Other privacy questions would need to be addressed as well. As we wrote in 2001, "Our vision is futuristic ... incomplete and tentative. Many details remain to be worked out."

References

  • David H. Kaye, Michael E. Smith, and Edward J. Imwinkelried, Is a DNA Identification Database in Your Future?, Criminal Justice, Fall 2001, at 5-9, 19
  • Mark Walsh, "21st Century Fingerprinting," ABAJ, Aug. 2013, at 16-17

Posted by DH Kaye at 4:49 PM 2 comments
Labels:

Saturday, July 27, 2013

Contamination or Sasquatch in Forensic Laboratories -- Which Is it?

DNA Diagnostics, Inc., is “a laboratory that provides multi-species testing including human as well as animal DNA testing for individuals, law enforcement, breed associations, and state regulatory agencies.”The lab
  • “participates in both human and animal proficiency testing”
  • has staff who are “eminently qualified to provide superior DNA testing services” and of “unquestionable quality,” and
  • performs “testing ... of the highest quality” (including “high volume genetic testing”) with “state of the art instrumentation and modern facilities .. on the cutting edge of technology in DNA testing.
Thus, “[s]taff members ... have been accepted in court as experts in human and animal forensic DNA testing ... for the prosecution and the defense and in [state and federal] criminal and civil cases.”

These attributes are what one would want in a forensic laboratory, and the work of this laboratory is nothing short of amazing. Last November, it issued a press release that “calls on public officials and law enforcement to immediately recognize the Sasquatch as an indigenous people.”

The Sasquatch? As in Bigfoot?

Absolutely. A “team of experts in genetics, forensics, imaging and pathology, led by Dr. Melba S. Ketchum ... sequenced 3 complete Sasquatch nuclear genomes and determined the species is a human hybrid” living in North America. The team discovered that “the legendary Sasquatch is a human relative that arose approximately 15,000 years ago as a hybrid cross of modern Homo sapiens with an unknown primate species.” Mitochondrial and “next generation sequencing [of] 3 whole nuclear genomes ... indicate that the North American Sasquatch is a hybrid species, the result of males of an unknown hominin species crossing with female Homo sapiens.”

This might seem amusing, but the study director’s responses to other scientists who interpret the results, not as proof of a hybrid species, but as an indication of contaminated samples, sound eerily like what one hears in court. John Timmer, a science writer and molecular biologist, describes what he found when he examined the “genomes” and conferred with her:
In cases where the hair comes attached to its follicle, it's possible to extract DNA from its cells. And that is exactly what the bigfoot team did, using a standard forensic procedure that was meant to remove any other DNA that the hair had picked up in the interim. If everything worked as expected, the only DNA present should be from whatever organism the fur originated from.

And, in Ketchum's view, that's exactly what happened. They worked according to procedure, isolating DNA from the hair follicles and taking precautions to rule out contamination by DNA from anyone that was involved in the work. Because of this, Ketchum is confident that any DNA that came from the samples once belonged to whatever creature deposited the fur in the woods—no matter how confusing the results it produced were. "The mito [mitochondrial DNA results] should have done it," she argued. "It's non-human hair—it's clearly non-human hair—it was washed and prepared forensically, and it gave a human mitochondrial DNA result. That just doesn't happen."

Ketchum was completely adamant that contamination wasn't a possibility. "We had two different forensics labs extract these samples, and they all turned out non-contaminated, because forensics scientists are experts in contamination. We see it regularly, we know how to deal with mixtures, whether it's a mixture or a contaminated sample, and we certainly know how to find it. And these samples were clean."
Timmer’s article on how science went wrong is well worth reading — and chilling if you think about how DNA Diagnostics’ director -- an eminently qualified expert witness -- might testify about contamination in a more mundane case.

References

Acknowledgment: Thanks to Joe Cecil for calling John Timmer's article to my attention.
Posted by DH Kaye at 1:12 PM 0 comments
Labels: ,

Friday, July 26, 2013

Good Point, Bad Math: DNA Database Statistics Misunderstood (Again)

In a New York Times op-ed article, High-Tech, High-Risk Forensics, Hastings Law School Professor Osagie K. Obasogie mentions a false arrest in the investigation of the murder near San Jose of millionaire investor Raveesh Kumra. A database trawl seemed to implicate Lukis Anderson, a homeless man who then “spent more than five months in jail with a possible death sentence hanging over his head” before prosecutors received records showing that Mr. Anderson was in a hospital for alcohol intoxication on the night of the murder. Prosecutors have suggested that paramedics who treated Anderson for intoxication at a liquor store in San Jose a few hours before the murder inadvertently transferred his DNA to Mr. Kumra’s fingernails.

This is by no means the first instance of secondary transfer of DNA to a crime scene (if that is the correct explanation), and forensic scientists have been conducting studies to see how often such transfer occurs. The case shows that police and prosecutors, as well as defense counsel and jurors, should not be overawed by matches from DNA database trawls. This is the good point that Professor Obasogie makes.

The rest of the op-ed article perpetuates mathematical fallacies about DNA databases. First, Professor Obosagie questions “the frequent claim that it is highly unlikely, if not impossible, for two DNA profiles to match by coincidence.” Nothing is impossible, but not many people share the same DNA identification profiles. The op-ed tries to deny this with the observation that “[a] 2005 audit of Arizona’s DNA database showed that, out of some 65,000 profiles, nearly 150 pairs matched at a level typically considered high enough to identify and prosecute suspects. Yet these profiles were clearly from different people.”

The 150 or so matches were, in fact, mismatches. That is, they were partial matches that actually excluded every “matching” pair. Only if an analyst improperly ignored the nonmatching parts of the profiles or if these did not appear in a crime-scene sample could they be reported to match.

Moreover, even we treat all 150 partial matches as tantamount to false full matches in casework, a proper analysis must account for artificially pairing every profile with every other profile and for the many ways to find some kind of partial match. The scientific and legal literature clearly shows that many partial matches are to be expected under these conditions. The 65,000-some samples then in the database gave rise to over a trillion possible partial matches. (The same combinatorial explosion explains the well-known “Birthday paradox” in probability theory.) A mere 150 partial matches out of a trillion opportunities to make such matches represents a quasi-false match rate on the order of about 100 per trillion (0.0000000001). This number is not quite zero, but neither is it “high risk.”

Second, Professor Obosagie notes that “There are also problems with the way DNA evidence is interpreted and presented to juries.” True enough. A common problem is the confusion of a random match probability with a source probability, and the op-ed makes this very mistake when it claims that jurors in the San Francisco prosecution of John Puckett five years ago were “told that there was only a one-in-1.1 million chance that this DNA match was pure coincidence.”

According to Mr. Puckett’s brief on appeal, the DNA analyst testified not to this source probability, but only that “the profile would occur at random among unrelated individuals in about 1 in 1.1 million U.S. Caucasians ... .” (The op-ed also mistakenly asserts that the defendant, who died before his appeal was heard, “is now serving a life sentence.”)

How prosecutors should present statistics in the cases in which the match came about from a database trawl is an important question, but it would have been misleading to tell jurors that the “chance that this DNA match [to John Puckett] was pure coincidence” was “one in three,” as Puckett wanted to do. Putting the possibility of laboratory error to the side (as the op-ed does), the probability of a database trawl suggesting that John Puckett was the killer of 22-year-old Diana Sylvester is 1 if Puckett was in fact the killer. It is about 1 in 1.1 million if he was not.

A figure like 1 in 3 therefore is not suitable for presenting to a jury as a measure of how revealing a database match is, but perhaps it is useful for another purpose. Puckett produced the probability by multiplying the tiny random-match probability of 1 in 1.1 million by the size of the database. This multiplication yields an estimate of how often trawling a database populated entirely by people innocent of every crime for which the database is used would produce matches to anyone. The larger the number, the greater the risk that “innocent databases”—those that fail to contain profiles of the individuals leaving their DNA at crime-scenes—will lead to false accusations.

Of course, we know that this is not a meaningful estimate of the false positive rate of the real databases, for most matches are corroborated with other evidence. If all databases were innocent, and the 1-in-3 number applied, we would be seeing lots of matches to people too young or too old to have committed the crime being investigated, in prison at the time, or having other solid alibis like Mr. Anderson’s.

Thus, it is improbable that strictly coincidental database matches are common—particularly in the run-of-the-mill cases in which the crime-scene DNA is not a mixture or too small or too degraded to be fully typed. Nevertheless, no one can say exactly how often DNA database searches turn up plausible suspects who are, in fact, innocent. This takes me back to Professor Obasogie’s one good point—let’s not get carried away with DNA database matches. There can be innocent explanations for reported matches. But let’s not become overly skeptical of the value of DNA databases to generate investigative leads—and let’s not use bad math to prop up our conjectures.

References

Posted by DH Kaye at 12:32 PM 2 comments
Labels: , ,

Thursday, July 25, 2013

Ninth Circuit Upholds Indefinite Retention of DNA Samples: More Problems with Judge Reinhardt’s Dissenting Opinion

In United States v. Kriesel, No. 11–30197, 2013 WL 3242293 (9th Cir. June 28, 2013), Judge Stephen Reinhardt characterized the court’s rejection of a convicted felon’s motion for the return of a DNA sample as the greatest judicially condoned infringement “on the privacy rights of so many Americans” in the history of this country. I suggested that to the extent that the opinion identified legitimate privacy interests, it overstated the actual infringement. As I explained, the abuses that worried Judge Reinhardt are either remote or prohibited by statute.

The last point makes the Kriesel dissent all but impossible to square with the reasoning of the Supreme Court in Maryland v. King. The King majority relied on statutory prohibitions on the kinds of abuses enumerated by Judge Reinhardt in maintaining that the individual privacy interests in preventing the state from acquiring DNA samples were too weak to justify invalidating the state’s pretrial DNA-sampling law.

Judge Reinhardt’s response seems to be that even if actual, widespread abuse is unlikely, “[i]t is the government's possession and control of Kriesel's most intimate genetic information that invades his right to privacy. Thus, Kriesel's concerns are real and legitimate, not speculation or mere conjecture.” But granting that some information on individual genomes is private, how can the interest in this privacy be “real and legitimate” if the prospect that the government will access and misuse the intimate information is (according to the King Court) unreal and illegitimate?

Before turning to the topic of the government’s reasons for wanting to retain samples indefinitely, a few other misleading parts of the dissenting opinion should be corrected. A trivial observation is that the opinion tries to wrap itself in apparently neutral or deep scholarship when it relies on views in the book Genetic Justice: DNA Data Bases, Criminal Investigations, and Civil Liberties. To create this impression about this book (which I have discussed here and here), the opinion elevates the authors—a professor of urban and environmental planning and policy, and an FDA employee formerly with the ACLU—to the ranks of  “prominent bioethicists.”

More significantly, the opinion seems to suggest that the FBI permanently houses DNA samples from “persons arrested or detained ... for the most minor of infractions committed on federal land, such as: cleaning or washing any personal property, fish, animal, or bathing at a faucet not provided for that purpose; water skiing in an area where prohibited by order; allowing a pet dog off its leash; distributing handbills without permission; and parking illegally.” According to Judge Reinhardt, “none of these people may seek the return of their DNA sample.”

But the government does not have “indefinite access to stored blood samples” (or buccal samples) if there is no conviction. In that event, the statute requires expungement of “the DNA analysis” from the “index” (if the Attorney General receives notice from the courts of this outcome). I assume that the actual samples are outside this provision, but (1) my understanding might be amiss, since one part of the statute (§ 14132(a)(1)(C)) speaks of “samples ... in the National DNA Index System” and the statute’s definition of “DNA analysis” is opaque, and (2) I understand that FBI policy is to destroy samples when expunging records. In any event, my reading of the majority opinion reveals little to suggest that the Kreisel majority countenances indefinite retention in this situation.

Finally, the dissent complains that “perhaps worse, the government retains blood samples given voluntarily to help solve crimes ... .” In Judge Reinhardt’s opinion, an individual who voluntarily submits DNA to police “surrenders his most fundamental privacy interest in not having his basic genetic information fall permanently into the hands of a government that is not always sensitive to the importance of the constitutional right to privacy.”

I am not going to pretend that the U.S. government is always sensitive about privacy (or anything else), but there are two glaring defects in this complaint. First, if the Fourth Amendment justification for permanent retention is consent, then the individual must consent to permanent retention. Just saying, "yes, you may use my sample" in connection with a particular investigation (such as the mass screening in Truro, Massachusetts, cited in the dissent) should not establish such consent.

Second, even if an individual wanted to surrender his constitutional rights, the government could not store the sample. The statute provides “that DNA samples that are voluntarily submitted solely for elimination purposes shall not be included in the National DNA Index System.” 42 U.S.C. § 14132(a)(1)(C) (emphasis added). Omitting this express limitation is especially puzzling given that the opinion cites § 14132(a) as the authority for its claim that the opposite is true.

Having lingered over the blemishes and flaws in the dissent, I need to turn to the more penetrating facets of the opinion. Stay tuned.
Posted by DH Kaye at 9:17 PM 0 comments
Labels: ,