Saturday, May 25, 2013

Bad Math or Passable Law? DNA Testing in the Continuing Prosecution of Amanda Knox and Raffaele Sollecito

In an previous posting, I raised some questions about an op-ed ("Justice Flunks Math") on the judge's refusal to depart from the court-appointed expert's written report in the prosecution of Amanda Knox and Raffaele Sollecito. This week, a flurry of opinionated comments appeared, and I let those that seemed to have at least some analysis or substance through the gate.

In my previous posting, I took issue with the op-ed's assertion that the trial judge "demonstrated a clear mathematical fallacy: assuming that repeating the test could tell us nothing about the reliability of the original results" and its apparent suggestion that retesting the same DNA sample would be comparable to testing a coin for bias by repeatedly tossing it. I argued that "[w]ithout some specification of precisely what made the initial testing problematic and whether those problems could be reduced sufficiently with retesting, it seems precipitous to convict the judge who overturned the guilty verdict of 'bad math.'"

Whatever the merits of the indictment of the judge, my thanks to those who offered information on whether retesting might be significantly more revealing than the initial testing. That is an interesting question in its own right.

In this regard, an author of the op-ed, Professor Leila Schneps kindly explained that the "confirming retest" (the phrase in her op-ed) did not mean a retest of the same sample (like flipping a coin again) but rather an analysis of a "new knife blade sample," a "rich sample ... from the place where the blade joins the handle of the knife." This new sample, she suggests, might be "positive for Meredith Kercher," in which case, "it would have correctly settled two of the questions left outstanding in the courtroom: was the first electropherogram showing the DNA on the knife correctly interpreted as Meredith's, and was Meredith's DNA actually on the knife?"

If we posit that the new sample is large enough to produce unambiguous results, then it could reveal whether "Meredith's DNA [was] actually on the knife." But Professor Schneps also states that the "rich sample" was "significantly lower than the quantity 'advised' by the kit, although the kit's website shows many examples of tests on smaller samples, some even smaller than the knife blade DNA, that gave positive and accurate results."

If the sample is this impoverished, are we not back in the realm of low-template DNA testing, where the worry is that stochastic effects can be dominant? The mathematical argument here seems to be that even though it might not be surprising to spot, by chance alone, some peaks in a new test that also are present in Meredith's genotype, the probability of those peaks plus the ones seen in the original testing of a different sample from the knife would be negligible unless Meredith's DNA was on the knife. In this way, the additional testing overcomes the low signal-to-noise ratio in each sample. That is a fair argument (as far as it goes), and the same logic underlies some protocols for testing contact DNA.

Still, given the difficulties and the level of discord over the best approaches to conducting and interpreting LT-DNA testing (see, e.g., A. Carracedo, P.M. Schneider, J. Butler & M. Prinz, Focus issue—Analysis and Biostatistical Interpretation of Complex and Low Template DNA Samples, Forensic Science International: Genetics 6 (2012) 677–678), and the court's experts' concerns about contamination, I wonder whether even the most mathematically erudite judge would have been so quick to order additional DNA testing in this case. Consequently, I am not yet prepared to give the judge a flunking grade for "a clear mathematical fallacy."

4 comments:

  1. Retesting the knife would not be very meaningful for those who already accept Meredith Kercher's DNA is on the blade, but Dr. Coralie and her son are absolutely right about Judge Hellmann. The following translated testimony is from John Follain's book "A Death In Italy", page 409:

    Judge: "Is there a trace that which can be attributed to Meredith?"
    Dr. Vecchiotti: "... We don't know if Meredith's DNA was there."

    It appears from his report that Judge Hellmann accept his expert's wrong answer. Repeating the test with the latest technology would have likely cleared up his misapprehension and ruled-out some of the forms of contamination that have been suggested by skeptics.

    ReplyDelete
    Replies
    1. If tests for blood prove negative and tests for human DNA prove negative, and then miraculously DNA appears when the blade is swabbed, there is justification for doubts that the DNA originated on the blade.

      In addition, the initial runs produced results that the counts were to low. All protocol had efforts stopping at that point.

      The science of DNA crime investigation has to stand rigorous scrutiny and going to the extreme of science fiction to produce DNA and test results (that cannot be duplicated or even produce consistent results) reeks of desperation by the prosecution and a lab willing to do anything to appease them.

      Delete
  2. Judge Hellmann's reasoning is indeed a bit more complex than that: he writes in his report that there is not enough consensus in the scientific community with respect to the results obtained by the Italian Scientific Police (LCN DNA tested without double amplification)bto use them as evidence in a criminal trial and that also testing another LCN DNA trace with a new more sensitive kit is a not well enough established technique.
    The Italian Supreme Court may have disagreed with him but it is a sensible position and above all quite different from what Schneps and Colmez assume in their book.

    ReplyDelete
  3. When the knife was resampled, the amount of DNA detected by Conti and Vecchiotti was 20-25% of the concentration of DNA found in the lowest point in their standard curve. Thus, the estimate of the concentration of DNA this sample is an extrapolation, not an interpolation, and therefore the estimate they obtained is less accurate than it would be if it were an interpolation. Maybe the value they estimated, 5 pg/µL, is significantly different from zero, but maybe not.

    ReplyDelete