Voice Stress Analysis and the Investigation of Trayvon Martin's Death

Today’s New York Times has a lengthy article [1] on the flaws in the Sanford, Florida, police investigation of the shooting of black teenager, Trayvon Martin, by the 28-year-old neighborhood watch volunteer, George Zimmerman. When the 16-day investigation did not produce any charges, Florida’s governor responded to a national outcry by appointing an aggressive prosecutor. Now indicted, Zimmerman continues to maintain that he shot Martin in self-defense.

Tucked away at the end of the article is the single sentence: “The police conducted a lie-detection procedure, known as voice stress analysis, on Mr. Zimmerman that [sic] he passed.” A voice-stress test? By “a small city police department that does not even have a homicide unit and typically deals with three or four murder cases a year”? [1]

Yes, law enforcement agencies across the country have invested millions of dollars in voice stress analysis (VSA) software programs—despite a widely known lack of evidence to show that they work. For example, after conducting a field study of jail inmates using urinalysis to check their statements about whether they had used specific drugs, a University of Oklahoma researcher wrote that “two of the most popular VSA programs in use by police departments across the country are no better than flipping a coin when it comes to detecting deception regarding recent drug use [2]." 1/

The response of the National Institute for Truth Verification—the company that bills itself as “the world leader in voice stress analysis”—is instructive. The company’s website insists that

the vast majority of VSA studies funded by pro-polygraph elements of the US Government were significantly flawed. One of the many flaws of these studies . . . was that they lacked real-life consequence and thus lacked jeopardy. . . . [C]onsequence and jeopardy found in “high stakes lies” are required to accurately and consistently detect deception. [¶] . . . VSA research conducted by the University of Florida, and a second study conducted by researchers from the University of Oklahoma, both utilized “low stakes lies” in an attempt to measure the results of various VSA instruments. [3]

Another webpage on validity lists many studies, but the descriptions indicate that they merely demonstrate that VSA can detect stress and anxiety. [4] The question, as with the polygraph, is whether an examiner can ascertain the cause of the stress. Thus, the defense of VSA seems be this: we have no scientifically respectable body of proof showing that VSA is highly sensitive and specific in detecting deception, but, then again, nobody has proven to our satisfaction that it does not work for this purpose.

Despite the inability to validate VSA, some police love it. A detective in the Sex Crimes Section of the Metropolitan Nashville Police Department explained that:

We purchased 10 CVSA's and trained 20 examiners and in my opinion, the instruments and training have been one of the greatest assets our department has ever acquired. Not only has it helped us solve many crimes from major thefts to homicides, but has also helped expose false reports from victims, thus saving our department many man-hours of investigation. [5]

Considering that the fundamental question about VSA is whether it can distinguish between stress caused by intentional deception and stress caused other factors—think about the rape victim who has to describe the events to the police—this kind of screening is a little frightening. It helped the Sanford police and prosecutor, though. Or did it?

Notes

1. The comparison to a coin may be misleading. Consider Table 12 in the Oklahoma study. Kelly R. Damphousseat et al., Assessing the Validity of Voice Stress Analysis Tools in a Jail Setting, Mar. 31, 2007, at 53 (NCJRS doc. no. 219031, available at https://www.ncjrs.gov/pdffiles1/nij/grants/219031.pdf). It indicates that 87 subjects tested positive for cocaine use within the past 72 hours. Of these, 40 deceptively stated that they had not used cocaine in this period. However, VSA programs only correctly indicated deception for eight of the 40. This is a sensitivity of 20%. Given an individual lying about recent cocaine use, the programs had only one chance in five of recognizing the deception. However, Their specificity was much better. Of the 47 respondents who were not deceptive, the programs correctly classified 42 (89%) as truthful. A coin flip, on the other hand, would have a specificity and sensitivity of 50%. See, e.g., David H. Kaye, The Validity of Tests: Caveant Omnes, 27 Jurimetrics J. 349 (1987).

References

1. Serge F. Kovaleski, Trayvon Martin Case Shadowed by Series of Police Missteps, N.Y. Times, May 16, 2012

2. Kelly R. Damphousse, Voice Stress Analysis: Only 15 Percent of Lies About Drug Use Detected in Field Test, NIJ Journal No. 259, Mar. 17, 2008

3. National Institute for Truth Verification, Research Casts Doubt on US Government-Funded VSA Studies

4. National Institute for Truth Verification, Studies Validating Voice Stress Analysis

5. Kent McAllister, quoted in National Institute for Truth Verification, Law Enforcement Reviews

ASCLD Meets Frontline

On April 17, PBS aired a Frontline documentary on “The Real CSI.” Within the week, the president of the American Society of Crime Lab Directors (ASCLD), Jill Spriggs, gave the following assessment to her fellow crime lab directors:

In the last week, published reports dusted off the same old forensic cases from the past and regurgitated negative information from years ago in order to provide doubt in the minds of the public and the court. I don’t know about you but some of my neighbors watched the Frontline documentary and had many questions. “Were latent prints really not a validated science?” “Is latent print analysis not what I thought it is?” “Were bite marks really a product of bad forensic science?” “How could anyone rely on bite marks?” “And, how many employees use certificate mills to obtain a forensic science certification?” My answer—You can be confident about latent print evidence. Latent prints are a validated science. Of all of the millions of fingerprint samples in the databases throughout the world, no two people have ever matched the same fingerprint. Bite marks are not an accredited crime laboratory discipline and “no” we don’t use forensic science certification mills to certify our analysts.

Watching the documentary, I shared her sense of boring familiarity. What’s new here? Same faces, same criticisms. But Ms. Spriggs’ rejoinder likewise is outdated in the logic it uses to shrug off the criticisms. Frontline summarized real problems with the way the American criminal justice system produces and consumes forensic science evidence. Although ASCLD is not responsible for these problems, let’s look more carefully at the three matters that Ms. Spriggs mentions.

1. Credible Credentials

In case after case, witnesses bolster their credentials as forensic experts with credentials from the American College of Forensic Examiners Institute—the organization depicted as a diploma mill on the Frontline show (and elsewhere). Courts often take the bait. For example, in People v. McWhorter, 212 P.3d 692 (Cal. 2009), the California Supreme Court juxtaposed a defense expert’s seemingly strong credentials—he “was certified by an organization known as the American Board of Recorded Evidence; and was a member of the American College of Forensic Examiners” with his puzzling inability to “identify the computer program he used to enhance or ‘electronically emboss’ the image in question [and to] satisfactorily explain the full nature of the process he used to create it.” Id. at 726.

It is appropriate to note that “we [all government crime labs?] don’t use forensic science certification mills to certify our analysts.” But what about the rest of the forensic science community—and the courts? Some prosecutors seem to have no compunction about presenting their witnesses as qualified at least in part because they are “a life fellow for The American College of Forensic Examiners” or some such thing. Chavarria v. State, 307 S.W.3d 386, 387 (Tex. Ct. App. 2009).

Judges and lawyers need to learn which organizations have meaningful standards and which do not. If there were less demand for dubious credentials, expert witnesses—government employees and private consultants alike—would have less incentive to pad their CVs with such credentials.

2. “Bite marks are not an accredited crime laboratory discipline”

Disavowing bite-mark evidence because it comes from outside the public laboratory is not responsive to the question, “Were bite marks really a product of bad forensic science?” Does ASCLD believe that forensic science is limited to the reports of crime laboratories? That forensic odontologists are not accredited? The American Board of Forensic Odontology "was organized in 1976 under the auspices of the National Institute of Justice" and "is accredited by the Forensic Specialties Accreditation Board (FSAB) as a forensic specialty offering board certification to qualified forensic dentists." If the public cannot trust this field's accredited expertise, why should it trust "an accredited crime laboratory discipline"? (There are a number of possible answers, but Ms. Spriggs does not provide them.)

3. “You can be confident about latent print evidence. Latent prints are a validated science.”

Latent prints often contain valuable information for ascertaining the origin of the print. However, the Frontline interviewees pointed to the lack of objective criteria for deciding when prints do or do not come from the same source and the danger that unconscious bias could influence these judgments. These problems notwithstanding, there are scientifically sound studies suggesting that latent fingerprint examiners can get the correct answers most of the time. The Frontline show could be faulted for failing to acknowledge this research while publicizing the NIJ’s refusal to fund a rigorous, comprehensive audit of actual laboratory work. Such a study should be done to complement the recently acquired experimental data, but why would Frontline omit these findings from its televised and online materials?

Yet, Ms. Spriggs does the same thing. Rather than refer to the scientific research that is responsive to the calls for better validation, she repeats the same old story: We can have great confidence in the work of latent print analysts because “[o]f all of the millions of fingerprint samples in the databases throughout the world, no two people have ever matched the same fingerprint.”

What is wrong with this proof of validity? First, whether any individuals have the same fingerprint tells us virtually nothing about the ability of analysts to compare prints accurately. In the Frontline show, Jennifer Mnookin made this point when she explained that the task confronting a latent print analyst is to discern when a partial print comes from a particular person’s finger. That task is distinct from deciding whether pairs of full prints come from different fingers.

Second, just because there are millions of prints on file, it hardly follows that “the millions of fingerprint samples in the databases throughout the world” match. The FBI alone has over 71 million prints in its database. To verify that no pair of these exemplar prints match would require approximately 2.5 x 10¹⁵ (2.5 quadrillion) comparisons of pairs of 10 prints. It would take an examiner, working at the incredible pace of one comparison per digit per second, some 800 million years to complete this task. When the FBI’s contractor, Lockheed Martin, tried to prove the non-existence of matching prints in the database using an automated matcher, it limited itself to a mere 51,000 prints and failed to establish uniqueness.

Ms. Spriggs importuned her fellow crime lab directors to “speak out on these issues. . . . Don’t wait another day! Get started!” Indeed, there are important things to be said.

Earprints Leave Their Mark in Germany

Der Spiegel recently reported that "earprints . . . are as useful to the police as finger prints. A burglar in Germany made the mistake of pressing his ear to front doors to check if anyone was home. The unique prints have allowed the police to pin 96 burglaries on him" [1].

The article makes no mention of the flap over earprints in England [2]. In 1998, a small-time burglar named Mark Dallagher made legal history when he became the first man to be convicted of murder by earprint evidence [3]. A jury found him guilty of murdering 94-year-old Dorothy Woods after Cornelis van der Lugt, a Dutch earprint expert, said he was "absolutely convinced" that Dallagher had made the prints found on the window of Miss Wood's house [3]. Eventually, however, DNA was recovered from the earprints, and it did not match Dallagher's. Following "an appeal, a retrial and a fresh police investigation the Crown said it had 'anxieties' about the case and was offering no evidence against Dallagher'" [3]. Thus, he was acquitted of the murder in 2004 [3].

Despite the bad press the Dallagher case generated, "earprints" are not without value in criminal investigations. Obviously, earmarks vary across individuals. Of course, so do repeated impressions of the same ear, but studies indicate that there is discriminating value in the marks [4-7]. Interestingly, in light of Dallagher, another study suggests that DNA recovered from earmarks can produce false exclusions. When DNA in "60 earprints collected from three healthy adult volunteers under controlled laboratory conditions" was analyzed, "high levels of non-donor alleles [were] observed" [8]. (This is from the abstract of the study. I have not had time to read the body of the paper or check for follow-up work.)

It seems clear, however, that reports like the one in Der Spiegel unconditionally praising the power of "unique prints" cannot be reconciled with the research to date. Fortunately, the German police in the case against the 33-year-old Macedonian linked to some 100 robberies have more to go on than the man's ears. A Hamburg police spokeswoman also referred to fingerprints and DNA [9].

References

1. German Police Identify Burglar by His Earprints: Ninety-Six Break-Ins Solved, Spiegel Online Int'l, April 30, 2012, http://www.spiegel.de/international/zeitgeist/0,1518,830659,00.html
2. David Bamber, Prisoners to Appeal as Unique 'Earprint' Evidence Is Discredited, The Telegraph, Dec 2, 2001
3. Sean O'Neill, Expert Evidence Flaws Clear 'Earprint Killer', Telegraph, Jan. 23, 2004
4. Cornelis Van Der Lugt, Andrew Thean, Lynn Meijerman, & George J. R. Maat, Earprints, in Forensic Human Identification: An Introduction 73-84 (Sue Black & Tim Thompson eds. 2006)
5. Lynn Meijerman, Andrew Thean & George Maat, 1 Forensic Science, Medicine, and Pathology 247-256 (2005)
6. Ivo Alberink & Arnout Ruifrok, Performance of the FearID Earprint Identification System, 166 Forensic Sci. Int'l 145–154 (2007)
7. Ivo Alberink & Arnout Ruifrok, Repeatability and Reproducibility of Earprint Acquisition, 53 J. Forensic Sci. 325-330 (2008)
8. E.A. Graham, V.L. Bowyer, V.J. Martin & G.N. Rutty, Investigation into the Usefulness of DNA Profiling of Earprints, 47 Sci. & Justice 155-159 (2007)
9. Andy Eckardt, Earprints Allow German Cops to Nab Alleged Serial Burglar, World News on MSNBC.com, Apr. 30, 2012

Acknowledgments

Thanks to Ira Ellman and ASU student Seth Reeker for the link to the Der Spiegel article. Cross-posted to the Double Helix Law blog.