Saturday, May 28, 2016

SWGFAST on "a decision to reach a conclusion" of individualization for latent fingerprints

Before SWGFAST (the Scientific Working Group on Friction Ridge Analysis, Study, and Technology) dissolved to make way for OSAC (the Organization of Scientific Area Committees in Forensic Science), it produced a working draft of a Guideline for the Articulation of the Decision-Making Process for the Individualization in Friction Ridge Examination (Latent/Tenprint) (dated 3/13/13). The Guideline tells examiners how to explain the thinking that leads to what SWGFAST called “a decision to reach a conclusion of individualization” (§ 10.1) and how to report this decision (§ 11). Some interesting parts follow:
11.2.2. The SWGFAST Individualization/Identification Position Statement (Document #103) states that: “The ability of a latent print examiner to individualize a single latent impression, with the implication that they have definitely excluded all other humans in the world, is not supported by research and was removed from SWGFAST’s definition of individualization.”
This is a major and salutary departure from historical practice, but what remains of the definition? SWGFAST’s last definition of “individualization” was “[t]he determination by an examiner that there is sufficient quality and quantity of detail in agreement to conclude that two friction ridge impressions originated from the same source.” SWGFAST, Standard Terminology of Friction Ridge Examination (Latent/Tenprint), Mar. 13, 2013, at 5. “If two friction ridge impressions originated from the same source,” however, they did not originate from any other source, that is, from any “other human[] in the world.”

The only interpretation of the Guideline that seems to make sense is that an “individualization” is a tentative rather than a “definitive” conclusion that all other humans in the world have been excluded. Indeed, the Guideline continues:
11.2.3. Specific words and phrases conveying absolute certainty are inappropriate or misleading and should not be used to express conclusions. Specific problematic phrases include: [e]xclusion of all others, 100% certainty (as an absolute fact), [and z]ero error rate / infallible method.
But it is easier to say what should not be said than to prescribe what should be stated. The Guideline offers the following positive-plus-negative advice:
11.2.4. These concepts should rather be expressed as the conclusion of the examiner, based upon data observed and interpreted through the examiner’s training and experience. The examiner has a level of personal confidence associated with the accuracy and reliability of this conclusion; however, the accuracy and precision of this personal level of confidence cannot currently be measured and reported. For this reason, certainty should not be reported numerically or in absolute terms.
Evidently, the examiner should say something like "My decision is that X is the source of the latent print. I base this decision on my training and experience. I have [enormous? high? medium? low?] confidence in my personal decision." Of course, if a categorical conclusion is given, what the judge or jury would like to hear is the probability that this conclusion is correct, not that an examiner has some personal belief. But how a latent print examiner could provide such a posterior probability without considering a case-specific prior probability is an unsolved problem. A more measurable quantity is a conditional error probability—here, the probability that an examiner would identify X when X is not the source. Although some experiments have examined the rate of false positive (and negative) errors by latent print examiners, the Guideline does not propose using these figures.

Instead, it implies in the future it will be possible "to measure a personal level of confidence." I suppose an examiner could be forced to pick the probability at which he or she is indifferent to a gamble in a game of chance that has a well-defined probability for winning and losing. Would that make the personal confidence measurable? If so, why can't it be done currently?

In any event, the lack of a workable mechanism to measure personal confidence is not why claims of absolute certainty should be avoided. The reason is that nothing empirical is absolutely certain. Tomorrow, the laws of physics could change and heavy objects could float upward.

Neither does the difficulty of devising a suitable psychometric instrument for personal certainty explain why an examiner cannot report any numbers. People quantify personal judgments that cannot be measured objectively when their doctors ask them to report the severity of pain on a scale of 1 to ten. Why cannot an examiner report a defensible level of confidence numerically if the subjective precision is provided? E.g., “I think X is the source, and the strength of my belief is 9 on a scale of 1-10, but obviously this is not an exact number. I could have said 8, or even 7 as well, but I would not want to go below that.”

I am not arguing that examiners should grade their personal belief in a source attribution on a scale of 1-10 or express it as a numerical probability. My point merely is that SWGFAST's argument about reporting a number for relative certainty is superficial and underdeveloped.

Sunday, May 1, 2016

BIT Global Group's Next Flaky Forensics Conference

BIT Global Group is notable for the scope and intensity of its spam promoting what appear to be parodies of academic or professional conferences. The latest email "on behalf of the organizing committee" cordially invited me "to attend BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016, which will be held during October 27-29, 2016 at International Conference Center, Dalian, China."

The organizing committee (or, as the conference's website designates it, the program committee) consists of one individual, Dr. Xiaodan Mei, President of BIT Group Global Ltd., China. But the "Advisory Board Members" are "coming soon," and already, "the Plenary Session will invite a distinguished international panel including Dr. Henry Lee and many others to talk about the impact of recent forensic scientific and technological breakthroughs around the world."

Indeed, if the website is accurate, the "Renowned Speakers of WCF-2016" who have signed on are
  • "Dr. Henry C. Lee, Chief Emeritus for Scientific Services, University of New Haven, USA" and
  • "Dr. John Zheng Wang, Professor and Director of Certificate Programs, California State University [Long Beach], USA,"
Postscript

On May 16, 24, and 29, 2016, Ms. Cherry Dong, Organizing Committee of the Forensics & Police Tech Expo 2016, advised me that
I’m writing to follow up my last invitation letter several weeks ago, it is regretful we couldn’t receive your reply. Now we would like to extend our cordially invitation to you again to join the BIT’s 3rd International Congress of Forensics & Police Tech Expo 2016 ... . Forensic science has attracted enormous attentions ... . Therefore, we establish this focused event ... So please kindly inform us if you are available to give us a presentation in the conference, it will be highly appreciated you can give us a prompt reply with a title and brief (3-5 sentences) summary on your recent work in the area.
As of June 13, 22, and July 3, 2016, the wording changed to
How are you doing? I am Cherry, the Program Coordinator ... . I am writing the letter to make sure if you have received my letter previously, it is about the conference: WCF-2016 which will be held during October 27-29, 2016 ... . Maybe there some problems with my mailbox ... . Now we have invited some distinguished experts including Dr. Henry Lee and Dr. John Zheng Wang and many others ... . In the past two congresses, we have invited more than 300 worldwide leading experts, famous police officers, and law practitioners ...
In June, the renowned speakers list had grown with the addition of
  • Mathias Gaertner, Publicly Accredited and Sworn in Expert Witness for Information Technology, Germany
  • Eric Kreuter, a Partner in the Financial Advisory Services Group at Marks Paneth LLP
  • Erik Laykin, Managing Director, Duff & Phelps LLC
  • Frank Prieels, Professor, University of Duesseldorf, Belgium
  • Andre Stuart, CEO, 21st Century Forensic Animation
  • Linda Xiao, Professional Officer (Technical), University of Technology, Australia.
and by early July, another keynote speech, from Raymond Hsieh, Professor and Director of Justice Studies, Department of Criminal Justice, California University of Pennsylvania, was added to the schedule.

For more information on BIT's "congresses" and practices, see Wikipedia and Flaky Academic Conferences.