Exposing Source Code in Law and Science

With the increasing use of "probabilistic genotyping software" has come a push for the making the software public. 1/ Indeed, one federal judge in the Southern District of New York went so far as to compel New York City's Office of the Chief Medical Examiner to make public the source code of the program it created to interpret the pattern of peaks and valleys in the graphs used to determine the nature of the DNA giving rise to them. 2/

Whether this will produce a more thorough analysis of how well the software performs (compared to other ways of testing the program, including releasing the code for inspection by defense experts subject to protective orders) remains to be seen. I have heard more than one expert involved in developing such software observe that releasing the source code is not going to do much to help anyone understand how well the program works. Very lengthy and complex programs may not consist entirely of modules that can be tested separately, and it can be difficult for testers who are not developing the software to develop the insight to conduct effective white-box testing. As far as I know, the FAA and the FDA do not insist on disclosure of all source code to approve the use of avionics and medical-device software, respectively.

Thus, a new editorial policy announced by a preeminent scientific journal, Nature, is noteworthy. 3/ It expresses a strong preference — but not an absolute requirement — for making the source code available to peer reviewers and then to the scientific community. Excerpts follow:

... From this week, Nature journal editors handling papers in which code is central to the main claims or is the main novelty of the work will, on a case-by-case basis, ask reviewers to check how well the code works .... Computational science — like other disciplines — is grappling with reproducibility problems, partly because researchers find it difficult to reproduce results based on custom-built algorithms or software. ...

Some journals have for years ... ensured that the new code or software is checked by peer reviewers and published along with the paper. When relevant, Nature Methods, Nature Biotechnology and, most recently, journals including Nature and Nature Neuroscience encourage authors to provide the source code, installation guide and a sample data set, and to make this code available to reviewers for checking. ...

According to the guidelines, authors must disclose any restrictions on a program’s accessibility when they submit a paper. [I]n some cases — such as commercial applications — authors may not be able to make all details fully available. Together, editors and reviewers will decide how the code or mathematical algorithm must be presented and released to allow the paper to be published.

... We also recognize that preparing code in a form that is useful to others, or sharing it, is still not common in some areas of science.

Nevertheless, we expect that most authors and reviewers will see value in the practice. Last year, Nature Methods and Nature Biotechnology between them published 47 articles that hinged on new code or software. Of these, approximately 85% included the source code for review.

[A]lthough many researchers already embrace the idea of releasing their code on publication, we hope this initiative will encourage more to do so.

Notes

1. See Jason Tashea, Code of Silence: Defense Lawyers Want to Peek Behind the Curtain of Probabilistic Genotyping, ABA J., Dec, 2017, at 18. The article asserts that "companies developing these tools 'black-box.' This means there is limited or no capacity to review the math;therefore, it cannot be independently challenged." Id.at 19. In computer scientists and engineering, however, "black box" is not a pejorative term, and it certainly does not mean that no testing is possible. Quite the contrary, it refers to a type of testing (also known as "behavioral testing") that "independently challenges" the ;program. It does so by checking whether the program's output is correct for different inputs. This type of testing has advantages and disadvantages compared to "white box" testing. Obviously, a combination of both types of testing is more complete than either one by itself.
2. E.g., Jason Tashea, Federal Judge Releases DNA Software Source Code That Was Used by New York City's Crime Lab, ABA J., Oct. 20, 2017, http://www.abajournal.com/news/article/federal_judge_releases_dna_software_source_code. The article asserts that "[p]robabilistic genotyping does not define a DNA sample itself; rather, it is an interpretive software that runs multiple scenarios, like risk analysis tools used in finance, to analyze the sample." The explanation could be clearer. The New York City OCME's program that was the subject of the judge's order models the stochastic process that produces the "peaks" that indicate the presence of certain features of the DNA in the sample. See David H. Kaye, SWGDAM Guidelines on "Probabilistic Genotyping Systems" (Part 2), Forensic Sci., Stat. & L., Oct. 25, 2015, http://for-sci-law.blogspot.com/2015/10/guidelines-on-probabilistic-genotyping.html. 
3. Editorial, Does Your Code Stand Up to Scrutiny?, 555 Nature 142 (2018), https://www.nature.com/articles/d41586-018-02741-4?WT.ec_id=NATURE-20180309.