As latent print examiners, we are aware of the critical role that forensic science plays in the criminal justice system. Our expertise in the science of fingerprints is often called upon to provide valuable evidence in criminal investigations and trials. However, recent years have seen increasing scrutiny of the validity of various forensic feature-comparison methods, including fingerprint analysis. In this blog post, we will explore a recent paper by Scurich, Faigman, and Albright (2023) that proposes scientific guidelines for evaluating the validity of these methods and discuss the implications for our field.
The paper, titled "Scientific guidelines for evaluating the validity of forensic feature-comparison methods," was published in the Proceedings of the National Academy of Sciences (PNAS). The authors argue that courts should employ ordinary standards of applied science when considering questions of fact in a legal context, particularly those related to measurement, association, and causality. They propose four guidelines for establishing the validity of forensic comparison methods: plausibility, soundness of research design and methods (construct and external validity), intersubjective testability (replication and reproducibility), and the availability of a valid methodology to reason from group data to statements about individual cases.
One of the key issues highlighted in the paper is the need for forensic feature-comparison methods to be grounded in basic science. The authors argue that many forensic science disciplines, including fingerprint analysis, have few roots in basic science and lack sound theories to justify their predicted actions or empirical tests to prove their effectiveness. This lack of scientific foundation has led to increasing challenges to the admissibility of forensic evidence in court, particularly in the wake of the Daubert decision, which requires judges to examine the empirical foundation for expert testimony.
The paper's discussion of plausibility is particularly relevant for latent print examiners. While the idea that fingerprints can be used to identify individuals is intuitively plausible, the authors argue that the theory and methods used to make these identifications must also be scientifically plausible. They cite the example of the Association of Firearm and Tool Mark Examiners (AFTE) theory, which assumes that examiners can compare the marks on a piece of evidence to "libraries" of marks produced by the same and different tools in their minds. The authors argue that this theory is implausible given what we know about human memory and analytical capabilities.
The paper also emphasizes the importance of sound research design and methods, including construct validity (the extent to which a test measures what it is supposed to measure) and external validity (the extent to which results can be generalized to a population). The authors discuss several studies on firearms and toolmarks analysis, highlighting the limitations of set-to-set designs and the failure to account for inconclusive results. These issues are also relevant for latent print examiners, as we must ensure that our research designs accurately measure the performance of our methods and that our results can be generalized to real-world casework.
Intersubjective testability, which encompasses replication and reproducibility, is another critical guideline proposed in the paper. The authors argue that the validity of a forensic feature-comparison method must be tested by multiple researchers using a variety of testing paradigms to overcome subjective errors and biases. They note that much of the research in the firearms and toolmarks field has been conducted by members of the same professional organization (AFTE) and published in its own trade journal, raising concerns about the lack of independent testing. As latent print examiners, we must also strive for greater intersubjective testability in our research, seeking out collaborations with researchers from other disciplines and subjecting our methods to rigorous, independent testing.
Finally, the paper addresses the challenge of reasoning from group data to statements about individual cases, a process known as "G2i." The authors argue that forensic examiners often make this leap without recognizing the limitations of their data, such as the lack of information about base rates in the population. They suggest that examiners should be limited to making general, group-level statements rather than individualized conclusions, and that future research should focus on developing methods for estimating the frequency of different features in a population.
In conclusion, Scurich, Faigman, and Albright's paper provides a framework for evaluating the validity of forensic feature-comparison methods, including fingerprint analysis. As latent print examiners, we must engage with these guidelines and work to ensure that our methods are scientifically plausible, rigorously tested, and appropriately applied to individual cases. By embracing a more scientific approach to our work, we can strengthen the credibility of our discipline and provide more reliable evidence in the pursuit of justice. This will require a commitment to ongoing research, collaboration with experts from other fields, and a willingness to question and refine our methods considering new evidence. While the challenges are significant, the potential benefits – for our field, for the criminal justice system, and for society as a whole – are well worth the effort.
FAQ's About this Paper
What are the primary concerns of forensic feature-comparison methods in a legal context?
The primary concerns involve questions of fact related to measurement, association, and causality, and how courts should employ ordinary standards of applied science to address these concerns.
Why do most forensic science disciplines lack roots in basic science?
Most forensic science disciplines lack roots in basic science because their inventions do not have sound theories to justify their predicted actions or empirical tests to prove their effectiveness.
What inspired the creation of the four guidelines for establishing the validity of forensic comparison methods?
The creation of the four guidelines was inspired by the "Bradford Hill Guidelines," which are the dominant framework for causal inference in epidemiology.
What is the significance of empirical validation in forensic science?
Empirical validation is crucial to determine that an instrument or method achieves the intended effect, ensuring the reliability and validity of forensic comparison methods.
How do forensic claims of individualization pose a problem?
Forensic claims of individualization, such as linking a latent fingerprint to a specific person, are inherently problematic because applied science is probabilistic and such claims lack robust empirical support.
What shifted the practice of admitting forensic comparison evidence in courts?
The practice shifted with the U.S. Supreme Court's decision in Daubert v. Merrell Dow Pharmaceuticals, Inc., which required judges to examine the empirical foundation for proffered expert opinion testimony.
What are the four guidelines proposed for evaluating forensic feature-comparison methods?
The guidelines include plausibility, soundness of research design and methods, intersubjective testability (replication and reproducibility), and the availability of a valid methodology to reason from group data to individual cases.
Why is plausibility a fundamental starting point for hypotheses in science?
Plausibility is essential because there ordinarily exists a theory or potential mechanism to explain how an effect comes about, which is crucial for considering the hypothesis that one factor is a cause of another.
What challenges do forensic pattern comparison disciplines face in testing and measuring error?
These disciplines face unique challenges regarding testing, measuring error, devising standards, providing peer review, and reaching consensus among knowledgeable individuals in the field.
How does the concept of intersubjective testability apply to forensic science?
Intersubjective testability, identified as a cornerstone of science by Sir Karl Popper, requires that conclusions from any study can be mitigated by demonstrating the same results under different conditions and by other investigators, ensuring consistency and reliability in forensic science methodologies.