We see the paper on a topic presented at the North East Division, IAI in Vermont last week;
_________________________________________
Contextual Information Renders Experts Vulnerable to Making Erroneous Identifications

In Press, Forensic Science International

Itiel E. Dror, David Charlton, and Ailsa E. Péron

School of Psychology; Faculty of Medicine, Health and Life Sciences,

University of Southampton, Southampton, United Kingdom

E-mail: id@ecs.soton.ac.uk

URL: www.ecs.soton.ac.uk/~id

From: http://www.ecs.soton.ac.uk/~id/FSI%20contextual%20influences.doc

Abstract:

We investigated whether experts can objectively focus on feature information in fingerprints without being misled by extraneous information, such as context. We took fingerprints that have previously been used to positively identify suspects. Then we presented these same fingerprints again, to the same experts, but gave a context that suggested that they were a no-match, and hence the suspects could not be identified. Within this new context, most of the fingerprint experts made different judgements, thus contradicting their own previous identification decisions. Cognitive aspects involved in biometric identification can explain why fingerprint experts are vulnerable to make erroneous matches.

Introduction

Being a scientist or forensic expert is rooted in the ability to examine evidence reliably and objectively. To do this, these professionals must be able to dissociate themselves from extraneous contextual and other influences that may interfere with their ability to examine, evaluate, and judge the relevant information. Their decisions should be based on the information relevant to the task at hand and its unbiased interpretation. This involves independent thought that ignores to a large extent extraneous pressures and influences.

External pressures and influences are many and varied.  History of science is full of examples of extraneous influences, and today too, scientists work within, and are influenced by, political, economical and other agendas (e.g., global warming, Genetically Modified crops, and Measles Mumps Rubella vaccine).

Terrorism has brought about a wave of contextual influences. These include, among others, heightened suspicion of Muslims, fear, anger, helplessness, as well as pressure on governments to control (or at least appear to control) such threats. Such contextual influences provide strong and ample opportunities to contaminate objectivity, leading to distortions and errors of judgement beyond the unavoidable. Indeed, within this context we have witnessed major misevaluations and misjudgements by intelligence experts.

Within a similar extraneous context the United States Federal Bureau of Investigation (FBI) positively, but erroneously, identified a Muslim as the Madrid bomber (see Figure 1). This incorrect identification was further verified by a number of FBI and other fingerprint experts and led to the arrest of an innocent person. It was only due to rare and exceptional circumstances that this error was ever revealed and eventually acknowledged by the FBI [1]. Errors can occur across forensic science evidence, including DNA [2].

Empirical cognitive research in these areas has been largely neglected (if not basically ignored), partially because professional expert assessment of evidence (as in the criminal justice system) is believed to be relatively objective. With the growing number of anecdotal cases that question this belief and suggest that forensic assessment is far from being as objective as it can and should be, it is important to conduct cognitive scientific research in this area. Laboratory experiments performed by our group have already suggested that emotional context may bias fingerprint identification. These studies found that university students were more likely to judge that there was a positive match between pairs of fingerprints that were presented within an emotional context than those presented within an emotionally neutral control context [e.g., 3]. However, this vulnerability was apparent only when the prints were ambiguous and lacked clarity. The emotional context had minimal effect when there was a clearly matching pair (or a clearly non-matching pair). These studies, however, were based on non-experts and conducted in a laboratory setting.

The study that we report here presents empirical data on whether actual fingerprint experts in their normal everyday working routines and environment are susceptible to extraneous contextual influences. We employed a within-subject design in which the same experts made judgements on identical pairs of fingerprints, but in different contexts. Our aim was to focus on and to examine the contextual influences themselves rather than reveal possible individual differences between experts. Accordingly, we collected and used pairs of fingerprints from archives that the same experts had examined and judged approximately five years earlier as a clear and definite match. These previous identification matches were taken from real criminal investigations and have been used in court.

In this study we re-presented these very same pairs of fingerprints to the same experts who had originally evaluated them as a match, but we now provided them within an extraneous context that might bias them to evaluate the prints as a non-match. We wanted to test whether their decisions were independent and relatively objective, and thus consistent regardless of extraneous influences. Alternatively, if they contradicted their previous decisions, this would demonstrate vulnerability to bias.



Figure 1: The FBI’s erroneous identification of the Madrid bomber. The latent print from the crime scene (left panel) and the fingerprint of the innocent suspect who was positively identified by a number of fingerprint experts (right panel).

Method: 

Participants: Participants were five certified fingerprint experts. Together they represent over 85 years of experience in examining fingerprints (mean of 17 years). The participants were taken from our international fingerprint expert pool of volunteers. This pool of participants includes fingerprint experts from a variety of Fingerprint Bureaus, Agencies, and Laboratories from across the world (including the USA, UK, Israel, the Netherlands and Australia). We only used experts that were not familiar with Mayfield’s fingerprint and that we could covertly access past archival identification matches that they made in the past (see Procedure, below)

Materials: A different pair of fingerprints was prepared for each of the expert participants. Each pair of prints had been previously identified as a match by that same expert in the year 2000, within the normal course of their work. The latent fingerprints had been obtained from the crime scenes and were all presented again to the experts in their original format.

We further established that all of the pairs of fingerprints were indeed a match by submitting them for verification, ‘context free’ to two experienced fingerprint experts who were not involved in or aware of our study (each had over 20 years of experience). Both experts independently verified that all five pairs of fingerprints were indeed matches.

Procedure: Participants signed a consent form a few months prior to the experiment. In this form they consented to being tested sometime within the next 12 months without their knowledge. Thus we were able to obtain consent but yet test the experts within their normal working environment without them knowing that they were in an experimental situation. We pre-screened our participants and used only participants that were not familiar with the fingerprint of Mayfield.

Participants were asked by one of their colleagues to examine a set of fingerprints, composed of a latent print (from the crime scene) and a print exemplars (a print obtained from a suspect). They were told that the pair of prints was the one that was erroneously matched by the FBI as the Madrid bomber, thus creating an extraneous context that the prints were a non-match.

The fingerprint experts were asked to decide whether there was sufficient information available in the pair of prints to make a definite and sound decision, and if so, what that judgement was (a match or non-match). They were allowed to evaluate the prints as they would do routinely –handling of the prints, magnifying, lighting equipment, and so forth. The experts were allowed an unlimited amount of time to make their evaluation. The fingerprint experts were further instructed to ignore the context and background information, and to just focus solely on the actual print in their evaluation and decision making.

Results:

Only one participant (20%) judged the prints to be a match, thus making a consistent identification regardless of the extraneous context. The other four participants (80%) changed their identification decision from the original decision they themselves had made five years earlier. Three of these four participants directly contradicted their previous decision and now judged the fingerprints as definite non-matches, whereas the fourth participant now judged that there was insufficient information to make a definite decision (either a match or a non-match).


Discussion: 

This study shows that fingerprint identification decisions of experts are vulnerable to irrelevant and misleading contextual influences. Our study specifically demonstrates that the extraneous context in which fingerprint examinations occur can determine the identification decision. When presented within a different context four out of five experts made different identification decisions. One of the four decided that there was insufficient information available in the latent print to make either a ‘match’ or ‘non-match’ decision, whereas the other three fingerprint experts decided that the fingerprints were a definite ‘non-match’. This is striking given that all five experts had seen the identical fingerprints previously and all had decided that the prints were a sound and definite match.

This is the first research study to experimentally examine the possible impacts of extraneous context in the real world of biometric and forensic science. One reason for the lack of research in this area is the difficulty in conducting proper scientific research with experts without their knowledge and in their real working environment, while obtaining their consent. We could only use experts for whom we could covertly access and obtain archival files of their own past judgements and who were not familiar with the Mayfield fingerprint. This stipulation further decreased the availability of suitable participants, but had the added advantage of providing a unique opportunity to conduct a within-subject study. The magnitude of the contextual effect and the fact that the experts had judged the same fingerprints in the past enabled the sample to provide clear findings with a high level of confidence. Furthermore, given that we conducted our experiment within the real world conditions of the criminal justice system even if only one expert out of five was susceptible to such effects that in itself would have serious implications.

Even if we were able increase our sample of expert participants ten fold (which is unrealistic, given all the constraints detailed above) and assuming that none of the additional participants would have been vulnerable to our manipulation and changed their judgements (which is statistically highly unlikely), our data would still demonstrate that approximately ten percent of the experts were susceptible to misleading extraneous contextual information. Thus, our results are striking even though we used five expert participants and a strong extraneous context.

The critical question is what do these results reflect and what do they imply. Are the inconsistent fingerprint identification decisions a reflection of practitioners’ errors? Do they reveal deeper methodological and procedural problems in the way that fingerprint experts are trained and identifications are conducted? Or do the results point out basic flaws in the scientific basis and assumptions underlying fingerprint identification altogether?

The data presented in this study, along with some of the rare examples where erroneous identifications are publicly revealed and acknowledged, do not necessarily indicate basic flaws in the scientific underpinning of fingerprint identification. The fundamental question as to whether fingerprint identification is a science is not addressed in this study, since that raises a different set of issues that pertain to a variety of “sciences” [4]. Our results also do not reflect or reveal practitioners’ errors whereby experts’ negligence, carelessness, and personal fault (intentional or not) produce erroneous identifications. Such causes are often used to deflect deeper scrutiny and discussion.

Rather, it seems that our findings of erroneous and inconsistent identification decisions may reflect cognitive flaws and limitations in conducting objective and independent processing and evaluation of the information. It is important to note that such problems arise mainly in the more difficult and challenging cases, such as with latent fingerprints collected at crime scenes that are distorted, partially missing, and contaminated. In such cases subjectivity is more pronounced [3, 5].

As extraneous contextual effects are more pronounced, greater distortions can arise. The sources of such distortions are many and varied, including emotional context, pressure, contextual information, group think, biases, hopes and expectations, self fulfilling prophecies, and peer pressure. In this study we used a strong misleading extraneous contextual influence, but such influences do occur.

It is important first to establish empirically that experts can be strongly influenced by extraneous contexts. Now that we have demonstrated such an effect, further research can and should use different and more subtle manipulations to examine in greater depth when such factors affect performance and render the experts vulnerable to misjudgements, and when such factors are unlikely to affect performance. When vulnerable, these effects can cause a variety of distortions that arise from ignoring parts of the evident information, over-emphasising and over-evaluating other parts of the information, and changing decision criteria, to name but a few.

Vulnerabilities in fingerprint identification can be minimized by better initial selection and screening of fingerprint experts; appropriate training and professional development, and the adoption of methodological procedures that adequately address potential pitfalls. Our results show that even in the face of strong extraneous contextual information one expert nevertheless did maintain their original judgement. That expert was indeed able to focus objectively and consistently on the data, ignoring the extraneous misleading contextual information. This clearly demonstrates that it is possible to be much more objective, and that some experts may not be optimizing objectivity in their professional domain.

The reliability and validity of a scientific method such as fingerprint identification is maintained only when analysis is relatively objective, and hence consistent, across individuals, times, and extraneous contexts. For fingerprint examination to remain a credible forensic science, it must achieve this level of objectivity of analysis. Our study shows that it is possible to alter identification decisions on the same fingerprint, solely by presenting it in a different context. This does not imply that fingerprint and other forensic identifications are not a science, but it does highlight problems of subjectivity, interpretation, and other psychological and cognitive elements that interact and may distort any scientific inquiries.

One of the main sources of weaknesses in biometric and other forensic sciences is the lack of research, attention, and application of psychological elements that play a key role in the identification processes. These range from the ways in which perceptual factors (such as similarity and orientation) affect the process of pattern recognition [6] to how we consider decision alternatives and shift response criteria [7]. With new and future statistical tools and technologies the face of fingerprint and biometric identification is changing; however, psychology and cognitive elements continue to play a critical role in their implementation and success [8]. To highlight and address such potential pitfalls, cognitive research needs to be applied systematically to the world of biometrics and forensics. This is all the more necessary in view of our findings that extraneous contextual information is able to determine experts’ evaluation of fingerprints, a well-established and relatively objective forensic discipline, then distorting effects are undoubtedly as prevalent, if not more so, in other biometrics and forensic disciplines [9].

Acknowledgements:

We want to thank all the fingerprint experts who are working together with us and support our efforts to try and understand the cognitive elements involved in fingerprint identification, and to thank Arie Zeelenberg, Robert Rosenthal, and Nick Donnelly for helpful comments on an earlier version of the manuscript.

References:

[1] Stacey R. B. (2004). Report on the erroneous fingerprint individualization in the Madrid train bombing case. Journal of Forensic Identification, 54 (6), 706-718.

[2] Thompson, W.C. (1995). Subjective interpretation, laboratory error and the value of DNA evidence: Three case studies, Genetica , 96, 153-168.

[3] Dror, I. E., Peron, A., Hind, S., & Charlton, D. (2005). When emotions get the better of us: The effect of contextual top-down processing on matching fingerprints. Applied Cognitive Psychology, 19(6), 799-809.

[4] Dror, I. E. & Thomas, R. D. (2005). The cognitive neuroscience laboratory: A framework for the science of mind. In C. Erneling & D. Johnson (Eds.), The Mind as a Scientific Object: Between Brain and Culture (pp. 283-292). Oxford University Press.

[5] Dror, I. E. (2005). Perception is far from perfection: The role of the brain and mind in constructing realities. Brain and Behavioural Sciences 28 (6).

[6] Ashworth, A.R.S. & Dror, I. E. (2000). Object Identification as a Function of Discriminability and Learning Presentations: The Effect of Stimulus Similarity and Canonical Frame Alignment on Aircraft Identification. Journal of Experimental Psychology: Applied, 6 (2), 148-157.

[7] Dror, I. E., Busemeyer, J. R., & Basola, B. (1999). Decision making under time pressure: An independent test of sequential sampling models.  Memory and Cognition, 27 (4), 713-725.

[8]  Dror, I.E. (2005). Technology and human expertise: Some do’s and don’ts. Biometric Technology Today, 13 (9), 7-9.

[9] Risinger, D.M., Saks, M.J., Thompson, W.C. & Rosenthal, R. (2002). The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion. California Law Review , 90(1), 1-56.

______________________________________________________________________
Feel free to pass The Detail along to other examiners.  This is a free newsletter FOR latent print examiners, BY latent print examiners. There are no copyrights on The Detail (except in unique cases such as this week's article), and the website is open for all to visit.  FAIR USE NOTICE: This newsletter contains copyrighted material from the public domain, the use of which has not been specifically authorized by the copyright owner.  www.clpex.com makes such material available in an effort to advance scientific understanding in the field of latent prints, thus constituting a 'fair use' of copyrighted material as provided for in section 107 of the US Copyright Law.  In accordance with Title 17 U.S.C. Section 107, the material on this site is displayed without profit to those who wish to view this information for research and/or educational purposes.  If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.

If you have not yet signed up to receive the Weekly Detail in YOUR e-mail inbox, go ahead and join the list now so you don't miss out!  (To join this free e-mail newsletter, log on to "www.CLPEX.com" and click on the "Subscribe" link on the left side of the home page. After you enter your e-mail address, you will be sent a Confirmation e-mail... just click on the link in that e-mail, and you are signed up!)  If you have problems receiving the Detail from a work e-mail address, there have been past issues with department e-mail filters considering the Detail as potential unsolicited e-mail.  Try subscribing from a home e-mail address or contact your IT department to allow e-mails from Topica.  Members may unsubscribe at any time.  If you have difficulties with the sign-up process or have been inadvertently removed from the list, e-mail me personally at kaseywertheim@aol.com and I will try to work things out.

Until next Monday morning, don't work too hard or too little.

Have a GREAT week!