We look at a recent episode of "Talk of the Nation", airing Friday on National Public Radio.  You can listen to the entire archived presentation by going to the following link:
(http://www.npr.org/templates/story/story.php?storyId=4798054)
_________________________________________
Talk of the Nation: Faulty Forensics
guests Jonathan Koehler and Barry Fisher
Friday, August 12, 2005

(The below is paraphrased in concept from dialogue, and does not represent exact transcription)

It’s an exciting world in the TV world of forensics, but how close to the Hollywood sets and actors match forensic science in the real world?  And how well do real forensic techniques meet the validation techniques required in scientific research?  According to some critics, it’s not well enough.  According to the journal “Science”,  forensic scientists may not be getting enough training, and faulty forensic science is a leading cause of faulty forensic evidence.   There isn’t enough research to show that even fingerprinting is a reliable enough way to link someone to a crime.  This hour we are going to take a look at the evidence for and against some of the forensic techniques we take for granted; do these methods hold up under scientific scrutiny, and what can be done to improve the quality of science coming out of our crime labs.

Guests: Dr. Jonathan Koehler, University of Texas, Austin
                Barry Fisher, Director of the LA Sheriff’s Department Crime Laboratory

On the graph on the first page of your article, you look at wrongful convictions using data provided by the Innocence project shows 86 examples where people were exonerated, and in 60% faulty forensics was involved.  Were they deliberate?

Koehler: Most often where there were mistakes, they were committed by well-meaning people.  It’s hard to know whether training or procedures were involved, but better protocols and more testing is necessary to know how good these examiners really are. 

Fisher: Forensic science is growing, but in many areas there are lack of funds necessary to conduct the type of research Dr. Koehler is talking about.  Hopefully the senators and Congress we have been working with recognize that there are a lot of needs and will move those efforts forward.  There has been a large number of people hired in the field, and we need to conduct “capacity building” where they are replacing a large number of very experienced retirees.  We need better ways to train people to do quality laboratory work.  Local laboratories don’t necessarily have the funds available just because of 911.  We need research to say that facial or hand geometry, for example, are valid.  There is a wide range of research that needs to be done, but it comes down to a matter of money.  Typically, Congress needs to define areas and funding and the issue becomes how we go about educating people in administration and Congress on the importance of what we do and what role Forensics plays in criminal justice.

Koehler: All new technologies are exciting and should be pursued with vigor, and I agree there should be funding available for research and training.  Further, courts should decide whether these techniques are good enough to bring in to court.  Daubert requires that all techniques used in court should be founded in the scientific method.  For example, research on new facial recognition technology should be published and peer reviewed before admitted into court.  What we need is validation of the technique and validation of the application of that technique to the case at hand.  We also need to know what is the error rate of that examiner.  If I can’t have the individual analysts error rate, I would like to see the error rate of the laboratory or the industry-wide error rate.  How often when they say “match” is it in fact not a match.  We really don’t have a very good idea for fingerprinting, because until recently the forensic sciences have resisted such a test.  There are some proficiency test results in fingerprinting, and the news is good or bad, depending on how you look at it.  The error rate seems to be on the order of a couple of percent.  Since 1995, about 4-5% of fingerprint analysts make at least one error.  Most people do a pretty good job, but somewhere between 4 and 6% make a mistake.  One one hand, 95% is pretty good.  On the other hand, most of us wouldn’t fly on an airplane that had a 95% chance of landing safely.   I would want to identify who the people are and what the circumstances are that lead to the errors.  Sometimes a really nice clear print might be a little easier to match than a partial print, for example.

There have been some proficiency tests in hair and bite marks that have been fairly disastrous.  Fingerprinting does pretty good compared to other forensic sciences, but for example voice print identification error rates are about 63%, handwriting error rates at 40%.  A lot of time this information on error rate doesn’t make it to the jury.  It’s not perfect, and it’s based on proficiency testing, but it doesn’t mean we shouldn’t pay attention to these error rates.  I think it’s signaling that there is a big problem in traditional forensic sciences.

Traditionally, defense counsels were reluctant to challenge fingerprinting because it was so well established in the courts that a challenge wouldn’t be taken seriously.  But recently, we have seen attorney’s and even judges challenge fingerprinting without any motion from the defense attorney.  For the most part, these challenges are failing, and fingerprint evidence is still sneaking in even with the Daubert criteria intact.  Judges tend to review the available evidence, find it’s lacking, but find some way around and let it into court, like referring to 100 years in court or its ability to convict the guilty.

Just because fingerprints or others are unique, what is at issue is our ability in a specific case to arrive at such a conclusion: that this is a print of this particular individual.

Fisher: There are a couple of things in play… the Daubert decision decides whether the evidence can even get to the jury.  Once there, other information can diminish the weight or value of the evidence.  Let’s assume that people who work in crime labs make errors, and one of the things we have to grapple with how probative the evidence is.  One of the conundrums we face is that these procedures that don’t follow traditional science were developed to solve problems that police officers face.  There is no doctoral degree in handwriting or fingerprints; these procedures developed over a number of years.  Since Daubert, the way in which courts looked at evidence has changed.  Since Frye, requirements are being made for reliability and other things.  We research and publish in peer review journals, and entities need to fund additional research; there hasn’t been the investment in this particular area to address these problems.  It’s OK to talk about the failings in an academic sense, but funding needs to be allocated to address these problems.

Koehler: We agree… money should be provided.  And I also agree that Daubert is designed to curb “junk science”.  There is some probative value to DNA and fingerprinting and others, but we aren’t going to know the value of that until the research is conducted.  One of my concerns is the way the match evidence is described by analysts in court.  There seems to be a sort of culture of exaggeration in traditional forensics sciences.  You very often hear “100% positive” match, no possibility that this fingerprint or this hair can come from someone else, there is no such thing as a mistake with this technology.  These sorts of things are complete nonsense.  Just one example is the Mayfield case who was matched to a fingerprint that the Spanish police lifted off detonators in the Madrid Spain bombing case.  Three FBI examiner said 100% “that’s a match, another retired examiner also said “That’s a match” but the Spanish police didn’t agree, and they ultimately matched it to someone else.  But had it not been for the Spanish police, we would have had 4 top experts in the United States saying it’s a match.  I don’t know what would have happened to Brandon Mayfield.  It makes no sense at all to describe matches in terms of 100% certainty.  It’s just not the scientific way.

Fisher: In general, I agree.  I think there are difficulties in maintaining a 100% point of view.  An examiner can be wrong.  These arguments are played out in the courtroom, and it’s a very strange place to have scientific arguments.  You have lawyers who aren’t scientists arbitrated by judges who are not scientists.  These are best suited to play out in an academic environment.  There is a certain amount of partisanship that is bound to play out in a courtroom environment.  If the right kinds of research designs and institutions can be put together, they are solvable or at least boundable.  If you go back to fingerprints vs. DNA testing, should we make a statement about whether these samples are 100% or should we assign some statistic.  There are limits to any measurement you make in science, and you have human error that crops into any human endeavor you are doing.  I don't think  it’s proper to offer statistical testimony in court.  I think at the end of the day, fingerprints are reliable, do have probative value, solve crime and exonerate innocent people.  There are some times when evidence doesn’t do what we hoped it would do.  I think that juries probably at least need to know this, take that into the jury room, and put that into the calculus they use to make their decision.

Caller: I see all this on TV and see the cool tests they are doing, and the Chief of police says we just don’t have the money to do all of these things.  Are these things going to continue to be cost prohibitive, or will we start to see these in smaller departments?

Fisher: CSI is entertainment, and although there is some basis of fact, they do exercise literary license.  You can read about the CSI Effect online, and there needs to be some study to see how real that is.

Koehler: Doing things right is expensive.  I don’t know where the funding will come from, but it is very important.  Once statistics is introduced, often juries are confused or misled.  As it becomes complex, juries are likely to turn to other things that are easier to understand.  When a juror hears that a body is found, there is a match, and 1:1million share that blood type, I have heard of jurors say that it must be him because those chances are like winning the lottery.  And even the statistics are limited to the fact that examiners can still make errors.

Caller: why do these forensic scientists know that they are testifying to in court is wrong, and go forward with false statements?  How come they can’t be held accountable for false testimony?

Fisher: Perjury and falsifying evidence are serious felonies that, if convicted, they can be sent to prison on.

Koehler: The forces are conspiring to giving us the information we need.  As the public becomes more aware that many of these errors are occurring, and judges get the courage to implement the Daubert criteria, we think that the funding will come.  But in general on faulty forensics, we are taking a much more optimistic view about this subject than other academics who have looked at this and said “it’s a disaster”.

______________________________________________________________________
Feel free to pass The Detail along to other examiners.  This is a free newsletter FOR latent print examiners, BY latent print examiners. There are no copyrights on The Detail (except in unique cases such as this week's article), and the website is open for all to visit.

If you have not yet signed up to receive the Weekly Detail in YOUR e-mail inbox, go ahead and join the list now so you don't miss out!  (To join this free e-mail newsletter, send a blank e-mail from the e-mail address you wish to subscribe, to: theweeklydetail-subscribe@topica.email-publisher.com)  If you have problems receiving the Detail from a work e-mail address, there have been past issues with department e-mail filters considering the Detail as potential unsolicited e-mail.  Try subscribing from a home e-mail address or contact your IT department to allow e-mails from Topica.  Members may unsubscribe at any time.  If you have difficulties with the sign-up process or have been inadvertently removed from the list, e-mail me personally at kaseywertheim@aol.com and I will try to work things out.

Until next Monday morning, don't work too hard or too little.

Have a GREAT week!