Can Forensic Experts Overcome Their Biases?

Is Recognizing Cognitive and Motivational Biases Enough?

The National Academy of Sciences reported that bias is a severe problem in forensic sciences. Cognitive biases were described as, “common features of decision making, and they cannot be willed away.” Is recognizing bias, alone, sufficient to address cognitive, motivational or other biases? What can a forensic expert do to avoid the bias trap? In this article, the author answers these questions.

Black’s Law Dictionary defines bias as, “[a] predisposition to decide a cause or an issue in a certain way (Garner, 2009).”  Prior experiences, learning paradigms, individual beliefs, and other biases can cloud the understanding of what is important.  There are two types of biases: cognitive biases and motivational biases (Giannelli, 2008).  Cognitive biases occur at the subconscious level and frequently interfere with the ability of people to make good decisions.  Motivational biases can occur at the conscious or subconscious level and result from a person’s desire to deliver expected results.

Black’s Law Dictionary offers four types of courtroom bias.

1. Actual bias occurs when “[g]enuine prejudice that a judge, juror, witness, or other person has against some person or relevant subject” is present. Examples of this bias are when a person believes a member of an ethnic or racial group possess certain behavioral traits.  Actual bias may cause a trier of fact to believe or not believe a person committed a certain act.
2. Advocate’s bias is present when an advocate for a person or cause becomes too involved with the person or cause being advocated. As an attorney, your job is to advocate for your client’s case, whether or not you believe the client is guilty of the charges or claims.  The retained expert is an advocate for the evidence rather than for a party to the case.  When an attorney becomes too involved with a case, advocate’s bias may develop, causing the attorney to make mistakes and overlook important issues.
3. Implied bias can result from relationships among parties. The expert witness must avoid any implied bias by receiving full payment for services ahead of offering testimony.  The implied bias is that the expert is testifying in a particular manner to ensure receipt of payment.  Another form of implied bias is if the expert has a relationship with a party to the case.
4. Judicial bias occurs when the judge or trier of fact has a bias in favor of one of the parties in a case. Judicial bias may take the form of the other forms of bias.

Experts make decisions that are expressed as opinions by applying analytical methods developed through training, education, and practice.  Such prior experience may induce biases that cause the expert to use trusted methods without considering alternatives.  Forensic science seeks to produce reliable evidence which is clearly reported (Sjerps and Meester, 2009).  Experts must recognize when their biases and those of others influence their decisions.

Laboratory Bias

Forensic examiners work in private, crime, or other forms of laboratories.  ASTM reported that eighty percent of studied laboratories showed laboratory bias (Lawrey, 2009).  Twenty percent of the laboratories displayed “significantly high bias.”  This bias was the result of interactions among many people.  Griffen and Tversky (1992) attributed similar bias to people’s tendency toward being more overconfidant in their judgments than is warrented by the facts.  When we select evidence that is not independent of the forensic analysis, problems occur (Sjerps and Meester, 2009).  Schwab (2008) showed that bias induces experts to be overconfidant in rating their abilities.

The National Academy of Sciences (NAS) (National Research Council, 2009) reported that bias is a severe problem in forensic sciences.  Cognitive biases were described as, “common features of decision making, and they cannot be willed away.”

NAS reported that judges are subject to bias in their rulings.  The report cites studies that half the fingerprint examinations had bias introduced into the procedures.  A recommendation was made to remove the association of crime laboratories from police agencies to reduce the motivational bias.  The expert bias can be reduced if the expert is not aware of the side which has hired him or her (Baer, 2005).

According to research, awareness of the source of cognitive bias is insufficient to prevent a person from being trapped by biases (Ariely, 2008; Cialdini, 2001).  Arzy, et al., (2009) discovered that by including one misleading detail about a patient, the misdiagnosis rate in emergency room cases was ninety percent by practicing physicians.  Telling a control group there was one misleading detail did not reduce the diagnostic error.  When the misleading detail was omitted from the information, the misdiagnosis rate reduced to thirty percent.

Forensic examiners must sort through evidence so as not to follow the trail of misleading information, resulting in a flawed opinion.  An example is when the retaining attorney offers details about the case that are not pertinant to the forensic examination.  Details such as a confession, witness’s statement, or the place where the evidence was discovered may unconsiously bias the forensic examiner’s perception of the authenticity of the evidence in question.

Contextual Bias

Research shows that document examiners who work for the government tend to focus on differences in writing, whereas private practice document examiners tend to focus on similarities in writing (McAlexander, 1999).  The reason for this difference of approach is the government examiners typically are retained by the prosecution.  The prosecution’s emphasis is convicting people accused of having committed crimes.  The private examiner is typically retained by defense counsel.  The defense’s focus is exonerating the accused party.  Each case is an example of contextual bias.

Research shows when a person is exposed to a concept, they anchor to the concept at a subconscious level.  Being exposed to the context of a case causes the forensic examiner to anchor to that context.  Anchoring is the psychological tendency to set the given context as the starting point for thinking.  As an example, when a person sees the price of a product, that price becomes the anchor from which they determine the price shown for the product by other merchants.

Fingerprint examiners were presented with cases in which they were told the suspect had been erroneously identified.  Peer review was required.  Four of the five review examiners excluded the suspect.  In each instance, the examiner was presented with fingerprints they had identified as the suspect’s prints in cases years earlier (Droor, 2010).  They were affected by the stated context.

Bias starts when a forensic examiner is hired by an attorney.  An example of this is an attorney approaches the document examiner saying, “I have a case where my client claims they did not sign this check,” or “my client claims they did not sign this contract.”  These statements set the foundation for cognitive, or subconscious, bias.  One of the jobs of a document examiner is to disengage from the bias.  This can be done by taking a step back from the scope of the examination request and focusing on the evidence.  Application of an objective scientific approach reduces cognitive bias.

The document examiner cares about the writing on the document, potential alterations, or other attributes of the scope of the analysis.  The context of the case is the purview of the retaining attorney, not the document examiner.

The forensic examiner may not be interested in the contents of the court case.  The less information you provide to the examiner, the more objective the results are.  The only point of interest is the evidence.  Specifics of the origin or history of the evidence may be requested by the examiner if it is pertinent to examination of the evidence.  An example is a case in which a party stated the reason the person’s signature changed was due to brain surgery.  Examination of the medical literature showed surgery on this portion of the brain has no effect on handwriting.  Therefore, the signature had not changed due to surgery.

None of us wants to feel our opinions are tainted by bias.  The ability to recognize when bias is an influence in an expert’s opinion and the skill of an expert to overcome his or her biases is integral to an expert’s credibility.

Framing the Problem

The presentation of information is known as framing.  When a problem is framed in a manner that appears to be logically sound, the problem solver accepts the framing and attempts to solve the problem in conjunction with the way the problem is framed (Bernstein, 1996).

A study conducted at Stanford University testing the impact of framing a situation and then adding additional information about the decision that is to be made, expose how bias creeps in and affects judgment.  In the Stanford University study, subjects were given sufficient information regarding a courtroom trial (Kahneman and Tversky, 1995).  One group was given more detail regarding the defendant, and another group was given additional information regarding the plaintiff.  Although the groups knew the data was biased, they were unable to mentally balance the information.  The biased groups were more confident about the outcome in favor of the side whose information was more voluminous than the group with balanced information (Kahneman and Tversky, 1995).

Bias of Juries

Webber (2008) reported, “Juries…typically base their decisions on whichever story seems most plausible to them, rather than weighing the evidence.”  These decisions are made regardless of whether the information is accurate.  McAuliff, Kovera, and Nunez (2009) expanded on Webber’s findings, stating that when jurors’ motivation is low or their ability to understand the presented information is poor, they rely on heuristics and that which they understand as real-life situations (McAuliff, et al., 2009).  McAuliff, et al., discovered that jurors are, “insensitive to the presence of a confound or experimenter bias in the expert’s research.”  The jurors relied on their flawed analysis of the expert’s evidence when rendering a verdict.

A sharp attorney can bias a jury by framing questions to the witness.  Framing bias can cause the jury to view the expert as qualified or not qualified.  McAuliff, et al., (2009) found a positive relationship between verdicts and juror’s evaluation of expert’s evidence.  McAuliff reported that jurors are not able to evaluate statistical evidence and methodologies.  He also reported that, “Judges are unable to differentiate between valid and junk science…leading to, admission of invalid research at trial.”  The forensic examiner must reduce these potential biases by presenting clear and easy to understand evidence to support an opinion.

Confirmation Bias

Confirmation bias results when a person accepts evidence that supports their position and rejects evidence that does not support their position.  An example of confirmation bias occurred in a court case requiring authentication of a will.  The opposing document examiner had forty-three exemplars.  He showed seven of the exemplars in his report.  By using only seven exemplars, he offered an opinion the decedent did not author the questioned signature (exclusion).  After we exchanged evidence, I used his other thirty-six exemplars to create exhibits that contradicted his opinion.  The decedent had wide variability in his signatures.  The Court agreed with my testimony.

Many forensic examiners are sole proprietors.  They tend to interface directly with the retaining party.  As a result, they must be vigilant not to be swayed by the claims of the attorney’s client.  The evidence is all that matters.  This approach reduces the effects of confirmation bias.

References

• Ariely, D. (2008). Predictably irrational; The hidden forces that shape our decisions. New York, N.Y: Harper Perennial.
• Arzy, S., Brezis, M., Khoury, S., Simon, S., and Ben-Hur, T. (2009). Misleading one detail: a preventable mode of diagnostic error? Journal of Evaluation in Clinical Practice, 15, 804-809.
• Baer, M.A. (2005). Is an independent medical examination independent? The Forensic Examiner. Winter, 33.
• Garner, B.A. (2009). Black’s Law Dictionary, Ninth Edition. St. Paul, MN: West Publishing Co.
• Giannelli, Paul C. “Confirmation bias in forensic testing.” GP Solo 25.2 (2008): 22. General OneFile. Web. 13 Jan. 2010.
• Cialdini, R. (2001). Influence: science and practice. Boston: Allyn & Bacon.
• Droor, I.E., and Charlton, D. (2006). Why experts make errors. Journal of Forensic Identification. 56 (4), 600-616.
• Kahneman, D., and Tversky, A. (1995). Conflict resolution: a cognitive perspective. Arrow, K., et al., (Eds). (1995) Barriers to conflict resolution. New York, NY: W. W. Norton & Company, Inc.
• McAuliff, B.D., Kovera, M.B., and Nunez, G. (2009). Can jurors recognize missing control groups, and experimenter bias in psychological science? Law and Human Behavior. 33, 247-257.
• National Research Council (2009). Strengthening forensic science in the United States: A path forward. Washington, D.C. The National Academy Press.
• Schwab, A.P. (2008). Putting cognitive psychology to work: Improving decision-making in the medical encounter. Social Science & Medicine. 67, 1861-1869
• Webber, S. (2008). The dark side of optimism; why looking on the bright side keeps us from thinking critically. The Conference Board Review. 45, 30-36.

Mike Wakshull, MS, CQE, PMP, CISA, is a forensic document examiner based in Temecula, CA. He holds a graduate school certificate in forensic document examination from East Tennessee State University and a master’s degree in Technology Management from the University of Denver. Mr. Wakshull was Chair of the 2012 National Association of Document Examiners Conference in San Diego and co-Chair of the 2015 Forensic Expert Witness Association’s conference. As a member of National Speakers Association, he regularly speaks to local and international legal and forensic meetings and conferences. He has been qualified as an expert witness in California Superior Courts and Federal District Court, testifying in cases involving will contests, contract disputes and others. Visit his website at http://www.quality9.com.
Mr. Wakshull can be contacted at (951) 252-4929 or by e-mail to MikeW@quality9.com.