Evaluating the Valuation Expert

Challenges and the Emergence of Peer Review

Judges are often required to pick apart complicated expert analysis to assess the validity and reliability of an expert’s work when its admissibility is challenged. But in the professional domain, we would not expect a tax auditor to be able to analyze a report on macroeconomic theory. Why, then, do we ask courts to perform these analyses in fields from accounting to zoology? It stands to reason our courts need help. We propose that litigators and experts now have the chance to help courts, and in the process, serve their clients better. Litigators need a system to provide judges with independent and neutral analyses on reliability. In academics, this system already exists: it is called peer review.

Litigation and expert testimony are often intertwined; this is especially true in civil cases where business valuation and damages must be calculated.  Indeed, few civil actions seeking to recover damages could go forward without some assessment of the value of the loss.  Yet, as valuators and accountants who have testified know, the estimates the parties put forth often differ wildly.  This is no surprise since the valuator is testifying only because he or she agreed with the position of the party that hired him or her.  The large question that remains, however, is how mainstream and reliable is any one valuator’s analysis?  The question is quite fundamental, since judges are tasked with deciding the admissibility of an expert’s testimony.

Today, judges are often required to pick apart complicated expert analysis to assess the validity and reliability of an expert’s work when its admissibility is challenged.  But in the professional domain, we would not expect a tax auditor to be able to analyze a report on macroeconomic theory.  Why, then, do we ask courts to perform these analyses in fields from accounting to zoology?  It stands to reason our courts need help.  We propose that litigators and experts now have the chance to help courts, and in the process, serve their clients better.  Litigators need a system to provide judges with independent and neutral analyses on reliability.  In academics, this system already exists: it is called peer review.

Methodological Failings in Frye and the Daubert Trilogy

Expert witnesses have a special and important role in the legal system.  As stated in Daubert, opinions based in expert knowledge “can be both powerful and misleading because of the difficulty in evaluating it.”  Indeed, Frye and Daubert both created threshold admissibility rules precisely because of the large influence experts can have on juries.  All federal courts, and most state courts, follow Daubert as the standard by which to evaluate the “methods and principles” underlying expert testimony.[i]  Yet, both Frye and Daubert have methodological failings.

Frye v. U.S. was decided in 1923, in response to an early “lie detector” test.  It held that in order to be admissible, “the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.”  (emphasis added)  But how does a court ask the “particular field” whether an expert’s opinions are “generally accepted?”  It generally does not.  On appeal, amici could submit briefs analyzing the expert opinions, but district courts typically ask the experts themselves.  The answers are usually not surprising.  Even when experts provide peer-reviewed literature, texts and treatises, and other general resources, the other side surely has plenty of rebuttal material in stow.  In many jurisdictions, accountants are not immune to this scrutiny and confusion.

Daubert suffers from its own set of methodological flaws.  The Daubert trilogy (Daubert, Joiner, and Kumho Tire) led to a shift in how numerous courts assess valuators and accountants.  This shift was codified in an amended Fed. R. Evid. 702 in 2000.  Judges were charged with analyzing the “methods and principles” underlying the opinions themselves, instead of deferring to the “particular field.”  The facts and data, the methods and principles, and their application to the case at hand, were all to be reviewed.  This “gatekeeping” task, as Judge Rehnquist protested, required judges to become “amateur scientists.”  Yet, after more than 20 years of experience, “gatekeeping” has proven to be difficult for the average court.  Many lawyers are brilliant but mathematically disinclined.  This holds true for judges—all former lawyers.  Many judges readily admit that analyzing the methods and principles of valuations and economic damages is a daunting and confusing task.  Further, there appears to be no consistent set of principles in the case law, which instruct whether an issue is one for admissibility or weight.

Regardless of jurisdiction or admissibility, accountants maintain certain standards for the profession—a standard that goes beyond the baseline admissibility determination courts make.  Still, as valuators and accountants involved as experts know all too well, interpretation of the standards is highly variable.  Hence, the admissibility analysis is fraught with risk for experts and litigators alike.  In the current adversarial system, there are no assurances that the judge will understand what is accepted in the profession.  The judge simply sees that two legitimate experts are debating.  How can one separate the wheat from the chaff?

Moving Past Adversarial Bias

When evaluating damages, experts and litigators bear many risks.  A major risk stems from the fact that even if the valuations are prepared with the utmost rigor and fairness, there is no guarantee the story will hold up in court.  Indeed, the strength of the analyses are secondary to the credibility with which they are presented.  Credibility is the real reason experts are used in court.  It is essential to victory.  How else would a non-accountant be able to evaluate dueling experts?  Attorneys have known this for a long time.  It is why so much emphasis is placed on vetting and selecting experts.  Yet many courts, when evaluating validity and reliability, evaluate expert opinion on credibility as well.  Unfortunately, credibility does not equal validity or reliability.

Many will argue the law has a way to ensure that experts toe the line in their analyses: cross-examination.  To be sure, this venerable institution does have a proper role in keeping experts true to the knowledge of the field.  The Daubert court noted, “[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence.”  However, practical experience suggests that its ability to find truth is limited with experts.  Both sides use this technique to chip away at an expert’s credibility.  Cross-examination is often a zero-sum game.  Further, given the complexity of many expert analyses, judges and juries cannot be faulted for being more confused at the end than the beginning.

What is needed, as both Frye and Daubert envision, is a way for the accounting and valuation communities to weigh in on the reliability on any expert’s opinions.  Under Frye, such analysis goes to the heart of what “general acceptance” means—it is general acceptance among the profession, not the courts.  Under Daubert, a frank and neutral assessment of the methods and principles can inform judges of the views of mainstream accountants and valuators.  These views are surely to be of great weight in the ultimate gatekeeping analysis that courts must undertake.  This “peer review” of expert testimony, if done correctly, stands to reduce the level of “junk” expert testimony in courts and make admissibility determinations more just and accurate.

Peer Review in Practice

Direct peer review of experts is a budding concept.  It is different from the peer review that Daubert envisions, whereby experts are evaluated based on their use of peer-reviewed literature (among other things).  In any case, valuation and damage assessment is not a “hard” science, and many different approaches can be taken.  Reliance on peer-reviewed literature is not necessarily a useful way to evaluate such expert opinion.  The idea of peer reviewing a valuation or damage analysis is different, however, and would require a much larger commitment from the field at large.  For starters, these peer reviewers would need to be considered consultants to gain protection from discovery.  Thankfully, early indicators show that such a system is feasible and practical.

There are two areas in the litigation timeline that could benefit from the peer review process.  The first is before an expert has submitted his or her report.  During the drafting phase, adversarial bias can open up a client to many risks.  Accountants who testify are exceedingly cautious, given the intense scrutiny their work will undergo.  Yet, no analysis is perfect.  Further, all experts have experienced clients who “push” them to take certain positions.  Sometimes, a line is crossed.  Other times, the effects are more subtle.  In any case, the accumulation of these biases can lead to a false sense of security in the position being taken.  Often, it is this sense of security which can result in an expert challenge.

With peer review, experts and litigators alike can combat these critical risks to the case.  Many experts fear the “traps” they might be lead into during deposition that could impeach their opinions.  These traps could be fatal, for both the litigator and the expert.  Even distinguished experts are not bulletproof.  In more general terms, many factors in the attorney-expert relationship can lead to “groupthink” that can blind both to important problems in the analysis.  By getting a neutral and blind peer review of an expert report by a colleague, experts and litigators alike can gain a critical advantage in anticipating the opponent’s arguments and protecting the expert against a challenge.

The second point in litigation that peer review could transform is the expert challenge itself.  Often, litigators know that the problem with expert challenges is not their expert’s analysis, but is that judges cannot make sense of two opposing but seemingly equal arguments.  Even with the best experts, credibility is hard won.  By subjecting expert reports, either your own or the report of the opposing party, to blind peer review, expert challenges can become the weight that tips the scales in your favor.

The process to utilize peer review in expert challenges is straightforward.  If you are challenged, your attorney can submit your report for blind peer review.  If the reviews are favorable to your challenge, the attorney can attach them as appendices along with an affidavit describing the process that was used in obtaining the reviews.  Since all motions to exclude are pre-trial, the rules of evidence do not apply.  The opposing party might depose the peer reviewers, but the reviewers would be aware of this scenario and be represented independently during depositions.  The depositions would not be very risky to your defense since the peer reviews would only be utilized if they were helpful.  Further, it is important to note that the peer review is blind.  It could be that taking a reviewer deposition could cost the opponent dearly.

Examples of Peer Review

To ground peer review in reality, examples prove instructive.  In City of Pomona v. SQM North America, which is currently on appeal after a verdict for the defendant, an amicus brief was filed where the expert reports of both plaintiff and defendant were subject to peer review.[ii]  The case revolves around the contamination of groundwater by perchlorate; the city seeks damages to pay for the cleanup from SQM North America.  On appeal, a central issue is whether the defense expert was improperly admitted.  In analyzing the reports, the peer reviewers essentially concluded that the defense expert’s opinions were irrelevant.  The expert claimed that myriad sources of perchlorate were not considered by the city, and, as a result, the city’s methods are undependable.  The peer reviewers disagreed unanimously, saying that the city did consider alternative sources, and ruled them out, in tune with a differential methodology the defense expert espoused.

Other work has also been done in the accounting and valuation arena.  For example, a recent case involving auditor liability was peer reviewed.  This peer review required the analysis of two issues.  The first issue involved whether an audit firm did not follow GAAP in providing auditing services.  The second issue involved the lost profit and business value that resulted because of unreported material issues in various financial statements.  The end result of the peer review was dispositive in the case.  Had the experts received the peer review before the reports were submitted, the outcome would likely have been very different.

Peer Review Goes to Court

Peer review stands to benefit experts and litigators in myriad ways.  Valuators and accountants can benefit immensely from power-testing their expert reports in the draft stage.  In expert challenges, peer review allows litigators to protect their experts and effectively advocate for their client using neutral and mainstream input from the professional community.  The main feature of peer review is that litigators now have the power to tilt the scales in their favor by helping judges decipher complex expert opinions and come to accurate and ultimately cost-saving decisions.  Many other applications of peer review are possible, from IP litigation to administrative law.  Advocating with experts opens parties up to large amounts of risk.  Peer review affords experts and attorneys the opportunity to mitigate this risk and prevent major losses.

[i] For a comprehensive overview of expert admissibility law in the 50 states, see http://jurilytics.com/50-state-overview

[ii] See http://jurilytics.com/blog/daubert-in-the-ninth-circuit-amicus-brief-by-david-faigman

David Faigman (also Acting Chancellor and Dean of UC Hastings College of the Law) and Amit Lakhani are co-founders of JuriLytics (www.jurilytics.com). JuriLytics was founded on the premise that expert opinion presented in courtrooms today is not optimal (and sometimes incorrect), and that having access to expert peer review can give litigators an unparalleled advantage. JuriLytics’ core mission is to increase the quality of expert opinion that is presented in the courtroom. We aim to not only help litigators, but also better the judicial system.
Messrs. Faigman and Lakhani can be reached at (650) 273-6476 or by e-mail at alakhani@jurilytics.com or dfaigman@jurilytics.com.