C:\documents and settings\eohcnm.001\local settings\temporary internet files\content.ie5\ps4vpx01\s66[1].pdf

Trial and Error: The Supreme Court’s Philosophy of Science Frye in excluding the plaintiffs’ experts’ testi- Apparently equating the question of whether expert testimony is reliable with the question of whether it is genuinely scientific, in Daubert v Merrell Dow Phar-maceuticals, Inc (1993) the US Supreme Court ran together Karl Popper’s and dectin was teratogenic. So the Supreme Court Carl Hempel’s incompatible philosophies of science. But there can be no criterion was to determine whether the Federal Rules of discriminating scientific, and hence reliable, testimony from the unscientific and Evidence had superseded Frye, and in particu- unreliable; for not all, and not only, scientific evidence is reliable.
lar how Rule 702 was to be interpreted.
In subsequent rulings (General Electric Co v Joiner, 1997; Kumho Tire Co v Yes, Justice Blackmun wrote for the unani- Carmichael, 1999) the Court has backed quietly away from Daubert’s confused phi- mous court, the Federal Rules of Evidence had losophy of science, but not from federal judges’ responsibilities for screening superseded Frye; but the Rules themselves re- expert testimony. Efforts to educate judges scientifically, and increased use of quire judges to screen proffered expert testi- court-appointed experts are, at best, only partial solutions to the problems with mony not only for relevance, but also for relia- scientific testimony. (Am J Public Health. 2005;95:S66–S73. doi:10.2105/AJPH.
bility. In doing this, he continued (in a part of the ruling from which Justice Rehnquist and Jus-tice Stevens dissented), courts must look, not to It seems to me that there is a good deal of ballyhoo dice, waste of time, or confusing or misleading an expert’s conclusions, but to his “methodol- about scientific method. I venture to think that the peo- the jury). In Barefoot v Estelle, a 1983 constitu- ogy,” to determine whether proffered evidence ple who talk most about it are the people who do least tional case, the Supreme Court affirmed that is really “scientific . . . knowledge,” and hence about it. . . . No working scientist, when he plans an ex- the rights of a Texas defendant were not vio- reliable. As to what that methodology is, citing periment in the laboratory, asks himself whether he is lated by the jury’s being allowed to hear psy- law professor Michael Green citing philosopher being properly scientific. . . . When the scientist ventures chiatric testimony of his future dangerousness of science Karl Popper, and quoting an observa- to criticize the work of his fellow scientist, he does not at the sentencing hearing—even though an ami- tion of Carl Hempel’s for good measure, the base his criticism on such glittering generalities as fail- cus brief from the American Psychiatric Associ- Daubert ruling suggested four factors that courts ure to follow the “scientific method,” but his criticism is ation reported that two out of three such pre- might use in assessing reliability: “falsifiability,” specific. . . . The working scientist is always too much dictions are mistaken. Writing for the majority, that is, whether proffered evidence “can be and concerned with getting down to brass tacks to be willing Justice White observed that state and federal has been tested”; the known or potential error to spend his time on generalities. rules of evidence “anticipate that relevant, un- rate; peer review and publication; and (in a nod privileged testimony should be admitted and its to Frye), acceptance in the relevant community.5 weight left to the fact-finder, who would have In partial dissent, however, pointing out that In Frye v United States 2 (1923), the Washington, the benefit of cross-examination and contrary the word “reliable” nowhere occurs in the text DC court upheld the exclusion of testimony of evidence by the opposing party.”3 Justice Black- of Rule 702, Justice Rehnquist anticipated diffi- the results of a then-new blood-pressure decep- culties over whether and, if so, how Daubert tion test on the grounds that novel scientific should be applied to nonscientific expert testi- testimony “crosses the line between the experi- that the tort system was getting out of hand, mony; worried aloud that federal judges were mental and the demonstrable,” and so is ad- Peter Huber argued in his influential Galileo’s being asked to become amateur scientists; and missible, only if it is “sufficiently established to Revenge that, under the Federal Rules, worth- questioned the wisdom of his colleagues’ readi- have gained general acceptance in the particu- less “junk science,” which would have been ness to get involved in philosophy of science.
lar field to which it belongs.”2 Ignored for a excluded by the Frye test, was flooding the I think he was right to suspect that something decade, rarely cited for a quarter-century, over courts. In 1992, proposals to tighten up the was seriously amiss; in fact, what I shall have time the “Frye test” became increasingly influ- Federal Rules were before Congress. In 1993, to say here might be read as an exploration, ential, until by the early 1980s it had been the Supreme Court issued its ruling in Daubert amplification, and partial defense of his reser- v Merrell Dow Pharmaceuticals, Inc,4 the first vations about that philosophical excursus.
case in its 204-year history where the central Rules of Evidence had set a seemingly less re- questions concerned the admissibility of scien- DAUBERT ’S CONFUSIONS: POPPER
strictive standard: the testimony of a qualified tific testimony. The Frye rule arose in a crimi- expert, including a scientific expert, is admissi- nal case and had for most of its history been ble provided it is relevant (unless it is excluded, cited in criminal cases; but Daubert was a tort under Rule 403, on grounds of unfair preju- action in which the trial court had relied on whether expert testimony is reliable with the S66 | Public Health Matters | Peer Reviewed | Haack American Journal of Public Health | Supplement 1, 2005, Vol 95, No. S1 question of whether it is genuinely scientific, long ago that induction is unjustifiable. But of evolution is, apparently persuaded Judge taking for granted that there is some scientific science doesn’t need induction; the method of “methodology” which, faithfully followed, conjecture and refutation requires only de- But there is an even more serious problem guarantees reliable results, and casting about ductive logic—specifically, modus tollens, the with the Daubert Court’s reliance on Popper, for a philosophy of science to fit this demand- rule invoked when an observational result of which Justice Rehnquist didn’t seem to be ing bill, the Daubert Court settled on an un- aware: Popper’s philosophy of science is sig- stable amalgam of Popper’s and Hempel’s Theories that have been tested but not yet nally inappropriate to the Court’s concern very different approaches—neither of which, falsified are “corroborated,” the degree of cor- with reliability. When Popper describes his however, is suitable to the task at hand.
roboration at a time depending on the num- approach as “critical rationalism,” it is to em- Popper describes his philosophy of science ber and severity of the tests passed. That a phasize that the rationality of the scientific as “Falsificationist,” by contrast with the Veri- theory is corroborated, to however high a de- enterprise lies in the susceptibility of scientific ficationism of the Logical Positivists, because gree, doesn’t show that it is true, or even theories to criticism, that is, to testing, and po- his key theme is that scientific statements can probable; indeed, the degree of testability of a tentially to falsification, not in their verifiabil- never be shown conclusively to be true, but hypothesis is inversely related to its degree of ity or confirmability. True, early on Carnap can sometimes be shown conclusively to be logical probability.8 Corroboration is not a translated Popper’s word Bewährung as “con- false. Hence, his criterion of demarcation: measure of verisimilitude, but at best an indi- firmation”; for a while, thinking the issue to be genuinely scientific, a statement must cator of how the verisimilitude of a theory merely verbal, Popper let it go—even occa- be “testable”—meaning, in Popper’s mouth, appears, relative to other theories, at a partic- sionally using “confirm” himself. But in a foot- “refutable” or “falsifiable,” i.e., susceptible to ular time9; and the fact that a theory has note to the English edition of The Logic of Sci- evidence that could potentially show it to be been corroborated doesn’t mean it is rational entific Discovery, he comments that this had false (if it is false). Curiously, Popper acknowl- to believe it. (It does mean, Popper writes, that been a bad mistake on his part, conveying the edged from the beginning that his criterion of it is rational to prefer the theory as the basis false impression that a theory’s having been demarcation is a “convention”; and in his for practical action; not, however, that there corroborated means that it is probably true.14 1959 introduction to the English edition of are good reasons for thinking the theory will The Logic of Scientific Discovery, he affirmed be successful in the future—there can be no doned Carnap’s (mis)translation,15 Popper in- that scientific knowledge is continuous with good reasons for believing this.10 So it seems sisted that corroboration must not be con- commonsense knowledge.6 Nevertheless, his that all this “concession” amounts to is that in whole philosophy of science turns on his cri- deciding how to act, we can do no better than terion of demarcation. Falsifiability is to dis- go with theories that we don’t so far know to represents its past performance only, and criminate real empirical science, such as Ein- “says nothing whatever about future perform- stein’s theory of relativity, from prescientific The first problem with the Daubert Court’s ance, or about the ‘reliability’ of a theory ”; even myths, from nonempirical disciplines like pure reliance on Popper is that applying his crite- the best-tested theory “is not ‘reliable’”16—so mathematics or metaphysics, from nonscien- rion of demarcation is no trivial matter; as Jus- scornful is Popper of the concept of reliability tific disciplines such as history, and from such tice Rehnquist pointed out, observing wryly that he refuses even to use the word without pseudosciences as Freud’s and Adler’s psy- that, since he didn’t really know what is meant putting it in precautionary scare quotes! choanalytic theories and Marx’s “scientific by saying that a theory is “falsifiable,” he Reiterating that he puts the emphasis “on neg- socialism.”7 Falsifiability is also central to Pop- doubted federal judges would, either.11 In- ative arguments, such as negative instances or per’s account of the scientific method as “con- deed, Popper himself doesn’t seem quite sure counter-examples, refutations, and attempted jecture and refutation”: making a bold, highly how to apply his criterion. Sometimes, for ex- refutations—in short, criticism—while the in- falsifiable guess, testing it as severely as possi- ample, he says that the theory of evolution is ductivist lays stress on ‘positive instances’, ble and, if it is found to be false, giving it up not falsifiable, and, so, is not science; at one and starting over rather than protecting it by point, he suggests that “survival of the fittest” inferences’, and which he hopes will guarantee ad hoc or “conventionalist” modifications.
is a tautology, or “near-tautology,” and else- the ‘reliability’ of the conclusions of these in- (This readiness to accept falsification and es- where that evolution is really a historical ferences,” Popper specifically identifies Hempel chew ad hoc stratagems is Popper’s “method- as representative of those inductivists with ological criterion” of the genuinely scientific.) changes his mind: evolution is science, after Popper also describes his philosophy of sci- all.12 It’s ironic; for Popper’s criterion of de- Hempel is not, perhaps, the prototypical in- ence as “deductivist,” by contrast with “induc- marcation had already found its way into the ductivist; he describes the method of science tivism,” whether in the strong, Baconian form US legal system, a decade before Daubert, in as “hypothetico-deductive,” he affirms that sci- that posits an inductive logic for arriving at a 1982 First Amendment case, McLean v entific claims should be subject to empirical hypotheses or in the weaker, Logical Positivist Arkansas Board of Education, where Michael check or testing, and he doesn’t follow Re- form that posits an inductive logic of confir- Ruse’s testimony that creation science is not ichenbach and Carnap in explaining confirma- science, by Popper’s criterion, but the theory tion by appeal to the calculus of probabilities.
Supplement 1, 2005, Vol 95, No. S1 | American Journal of Public Health Haack | Peer Reviewed | Public Health Matters | S67 Nevertheless, Popper is surely right to see DAUBERT’S CONFUSIONS:
evolving local ways and means of stretching Hempel’s approach as very significantly at the imagination, amplifying reasoning power, odds with his own: Hempel is not centrally extending evidential reach, and stiffening re- So, the Daubert Court mixes up its Hoppers and its Pempels. But isn’t this just a slip, of Every kind of empirical inquiry, from the verification and falsification, and argues that merely scholarly interest? No; it is sympto- simplest everyday puzzling over the causes of Popper’s criterion “involves a very severe re- matic of the serious misunderstanding of the delayed buses or spoiled food to the most striction of the possible forms of scientific place of the sciences within inquiry generally complex investigations of detectives, of histo- hypotheses,” for example in ruling out purely revealed by the Court’s equation of “scien- rians, of legal and literary scholars, and of sci- existential statements18; when he speaks of entists, involves making an informed guess “testing” he envisages both disconfirmation about the explanation of some event or phe- and confirmation of a hypothesis; and one of been that the words “science,” “scientific,” and nomenon, figuring out the consequences of its his chief projects was to articulate the “logic “scientifically” are often used as generic terms being true, and checking how well those con- of confirmation,” i.e., of the support of general of epistemological praise, meaning vaguely sequences stand up to evidence. This is the “strong, reliable, good”— as, in television procedure of all scientists; but it is not the advertisements, actors in white coats urge procedure only of scientists. Something like combining Hempel’s account of confirmation viewers to get their clothes cleaner with new, the “hypothetico-deductive method,” really is with Popper’s criterion of demarcation, to “scientific” Wizzo. This honorific usage is un- the core of all inquiry, scientific inquiry in- craft a crisp test to identify genuine, and mistakably at work in the Daubert ruling; in- cluded. But it is not distinctive of scientific in- deed, it seems to be implicit even in the way quiry; and the fact that scientists, like inquir- Hempel’s philosophy of science is more posi- Justice Blackmun writes of “scientific . . .
ers of every kind, proceed in this way tells us tive than Popper’s, it isn’t much more helpful knowledge,” strategically excising three not nothing substantive about whether or when regarding the question of reliability. For one insignificant words from the reference in Fed- thing, the confirmation of generalizations by eral Rules of Evidence 702 to “scientific or positive instances that preoccupies Hempel is other technical knowledge,” apparently signal- with specialized instruments; stretched the just too simplified to apply to the enormously ing an expectation that a criterion of the gen- imagination with metaphors, analogies, and complex congeries of epidemiological, toxico- uinely scientific will also discriminate reliable models; amplified reasoning power with nu- logical, etc. evidence at stake in a case such merals, the calculus, computers; and evolved as Daubert. For another, Hempel himself If “scientific” is used honorifically, it is a a social organization that enables coopera- seems eventually to have concluded (rightly, tautology that “scientific” equals “reliable”; I believe) that the “grue” paradox shows that but this tautology, obviously, is of no help to allowing each scientist to take up his investi- confirmation isn’t a purely syntactic or logical a judge trying to screen proffered scientific gation where others left off. Astronomers notion after all,19 and late in life began to testimony. If “scientific” is used descriptively, devise ever more sophisticated telescopes, however, “scientific” and “reliable” come chemists ever more sophisticated techniques the right track in focusing on historico- apart: for, obviously, physicists, chemists, biol- of analysis, medical scientists ever more politico-sociological, rather than logical, as- ogists, medical scientists, etc., are sometimes sophisticated methods of imaging bodily states incompetent, confused, self-deceived, dishon- and processes, and so on; scientists work out est, or simply mistaken, while historians, de- what controls are needed to block a potential tectives, investigative journalists, legal and source of experimental error, what statistical of supportiveness of evidence or, as he said, literary scholars, plumbers, auto mechanics, techniques are needed to rule out a merely of “relative confirmation,” the relation be- etc., are sometimes good investigators. In coincidental correlation, and so forth. But tween observational evidence and hypothesis, short, not all, and not only, scientists are reli- these scientific “helps” to inquiry are local expressible as “E confirms H [to degree n],” able inquirers; and not all, and not only, sci- and evolving, not used by all scientists.22 or “H is confirmed [to degree n] by evidence entific evidence is reliable. Nor is there a You may object that, since I have acknowl- E.” This, as Hempel acknowledged, falls “scientific method” in the sense the Court as- edged that scientific inquiry is continuous short of an account of “absolute confirma- sumed: no uniquely rational mode of infer- with everyday empirical inquiry, I have in ef- tion,” the warrant of a scientific claim, which ence or procedure of inquiry used by all sci- fect agreed with Popper that science is an ex- would be expressed in nonrelative terms as entists and only by scientists. Rather, as tension of common sense. Indeed, I think sci- “H is confirmed [to degree n], period.” To Einstein once put it, scientific inquiry is “noth- ence is well described, in Gustav Bergmann’s discriminate reliable from unreliable testi- ing but a refinement of our everyday think- wonderfully evocative phrase, as the “Long ing,”21 superimposing on the inferences, Arm of Common Sense.”23 But the continuity desiderata, and constraints common to all se- is not between the content of scientific and of rious investigation a vast variety of constantly S68 | Public Health Matters | Peer Reviewed | Haack American Journal of Public Health | Supplement 1, 2005, Vol 95, No. S1 basic ways and means of everyday and of sci- The honorific use of “science” and its cog- the Joiner Court (with the exception of Justice entific inquiry; and it is precisely because of nates tempts us, as it did the Daubert Court, Stevens) flatly denied the legitimacy of the this continuity that the Popperian preoccupa- to criticize poorly conducted science as not distinction between methodology and conclu- tion with the “problem of demarcation” is a really science at all; but “not scientific” is as sions. Opining that this is No Real Distinction, unhelpful as generic epistemic criticism as the Court sounded like nothing so much as a Or you may object that the Daubert Court’s “scientific” is as generic epistemic praise. The conclave of medieval logicians; but given their citation to Paoli,28 it seems likely that proffered scientific testimony “can be and has science,” which presumably refers to activi- they didn’t really intend to make a profound been tested” surely is potentially helpful. This ties that purport to be science but aren’t re- is true; but it is no real objection. “Check ally, derives in part from its imputation of knowledge, as Judge Becker had, that if an whether proffered testimony has been tested” false pretenses, generally, and in part from expert’s conclusions are problematic enough, is very good advice when a purported expert the favorable connotations of “scientific,” spe- this alerts us to the possibility of some meth- hasn’t made even the most elementary effort cifically. But rather than sneering unhelpfully to check how well his claims stand up to evi- that this or that work is “pseudoscientific,” it This focus on “methodology”—an accordion dence: such as the knife-mark examiner in is always better to get down to those “brass concept expanded and contracted as the argu- Ramirez v State,24 who testified that he could tacks” Bridgman talks about, and specify infallibly identify this knife, to the exclusion what, exactly, is wrong with it. Is it dishon- epistemological question. Joiner’s attorneys of all other knives in the world, as having estly or carelessly conducted? Does it rest on proffered a collage of bits of information, made the wound— though no study had es- flimsy or vague assumptions—assumptions for none sufficient by itself to warrant the conclu- tablished the assumed uniqueness of individ- sion that exposure to polychlorinated bi- ual knives, and his purported ability to make aren’t even susceptible to evidential check? phenyls promoted Joiner’s cancer, but which, such infallible identifications had never been Does it seek to impress with decorative or they argued, taken together gave strong sup- tested. This is not, however, because falsifia- distracting mathematical symbolism or elabo- port to that conclusion. General Electric’s at- bility is the criterion of the scientific, but be- rate-looking apparatus? Does it fail to take es- torneys replied, in effect, that piling up weak cause any serious inquirer is required to seek evidence can’t magically transform it into out all potentially available evidence and to strong evidence. In response, Joiner’s attor- go where it leads, even if he would prefer to neys referred to the EPA guidelines for as- avoid, ignore, or play down information that DAUBERT’S LEGAL PROGENY
sessing the combined weight of epidemiologi- pulls against what he hopes is true.
cal, toxicological, etc. evidence. But no one Yes, this is a requirement on scientists; as So, the Daubert Court’s philosophy of sci- addressed the key question: Is there a differ- Darwin recognized when he wrote in his au- ence was muddled; but haven’t subsequent tobiography that he always made a point of Supreme Court rulings cleared things up? Not interrelated that the whole really is greater recording recalcitrant examples and contrary exactly: it would be more accurate to say that than the sum of its parts, and a collection of arguments in a special notebook, to safe- in General Electric Co v Joiner (1997) and unrelated and insignificant bits of information guard against his tendency conveniently to Kumho Tire Co v Carmichael (1999), the Su- between true consilience and the “faggot fal- forget negative evidence.25 But it is no less a requirement on other inquirers, too; as we all Daubert’s confused philosophy of science.27 There is a difference. Evidence of means, realized a few years ago when a historian Pompel, falsifiability, etc., so prominent in support the claim that the defendant did it Daubert, are conspicuous by their absence much more strongly than any of these pieces Kennedy turned out to have ignored the fact from Joiner and Kumho. But there are points of evidence alone could do. Similarly, evi- that the supposedly incriminating letters were dence of increased incidence of a disease typed with correction ribbon, and the ad- In Joiner, there was a bit of a kerfuffle dress included a zip code—when neither ex- about “methodology”: Joiner’s attorneys had stance may interlock with evidence that ani- isted at the time the letters were purportedly argued that the lower court erred in exclud- ing their proffered expert testimony because, “Nonscience” is an ample and diverse cate- instead of focusing exclusively on their ex- gory that includes the many human activities perts’ methodology—which, they maintained, was the very same “weight of evidence” claim that this substance causes, promotes, pseudo-inquiry, inquiry of a nonempirical character, and empirical inquiry of other (General Electric’s) experts—improperly con- strongly than any of these pieces of evidence kinds than the scientific; and of course there cerned itself with the experts’ conclusions.
alone could do. However, the interlocking are plenty of mixed and borderline cases.
Apparently anxious to sidestep this argument, will be less robust if, for example, the animals Supplement 1, 2005, Vol 95, No. S1 | American Journal of Public Health Haack | Peer Reviewed | Public Health Matters | S69 are unlike humans in some relevant way, or subject only to review for abuse of discretion, Daubert’s policy ramifications, and pick up if the mechanism postulated to cause damage not to any more stringent standard; and the the epistemological thread once more.
is also present in other chemicals not found Kumho Court, pointing out that, depending on to be associated with an increased risk of the nature of the expertise in question, the Daubert factors may or may not be appropri- “Interlocking” is exactly the right word; for ate, held that it is within judges’ discretion to So, since Kumho’s epistemological step for- evidence is structured like a crossword puz- use any, all, or none of them. A year later, re- ward, the other problem Justice Rehnquist zle, with warranted claims anchored by expe- vised Federal Rules of Evidence made explicit worried about—that judges generally lack the riential evidence (the analogue of clues) and what according to Daubert had been implicit background knowledge that may be essential enmeshed in reasons (the analogue of com- in Rule 702 all along: admissible expert testi- to a serious appraisal of the worth of scientific pleted intersecting entries). How reasonable a mony must be based on “sufficient” data, the (or other technical) testimony—looms larger crossword entry is depends on how well it is product of “reliable” testimony “reliably” ap- than ever. But hasn’t the legal system by now supported by the clue and completed inter- plied to the facts of the case. Federal judges found ways to help judges handle their quite secting entries, how reasonable those other now have large responsibilities and broad dis- burdensome responsibilities for keeping the entries are, independent of this one, and how cretion in screening not only scientific testi- gate against unreliable expert testimony? Up much of the crossword has been completed; mony but expert testimony generally—but to a point; but only up to a point. Ways have similarly, how warranted a claim is depends very little guidance about how to perform this on how supportive the evidence is, how se- cure the reasons are, independent of this Post-Daubert courts have apparently been to enable them to call on the scientific com- claim itself, and how much of the relevant ev- significantly tougher than before on expert munity for help; but these have been rela- idence the evidence includes.31 Because of testimony proffered by plaintiffs in civil tively small steps, and sometimes (under- the ramification of reasons, the desirable cases. This isn’t the place for a full-scale dis- kind of interlocking of evidence gestured at cussion of the frequently heard criticism that Daubert prompted various efforts to edu- in Joiner is subtle and complex, not easily cap- Daubert and its progeny tend to favor defen- cate judges scientifically. In May 1999, for ex- tured by any mechanical weighting of epi- dant corporations over plaintiffs; but I will ample, about two dozen Massachusetts Supe- demiological data relative to animal studies or say that I think things are a lot more compli- rior Court judges attended a 2-day seminar toxicological evidence. Nor, moreover—as Jus- cated than this criticism suggests. No doubt on DNA at the Whitehead Institute for Bio- tice Rehnquist already saw in the context of there are heartless and unscrupulous compa- medical Research. A report in the New York Daubert—can its quality readily be judged by nies more concerned with profit than with Times quoted the director of the Institute as- the dangers their products may present to suring readers that, while in the O. J. Simpson the public; and it is certainly easier to sym- case, lawyers had “befuddled everyone” over In Kumho, the Supreme Court made a real the DNA evidence, after a program such as epistemological step forward. In this products- poor Mr. Joiner than with a vast, impersonal this judges would “understand what is black liability case, which focused on the proffered outfit like Merrell Dow or General Electric.
and white . . . what to allow in the court- testimony of an expert on tire failure, the But no doubt there are also greedy and op- room.”33 To be candid, this report leaves me a Court tried to sort out the problems with non- portunistic plaintiffs and plaintiffs’ attor- little worried about the danger of giving scientific experts which, as Justice Rehnquist judges the false impression that they are qual- had anticipated, soon arose in the wake of when meritless litigation forces a company ified to make subtle scientific determinations; Daubert; and ruled that judges can’t evade to downsize or close also deserve our sym- when it is hardly realistic to expect that a few their gatekeeping duty on the grounds that pathy. Moreover, although we certainly hope hours in a science seminar will transform proffered expert testimony is not science; the that the tort system will discourage the man- judges into scientists competent to make sub- key word in Federal Rule of Evidence 702, ufacture of dangerous substances and prod- tle and sophisticated scientific judgments—any after all, is “knowledge,” not “scientific.” No ucts, we also want it not to discourage the longer fussing over demarcation, recognizing manufacture of safe and useful ones. And I could transform scientists into judges compe- the gap between “scientific” and “reliable,” in will add that, although it seems that since tent to make subtle and sophisticated legal Kumho the Supreme Court acknowledged that Daubert courts have not—at least not yet— what matters is whether proffered testimony is been as tough on expert testimony proffered It really isn’t feasible to bring—let alone reliable, not whether it is scientific. Quite so.
by prosecutors in criminal cases as they have keep—judges up to speed with cutting-edge Far from backing away from federal courts’ on plaintiffs’ experts in civil cases, we surely genetics, epidemiology, toxicology, and so on.
gatekeeping responsibilities, however, the also want to avoid convicting innocent crimi- (This is not in the least to denigrate judges’ Joiner Court affirmed that a judge’s decision nal defendants on flimsy forensic testimony— abilities, but rather to draw the analogy with to allow or exclude scientific testimony, even and leaving the real offenders at liberty.32 expecting a few lessons to turn a professional though it may be outcome-determinative, is That said, I will leave it to others to pursue football player into a ballet dancer, or me S70 | Public Health Matters | Peer Reviewed | Haack American Journal of Public Health | Supplement 1, 2005, Vol 95, No. S1 into a concert pianist.) It ought to be possible, nical Advisors also provides the names of in- however, to educate judges in the elements of dant company or by plaintiffs’ attorneys, to probability theory, to give them a sense of his simply having such acquaintances . . . to appointed experts is “elitist” and “undemocra- his being completely out of any professional that kind of mistake made at the laboratory, tic,” even “totalitarian”38; but this strikes me as something of an exaggeration. Certainly, Yes, it is disturbing that, while serving on probability that the lab made a mistake is trial by jury is a better way of getting at the Judge Pointer’s panel, one scientist signed a such-and-such affects the significance of a truth than trial by oath or ordeal; certainly letter asking for financial support for another random-match probability. More generally, it citizens’ service on juries is an expression of project from one of the defendant companies; seems both feasible and useful to try to en- and worrying that just four scientists were, in sure that judges understand the more com- strange to deny that the Netherlands, say, is a effect, responsible for the disposition of sev- democracy, simply because the Dutch judicial eral thousand cases. Moreover, given that likely to encounter most frequently: the role system routinely relies on experts appointed even competent and honest scientists will of suggestion, for example, and its significance by the courts). Still, especially considering sometimes legitimately disagree, we need to for how DNA samples or suspect knives, etc.
how tiny the proportion of federal cases de- should be presented or how photo-arrays or cided by juries now is,39 it seems reasonable appointed scientists are not of one mind.
lineups should be conducted. Of course, when to be willing to consider adapting the adver- Both legal issues and practical questions the issues are subtle, the subtleties need to be sarial culture a little in this way,40 if and conveyed; one would hope that judges under- when this would better serve the fundamental Should court-appointed experts help judges stand the concept of statistical significance, purpose of protecting against arbitrary and with their Daubert screening duties, or for instance—but also grasp the element of should they testify before juries, along with Sometimes it is thought that there are no neutral experts. If neutrality is taken to mean many state equivalents, courts have had the freedom from all preconceptions, it is true that should pay for their services? How should power to appoint experts of their own selec- there are few if any neutral experts: anyone they be instructed about conflicts of interest? tion. Used in a number of asbestos cases in competent to the task of a court-appointed sci- We could learn a lot from Judge Pointer’s 1987 and 1990,34 the practice came to pub- entist is virtually certain to have some view at experience—and (if we are careful to avoid lic attention in the late 1990s in the context the outset. And if neutrality is taken to mean the pitfalls of facile crosscultural compar- of a wave of lawsuits against the manufactur- freedom from all contact, direct or indirect, isons) from the experience of other legal sys- ers of silicone breast implants, when it was with either party, again there probably won’t tems—about how and when court-appointed adopted by Judge Jones in Hall ,35 and most be many neutral scientists; for, given the de- pendence of much medical research on drug Such experts are potentially very useful in 1996 appointed a National Science Panel to company funding,41 most scientists competent some kinds of cases; but of course they are help him sift through the scientific evidence to the task will probably know people in- no panacea—in fact, I don’t suppose for a mo- in the several thousand federal silicone breast volved with one party or the other. But it ment that there is a panacea. Rather, there is implant cases that had been consolidated to doesn’t follow, and it isn’t true, that some ex- a range of possibilities worth pursuing. Think- his court. And it seems that, as their gate- perts aren’t, in the essential sense, more neu- ing about the unhappy interaction of the FDA keeping responsibilities have grown, more tral, less biased than others: that is, more will- and the tort system in the silicone breast im- judges have been willing, as Justice Breyer ing to go where the evidence leads, even if it plants affair, for example, you might wonder urged in Joiner, to call directly on the scien- pulls against what they were initially inclined how the FDA could have acted to prevent the tific community for help36: court-appointed panic in the first place43; thinking about the experts have advised judges on the potential Bias, in the sense at issue here, is not the willingness of the American Association for dangers of seatbelt buckles, the diet drug fen- same as conflict of interest; nevertheless, we phen, and the antilactation drug Parlodel; certainly want to avoid conflicts of interest, and, in the Court of Appeals in Michigan, on both because they may lead to bias in the relevant sense, and because, even if they when legal disputes turn on scientific issues don’t, we want to avoid the appearance of irresoluble by the presently available evi- Appointed Scientific Experts Project makes such bias. But we should be conscious that dence; thinking of the weaknesses of other available “independent scientists . . . [to] edu- techniques of forensic identification, and the cate the court, testify at trial, assess the liti- appointed scientist’s being financially sup- mistakes made by crime labs, etc., revealed in gants’ cases, and otherwise aid in the process ported in some way by a defendant company of determining the truth”; Duke University’s or plaintiffs’ attorneys, to his discussing his tions, one might wonder how we could make Registry of Independent Scientific and Tech- court-appointed work with an acquaintance the forensic science business more rigorous Supplement 1, 2005, Vol 95, No. S1 | American Journal of Public Health Haack | Peer Reviewed | Public Health Matters | S71 (the temptation to say “more scientific” is Century. London, UK: Allen & Unwin; 1957; reprinted criteria of cognitive significance: problems and changes.
under the title Science: conjectures and refutations in Adapted from two papers originally published in 1950 Popper KR. Conjectures and Refutations: The Growth of and 1951. In: Hempel, Aspects of Scientific Explanation Scientific Knowledge. New York, NY: Basic Books; 1962: and Other Essays in the Philosophy of Science. New York, 33–69; and in part, under the title Falsificationism, in NY: Free Press; 1965: 101–120, with the addition of Justice requires not only just laws, and just Klee R, ed. Scientific Inquiry: Readings in the Philosophy Postscript (1964) on cognitive significance, 120–122.
of Science. Oxford, UK: Oxford University Press; 1999: administration of those laws, but also factual 19. Hempel CG. Postscript (1964) on confirmation. In: 65–71; see also Popper KR. The problem of demarca- Hempel CG. Aspects of Scientific Explanation and Other truths—which, increasingly often, courts must tion (1974) in: Miller D, ed. A Pocket Popper. London, Essays in the Philosophy of Science. New York, NY: Free rely on science to discover. As Learned Hand once put it, “No one will deny that the law Popper KR. The Logic of Scientific Discovery, Lon- 20. Hempel CG. The irrelevance of truth for the criti- don, UK: Hutchinson; 1959: section 83.
should in some way effectively use expert cal appraisal of scientific theories (1990). Reprinted in: Popper KR. Objective Knowledge: An Evolutionary Jeffrey R, ed. Selected Philosophical Essays [by] Carl G. knowledge wherever it will aid in settling dis- Approach. Oxford, UK: Oxford University Press; Hempel. Cambridge, UK: Cambridge University Press; putes. The only question is as to how it can do 2000: 75–84. Kuhn T. The Structure of Scientific Revo- so best.”44 Now, more than a century after lutions. Chicago, Ill: University of Chicago Press; 1962.
10. Popper KR. Objective Knowledge: An Evolutionary Hand posed the essential question, and more Approach. Oxford, UK: Oxford University Press; 21. Einstein A. Physics and reality. Journal of theFranklin Institute, 221, No. 3 (1936). Reprinted in: than a decade after Daubert, we are still fum- Bargmann S. Ideas and Opinions of Albert Einstein. New 11. Daubert v Merrell Dow Pharmaceuticals, Inc, 509 York, NY: Crown Publishers; 1954: 290–323, 290.
US 579,600; 113 SCt 2786,2800 (1993). Some fed-eral judges evidently understand falsifiability better 22. For a detailed development of the conception of than others. In US v Havvard, 117 FSupp2d 848,854 scientific method on which I have relied here, see (2000), admitting fingerprint identification testimony, Haack S. Defending Science—Within Reason: Between The author is Professor of Law and of Philosophy at the Judge Hamilton observes that “the methods of latent Scientism and Cynicism. Amherst, NY: Prometheus; print identification . . . have been tested . . . for roughly Requests for reprints should be sent to Susan Haack, PhD, 100 years . . . in adversarial proceedings.” But in Llera- 23. Bergmann G. Philosophy of Science. Madison, School of Law, University of Miami, 1311 Miller Drive, Plaza I, 2002 WL 27305 (ED Pa 2002), 27310 im- Wis: University of Wisconsin Press; 1957: 20.
Coral Gables, FL 33124 (e-mail: [email protected]). posing restrictions on fingerprint identification testi- This article was accepted July 27, 2004. 24. Ramirez v State, 542 So2d 352 (Fla 1989); mony, Judge Pollak points out that “ ‘adversarial’ testing Ramirez v State, 651 So2d 1164 (Fla 1995); Ramirez v in court is not . . . what the Supreme Court meant State, 8120 So2d 836 (Fla 2001). Florida remains offi- when it discussed testing as an admissibility factor.” cially a Frye state, but it seems to be rapidly evolving (Shortly thereafter, Judge Pollak reconsidered and re- This work was supported in part by the Project on Sci- in the direction of (as Michael Saks puts it) “Fryebert.” vised his ruling, but on grounds unrelated to the point entific Knowledge and Public Policy.
25. Darwin F, ed. Charles Darwin: Autobiography and The author is grateful to Mark Migotti for helpful Letters. New York, NY: D. Appleton and Company; 12. See Popper KR. Natural selection and its scientific 1893:45. Reprinted New York, NY: Dover; 1952.
status; reprinted from sections 1 and 2 of a lecture of1977 in: Miller D, ed. A Pocket Popper. London, UK: 26. See Thomas E, Hosenball M, Isikoff M. The JFK– Marilyn hoax. Newsweek. June 6, 1997:36–37.
Bridgman P. On “scientific method.” Originally 13. McLean v Arkansas Board of Education, 529 27. General Electric Co v Joiner, 522 US 136, 118 SCt published in 1949; reprinted in: Bridgman P. Reflec- FSupp 1255 (1982). Judge Overton’s ruling and Ruse’s 512 (1997); Kumho Tire Co v Carmichael, 526 US 137, tions of a Physicist. New York, NY: Philosophical Li- testimony, along with Larry Laudan’s properly scathing critique, can be found in Ruse M, ed. But Is It Science? 28. In re: Paoli R.R. Yard PCB Litig., 35 F3d 717 (3d Frye v United States, 54 App DC 46, 293 F The Philosophical Question in the Creation/Evolution Controversy. Amherst, NY: Prometheus; 1996.
29. The term “accordion concept” was introduced in Barefoot v Estelle, 463 US 880,898, 103 SCt 14. Popper KR. The Logic of Scientific Discovery. Lon- Sellars W. Scientific realism or irenic instrumentalism? 3383,3397 (1983). Barefoot was executed in 1984.
don, UK: Hutchinson, 1959: 251–252, Note number In: Cohen R, Wartofsky M, eds. Boston Studies in thePhilosophy of Science. Vol 2. New York, NY: Humanities Daubert v Merrell Dow Pharmaceuticals, Inc, 509 *1, added in the English edition. When Popper uses “confirm” for “corroborate,” as he does in his Philoso-phy of science: a personal report (1957), the effect is 30. The word “consilience,” meaning etymologically The Daubert Court did not itself scrutinize the dis- “jumping together,” was coined by the 19th century puted testimony; on remand, Judge Kozinski again ex- philosopher of science William Whewell, and recently cluded the plaintiffs’ proffered experts, this time under 15. I am being deliberately noncommittal about made famous as the title of a bestselling book by E.O.
Daubert rather than Frye. Because of litigation costs, whether this really is a mistranslation. Pons’ Global- Wilson, Consilience: The Unity of Knowledge. New York, Merrell Dow had already taken Bendectin off the mar- wörterbuch Deutsch-Englisch (1983), explains Be- NY: Knopf, 1998. The phrase “faggot fallacy,” adopted ket in 1984. In 2000, the FDA again declared the wärhung as “proving one’s/its worth”; a secondary by General Electric’s attorneys in Joiner, was intro- duced in Follies and Fallacies in Medicine by P. Skra- Popper KR. Logik der Forschung. Vienna, Austria: 16. Popper KR. Objective Knowledge: An Evolutionary banek and J. McCormick. Originally published in 1989; Julius Springer, 1934. English edition, The Logic of Sci- Approach. Oxford, UK: Oxford University Press; 1972: reprinted Amherst, NY: Prometheus Books, 1997.
entific Discovery. London, UK: Hutchinson, 1959. The 31. I first introduced the analogy in Haack S. Rebuild- observation that his criterion of demarcation is a con- 17. Popper KR. Objective Knowledge: An Evolutionary ing the ship while sailing on the water. In: Gibson R, vention, found in the original German edition, appears Approach. Oxford, UK: Oxford University Press; 1972: Barrett R, eds. Perspectives on Quine. Oxford, UK: on page 37 of The Logic of Scientific Discovery; the ob- 20; the reference to Hempel is in footnote 29.
Blackwell; 1990:111–128. It is articulated in more de- servation that science is continuous with commonsense 18. Hempel CG. Studies in the logic of confirmation.
tail in Haack S. Evidence and Inquiry: Towards Recon- knowledge appears only in the new Preface added to Mind. 1945; 54:1–26, 97–121; reprinted in Hempel struction in Epistemology. Oxford, UK: Blackwell; 1993: CG. Aspects of Scientific Explanation and Other Essays in 73–94; and developed further in Haack S. Defending See Popper KR. Philosophy of science: a personal the Philosophy of Science. New York, NY: Free Press; Science – Within Reason. Amherst, NY: Prometheus; report. In: Mace CA, ed. British Philosophy in Mid- 1965: 3–46, 43–45. See also Hempel CG. Empirist S72 | Public Health Matters | Peer Reviewed | Haack American Journal of Public Health | Supplement 1, 2005, Vol 95, No. S1 32. As one character says to another in a cartoon forwhich I have a particular fondness, “Politically, I supposeyou could say I’m a member of the lunatic middle.” 33. Goldberg C. Judges’ unanimous verdict on DNAlessons: Wow! New York Times. April 24, 1999:A10.
34. See Rubin CR, Ringenback L. The Use of CourtExperts in Asbestos Litigation, 137 FRD 35 (1991).
35. Hall v Baxter Healthcare Corp, 947 FSupp 1387(D Ore 1996).
36. See Erichson HM. Mass tort litigation and inquisi-torial justice. Geo Law J. 1999;87:1983–2024.
37. DePyper et al. v Paul V. Navarro, No. 19149, 1998WL 1988927 (Mich App. Nov. 6, 1998); Denial of Ex-pert Witness Testimony Violates Daubert, Appeal States.
DES Litig Rep., Dec. 1998; 38. Howard MN. The neutral expert: a plausiblethreat to justice. Crim Law Rev. 1991; 98–105; citedby: Van Kampen P. Expert evidence compared. In:Malsch M, Nijboer JF, eds. Complex Cases: Perspectiveson the Netherlands Criminal Justice System. Amsterdam,The Netherlands: Thela Thesis; 1999.
39. Only 4.4% of federal criminal cases end in a juryverdict, and only 1.4% of federal civil cases are re-solved by juries. Glaberson W. Juries, their powersunder siege, find their role is being eroded. New YorkTimes. March 2, 2001: A1.
40. I have written at greater length about tensionsbetween science and the culture of the law in “Inquiryand advocacy, fallibilism and finality: culture and infer-ence in science and the law,” Law, Probability and Risk.
2003; 2:205–214.
41. See Angell M. Is academic medicine for sale?N Engl J Med. 2000; 342(20):1516–1518. 42. See, for example, Cecil JS, Hooper LJ, WillgingTE. Assessing causation in breast implant litigation: therole of science panels. Law Contemp Prob. 2001;64:139–189. Monahan J, Walker L. Scientific author-ity: the breast implant litigation and beyond. Va LawRev. 2002;86:801–833.
43. The wave of litigation began after the FDAbanned silicone breast implants, formerly “grandfa-thered in”; they were not known to be unsafe, but themanufacturers had failed to submit evidence of theirsafety, as they had been required to do.
44. Learned Hand. Historical and practical considera-tions regarding expert testimony. Harv Law Rev. 1901;15:40–58, 40 (my emphasis).
Supplement 1, 2005, Vol 95, No. S1 | American Journal of Public Health Haack | Peer Reviewed | Public Health Matters | S73

Source: http://defendingscience.org/sites/default/files/upload/HaackSCPHILOSOPHY.pdf