Scientific Evidence

1
Scientific Evidence

• Paul Giannelli
• Weatherhead Professor of Law
• Case Western Reserve University

2
Developments in the 1990s

• DNA Litigation
• Daubert v. Merrell Dow Pharmaceuticals
• Supreme Courts junk science decision
• Abuse Cases
• W. Virginia, Oklahoma City, FBI

3
DNA Admissibility Wars

• From university science, not forensic science
• Science culture
• written protocols
• quality assurance/quality control
• proficiency testing
• Open science vs. adversarial science

4
DNA Exonerations

• Mistaken eyewitnesses 84
• Police misconduct 50
• Prosecutorial misconduct 42
• Tainted or fraudulent science 33
• Ineffective defense counsel 27
• False confessions 24
• Jailhouse snitches 21
• Scheck et al., Actual Innocence 246 (2000) (62
  cases)

5
Abuse Cases

• In re W.Va. State Police Crime Lab., Serology
  Div. , 438 S.E. 501 (W. Va. 1993) (Fred Zain)
  (perjured testimony, false lab reports)
• Mitchell v. Gibson, 262 F.3d 1036, 1044 (10th
  Cir. 2001) (Ms. Gilchrist thus provided the jury
  with evidence implicating Mr. Mitchell in the
  sexual assault of the victim which she knew was
  rendered false and misleading by evidence
  withheld from the defense.)

6
Daubert Trilogy

• Daubert v. Merrell Dow Pharm., Inc.
• 509 U.S. 579 (1993)
• establishes reliability test rejects Frye
  general acceptance test
• General Elec. Co. v. Joiner
• 522 U.S. 136 (1997)
• appellate review of Daubert issues abuse of
  discretion
• Kumho Tire Co. v. Carmichael
• 526 U.S. 137 (1999)
• Daubert applies to technical evidence i.e.,
  all experts

7
Daubert Factors

• (1) Testing (falsifiability)
• (2) Peer review publication
• (3) Known or potential error rate
• (4) Standards controlling use of technique
• (5) General acceptance (from Frye test)

8
Federal Evidence Rule 702

• If scientific, technical, or other specialized
  knowledge will assist the trier of fact jury to
  understand the evidence or to determine a fact in
  issue, a witness qualified as an expert by
  knowledge, skill, experience, training, or
  education, may testify thereto in the form of an
  opinion or otherwise . . .

9
Rule 702 Amendment (2000)

• if (1) the testimony is based on sufficient
  facts or data,
• (2) the testimony is the product of reliable
  principles and methods, and
• (3) the witness has applied the principles and
  methods reliably to the facts of the case.

10
Rule 702 Requirements

• (1) Subject matter requirement Is this topic a
  proper subject for expert testimony?
• (2) Qualifications requirement Is this witness
  qualified in this subject matter?

11
Subject Matter Requirement
12
Subject Matter Tests
13
Daubert Initial Reviews

• Astonishingly, all parties expressed
  satisfaction with the Daubert decision the
  lawyers for the plaintiff and defense, and
  scientists who wrote amicus briefs.
• Foster et al., Policy Forum Science and the
  Toxic Tort, 261 Science 1509, 1614 (Sept. 17,
  1993)

14
Comparison of Tests (1993)
15
Daubert Liberal v. Strict

• Given the Rules permissive backdrop and their
  inclusion of a specific rule on expert testimony
  that does not mention general acceptance, the
  assertion that the Rules somehow assimilated Frye
  is unconvincing. Frye made general acceptance
  the exclusive test for admitting expert
  scientific testimony. That austere standard,
  absent from, and incompatible with the Federal
  Rules of Evidence, should not be applied in
  federal trials. 509 U.S. at 589.

16
Daubert continued

• The Rules basic standard of relevance ... is a
  liberal one. Id. at 587.
• A rigid general acceptance requirement
  would be at odds with the liberal thrust of the
  Federal Rules and their general approach of
  relaxing the traditional barriers to opinion
  testimony. Id. at 588.

17
But Gatekeeper role

• In order to qualify as scientific
  knowledge, an inference or assertion must be
  derived by the scientific method. Proposed
  testimony must be supported by appropriate
  validation i.e., good grounds, based on what
  is known. In short, the requirement that an
  experts testimony pertain to scientific
  knowledge establishes a standard of evidentiary
  reliability. Id. at 588.

18
United States v. Bonds

• DNA admitted at trial under Frye test
• We find that the DNA testimony easily meets the
  more liberal test set out by the Supreme Court in
  Daubert.
• 12 F.3d 540, 568 (6th Cir. 1993)

19
Borawick v. Shay

• Repressed memory evidence
• by loosening the strictures on scientific
  evidence set by Frye, Daubert reinforces the idea
  that there should be a presumption of
  admissibility of evidence
• 68 F.3d 597, 610 (2d Cir. 1995)

20
Later Supreme Court Cases

• Joiner (1997)
• Daubert somewhat broader than Frye
• Kumho (1999)
• Daubert extends to nonscientific evidence
• Wisegram v. Marley Co., 528 U.S. 440 (2000)
• Daubert sets an exacting standard

21
U.S. v. Horn

• Under Daubert, ... it was expected that it would
  be easier to admit evidence that was the product
  of new science or technology. In practice,
  however, it often seems as though the opposite
  has occurred application of Daubert/Kumho Tire
  analysis results in the exclusion of evidence
  that might otherwise have been admitted under
  Frye.
• 185 F. Supp. 2d 530 (D. Md. 2002) (HGN)

22
Admissibility Challenges

• Supreme Court in Daubert and Kumho is plainly
  inviting a reexamination even of generally
  accepted venerable, technical fields.
• U.S. v. Hines, 55 F. Supp. 2d 62, 67 (D. Mass.
  1999)
• Courts are now confronting challenges to
  testimony whose admissibility had long been
  settled.
• U.S. v. Hidalgo, 229 F. Supp. 2d 961, 966 (D.
  Ariz. 2002)

23
Civil Cases

• In the Daubert case ... the Supreme Court
  rejected the deferential standard of the Frye
  Rule in favor of a more assertive standard that
  required courts to determine that expert
  testimony was well grounded in the methods and
  procedures of science.
• Kassierer Cecil, Inconsistency in Evidentiary
  Standards for Medical Testimony Disorder in the
  Courts, 288 J. Am. Med. Assn 1382, 1383 (2002)

24
Rand Institute Civil Cases

• Since Daubert, judges have examined the
  reliability of expert evidence more closely and
  have found more evidence unreliable as a result.
• Dixon Gill, Changes in the Standards of
  Admitting Expert Evidence in Federal Civil Cases
  Since the Daubert Decision, 8 Psychol., Pub.
  Poly L. 251 (2002)

25
Study of Criminal Cases

• Daubert decision did not impact on the admission
  rates of expert testimony at either the trial or
  appellate court levels.
• Groscup et al., The Effects of Daubert on the
  Admissibility of Expert Testimony in State and
  Federal Criminal Cases, 8 Pyschol., Pub. Poly
  L. 339, 364 (2002)

26
Forensic Community

• The Daubert Standard goes a step further than
  Frye and requires the forensic scientists to
  prove that the evidence is fundamentally
  scientifically reliable, not just generally
  accepted by his/her peers in the discipline.
• Jones, Presidents Editorial The Changing
  Practice of Forensic Science, 47 J. Forensic Sci.
  437, 437 (2002)

27
Comparison of Tests (2006)

• No reliability test
• E.g., Relevancy test

• Reliability tests
• E.g., Frye general acceptance test
• E.g., Daubert test
• E.g., Other reliability tests

28
Daubert in the States

• Frye jurisdictions Cal., N.Y., Fla., Ill., Pa.,
  Md.
• Daubert jurisdictions
• But not necessarily Joiner Kumho
• Relevancy test e.g., Wisconsin
• Other reliability tests e.g., N.C.

29
Strict v. Lax Approaches

• The choice is not between easy Frye and
  difficult Daubert it is between strict and lax
  scrutiny.
• Redmayne, Expert Evidence and Criminal Justice
  113 (2001)

30
Daubert Strict v. Lax

• U.S. v. Crisp, 324 F.3d 261 (4th Cir. 2003)
• Admitting handwriting comparison (lax)
• Admitting fingerprint identification (lax)
• The government has had ten years to comply with
  Daubert. It should not be given a pass in this
  case. (strict)
• Id. at 272 (Michael, J., dissenting)

31
Lee v. Martinez (lax Daubert)

• Admitting polygraph evidence under Daubert
• This liberal approach Daubert to the admission
  of evidence is consistent with the intent of the
  drafters of the Federal Rules of Evidence.
• 96 P.3d 291, 297 (N.M. 2004)

32
Ramirez v. State (strict Frye)

• In order to preserve the integrity of the
  criminal justice system in Florida, particularly
  in the face of rising nationwide criticism of
  forensic evidence in general, our state courts
  must apply the Frye test in a prudent manner to
  cull scientific fiction and junk science from
  fact. Any doubt as to admissibility should be
  resolved in a manner that minimizes the chance of
  a wrongful conviction, especially in a capital
  case. 810 So. 2d 836, 853 (Fla. 2001)

33
People v. Davis (lax Frye)

• Admitting lip print evidence under Frye
• QD expert testified that lip print comparison is
  an accepted method of scientific identification
  in the forensic science community . . . He is
  unaware of any dissent in the field regarding the
  methodology used to make a positive
  identification of a lip print.
• 710 N.E.2d 1251 (Ill. Ct. App. 1999)

34
Hair Comparisons

• This court has been unsuccessful in its
  attempts to locate any indication that expert
  hair comparison testimony meets any of the
  requirements of Daubert.
• Williamson v. Reynolds, 904 F. Supp. 1529, 1558
  (E.D. Okl. 1995) revd on this issue, Williamson
  v. Ward, 110 F.3d 1508, 1522-23 (10th Cir. 1997)
  (due process, not Daubert, standard applies in
  habeas proceedings)

35
Hair Comparison (cont.)

• Most courts still admit this evidence
• DNA evidence compared Microscopic analysis
  wrong 12 of time
• Houch Budowle, Correlation of Microscopic and
  Mitochondrial DNA Hair Comparisons, 47 J.
  Forensic Sci. 964 (2002)

36
Handwriting Comparisons

• Because the principle of uniqueness is without
  empirical support, we conclude that a document
  examiner will not be permitted to testify that
  the maker of a known document is the maker of the
  questioned document. Nor will a document
  examiner be able to testify as to identity in
  terms of probabilities.
• U.S. v. Hidalgo, 229 F. Supp. 2d 961, 967 (D.
  Ariz. 2002)

37
Handwriting (cont.)

• U.S. v. Prime, 363 F.3d 1028, 1033 (9th Cir.
  2004) (admitting)
• U.S. v. Crisp, 324 F.3d 261 (4th Cir. 2003) (same)

38
Fingerprints

• U.S. v. Llera Plaza, 188 F. Supp. 2d 549, 558
  (E.D. Pa. 2002) (excluding and then admitting)
• U.S. v. Mitchell, 365 F.3d 215, 247 (3d Cir.
  2004) (admitting)
• U.S. v. Abreu, 406 F.3d 1304 (11th Cir. 2005)
  (same)

39
U.S. v. Havvard

• Error rate is zero. ???
• Peer review is a second examiner reviewing the
  analysis. ???
• Adversarial testing scientific testing ???
• 117 F. Supp. 2d 848 (S.D. Ind. 2000)

40
Fingerprints Stephan Cowans

• Released after serving 6 years (Massachusetts)
  for nonfatal shooting of a police officer. First
  conviction overturned on DNA evidence in which
  fingerprint evidence was crucial in securing the
  wrongful conviction.
• Loftus Cole, Contaminated Evidence, 304 Science
  673, 959, May 14, 2004

41
Riki Jackson

• Convicted of murder in 1997 based on bloody
  fingerprints discovered on a window fan.
• 2 defense experts, retired FBI examiners,
  testified that there was no match.
• McRoberts et al., Forensics Under the Microscope
  Unproven Techniques Sway Courts, Erode Justice,
  Chi. Trib., Oct. 17, 2004

42
Brandon Mayfield

• Although F.B.I. found fingerprint match, Spanish
  officials matched the fingerprints to an Algerian
  national.
• Kershaw, Spain and U.S. at Odds on Mistaken
  Terror Arrest, N.Y. Times, Jun. 5, 2004 at A1

43
FBI Report (2004)

• Dissimilarities were easily observed when a
  detailed analysis of the latent print was
  conducted.
• inherent pressure of high-profile case
• confirmation bias

44
FBI Report (cont.)

• To disagree was not an expected response.
• Verifiers should be given challenging exclusions
  during blind proficiency tests to ensure that
  they are independently applying ACE-V methodology
  correctly
• Stacey, A Report on the Erroneous Fingerprint
  Individualization in the Madrid Train Bombing
  Case, 54 J. Forensic Identification 707 (2004)

45
Inspector General Report

• A significant cause of the misidentification was
  reasoning backward from features that were
  visible in the known prints of Mayfield.
• Having found as many as 10 points of unusual
  similarity, the FBI examiners began to find
  additional features in LFP 17 that were not
  really there, but rather were suggested to the
  examiners by features in the Mayfield prints.

46
I.G. Report (cont.)

• FBI examiners did not attempt to determine the
  basis of the Spanish National Polices doubts
  before reiterating that they were absolutely
  confident in the identification on April 15, a
  full week before the FBI Laboratory met with the
  SNP.

47
Fingerprint Mistakes

• Cole, More Than Zero Accounting for Error in
  Latent Fingerprint Identification, 95 J. Crim. L.
  Criminology 985 (2005) (documenting 23 cases of
  misidentifications)

48
The Experiment

• 5 fingerprint examiners asked to review their
  prior cases, believing them to be the Mayfield
  prints
• 1 still judged the print to be a match
• 3 directly contradicted their prior
  identifications
• 1 concluded insufficient data
• Itiel E. Dror et el., Contextual Information
  Renders Experts Vulnerable to Making Erroneous
  Identifications, 156 Forensic Sci. Intl 74
  (2006).

49
FBI Review

• 2 approaches
• quantifiable minimum threshold
• black box
• to be truly blind, the second examiner should
  have no knowledge of the interpretation by the
  first examiner (to include not seeing notes or
  reports
• Budowle et al., Review of the Scientific Basis
  for Friction Ridge Comparisons as a Means of
  Identification Committee Findings and
  Recommendations, 8 Forensic Sci. Communications
  (Jan. 2006)

50
Simultaneous Impressions

• Application of ACE-V to simultaneous
  impressions cannot rely on the more usual
  application of ACE-V for its admissibility, but
  must be independently tested .
• On the record before the motion judge, the
  Commonwealth has not yet established that the
  application of the ACE-V method to simultaneous
  impressions is generally accepted by the
  fingerprint examiner community .
• Commonwealth v. Patterson, 840 N.E.2d 12 (Mass.
  2005)

51
Firearms IdentificationAdmitting Evidence

• U.S. v. Hicks, 389 F.3d 514 (5th Cir. 2004)
• U.S. v. Foster, 300 F. Supp. 2d 375 (D. Md. 2004)
  
• But see Schwartz, A Systemic Challenge to the
  Reliability and Admissibility of Firearms and
  Toolmark Identification, 6 Colum. Science Tech.
  L. Rev. (2005)

52
Cartridge Case Ident. (cont.)

• Inadmissible because failed to follow standards
• No documentation - sketches or photo
• No technical review by 2d examiner
• U.S. v. Monteiro, 407 F. Supp. 2d 351 (D. Mass.
  2006)

53
Cartridge Case Ident. (cont.)

• OShea declared that this match could be made
  to the exclusion of every other firearm in the
  world. . . . That conclusion, needless to say,
  is extraordinary, particularly given OSheas
  data and methods.
• Admitting similarities, but not conclusion
• U.S. v. Green, 405 F. Supp. 2d 104 (D. Mass.
  2005)

54
Cartridge Case Ident. (cont.)

• I reluctantly come to the above conclusion
  because of my confidence that any other decision
  will be rejected by appellate courts, in light of
  precedents across the country . . . While I
  recognize that the Daubert-Kumho standard does
  not require the illusory perfection of a
  television show (CSI, this wasn't), when liberty
  hangs in the balanceand, in the case of the
  defendants facing the death penalty, life
  itselfthe standards should be higher than were
  met in this case, and than have been imposed
  across the country. The more courts admit this
  type of toolmark evidence without requiring
  documentation, proficiency testing, or evidence
  of reliability, the more sloppy practices will
  endure we should require more. U.S. v. Green,
  supra

55
Gunshot Residue Tests

• Baltimore PD used 2 (instead of 3) elements
• Also, contamination problems
• Bykowicz, Lawyers Call City Analysis of Gunshot
  Residue Flawed, Baltimore Sun, Mar. 5, 2005
• Nethercott Thompson, Lessons from Baltimores
  GSR Debacle, The Champion 36 (June 2005)

56
GSR (cont.)

• Bykowicz, FBI Lab Scraps Gunfire Residue Agency
  Wont Do Analysis, Putting Evidence in Doubt,
  Baltimore Sun, May 26, 2006 (The resulting FBI
  contamination study . . . documents the presence
  of hundreds of particles consistent with gunshot
  resulted in several areas of the lab.)

57
GSR (cont.)

• Judge Hall This court is not convinced that
  the relevant scientific community has a generally
  accepted standard for interpreting what
  conclusions can be drawn from GSR testing and
  analysis. . . . It is clear that significant
  questions exist . . . Concerning how many
  particles are required for there to be a positive
  test.)
• Orrick, Anoka Judge Rejects Gunshot Residue
  Evidence, St. Paul Pioneer Press, July 13, 2006
  (quoting

58
Bullet Lead Comparison

• Chemically indistinguishable
• Could have come from the same box.
• State v. Earhart, 823 S.W.2d 607(Tex. Crim. App.
  1991)
• Melt can range from the equivalent of as few as
  12,000 to as many as 35 million 40grain, .22
  caliber longrifle bullets)
• National Research Council, Forensic Analysis
  Weighing Bullet Lead Evidence (2004)

59
Bullet Lead (cont.)

• State v. Behn, 868 A.2d 329 (N.J. Super. A.D.
  2005)
• based on erroneous scientific foundations
• Clemons v. State, 896 A.2d 1059 (Md. 2006)
• Inadmissible under Frye
• Ragland v. Commonwealth,191 S.W.3d 569 (Ky. 2006)
• Inadmissible under Daubert

60
Bitemark Comparison

• Despite the continued acceptance of bitemark
  evidence in European, Oceanic and North American
  Courts, the fundamental scientific basis for
  bitemark analysis has never been established.
• Pretty Sweet, The Scientific Basis for Human
  Bitemark Analyses A Critical Review, 41 Sci.
  Just. 85, 86 (2001)

61
Bitemark (cont.)

• State v. Krone, 897 P.2d 621 (Ariz. 1995) (The
  bite marks were crucial to the States case
  because there was very little other evidence to
  suggest Krones guilt.)
• Krone exonerated through DNA profiling
• Hansen, The Uncertain Science of Evidence, ABA J.
  49 (July 2005)

62
Bitemark (cont.)

• Expert opined that upon finding five unique
  points of identity between a bite mark and the
  suspect's teeth, the chances of someone else
  having made the mark would be 4.1 billion to one.
  Mr. Otero was subsequently exonerated when DNA
  did not match.
• Ege v. Yukins, 380 F. Supp. 2d 852, 871 (E.D.
  Mich. 2005)

63
Bitemark (cont.)

• Expert concluded that Burke's teeth matched the
  bite mark on the victim's left breast to a
  reasonable degree of scientific certainty.
  DNA analysis showed that Burke was excluded as
  the source of male DNA found in the bite mark on
  the victim's left breast.
• Burke v. Town Of Walpole, 405 F.3d 66, 73 (1st
  Cir. 2005)

64
Forensic Science Oxymoron?

• Donald Kennedy, Editor-in-Chief, Editorial,
  Forensic Science Oxymoron?, 302 Science 1625
  (2003) (discussing the cancellation of a National
  Academy of Sciences project designed to examine
  various forensic science techniques because the
  Departments of Justice and Defense insisted on a
  right of review that the Academy has refused to
  other grant sponsors)

65
Houston Crime Lab

• Prosecutors in Mr. Suttons case had used
  DNA to convict him, submitting false scientific
  evidence asserting that there was a solid match
  between Mr. Suttons DNA and that found at the
  crime scene. In fact, 1 of every 8 black people,
  including Mr. Sutton, shared the relevant DNA
  profile. More refined retesting cleared him.
• Liptak Blumenthal, New Doubt Cast on Crime
  Testing in Houston Cases, N.Y. Times, Aug. 5, 2004

66
Independent Report (2005)

• Four instances of dry labbing in drug section
  (reporting results without doing tests)
• These instances were well known within the lab.
• One of the 2 examiners involved was still
  employed, having been reinstated by the police
  chief after being removed from his position by
  the lab.

67
DNA Unit

• Technical leader in the DNA unit had been
  removed and never replaced, leaving a 6-year
  vacancy in an important supervisory position.
• To compound matters, he was reassigned to head
  the quality control program, a critical task that
  even he admitted he did in a lackluster fashion.

68
Backlog

• March 2002 estimate that 19,500 sexual assault
  kits received by HPD that had never been
  processed, some dating as far back as 1980.
• Only known suspect cases tested i.e., cold
  cases not entered into DNA databases.
• Lynn Jones, arrested for sexual assault of a
  child, remained in jail for 9 months awaiting
  results.

69
Leaky Roof

• 34 homicide sexual assault cases badly water
  damaged by tropical storm in 2001 due to leaking
  roof.
• In 2003, Crime Lab Employees told I.A.
  investigators that this biological evidence had
  become so saturated with water that they observed
  bloody water dripping out the boxes and pooling
  on the floor.

70
Regulation of Crime Labs

• Accreditation of labs
• E.g., New York, Oklahoma, Texas, Virginia
• Justice for All Act
• Requires states to have an investigative entity
• DNA Identification Act
• Requires accreditation of DNA labs within 2 years

71
ABA Innocence Policies

• 1. Crime laboratories and medical examiner
  offices should be accredited, examiners should be
  certified, and procedures should be standardized
  and published to ensure the validity,
  reliability, and timely analysis of forensic
  evidence.
• 2. Crime laboratories and medical examiner
  offices should be adequately funded.
• Report of the ABA Criminal Justice Sections Ad
  Hoc Innocence Committee to Ensure the Integrity
  of the Criminal Process, Achieving Justice
  Freeing the Innocent, Convicting the Guilty
  (Giannellli Raeder eds. 2006)

72
ABA Defense Experts

• 3. The appointment of defense experts for
  indigent defendants should be required whenever
  reasonably necessary to the defense.
• Ake v. Oklahoma, 470 U.S. 68 (1985)
• due process right to expert for indigents
• Giannelli, Ake v. Oklahoma The Right to Expert
  Assistance in a Post-Daubert, Post-DNA World, 89
  Cornell L. Rev. 1305 (2004)

73
ABA DNA Standards

• Collection, preservation retention
• Pretrial disclosure
• Defense testing retesting
• Admissibility of DNA evidence
• Post-conviction testing
• Charging persons by DNA profile
• DNA databases

74
Part III Testing of DNA Evidence

• Standard 3.1 Testing laboratories
• 3.2 Testing interpretation of DNA evidence
• 3.3 Laboratory reports
• 3.4 Consumptive testing

75
DNA Standards (cont.)

• Accreditation every two years.
• Written policies, including protocols for testing
  and interpreting test results.
• Quality assurance procedures, including
• audits,
• proficiency testing, and
• corrective action protocols.
• Follow procedures designed to minimize cognitive
  bias when interpreting test results.

76
DNA Standards (cont.)

• Accreditation every two years.
• Written policies, including protocols for testing
  and interpreting test results.
• Quality assurance procedures, including
• audits,
• proficiency testing, and
• corrective action protocols.
• Follow procedures designed to minimize cognitive
  bias when interpreting test results.

77
Conclusion

• To put the point more bluntly if the state
  does not test the scientific evidence with which
  it seeks to convict defendants, it should forfeit
  the right to use it.
• Redmayne, Expert Evidence and Criminal Justice
  139 (2001)