Report of the Auditability Working Group

1
Report of the Auditability Working Group

• David Flater
• National Institute of Standards and Technology
• http//vote.nist.gov

2
Outline

• Presentation
• Charge to the working group
• The goal of software independence (SI)
• What was actually required in the 2007 TGDC draft
• Alternatives to SI and their consequences
• Paper, voter-verification, and accessibility
• Effectivity concerns
• 3 options
• Debate
• TGDC and EAC discussion
• Resolutions (choose an option)

3
Charge to the working group

• Alternatives to Software Independence (SI) EAC
  directs the TGDC to develop draft requirements
  for audit methods to achieve the goal of Software
  Independence (SI). The goal is to develop
  requirements for the auditability of the election
  system without requiring a specific technology.
  The starting point for these requirements should
  be the work already completed by NIST on
  alternatives to SI.

4
The SI rationale (abridged)

• The following is not the entire SI rationale, but
  it is the acid test that distinguishes SI from
  other forms of auditability
• Accept as plausible that there could be one rogue
  or coerced software engineer in each independent
  supplier of voting equipment to the jurisdiction
• Alternately, that each supplier relies on
  insecure COTS software that a third party can
  exploit, or that common mode failures exist, or
  etc.
• All electronic records potentially compromised
• If there are no other records, then it is not
  possible to compare records to audit the result
• The goal of SI, as abridged mitigate this
  threat (and others that are easier)

5
Mitigation independent voter-verifiable records

• Independent records enable a meaningful audit
• Voter-verification establishes independent
  validity
• Validated records must be protected from
  modification
• Paper records suffice
• Direct and indirect verification
• Ballots dropped into ballot box
• More difficult to achieve wholesale compromise of
  paper records without detection
• Alternatives that mitigated the threat without
  using paper were not prohibited in the 2007 TGDC
  draft

6
IVVR versus paper

• What the 2007 TGDC draft actually required
• Either independent voter-verifiable records
  (IVVR), or
• "Innovation class submission"
• Intent the term IVVR was introduced
  specifically to avoid mandating paper
• Extent paperless solutions are still researchy
• From absence of example, cannot conclude
• That the requirements are more restrictive than
  necessary to achieve the goal
• That no conforming paperless solution can
  possibly exist
• Do not have working group consensus on these
  assertions

7
Alternatives and consequences

• Electronic Independent Verification Devices (e.g.
  VoteGuard)
• At best incomplete response to the rogue
  programmers threat
• Parallel testing
• Arms race between complexity of testing and
  complexity of evading detection
• Cannot be required in the VVSG
• Punts the problem to poll workers
• Software assurance
• Would require invasive, expensive changes to the
  development process and all-new systems
• End-to-end crypto still a research topic
• Unknown unknowns ("innovation class")

8
Tried a different approach

• Previous state auditability SI
• Suggested new state auditability ability to
  do an automated, independent recount
• Automated, because manual counting is inaccurate
• Independent, so that it is a meaningful audit
• Want something comparable to shipping opscan
  ballots to neighboring county
• Falls short of the SI goal if voter-verifiability
  is not included
• Paperless approaches IVD
• At best incomplete response to the rogue
  programmers threat
• How much does "independent" entail
• Taking verification off the critical path or
  making it "random"

9
Paper, voter-verification, and accessibility

• There have been misunderstandings about what
  exactly the 2007 TGDC draft required and did not
  require
• Paper record accessibility requirements were
  intended to be more general (i.e. stronger) than
  in VVSG 1.0
• "Software independence" maybe conveyed that not
  allowed to use software for audio readback that
  was not the intent
• 2007 TGDC draft reflected a difficult compromise
• Identical experience for every voter is
  infeasible
• Prohibiting or limiting voter-verification would
  not be a win
• Absence of conforming implementations raised
  objections
• Rejecting paper entirely versus requiring paper
  record accessibility
• If there is agreement to pursue some alternative
  to SI, then agreement on reasons for rejecting SI
  is not required
• If there is not agreement to pursue some
  alternative to SI, then a better compromise has
  not yet been identified

10
Effectivity concerns

• There have been misunderstandings about the
  impact of VVSG 2.0 on already-deployed systems
• VVSG 2.0 intended to be "forward-looking" for new
  certifications after some date
• EAC determines the date once new guidelines are
  approved
• Certificates issued under previous versions of
  the guidelines will not be revoked automatically
  when new guidelines are approved
• No mandate to retrofit or replace
  already-deployed systems
• "Worst" case Assuming that a jurisdiction has
  voluntarily adopted a law that deployed systems
  must comply with latest EAC guidelines to be used
  in an electionapproval of VVSG 2.0 is not
  imminent
• V Voluntary

11
Option 1

• Endorse one or more of the existing paperless
  alternatives
• Different alternatives have different
  implications and consequences
• Implied policy decisions
• IVD reject the rogue programmers threat or
  accept an incomplete mitigation
• Parallel testing accept difficult and/or
  incomplete procedural mitigation that is outside
  the scope of EAC certification
• Software assurance commit to invasive,
  expensive changes to the development process and
  all-new systems
• Defining a higher-level auditability concept
  requires relaxing one of the constraints
• Otherwise auditability SI

12
Option 2

• Conclude that it was all a big misunderstanding
• Goal of SI no mandate for paper is what we had
  in 2007
• Paper ballot accessibility requirementsas
  intended
• No manual paper ballot handling (Acc-VS)
• Alternative format verification of complete paper
  ballot print content
• Accept that an example of a conforming system
  need not exist yet
• No mandate to retrofit or replace
• Engage Standards Board, Board of Advisors during
  the process
• Refocus on communication, first impressions

13
Option 3

• No misunderstandingconfirm the previous result
• Accept the SI argument accept the SI conclusion
• Market shifting to opscan
• Paper ballot accessibility requirementsas
  intended
• Manufacturers reportedly are responding to paper
  handling and readback requirements
• Fighting the previous battle?