opening the sources of accountability

1
opening the sources of accountability
4S-EASST, Shay David, August 2004 sd256_at_cornell.ed
u
2
do we need a new theory of invention and
innovation when we stop looking at steam engines
and start looking at search engines?
3
do we need a new theory of floss?
Weber any non-economic explanation of
open-source will be both boring and wrong
4
my work in progress

• a SCOT history 1969-2003
• a critique of the political economy
• normative claims about floss
• floss licenses
• case studies

5
one big question to think about when
contemplating invention and innovation relating
to flosshow do the social values and virtues
change as the technologies and methodologies
change?
6
how does the concept of accountability change?
7
key argument

• accountability, which in traditional software
  environments is seriously eroded because it does
  not scale, can be generalized to collective
  action in the context of floss if we understand
  it less as punishability and more as a culture
  and a standard of care that encourages the
  prevention of risk and harm

8
three motivations for accountability

• (1) accountability as a virtue that is desirable
  in its own right
• (2) accountability as a guideline for
  answerability which motivates precautionary
  behavior that, in turn, caters social welfare
• (3) accountability as a tracing tool that allows
  us, a posteriori, to identify the people involved
  in accidents and damage-inducing errors, punish
  the responsible if necessary and compensate the
  victims if possible

9
conceptual foundations of accountability
accountability
responsibility, fault, guilt
individuality, personhood
10
the co-production of values and software

• my claims - the meaning of accountability,
  responsibility, fault, guilt, individuality,
  personhood and our theory of causation change as
  the technology evolves. - software is no
  exception, and floss is a case in point.

11
four barriers to accountability Helen
Nissenbaum, "Accountability in a Computerized
Society," Science and Engineering Ethics 2, no.
2 (1996)1

• (a) The problem of many handsresponsibility,
  which is usually analyzed in terms of a single
  individual, does not generalize to collective
  action and, therefore, in a computerized society
  responsibility is thinned because in software it
  is customary for many people in many
  organizations to work collectively towards the
  end product
• (b) The problem of inevitable bugsglitches in
  the performance of software are viewed as
  inescapable and, consequently, a dangerous
  approach has developed that frees developers of
  their accountability
• (c) Views of the computer as scapegoatthe
  increased agency we tend to assign to non-humans
  absorbs the human accountability when the
  computer is blamed for faults and the
  investigation stops at that
• (d) Ownership without liabilitythe vocal debates
  on intellectual property rights (IPR) obscure the
  equally important discussion on the
  responsibility and liability that should be
  associated with those rights.

12
earlier mechanisms suggested to uphold
accountability

• adoption of an industry wide standard of care
• adoption of strict-liability principles for
  software development

13
what can floss teach us?
14

• my claims
• strict liability is inappropriate for software
  that departs from firms and markets
• the standard of care should be self-adopted and
  community based

15
inevitable bugs?
( Brian Smith, "The Limits of Correctness," ACM
SIGCAS Computers and Society 14-15, no. 1
(1985)20 ) Suppose the people want peace, and
the President thinks that means having a strong
defense, and the Defense department thinks that
means having nuclear weapons systems, and the
weapons designers request control systems to
monitor radar signals, and the computer companies
are asked to respond to six particular kinds of
radar pattern, and the engineers are told to
build signal amplifiers with certain circuit
characteristics , and the technician is told to
write a program to respond to the difference
between a two-volt and a four-volt signal on a
particular incoming wire. If being correct means
doing what was intended, whose intent matters?
The technician's? Or what, with twenty years of
historical detachment, we would say should have
been intended?

• correctness proofs are bound to fail since they
  test a model and not the real worldand they
  involve intent
• only real world testing, and not simulation, can
  provide assurance of software functioning
  correctly. this is why strict liability is
  inappropriate. it is not a balanced measure.
• flosss dictum to release often, release early
  and the sustained access to the code solves two
  problems
• testing is done in the real world
• interpretive flexibility is maintained

16
many hands, many eyes

• Linuss law given enough eyes every bug is
  shallow ? bugs are shallow phenomena, not
  inevitable barriers to accountability
• in a reputation economy, the many eyes that watch
  the many hands that code are a watchdog that
  prevents risk and harm
• a culture of responsibility is developed, if only
  because one can fix the errors of another
• the motivation (and ability) to prevent risk and
  harm is increased, not by the fear of punishment
  but rather by the desire to maintain ones
  standing within their social group.

17
accountability without ownership

• Most IPR advocates take for granted two premises
  
• (1) consumers and producers are two distinct
  groups with conflicting interests
• (2) intellectual property, like other types of
  property, requires some level of protection from
  appropriation
• floss communities overcome the ownership without
  liability barrier not by reducing liability, but
  by forfeiting ownership and relinquishing
  protection for IP

18
hybrid networks what can we learn from ANT?

• open source project is essentially a hybrid
  network a heteroclite assemblage of human and
  non-human actors, entangled in specific
  configurations that may vary over timeNicolas
  Ducheneaut, "The Reproduction of Open Source
  Software Programming Communities" (PhD
  Dissertation, Berkeley, 2003)
• Maybe the computer is the cause of error?

19
its not my fault

• Perry Hoberman, "Accept," (2003)

20
so whose fault is it?

• to be sure, the yardstick of causation (e.g.
  Feinberg) must not be deemed necessary to
  establish responsibility.
• the question becomes not who caused the error?
  but rather
• what did the human actors do in an error-prone
  environment to prevent risk and harm?
• how can this error be used to improve similar
  systems in order to improve the welfare of
  society?

21
case studies
22
floss-based voting

• existing situation
• black-boxed e-voting solutions dont meet even
  the rudimentary standards of security (Kohno et.
  al based on code review of Diebolds leaked code)
• motivation from floss transpires transparency
  which is a pre-requisite for the trust that is
  essential to democracy
• flosss potential
• Aussies do it right
• VoteHere semi-open-sourced their code

23
floss based EMR

• existing situation
• click-wrap licenses impose impossible limitation
  on accountability mandated by HIPAA
• Multitude of vendors whose software applications
  are incommensurable
• motivation 195,000 people die each year in the
  US alone from preventable medical errors (Health
  Grades Inc., 2004 )
• flosss potential
• SPIRIT, DebianMed
• higher connectivity among vendorsusing open
  standards
• solving the conflict of HIPAA regulated access

24
conclusions

• by ensuring that softwares source code remains
  free through sensible licensing agreements, by
  guaranteeing that enough eyes watch the many
  hands that fix bugs, and by accommodating new
  modes of collaborative activity through
  socialand not legalmechanisms, we can sincerely
  hope that the barriers to accountability will
  diminish.
• we need to build a framework for moral and
  ethical debates that can accommodate meaningful
  discussions regarding the rapidly changing
  technological practices which sometimes work
  towards the values we perceive as valuable and
  sometimes work against them.
• by contrasting floss with closed systems we can
  rethink the important virtue of accountability
  and unpack some of its conflicting meanings as we
  adopt the concept to the ever changing world of
  IT.

25
q a