Ethical Issues in Empirical Studies of Software Engineering

1
Ethical Issues in Empirical Studies of Software
Engineering

• Authors Janice Singer and Norman G. Vinson
• Presenter Leila Naslavsky

2
Introduction

• Researchers conducting Empirical Studies of
  Software Engineering (ESSE) often face ethical
  dilemmas
• Dr. Jonas collects data by observing SE teams
  at local companies. He then categorizes the team
  according to their success at collaboration . A
  few weeks into the research program, a manager
  asks to see Dr. Jonas field notes and wishes to
  know how his company compare to other companies

3
Introduction

• The paper is a first step towards the creation of
  a set of ESSE guidelines.
• This discussion is relevant to research that
• Employs human subjects or
• Collects information that allows identification
  of individuals.

4
Motivation

• Why should we (researchers) care?
• Risk losing cooperation and honesty if upset the
  subject
• Risk losing access to the subjects, funding or to
  other resources if upset subjects employers or
  managers.
• Lack of code of ethics that deals with humans as
  subjects and SE per se.
• Some ESSE peculiarities
• Organizations serving as research subject
• Long-term classroom studies
• Use of artifacts (e.g source code).

The paper aim at presenting the standards not
debating them
5
Outline

• The authors reviewed 22 existing codes (e.g by
  ACM, IEEE-CS) and abstracted high-level ethical
  principles
• Informed consent
• Scientific value
• Beneficence
• Confidentiality.

6
Informed Consent

• Full informed consent on the part of the subject
  to participate in the research.
• Must contain at least
• Disclosure
• purpose, procedure, risks to subjects,
  anticipated benefits, statement offering to
  answer subject questions.
• Comprehension and competence
• Voluntariness
• Right to withdraw

7
Informed Consent - Example

• Dr. Gauthier does not have access to industrial
  software engineers to test her tool. She uses the
  students as subjects
• No consent was obtained from students
• Problem
• Professor power over students grades
• Solution
• Use another professors students
• Do research after end of semester

SHOULD generally obtain informed consent of
subjects/host organizations
8
Scientific Value

• Scientific Value Components
• Importance of research topic evaluated through
  risks and benefits to subjects and society.
• Validity of results results should reliably and
  faithfully represent reality.
• Invalid results are produced by
• Misapplication of reputed methodologies
• Methodological oversights
• Use of disproved methods.
• Is critical some researchers are not completely
  familiar with the methodologies they use.

Weighted against possible harm to the subjects,
a study without merit should not be undertaken
9
Beneficence - Human

• Favorite balance of benefits to harm
• Tradeoffs that can adversely affect one
  stakeholder group will sometimes need to be made.
• Example
• Software engineers who maintain the source code
  have not been so appreciative of research on
  source code reengineering and automated
  translation.
• Minimized by
• Involve them in all issues surrounding the
  project schedule and new source code integration.
  
• Give them time to explore new code
• Training on new language.

10
Beneficence - Organizational

• Conflicts of interest between hosting
  organization and public at large
• Example
• Public negative evaluations of companies can
  cause financial harm.
• Disclosing of flaws on a companys software
  development process.
• Informing management of process problems could
  result in dismissals, thus harming individuals.

11
Confidentiality

• Confidentiality Components
• Anonymity no one can identify participants of
  experiment
• Use subject number or aliases
• Subject name not linked to their data.
• Confidentiality of data privacy of data
  collected.
• Consent document inform who will access raw data,
  and for what purposes.

12
Confidentiality - Example

• Research on how novice programmers gain expertise
  realized in a small company. Interviewed 10
  novices and 2 gurus.
• Difficulties for confidentiality
• Conducted study in workplace
• Few subjects
• Information in report could be used to identify
  individuals.
• ESSE studies are often on few subjects and occur
  in workplaces
• Researches should disclose the limits of
  confidentiality as part of the informed consent
  procedure.

13
Consent, Confidentiality, BeneficenceTaking the
Measures of Artifacts

• Artifacts can Identify individuals who created
  them, raises issues of
• Consent, confidentiality and beneficence.
• Need the consent of organization to access their
  source code and documentation.
• Programmers dont expect the their work to remain
  private.
• It is ADVISABLE to involve programmers in the
  research. (slide 9).

14
Exceptions

• Exceptions to any set of guidelines can occur
• Informed consent an confidentiality are not
  required when
• No information in the raw data identifies
  individuals.
• The expectation of privacy doesnt exist.
• More harm result in maintaining confidentiality
  than from breaching it.

15
Conclusion

• To avoid the dilemma presented in the
  introduction
• Consider the ethical implications of the research
  BEFORE
• Ensure rights and responsibilities understanding
  by hosting organizations and subjects
• ESSE community need to develop code of research
  ethics adapted to the peculiarities.

16
Discussion

• Since programmers expect that their code will be
  reviewed, it SHOULD NOT be necessary, from an
  ethical perspective, to obtain their consent to
  use their code. (agree? disagree?)
• In practice it is not possible to eliminate
  management coercion. Is it better if researcher
  or manager contacts the subjects of research?
• If the researcher knows an employee will be laid
  of next week. This employee says he cant be
  interviewed a certain day because he is going to
  buy his house would you warn him?