Guidelines for Empirical Research

1
Guidelines for Empirical Research

• Dieter Rombach
• ISERN WS 2005
• Noosa Heads, 14 November 2005

2
History

• Session _at_ ISERN 2004
• Discussion notes available
• Highlights
• How much context must be provided (beyond
  explicitly treated dependent independent
  variables) to allow for cross-study analysis?
  Minimal, but ???
• How will standards for documentation evolve and
  be accepted? Low evolution similar to other
  fields, but ISERN community can play a major role
  (paper reviews, ) in developing minimal set of
  guidelines
• How can we create better incentives for
  replication, and how can we aggregate results of
  study families? Exp packages hard to reuse,
  aggregation capability less than satisfactory
  (?inspections?),
• How can we provide better incentives for
  following guidelines? Main issues are
  ownership, requirements by funding agencies,
• No joint work on white-paper since 2004!!!

7 ISERN members on ICSE 2006 PC!
Who can convincingly summarize what we know about
inspections?
3
Session 2005

• 2 introductory presentations
• Carolyn Seaman (Qualitative Research Doc.
  Guidelines)
• Dieter Rombach (Experiences _at_ IESE)
• Brainstorming of issues
• Discussion
• Set up of working group

4
Issues related to Guidelines for ER (ISERN
2004)

• Context 1
• Cooperative culture 6
• Teaching ESE 7
• Rewards incentives 4
• Replication aggregation 2
• Document structure 1
• Motivation for practice 1
• How much is enough data 2
• Motivation/value 1
• Reporting 1
• Decision support 1
• Methods 2
• How to do 1