Prototyping for Requirements Elicitation and Validation

1
Prototyping for Requirements Elicitation and
Validation

• Ann M. Hickey
• Center for the Management of Information
• University of Arizona

2
Overview

• Background
• Why Prototype?
• Requirements Prototyping
• Prototype Development Issues
• Evaluation Methodology
• Preliminary Evaluation Planning
• Face-to-Face User Evaluations
• Distributed User Evaluations
• Benefits, Risks Implications
• Conclusions Future Research

3
Background

• U of A developed Steps to Target (STT)
  Methodology for DESCIM in FY96
• STT Goal Define streamlined process for
  incremental development of target systems
• DESCIM began implementation in FY97
• Prototyping is a key component of STT
• FY97 research focused on maximizing benefits of
  prototyping for STT

4
Why Prototype?

• Users often dont know or cant articulate their
  requirements until they see them
• Prototypes are an active model that users can
  see, touch, feel, and experience
• Prototyping can accelerate the life cycle, allow
  earlier detection of errors, and provide more
  rapid user feedback
• Prototypes can serve many purposes in all phases
  of systems development life cycle

5
Why Prototype?

• Requirements Phase
• Elicit and validate user requirements, especially
  following data/process integration
• Evaluate developers interpretation and
  implementation of requirements and their
  underlying assumptions
• Determine viability of user interface
• Assess initial system usability
• Respond to developers requirements questions

6
Why Prototype?

• Traditional Uses Design Programming Phases
• Analyze system performance
• Evaluate alternative architectures
• Test functionality, usability, and system
  acceptability formally by developers, test
  agencies, and users
• Focus shifts between requirements usability
• Depends on prototype maturity and life cycle
  phase
• Evaluation methodology geared towards
  Requirements Phase of life cycle

7
Requirements Prototyping
Business
Information System
8
Prototype Development

• Many rapid prototyping tools available
• Stand-alone tools support throw-away prototypes,
  generally not good for production systems
• Integrated prototyping development tools
  support evolutionary prototypes production
  systems
• Must choose prototyping approach before tools
• Prototyping approach impacts life cycle
• Throw-away prototypes generally used in early
  phases
• Evolutionary approaches preferred by DESCIM and
  many other developers for entire life cycle

9
Prototype Development Evolutionary vs. Throw-away
Throw-away Evolutionary
Development approach Quick and dirty Build in
quality No rigor Rigorous
Development environment Prototyping
tools Production languages
Design drivers Fast development Optimize
modifiability, maintainability
What to build Only difficult, Scenarios,
Visible or unclear parts Critical,
User Interface
Ultimate goal Throw it away Evolve it
10
Prototype Evaluation Methodology

• Prototype evaluations may be formal or informal
• Frequent informal evaluations by small groups of
  users essential for rapid feedback
• Formal evaluations by large user groups required
  for elicitation and validation
• Distinct phases/types of formal evaluations
  require different meeting structure methods
• Recommended structure provided for each prototype
  evaluation phase
• Detailed methods with supporting EMS tools
  defined for each evaluation type

11
Prototype Evaluation Phases

• Preliminary Evaluation Planning Session
• Generic evaluation by functional development
  teams
• Tailor plan for user evaluation
• Face-to-Face User Evaluation Meeting
• Tailored evaluation by knowledgeable users to
  elicit new/missing requirements
• Structure of evaluation based on user expertise
  and prototype maturity
• Must integrate user assessments/requirements
• Distributed User Evaluation
• Lacks richness of face-to-face user evaluation
• No provisions for group assessment/integration

12
Preliminary Evaluation

• Conduct generic evaluation
• Training for functional development team
• Identify evaluation purpose issues
• Use model of requirements/usability factors
• Select appropriate evaluation type
• Based on user/system maturity
• Develop evaluation scenarios
• Use prescribed guidelines to create
• Adapt final evaluation questionnaire
• Base on methodologys generic questionnaire

13
Evaluation Purpose

• Validate that current requirements have been
  accurately identified and clearly understood
• Identify usability requirements or problems
• Elicit new requirements for future increments

14
Evaluation Factors
Prototype System

Usefulness Willingness to Adopt
15
Evaluation Types

• Evaluation types must be clearly separated
• Demonstrations
• Chauffeured Scenarios
• Independent User Scenario Execution
• Comprehensive Prototype Evaluation
• Different techniques needed for each type
• Recommend start with demonstrations
• Follow with unstructured hands-on practice until
  users achieve sufficient familiarity with system
• Dont jump straight to scenarios

16
Face-to-Face EvaluationsProposed Agenda

• Introduction/Administrative
• Purpose of Evaluation
• System/Prototype Overview
• User Evaluations
• May do one or more different evaluation types
• Evaluation methodology differs for each type
• Integrate Document Prototype Evaluation Results

17
System/Prototype Overview

• System Overview
• Define system scope incremental development
  plans
• Summarize currently known requirements
• Prototype Scope
• Define what portion of known system requirements
  are included in prototype
• Identify what prototype doesnt do (e.g., error
  checking)
• Highlight any developer issues or questions
• Prototype Training
• Need just-in-time training as required for the
  evaluation
• Training must be tailored to expertise of users
  and system complexity

18
Chauffeured Scenarios

• Developer walks users through scenarios
• Display on public screen
• Two-user teams evaluate scenario
• One user runs prototype to try scenario
• Other user runs GroupSystems Categorizer to
  record teams comments
• Technographer supplements capture of requirements
  identified during discussions

19
Integrate Evaluation Results

• Must allow time for group review, analysis and
  comment on evaluation results to
• Increase clarity of requirements
• Achieve group consensus on validity of
  requirements
• Prioritize validated requirements
• Time limitations can result in unclear
  requirements and lack of consensus
• Negotiation prioritization may need to be
  deferred until developer analyzes results

20
Distributed User Evaluations

• Generally best as follow-on to face-to-face
  evaluations
• If no training required, may do distributed
  evaluations immediately
• Recommend independent scenario and comprehensive
  face-to-face methods as model for distributed
  evaluations
• Users will not be able to integrate results as in
  face-to-face sessions

21
Prototyping Benefits

• Requirements Prototyping
• Helps users think about and visualize their
  requirements
• Provides users concrete artifact for validation
  which is far superior to an abstract requirements
  list
• Can assess impact of data/process integration
• Responds to developers requirements questions
• Disciplined Prototype Evaluations
• Pre-planning preparation highlight important
  issues
• User evaluations ensure broad range of
  requirements surfaced and validated during
  structured review
• Process provides effective documentation of
  requirements

22
Prototyping Risks

• Encourages build fix approach to systems
  development
• May increase system scope/complexity
• Can lead to premature commitment to a design
  without evaluating alternatives
• May create unrealistic user expectations
• Users may focus on cosmetics vs. functionality of
  prototype

23
Implications

• Prototype development must be carefully managed
  IAW STT Methodology
• Evaluations must be scheduled, resourced, planned
  included in requirements phase
• Dont always need large group or groupware, but
  do need frequent small user group evaluations
• Primary decision evaluations do need broad user
  representation and will benefit from groupware
• May do in conjunction with other meeting
  activities
• Prototyping for requirements elicitation
  validation is a fundamental business change

24
Conclusions

• Prototyping is essential for accurate and
  complete identification of requirements
• Evaluations confirmed value of prototypes for
  requirements elicitation validation
• Prototype evaluation methodology is ready for
  implementation now