Fundamentals of Operations Research

1
Fundamentals of Operations Research

• Data for Action Workshop
• Child Survival Technical Support Project

Susan Adamchak, PhD
Julie Pulerwitz, ScD
2
The Horizons Program

• Global HIV/AIDS operations research program to
  learn about
• Which issues are most relevant to prevent HIV
  transmission and mitigation of AIDS impact
• What types of interventions work to prevent HIV
  and mitigate AIDS
• How and why interventions work
• How much do they cost

3
The FRONTIERS Program

• Global reproductive health OR program to
• Develop and test innovative solutions to global
  and regional RH service delivery problems
• Disseminate results to improve policy development
  and program management
• Enhance sustainability by improving OR capacity

4
What is Operations Research?

• OR diagnoses and evaluates problems
• OR compares service delivery approaches for their
  impact, cost-effectiveness, quality, and
  acceptability

5
Operations Research

• A continuous process with 5 steps
• Problem identification and diagnosis
• Strategy selection
• Strategy testing and evaluation
• Information dissemination
• Information utilization

6
Key Attributes of OR

• Improves service delivery
• Is innovative, risk-taking, cutting edge
• Serves both managers and researchers
• Fosters problem-solving mentality
• Seeks to ensure policy impact
• Documents impact of program effort

7
Goal of OR

• To increase the efficiency, effectiveness, and
  quality of services delivered by providers
• To increase the availability, accessibility, and
  acceptability of services desired by clients

8
Categories of OR Studies

• Exploratory or Diagnostic Studies
• Problem isnt known
• Field Intervention Studies
• Program approach not known
• Evaluation Studies
• Impact not known
• Cost-Effectiveness Studies
• Cost and effectiveness not known

9
Problem Identification

• Is it an important problem?
• To whom?
• How can the problem can be addressed by the
  program?
• What health improvements will result from solving
  this problem?

10
Is the Problem Researchable?

• Do I need to do research to solve the problem?
• Can the problem be corrected by common sense and
  experience?
• Do I have enough time, money and qualified
  persons to do research?
• Will decision-makers listen to research results?
• What ethical issues need to be considered?

11
Stating an OR Problem

• Describe the problem in terms of health issues
  facing the specified population(s)
• Describe the nature and extent of the problem
• Identify other program strategies that have been
  tried to solve similar problems
• Restate the problem as a question to be answered
  through research

12
Defining Strategic Approaches Implementation

• Describe all components of strategy, resources
  needed, implementation plan, and schedule for
  activities
• Implement activities as perfectly as possible

13
Choosing a Strategy to Test Feasibility

• Can the program afford it?
• Can it be easily implemented?
• Will it be effective?
• Is there enough time to test it?
• Do the staff and managers want to test it?

14
Choosing a Strategy to Test Utilization

• How will the results be used?
• Who will use the results and how will they know
  about it?
• Will other programs be interested in the results?
• If proved effective, can it be implemented
  routinely?
• If proved not to be effective, what will happen?

15
Testing a StrategyEvaluation

• Specify cause and effect between desired changes
  and strategy
• Specify research objectives and hypotheses
• Operationally define key variables
• Create research design to test hypotheses
• Describe sampling plan, data collection methods
  and analysis plan

16
Monitoring Intervention Studies

• Is the intervention being conducted equally in
  all units?
• How much variation is there in the independent
  variable?
• Do groups remain equivalent?
• Are observations collected on schedule?

17
Research Hypotheses, Objectives and Variables
What are They and Why are They Important?
18
Hypotheses

• A hypothesis is a statement about an expected
  causal relationship between independent and
  dependent variables that can be tested through
  collecting information, i.e., a tentative
  answer.

19
Causality

• Causality is when one event produces a second
  event
• Studying causality is essential for making
  decisions whether to
• Continue an intervention
• Expand an intervention

20
Causality Requirements

• A precedes B
• B is present only when A is present
• The intervention is the only difference between a
  set of observations or groups

21
Examples of Hypotheses

• Contraceptive use will be higher in villages
  where the fieldworker is married than in villages
  where the fieldworker is unmarried.

22
Examples of Hypotheses

• The performance of clinic staff who have received
  a five-week, field-based training course will be
  higher than the performance of clinic staff who
  have received a three-week, classroom-based
  training course.

23
Objectives

• Ultimate objectives relate reasonable and
  expected contributions of the study to broad
  social, economic or health concerns (reducing
  infant mortality).

24
Objectives

• Immediate objectives relate directly to the
  research problem situation stated in behavioral
  terms (improving prenatal care).

25
Independent Variables

• Programmatic factors than can be manipulated
• Strategy being tested
• The cause

26
Dependent Variables

• Characteristics we expect to change
• Outcomes and/or impact being measured
• The effect

27
Operational Definitions

• How will I know it when I see it?
• A description of how a variable or construct will
  be observed and measured (indicator)
• A description of how a hypothesis will be proved
  or disproved (specifying condition for
  significant difference)

28
Validity Reliability

• Validity when youre measuring what you think
  youre measuring.
• Construct vs. content vs. predictive vs. face
• Reliability when answers are consistent across
  time and circumstances.
• Test-retest vs. internal consistency vs.
  inter-rater

29
Validity of Measures

• The extent to which a measure actually measures
  what it is supposed to measure
• Is the measure appropriate for the
  variable/construct?
• Is the measure accurate?

30
Reliability of Measures

• The consistency of a measure from one use to the
  next
• Will the respondent answer the same question the
  same way twice?
• Will two interviewers record the same response
  the same way twice?
• Will the same observer score the same observation
  the same way twice?
• Will two observers score the same observation the
  same way twice?

31
Study Designs to TestHypotheses
32
Characteristics of an Experimental Design

• COMPARES experimental and control groups
• MANIPULATES intervention
• CONTROLS threats to validity through random
  assignment

33
True and Quasi-Experimental Designs

• True Experimental
• Uses random assignment
• Protects against threats to validity
• Demonstrates causality

• Quasi-Experimental
• Does not use random assignment
• Open to many threats to validity
• At best, only suggestive of causality

34
You Cannot Use an Experimental Design When

• Units cannot be randomly assigned (e.g., ethics,
  logistics, politics)
• Only 1 unit in sample
• Contamination due to proximity or other exposure
  is possible
• Situational factors (time, cost) are present

35
(No Transcript)
36
Units of Study in OR

• Individuals clients, providers, the general
  public
• Groups facilities (e.g., clinics), villages,
  districts
• Note When unit of study is a group, the group
  not the individual must be used as the variance
  measure.

37
Symbols
Experimental Group
Random Assignment of cases to the groups
RA
Control Group
The passage of time
Intervention
X
Observation
O
A single group of subjects
X and O in a row
Between groups not randomly assigned
- - - - - - - - - - - - - - -
38
Classic Experimental Design
Time
Experimental Group O1 X O2 Control Group O3 O4
RA
39
Pretest-Posttest Non-Equivalent Control Group
Design

• No random assignment

Time
Experimental Group O1 X O2 Control Group O3 O4
- - - - - - - - - - - - - -
40
Use Of Non-equivalent Control Group Design

• When random assignment is not possible
• When you have no more than two units to assign
  (two hospitals, two districts, etc.)
• Study units should be matched with a
  non-equivalent control group design (I.e., have
  similar characteristics)

41
Time Series Design

• Repeated measures on the same group over time
• No control or comparison group

Time
Experimental Group O1 O2 O3 X O4
O5 O6
42
Use Of Time Series Designs

• Whenever you cannot use a separate control group
  (only one facility in the study, etc.)
• Evaluate a mass media campaign

43
Non-Experimental Designs

• Case Study
• Experimental Group
• X O1
• One Group Pre-test/Post-test
• Experimental Group
• O1 X O2

Time
Time
44
Good Design to Study Integrated Services
45
Overview of Data Collection Methods

• Quantitative methods describe what people do
  (e.g., how many people have ever used ORT). Need
  a lot of people for statistical analysis.
• Qualitative methods explore why people think and
  behave as they do (e.g., why difficult to use
  ORT). Need a lot of in-depth information from few
  people.

46

• Qualitative Quantitative
• Insiders Perspective Outsiders perspective
• Exploratory Confirmatory
• Hypothesis generating Hypothesis confirming
• Words Numbers
• Less structure More structure
• Dynamic Static
• Small sample Large sample
• More depth Less depth
• Purposive Random
• Rich Less rich

47
Quantitative Methods

• Surveys
• Face to face or self-administered
• Service or clinic statistics
• Inexpensive to collect
• National statistics (e.g., census)
• Nationally representative

48
(No Transcript)
49
Qualitative Methods (1)

• Focus Group Discussions
• group of 8-10 sharing opinions
• requires skilled facilitator to solicit from
  everyone
• rich information in short period of time
• often reflects social norms, what should be
• Case Studies
• in depth information about 1 example

50
Qualitative Methods (2)

• In-Depth Interviews
• Interview guide with topics 45 min to 1 hour
• Requires skilled interviewer to probe, follow
  leads
• Rich data
• Reflects personal beliefs and experiences
• Mapping, Story Telling, Picture Cards
• Creative techniques
• Alternatives to direct questions

51
(No Transcript)
52
(No Transcript)
53
Disseminate the Results

• Develop a communications strategy to reach all
  stakeholders with appropriate messages
• Allow sufficient time, funds and qualified
  persons to do a good job
• Include a means to evaluate what people did with
  the information

54
Ensure Results are Used

• Plan a utilization strategy that anticipates what
  actions are needed to change the way services are
  currently implemented
• Allow sufficient time, funds and qualified
  persons to do a good job

55
(No Transcript)
56
Design Exercise 1

• You want to test alternative ways to provide
  post-partum contraceptive services to women.
• One alternative is to provide woman with
  individual counseling, and to schedule an
  appointment to receive contraception later.
• A less expensive alternative is to show a group
  of women a video and to provide an appointment
  form to fill out and return to the receptionist.

57
Design Exercise 2

• In the Ministry of Health, many officials feel
  that shifting antenatal care from nurses to
  nurses aides may lead to less care for pregnant
  women and result more negative outcomes of
  pregnancies.
• Design a study to provide information to the
  Ministry officials about the effect of shifting
  antenatal care from nurses to nurses aides.

58
Design Exercise 3

• Many women in the district use TBAs during their
  pregnancy. A recent survey of mothers and
  children detected a high number of infant deaths
  attributable to tetanus.
• Design a study to test whether training TBAs will
  lead to increased referrals to health services,
  and increase the proportion of women receiving
  antenatal tetanus toxoid vaccinations.

59
Design Exercise 4

• A PMTCT program provides HIV counseling and
  testing, and treatment to prevent the
  transmission of HIV from mother to baby. It is
  recommended that mothers exclusively breast-feed
  or formula-feed, but mixed feeding is common.
• Design a study to test whether involving male
  partners in PMTCT will increase recommended
  breast-feeding practices, and reduce the rates of
  MTCT.