Research: Conceptualization and Measurement

1
Research Conceptualization and Measurement

• Conceptualization
• Steps in measuring a variable
• Operational definitions
• Confounding
• Criteria for measurement quality
• Techniques of dealing with problems in
  measurement
• Ensuring reliability
• Validity
• Face validity
• Content validity
• Criterion related validity
• Construct validity

2
Conceptualization

• We want to speak of abstract things
• Intelligence
• Ability to cope with stress
• Life satisfaction
• Happiness
• We cannot research these things until we know
  exactly what they are.
• Everyday language often vague and unspecified
  meanings.

3
Conceptualization

• Specify exactly what we mean and dont mean by
  the terms we use in our research.
• No true (final) definitions of the stuff of
  life
• Conceptualization
• The process of identifying and clarifying
  concepts
• We specify what we mean by using certain terms
• Indicatorsthe presence or absence of the concept
  we are studying.
• These are often multi-dimensional
• More than one specifiable aspect or facet
• What do we mean by happiness?

4
Steps in reaching a measurement of a variable

• We all have conceptions of what we understand by
  compassion, prejudice, poverty, etc.
• People do not always agree about the meanings
• Begin by asking people to describe what they mean
  when they use certain terms such as
  intelligence
• Consult the EXPERTS
• Literature review
• Even the experts do not agree
• Coming to an agreement on what we understand is
  called conceptualization.
• Result of this process is a concept
• e.g., prejudice

5
Operational definitions

• Specifying exactly what we are going to observe,
  and how we will do it.
• Make the variable directly measurable
• Describe of the operations used to measure a
  concept

6
Examples

• Socio-economic Status (SES)
• What was your total family income during the past
  12 months?
• What is the highest level of school you
  completed?
• How would you operationalize success at
  university?
• If you operationalize badly, you end up not
  studying what you want (invalid operational
  definitions)
• E.g., operationalizing success in career by
  looking only at pay check

7
Confounding

• Intelligence tests require knowledge of the
  language in which they are given
• Also measuring acquired language skills
• Juvenile delinquency can be defined in terms of
  convictions in court
• But convictions are more frequent when they are
  not legally represented - thus also measuring
  economic status
• Confounding When operational definitions
  measure more than one thing

8
Criteria for measurement quality

• Reliability
• Does it yield the same result every time?
  Stability over time.
• If I measure you now and again in half-an-hour,
  do I get the same reading?
• Maximum reliability depends on the construct
  some constructs are unstable, e.g., heart rate.
• Single observers or raters

9
Techniques of dealing with problems in
measurement reliability

• Test-retest method Make the same measurement
  more than once (external)
• Split-half method Divide the instrument in two
  halves - Cronbach's alpha (internal)

10
Ensuring reliability

• Reliability suffers when respondents or
  researchers have to interpret
• Objective scales are always more reliable
• Allow for little interpretation
• Using a fixed-response format helps
• e.g. Multiple choice, Likert type response
  formats
• Researcher does not have to interpret what the
  respondent meant

11
Validity

• The extent to which an empirical measure
  adequately reflects the meaning of the concept
  under investigation
• For a scale The degree to which it measures what
  it is supposed to measure
• Validity is divided into many types
• Content validity
• Criterion-related validity
• Construct validity
• Face validity

12
Face validity

• How a measure conforms to our common agreements
• Examine the wording of the items
• Submit items to expert judges

13
Content validity

• How much a measure covers every element of a
  concept.
• Example Measuring only the affective aspect of
  love, but not the behavioral.
• Experts in a given domain generally judge content
  validity.
• For example, the content of the SAT Subject
  Tests is evaluated by committees made up of
  experts who ensure that each test covers content
  that matches all relevant subject matter in its
  academic discipline.

14
Criterion-related validity

• Sometimes called predictive validity
• How a measure predicts performance on an external
  criterion
• e.g., How ACT or SAT results predict academic
  success at university, as a way of saying they
  have validity

15
Construct validity

• Closely tied to into the theoretical
  underpinnings of the concept.
• Variables ought to be related, theoretically, to
  other variables.
• This kind of validity is based on logical
  relationships between variables
• So Does the instrument actually measure the
  concept (or construct)?
• e.g., Measure cranial circumference or brain
  weight to measure intelligence
• Most difficult to achieve, most important
  measures lacking in construct validity are almost
  useless

16
How to check for construct validity

• How can you show that a measurement truly
  measures what it claims to?
• How would you show that your depression scale has
  construct validity?

17
How to check

• 1. See how it relates to similar and dissimilar
  concepts
• Show that your depression scale relates
  positively to similar concepts
• e.g., People who score high on your depression
  scale will have many sad thoughts

18
How to check

• 2. Show that your depression scale relates
  negatively to opposite concepts. Examples
• People who score high on it will have very low
  energy levels
• Husbands who score high on a measure of marital
  satisfaction have fewer extra-marital affairs

19
Conclusion

• The more a scale or instrument has the qualities
  of reliability and validity, the better it is.
• Reliability and validity need to be sorted out
  before you run the study

20
Conclusion

• Tension between validity and reliability
• Richness of meaning of concepts
• Operational definitions and measurements seem to
  rob concepts of their richness of meaning
• But the more conceptual variation and richness we
  allow in our study, the more opportunity for
  disagreement on how it applies in this situation
• Related to the tension between quantitative,
  structured techniques such as surveys, and
  qualitative, semi-structured methods such as
  interviews