Translating English questions into Spanish: A randomized experiment

1
Translating English questions into SpanishA
randomized experiment

• Giorgina Piani University of Uruguay
• ESRA, Prague 2007

2
Importance of good translations

• Validity
• Equivalence
• A good design (translation) for the collection
  of comparative data should permit one to assume
  as much as possible that the differences
  observed cannot be attributed to the differences
  in the method being used (originated in the
  translation process)
• Suchman (1964)

3
Why Spanish?

• Oficial language in 21 countries
• Spanish speaking population one of the fastest
  growing areas
• 3rd most widely spoken language (English and
  Mandarin)
• 400 million Spanish speakers
• Year 2050 ? 530 million Spanish speakers

4
Spanish Regionalisms

• Castillan Spanish (Spain)
• U.S. Spanish
• Caribbean Spanish
• Mexican Spanish
• Andean Spanish (Venezuela, Colombia, Ecuador,
  Perú)
• Chilean Spanish
• Paraguayan Spanish
• River Plate Spanish (Argentina and Uruguay)

5
Translation of survey materials

• Is it possible to use a single Spanish Question.
  Standard Spanish?
• How many different versions are needed?
• How much different do they need to be?

6
Antecedents

• Quality assessment of ISSP Spanish translations
• 2004 Citizenship English Source Questionnaire
• Spanish translations
• Chile
• Dominican Republic
• Mexico
• Spain
• Uruguay
• Source and Target questionnaires were compared
  for the 5 countries
• Back translation (BT) was used to report quality
  assessment

7
Translation problems

• Word by word translations
• Change in the context
• Semantic equivalence
• Scale equivalence

8
Split-Ballot experiment(Randomized experiment)

• Scale Equivalence

9
Experimental Design

• English source question 2004 ISSP Citizenship
  Questionnaire
• 2 different Spanish Translations (2 countries)
• Both Spanish versions sound right in Uruguayan
  (River Plate) Spanish
• 2006 ISSP Role of Government Questionnaire

10
(No Transcript)
11
Thinking of the last national election in
country, how honest was it regarding the
counting and reporting of the votes?
12
Model Estimation

• Impact of the split-ballot experiment (question
  wording) on respondents evaluation of the
  counting and reporting of the votes
• Impact of individual variables on respondents
  evaluation

13
Ordered Probit Regression Model

• Experimental condition
• 0QA
• 1QB
• Sex
• 0Woman
• 1Man
• Age
• Education (years of schooling)
• Political affiliation
• 1Left party
• 0Else
• Place of residence
• 1capital city
• 0Rest of the country
• Type of Employer
• 1Public sector
• 0Else

INDEPENDENT VARIABLES
DEPENDENT VARIABLE

• 1 Not very trustful
• Not at all trustful
• OR
• Somewhat fraudulent
• Very fraudulent
• 2 Indiferent
• OR
• Neither clean nor fraudulent
• 3 Very trustfulFairly trustful
• OR
• Very clean
• Somewhat clean

14
Estimated Model Results
significant at 10 significant at 5
significant at 1
15
Main Results

• QB elicits more Trustful / Clean responses
  compared to QA
• The use of the world fraudulent seems to convey
  a more negative evaluation than its counterpart
  in QB not trustful
• The older the R. the more likely he/she is to
  choose Trustful / Clean responses
• Trustful / Clean responses increase with the
  educational level of Rs.
• Left Party sympathizers are more likely to trust
  the counting and reporting of votes (Winning
  Party)
• Rs. living in the capital city are more likely to
  trust the counting and reporting of votes than
  Rs. living in the rest of the country

16
Implications Cross-national surveys in Spanish

• ISSP (International Social Survey Program)
• English Source Questionnaire
• Individual-country translation
• Latinobarómetro
• Spanish Source Questionnaire (some questions are
  borrowed from Questionnaires in English)
• Single Questionnaire (Standard Spanish) applied
  to all country members
• World Values Survey
• English Source Questionnaire
• Individual-country translation
• Quality assessement (Back-Translation)
• LAPOP (Latin American Public Opinion Project)
• Spanish Source Questionnaire (Root Questionnaire)
• Root Questionnaire adjusted to show country
  language specificities

17
Research Problems

• Does the use of Regional Spanish(es) pay off?
• Advantages and Disadvantages?
• Measurement of the quality of country specific
  questionnaire translation into Spanish.
• Are some Spanish speaking populations more
  sensitive to the use of a Standard Spanish as
  opposed to the Regional Spanish?
• Are some survey issues and/or data collection
  modes more sensitive to this differences?

18
Next steps

• Design a new experiment (2007 ISSP Quest.)
• Standard Spanish Vs. Regionalisms
• Latinobarómetro Questionnaire
• Regional translation Vs. the Standard
  translation

19
Q A

• Thank you!
• Gracias!
• gpiani_at_umich.edu