Results in a Flash

1
Results in a Flash

• Zannette Uriell, Evangeline Clewis, Paul
  Rosenfeld
• Navy Personnel Research, Studies, and Technology
  (BUPERS-1)
• Bureau of Naval Personnel
• Millington, TN
• Presented at the Eighth Annual Navy Workforce
  Research Analysis Conference
• Arlington, VA
• 6 May 2008

The opinions expressed are those of the authors.
They are not official and do not represent the
views of the U.S. Navy.
2
Enabling Technologies Data Collection
Alternatives (DCA)
Product Description Develop a suite of data
collection tools to most effectively and
efficiently assess the Total Force. Suite will
include valid set of integrated algorithms to
optimally sample sectors of the Total Force, as
well as a collection of assessment methods and
decision support technologies. Develop measures
not currently available in Navy. Use alternative
methodologies for obtaining information from the
Total Force TRL at Start 4 TRL at Transition 6

• Planned Demos/Deliverables/Transitions
• Q4/FY08 Conjoint Survey, Lightning Poll Demos,
  TF Technology Survey
• Q4/FY09 Lightning Poll, Conjoint Deliverables,
  ICall Demo
• Q4/FY09 Total Force Sampling Algorithm Developed
• Feeds TAXSE

• Warfighting Payoff
• Supports CNP Strategic Vision of an integrated
  Total Force
• Develop sampling algorithms that are valid for
  the Total Force
• Provide comprehensive, integrated suite of
  instruments and tools required to accurately
  predict key outcomes within the Total Force
• Conjoint analysis, lightning poll, electronic
  suggestion box (ICall/ILocker), etc. both as new
  technologies and ways to inform agents for
  modeling and simulating outcomes

3
Data Collection Alternatives (DCA) Operational
Problem

• Desire to
• Improve assignment decisions for jobs and teams
• Forecast subgroup continuation behavior
• Predict the impact of policy decisions on
  retention and satisfaction
• Comprehensive, integrated suite of instruments
  and tools required to accurately predict these
  outcomes within the Total Force (active, reserve,
  and civilian)
• Need to determine best technologies and
  methodologies for collecting data to be used for
  prediction
• Different types of data may lend themselves
  better to non-traditional data collection methods

3
4
Lightning Poll Technical Objective

• Determine prevalence of and use of Short Message
  Service (SMS) capabilities in cell phones by
  Total Force (TF)
• Some populations (recruiters) have Navy-funded
  text messaging
• Determine if SMS surveys are a viable part of the
  integrated toolkit for data collection
• For overall Navy results and/or for select
  populations
• Compare to Navy-wide results

4
5
Lightning Poll Technical Approach

• Conduct an SMS survey of CNRC recruiters (May
  2008)
• Fine tune Lightning Poll methodology on personnel
  known to have Navy-funded text messaging
• Conduct technology survey to gather data on
  technology used at home by TF that can be
  leveraged for other data collection efforts (July
  2008)
• First scientific Navy-wide TF survey
• Web 2.0 (social media) of key interest, including
  use of blogging and chat rooms, as well as use of
  text messaging
• Expanded to include telework questions, related
  to Task Force Life-Work efforts
• Conduct concurrent TF-wide SMS survey for those
  who opt to participate (based on above technology
  survey July 2008)
• Five questions taken from a Navy-wide survey,
  enabling comparison of results

5
6
Text Messaging

• Text messaging (aka Short Message Service, or
  SMS) functions like e-mail messages are sent to
  your cell phone and stay on the phone until you
  delete them
• Generally arrive shortly after they are sent
  (dependent upon cell phone reception)
• Messages are about 140 characters long (actual
  length dependent upon cell provider)
• Each message (incoming and outgoing) counts
  towards monthly amount
• Some opt for unlimited text messages for a
  monthly fee while others choose to pay for 200 a
  month free and 10 cents for each beyond the 200
• Useful for voting (American Idol), organizing
  events (spontaneous presidential campaigns),
  notifications (Amber Alert), or purchasing
  (amazon.com)

7
Use of Cell Phones by Those in DEP
As of 2005, 45 of teens (ages 12-17) have cell
phones, and 33 have used it for text messaging.
69 of those in DEP have cell phones
Note DEP Survey Questions 37 38.
2006 CNRC Delayed Entry Program (DEP) Survey
8
Use of Text Messaging by Enlisted and Officers
In 2005, 93 of officers and 83 of enlisted have
either a personal or work cell phone (or both).
Thirty-five percent of American population uses
text messaging, ranging from 65 for 18-29
year-olds to 8 for those 65 and older.
Seventy-six percent of enlisted and 64 of
officers have text messaging capabilities on
their cell phones.
Text messages sent
Publicity Section Note Graphs show results for
those with cell phones and text messaging
capability.
2005 Navy MWR Customer Survey
9
Recent Technology Use
( Yes)

• Among Sailors, older generations are less likely
  to use newer technologies such as text messaging,
  instant messaging, and social networks.
• Compared to the general population, Sailors use
  much more technology, are more positive about its
  value, and discount the down side.

Items come from the Pew Research Center, A
PORTRAIT OF GENERATION NEXT, 9 January 2007.
The questions were asked of a national, random
sample, in a telephone/cell phone survey.
2007-2008 Retention Quick Poll
10
How Does a Lightning Poll Work?

• Respondents notified of poll topic and sponsor,
  as well as Privacy Act
• Through non-cell phone means
• Research computer (computer with special software
  that is attached to a modem and cell phone/SIM
  card) sends question to cell phones of all
  respondents
• Message stays on cell phone until respondent
  takes action (reply, delete, etc.)
• If response received, next question sent to cell
  phone
• Process continues until end of survey
• Navy personnel data (e.g., gender, paygrade,
  community) may be matched to responses prior to
  analyses

11
Risks/Barriers and Payoff

• Lack of knowledge of cell phone numbers in TF
• For Navy-wide results, requires potential
  respondents to provide cell phone number
• Concern that few will be interested in
  participating in SMS survey Navy-wide
• Mitigated by conducting additional survey of
  known population
• Navy will have increased knowledge of venues for
  data collection (and suggested populations) and
  what information may be collected through SMS, as
  well as how quickly results can be available with
  SMS survey

11
12
Specific Issues to Research

• Are there age differences in access to and use of
  cell phones?
• How do results compare to larger Navy-wide
  surveys?
• Can SMS be used aboard ship or overseas, or only
  for shore CONUS units?
• Are there age/gender differences in responses?
  Completion rates (especially for juniors, who are
  least likely to complete traditional surveys)?
• How long does it take for first answer? To
  complete survey?
• How should it be done (programming issues,
  cross-cell company issues, etc.)?
• What are attitudes towards cell phone survey
  (length, desire to do again, etc.)?

13
Project Deliverables

• Briefing to CNRC of recruiter results
• Briefing to N-134 and other component sponsors of
  Telework and Technology Survey results
• Briefing of usability of SMS and other technology
  for other ONR efforts
• Report on viability of SMS surveys as part of a
  data collection toolkit
• Journal article about SMS survey results

13
14
Contacts

• Zannette Uriell
• Institute for Organizational Assessment (NPRST)
• zannette.uriell_at_navy.mil 901 874-4641 DSN 882
• Evangeline Clewis
• Institute for Organizational Assessment (NPRST)
• evangeline.clewis_at_navy.mil 901 874-4943 DSN
  882
• Dr. Paul Rosenfeld
• Director, Institute for Organizational
  Assessment (NPRST)
• paul.rosenfeld_at_navy.mil 703 695-2582, DSN 225