EOSDIS Survey Overview

1
EOSDIS Survey Overview

• Carol.L.Boquist_at_nasa.gov
• HDF and HDF-EOS Workshop
• Nov. 4, 2009

2
Why we survey
NASAs Earth Observing System Data and
Information System provides data products and
associated services for interdisciplinary studies
to a diverse user community. Beginning in 2004,
the Earth Science Data and Information System
(ESDIS) Project has conducted annual surveys to
measure user satisfaction using the American
Customer Satisfaction Index (ACSI). The results
are one of the ESDIS Projects metrics sent to
the Office of Management and Budget (OMB).
3
The Easy Part
We start by setting up an Interagency Agreement
between NASA and the Department of Interior. The
survey is conducted by CFI Group under contract
with the Federal Consulting Group (FCG) of
Department of Interiors National Business
Center. The FCG is Executive Agent in government
for the ACSI. They work with OMB to ensure
adherence to applicable Federal regulations such
as the Privacy Act and the Paperwork Reduction
Act.
4
EOSDIS Data Centers
5
EOSDIS Data Centers
Alaska Satellite Facility DAACASF
DAAC http//www.asf.alaska.edu Synthetic Aperture
Radar (SAR) Products, Sea Ice, Polar Processes,
Geophysics
Land Processes DAACLPDAAC http//LPDAAC.usgs.gov
Surface Reflectance, Land Cover, Vegetation
Indices
Global Hydrology Resource CenterGHRC http//ghrc.
nsstc.nasa.gov/ Hydrologic Cycle, Severe Weather
Interactions, Lightning, Atmospheric Convection
Physical Oceanography DAAC PO.DAAC http//podaac.
jpl.nasa.gov Sea Surface Temperature, Ocean
Winds, Circulation and Currents, Topography and
Gravity
Oak Ridge National Laboratory DAACORNL
DAAC http//daac.ornl.gov Biogeochemical
Dynamics, Ecological Data, Environmental
Processes
National Snow and Ice Data Center DAACNSIDC DAAC
http//nsidc.org Snow and Ice, Cryosphere,
Climate Interactions, Sea Ice
6
EOSDIS Data Centers
Crustal Dynamics Data Information SystemCDDIS
http//cddis.gsfc.nasa.gov Space Geodesy
Atmospheric Science Data CenterASDC
LaRC http//eosweb.larc.nasa.gov Radiation
Budget, Clouds, Aerosols, Tropospheric Chemistry
MODAPS Level-1 Atmospheres Archive and
Distribution System MODAPS LAADS http//ladsweb.n
ascom.nasa.gov MODIS Level-1 and Atmosphere Data
Products
Ocean Biology Processing Group http//oceancolor.g
sfc.nasa.gov Ocean Biology, Sea Surface
Temperature, Biogeochemistry
Socioeconomic Data and Applications CenterSEDAC
http//sedac.ciesin.columbia.edu Human
Interactions, Land Use, Environmental
Sustainability, Geospatial Data, Multilateral
Environmental Agreements
Goddard Earth Sciences Data and Information
Services CenterGES DISC http//disc.gsfc.nasa.gov
Atmospheric Composition, Atmospheric Dynamics,
Global Precipitation, Hydrology, Solar
Irradiance, Global Modeling, Multi-Sensor
Research Products
7
Who do we survey?

• E-mail addresses are obtained from
• Orders from registered users
• Inquiries
• Anonymous FTP access
• Data center lists
• The number of e-mail addresses per data center
  varies greatly because the data centers vary
  greatly
• Discipline-specific
• Number or frequency of available products
• Size of user communities
• Number of anonymous ftp users
• Restricted access or cost
• Multiple invitations to the same user
• A user with multiple e-mail addresses will
  receive multiple invitations
• A user accessing multiple data centers will
  receive multiple invitations

8
Survey Questions

• The survey contains comment fields and several
  types of questions
• Demographic questions
• Questions to aid recall
• Rating questions for the ACSI and EOSDIS models
• Non-modeled rating questions
• The questions were originally based on a previous
  survey of the members of the User Working Group
  for each of the data centers.
• Formulating the questions is a combined effort of
  project personnel, EOSDIS User Services Working
  Group (USWG), and the CFI Group.
• The challenge is to keep the survey short enough
  so users are willing to respond, yet long enough
  to reflect the complexity of the data and
  differences in data centers, and more
  importantly, understand the needs of the diverse
  user community.

9
How did EOSDIS Compare in 2008?
The ACSI is the 1 national economic indicator of
customer satisfaction. The ACSI is produced by
the Stephen M. Ross Business School at the
University of Michigan, in partnership with the
American Society for Quality (ASQ) and the
international consulting firm, CFI Group. The
ACSI is used to measure over 40 industries and
200 organizations covering 45 of the U.S.
economy. Over 70 U.S. Federal Government agencies
have used the ACSI to measure more than 120
programs/services.
10
Survey Results 2004-2008
Although the survey contains over 50 questions,
the ACSI is based on responses for only three
questions. N number of responses used (sample
size). See Slide 9 for confidence levels.
11
The EOSDIS Model
2008 Results and Priorities

• A series of questions are asked for each of the 6
  elements of the EOSDIS model
• Product Search
• Product Selection and Order
• Delivery
• Product Quality
• Product Documentation
• Customer support
• CFIs methodology quantifiably measures and links
  satisfaction levels to performance and
  prioritizes actions for improvement.

12
EOSDIS ACSI Score Comparison 2004-Comparisons
2004-2008
YYR
ACSI 2004 2005 2006 2007 2008
EOSDIS 75 78 74 75 77
Federal Government 72 71 72 68 69
Overall 74 73 74 75 75

• Federal Government
• Notes
• These numbers are from the end of year results
  available at thecsi.org.
• The charts in our annual presentations may show a
  slightly different score because the EOSDIS ACSI
  score is computed before the end of the calendar
  year.
• Our presentations have either the score from a
  previous year or a quarterly ACSI average of the
  scores available at the time that our EOSDIS ACSI
  score was computed.

13
Top-line Results 2004-2008
2004 2005 2006 2007 2008
Customer Satisfaction Index 75 78 74 75 77
Likelihood to Recommend 86 89  86  85 86
Likelihood to use Services in Future 86 91 88 87  89
14
Top-line Results 2004-2008 (cont.)
2004 2005 2006 2007 2008
Product Search 70 74 70 72 75
Product Selection and Order 73 76 72 74 77
Delivery 84 85 79 79 81
Product Quality 68 71 71 72 74
Product Documentation - - 72 74 75
Customer Support 84 84 82 83 84
Product Quality includes format questions.
15
2004 HDF/HDF-EOS
In what format were data or products provided?
HDF-EOS 49
HDF 39
NetCDF 5
Binary 14
ASCII 12
GeoTIFF 19
Other 7
16
2005 HDF/HDF-EOS
Format preferred
Format received
HDF-EOS 39
HDF 30
NetCDF 3
Binary 6
ASCII 7
GeoTIFF 9
Other 3
HDF-EOS 22
HDF 20
NetCDF 7
Binary 8
ASCII 10
GeoTIFF 25
OGC, Other GIS, Other 6
17
2006 HDF/HDF-EOS
In 2005, 9 said products were provided in
GeoTIFF and 25 who said it was their preferred
method.
18
2007 HDF/HDF-EOS
In 2006, 67 said products were provided in
HDF-EOS and HDF and 42 said they were their
preferred method.

Multiple responses allowed
19
2008 HDF/HDF-EOS
Multiple responses allowed
20
2009 Format Questions

• What format(s) would/do you prefer? (select any
  that apply)
• HDF-EOS/HDF
• NetCDF
• Binary
• ASCII
• GeoTIFF
• JPEG, GIF, PNG, TIFF
• OGC Web services (WMS, WCS, WFS, etc.)
• GIS (e00, shp, etc.)
• CEOS
• KML, KMZ
• Other (Please specify another format or comment
  on specific version, etc.

• In what format(s) were your data products
  provided to you? (select any that apply)
• HDF-EOS/HDF
• NetCDF
• Binary
• ASCII
• GeoTIFF
• JPEG, GIF, PNG, TIFF
• OGC Web services (WMS, WCS, WFS, etc.)
• GIS (e00, shp, etc.)
• KML, KMZ
• CEOS
• Dont know
• Other (please specify and/or comment

• Still using the 10-point scale on which 1 means
  Poor and 10 means Excellent, how would you
  rate
• Ease of using the data product in the delivered
  format
• Overall quality of the data product
• Overall usability of the data product

21
2009 Documentation Questions

• What documentation did you use or were you
  looking for?
• Instrument specifications
• Science algorithm
• Product format
• Tools
• Science applications
• Data product description
• Production code
• Other

• Was the documentation
• Delivered with the data
• Available online
• Not found (Skip to Customer Services)
•  

•  Still using the 10-point scale on which 1
  means Poor and 10 means Excellent, how
  would you rate
• Overall quality of the document (i.e., technical
  level, organization, clarity)
• Extent to which the data documentation helped you
  use the data

22
Customer Service

•  Was it
• By phone
• By E-mail
• Both by phone and e-mail

• Have you requested assistance from ltData center
  namegts user services office during the past
  year?
• Yes, continue
• No Go to Closing 

• Think about the user services staff you
  interacted with when you requested assistance
  from ltData center namegt user services. On the
  same scale from 1 to 10 where 1 means Poor and
  10 means Excellent, how would you rate the user
  services staff on
• Professionalism
• Technical knowledge
• Accuracy of information provided
• Helpfulness in selecting/finding data or products
• Helpfulness in correcting a problem
• Timeliness of response

23
2009 Survey Responses
Data Center Number of addresses Bounced Back Received Invitation Responded Response Rate
GHRC 1174 161 1013 82 8
ASF SAR DAAC 1372 144 1228 127 10
CDDIS 1597 612 985 143 15
GES DISC 2248 812 1436 101 7
SEDAC 2338 184 2154 131 6
ORNL DAAC 2351 229 2122 164 8
LaRC ASDC 2386 410 1976 144 7
PO.DAAC 3770 1425 2345 173 7
OCDPS-OBPG 3841 381 3460 266 8
NSIDC DAAC 4673 911 3762 312 8
MODAPS LAADS 8198 1119 7079 696 10
LP DAAC 11585 916 10669 1503 14
Total 45533 7304 38229 3842 10
24
Thank you! any questions?
25
Some thoughts on users
http//www.humanfactors.com
26
Some thoughts on web sites
27
And just a few more