U.S. Charter Schools Website Usability Test

1
U.S. Charter Schools WebsiteUsability Test

• August 30, 2004
• Sutton Stern

2
Overview

• Goals
• Process
• Participants
• Summary Results
• Findings Recommendations

3
Goals

• Research Goal
• Assess the overall usability of the web site,
  with special focus on navigation, page layout,
  and content
• Design Goal
• Assess the web sites ability to meet the most
  important goals of its most important visitors
  then make design and navigation changes
  accordingly

4
Process

• 5 participants tested
• Usability session administered remotely via WebEx
  and recorded
• Test combined a free exploration activity with
  pre-determined tasks
• Post task assessment forms

5
Participants
6
Summary Results

• Many successes on the site
• The State Profiles section was easy to use and
  valuable to all 5 users
• The scope and content of the news items were very
  well received by all 5 users
• Just for Parents, NCLB, Guide to Successful
  Charter Schools content all accessible and highly
  rated
• Global assessment positive
• 4 of 5 users answered This is better than most
  of the web sites that provide information to the
  question Which of the following best describes
  the web site?
• Web Dialogues intrigued users
• Teachers thought to be good candidates to use
  this feature
• Administrators interested if the subject matter
  was relevant

7
Summary Results

• Homepage fails in presenting the global
  navigation
• Attention of users is on the center of the home
  page
• Homepage shortcuts used almost exclusively
• Resources section underutilized
• Site serves certain user profiles well, but
  under-serves others
• Site is good for school developers, especially
  those getting a school started, evangelists,
  and parents
• Site is less effective for administrators and
  teachers
• At a high level, the sites focus or purpose
  isnt clear
• Who is the site for?
• What is the site trying to help it users
  accomplish?
• Site needs to support specific user profiles and
  tasks as well as reflect larger Department goals

8
Summary Results

• Second-level pages should be index pages, not
  content pages
• Page layout not working for users at the 2nd
  level
• Indexes missing across the site
• Information is often presented by date not by
  subject matter
• No global site map or index
• Content is good except in representing the work
  and success of actual schools
• All users looking for examples from existing
  schools to either model or use as a tool for
  communicating CS value
• This was balanced by depth and breadth of
  well-targeted and well presented content

9
Summary Results

• For administrators time is the most precious
  commodity
• The big issue for me is time. I work 10 to 12
  hours a day.
• -- Participant 5
• This is making me work too hard.
• --Participant 2

10
Findings

• Homepage Layout
• Navigation
• User Profiles
• Content
• The Index
• Grants
• Community / Dialogues

11
Homepage Layout

• That home page is just not user friendly it
  doesn't take you to anything easily
• - Participant 3, CS Developer

12
Homepage

• Homepage fails to represent the sites
  organization to visitors
• 3 of 5 participants did not see or ignored the
  global navigation until the end of the session or
  until prompted
• 3 of 5 participants never saw the Resources tab
  until the last task and failed to complete key
  Resources-related tasks
• Users consistently used highlight and newsletter
  boxes in the middle of the homepage as the
  primary navigation
• Photos and roll-over graphics below the global
  navigation helped divert attention from global
  navigation and did little to enhance the user
  experience

13
Homepage
Global Navigation not very apparent
Critical resources section not seen
Resource Updates mistaken for Resources section
Feature navigation dominates the page and
becomes primary navigation
14
Homepage

• Simplify homepage so that it represents the
  overall site organization to visitors
• Restore Global Navigation primacy
• Move away from shortcut or feature based
  navigation
• Promote search and shortcut functionality in an
  obvious and consistent way
• Provide space for features but avoid making them
  dominant
• Design a lighter, less dense, less branded
  homepage
• Also
• Consider for homepage features funding, NCLB, and
  important new resources as subject matter

15
Homepage
Consider pull-down menu for key site destinations
to replace feature based navigation
16
Navigation

• "I have trouble finding things in here so I
  wouldnt look.
• - Participant 3, CS Developer

17
Navigation

• Limited value in current navigation scheme
• Further research needed for effective information
  architecture clear and focused strategy also
  needed
• Consider Subject Matter based navigation
• Starting a School
• Managing a School
• Charter Laws
• States
• Community
• News and Events
• Or Roles based navigation
• Developers
• Charter School Supporters
• Administrators
• Parents
• Teachers

18
Navigation

• Utilities need consistent definition and
  presentation. Possibilities include
• Search / Shortcuts
• Login / Register
• Contact
• Subscribe
• Home
• Glossary
• Secondary navigation needs to be presented
  clearly and consistently
• Current 2nd level pages mix content approach with
  index approach
• Selected secondary navigation choices need to be
  better represented than current bold arrow
  remove all deselected arrows
• Use index page format for each global navigation
  home page

19
Navigation
2nd level pages global navigation homepages)
fail to orient users and present clear navigation
choices to them.
Indexed text links should duplicate the In this
section navigation
Staring a CS home page provides good example of
indexed, text link approach to 2nd level pages.
Navigation choices are listed and explained to
enable an accurate next click.
20
User Profiles

• "all of the information needed to write a
  charter looks like it's thereLots of good stuff
  here.
• - Participant 3, CS Developer

21
User Profiles

• For some user types, the site provides both
  nearly ideal content and access
• For Developers starting a school, the Start a
  Charter School content was highly rated by 5 of
  5 users
• Content about creating new charter schools from
  models of successful schools would be valuable
  to founder/developers
• Parents also well served
• Evangelists can find much information to
  support their communications efforts
• USCS site managers should strongly consider
  Persona development as initial step in any new
  design cycle
• Key tasks and user profiles need to be identified
  and modeled to create a more user-centric site
  architecture and strategy

22
User Profiles

• Administrators have to balance severely limited
  time with the need/desire for information
• Want to stay abreast of funding opportunities
• Want to recruit great teachers
• Want ideas from other schools
• Want to say abreast of charter school news,
  nationwide
• Prefer to sort or search where possible, rather
  than
• Will scan rather read
• Have little time to locate all of this
  information
• Want things for their school and to know what can
  the DOE give them?

23
User Profiles

• CS Principals are more like directors than
  administrators and need strategic information
• Case studies and research are seen as valuable
• Current site does very little for teachers
• Most participants believed that teachers would
  have the greatest interest in Community type
  content
• Consider creating web dialogues oriented toward
  the needs of Charter School teachers
• Teachers at Charter Schools are also board
  members, future administrators, and future
  developers so the site must support the needs of
  teachers now
• One teacher wanted classroom materials available
  on the site

24
Content

• All along, Id like to look at schools that are
  ahead of us.
• - Participant 5, CS Principal
• I think successful examples are useful.
• - Participant 3, CS Developer

25
Content

• All users want to see examples of successful
  schools
• Users were looking for examples of a wide variety
  of practices from successful schools
• When asked about creating accountability
  programs, developing hiring plans, administrators
  in particular wanted first to see examples of
  what other schools had done
• When asked where to find information about
  sustaining and improving their schools, 4 of 5
  participants wanted to see how other schools had
  done it first
• A glossary of terms, especially acronyms, is
  essential
• Also set forth an acronym usage policy, then
  apply it consistently

26
Content

• Content should be integrated by subject matter
• Dont separate research, resources, case
  studies, news, and web dialogues by content type
• Visitors are looking for information on a
  subject, not looking for a category of
  information
• Surface ALL content around a single subject in a
  single place
• Consider adding content for special cases, like
  Military Schools, Music Schools, Charter
  Districts, etc.

27
Content

• Meaning of Resources a little unclear
• Whats the difference between content and
  resources
• Resources on other sites are often downloadable
  documents not html pages. On the USCS site they
  are both.
• Consider an IRS model, where hard info is in a
  PDF format and html pages present only summary
  info and context
• Time-strapped visitors want to find something on
  the site, then download it and read it later as a
  document

28
Content

• Recency is critical
• As participants browsed they observed dates that
  were in the past and doubts about the currency
  crept in
• Content Czar needs to ensure recency and remove
  dated materials

Even if the content isnt out-of-date, dates make
it feel as though it is.
Remove all dates and indicate current year
content with a New! symbol
29
Content

• Aggregate Access
• Many resources available here as well as on other
  DOE sites
• All DOE resources should be accessible to Charter
  school site users either through access to a
  searchable Data base or through clearly defined
  links

30
Indexing

• Im all about the Index
• - Participant 4, CS Teacher/Board Member
• It just has dates. It doesn't have titles. I'd
  have to look at everyone of these things to see
  if it said anything."
• - Participant 2, CS Principal

31
The Index

• The site underutilizes fundamental organizational
  method
• No site map or site index-- essential to most
  resources sites
• Participant 3 says "no place to go right now
  alphabetically to get me to go from one place to
  another."
• 2nd level pages should employ an index design to
  support secondary In this Section navigation
• Resource Updates should be organized by subject
  not by date

32
The Index

• Resource Updates should be organized by subject
  not by date

This content must be presented by topic. Users
very reluctant to click through dates to find
subjects of interest .
33
Resources
34
Resources

• The resources tab is was not easily seen by
  participants
• Resources need to aggregate all resource types by
  subject matter, e.g, under Managing a School
• Finance and Facilities
• Governance and Management
• Accountability
• Policy and Oversight
• Resource Updates should not be a separate section
  but should be available in all the resource topic
  areas
• Use indexes throughout the resource topics area
  to make many resources easily scannable to users

35
Resources
Successful Search Resources interaction should be
presented throughout the site as a global utility
36
Grants

• Grants are something that we are always looking
  at, so I would come back and use this page."
  - Participant 5, CS Principal

37
Grants

• Funding information is the most important
  information available on the site to critical
  users like administrators, board members, and
  developers
• Grants must be easily parsed
• Information design must make critical information
  easy to scan
• Eligibility (avoid acronyms)
• Deadline
• How or through what organization grants are
  awarded
• Funding information should be surfaced whenever
  possible on the homepage
• Feature timely individual grants from the
  homepage and a link to all federal grant
  information from that feature

38
Dialogues/Community

• This is an interesting thing to me. it would be
  interesting to try." - Participant 2, CS
  Principal

39
Dialogues

• The login box is a definite barrier
• 4 of 5 participants believed that having a login
  ID was required to view threads and summaries
• Reduce Login ID visibility and surface
  prominently only when a user attempts something
  that requires it
• Users believed content the dialogues was valuable
  and were interested in attending, but almost all
  lacked confidence in their web savvy
• There are good instructions to participate but
  these need to be surfaced at the top of the
  dialogues home page
• A message assuring users how easy it is to login
  and join and as well as the describing the option
  of anonymous participation should be prominent
  upon accessing the dialogues

40
Dialogues / Community

• Dialogues should be expanded and promoted as a
  permanent site service
• Current Community Tab is confusing
• Users associated the Calendar and the National
  conference with news NOT with Community
• Users were unable to find the National Conference
  Information
• News and Events Navigation tab will solve this
  problem
• Web Dialogues do not appear to be in Community,
  but should
• Consider making discussions a permanent Community
  feature targeted at teachers and administrators
• Make the discussions VERY functional and
  relevant, choose and promote bulletproof topics
  about hiring and obtaining funds, etc.

41
Dialogues
Remove this login box and depend on login utility
in upper right corner

This great explanatory content needs to be
featured prominently in a way that reassures
inexperienced potential participants.
42
Questions

• Sutton Stern
• sutton_at_easeability.com
• [phone] x308