Computer Science Undergraduates Address the Severely Handicapped

1
Computer Science Undergraduates Address the
Severely Handicapped
Michael Buckley Department of Computer Science
Engineering University at Buffalo
2
(No Transcript)
3
or... why we do what we do...

• we are problem solvers
• David is 43 yrs. old, suffered a stroke at 27,
  has not spoken since. Just wants to order a
  pizza.
• Jackie is 19 and has never communicated.
• Dean is 5 yrs. old, autistic, can select radio
  stations, does not make selections on anything
  else including a communicator.

4
(No Transcript)
5
(No Transcript)
6
The Problem Space

• People occupy the Problem Space, computers occupy
  the Solution Space
• High quality systems successfully model the
  problem space in the solution space.
• CS should stress problem space thinking.
• Computer solutions more naturally fall out of the
  Problem Space perspective.
• In this case, Quality of Life is the problem
  space

7
Looking for lessons in philosophy and sociology

• The best programs are simple and elegant
  solutions to complex problems.
• The best design is often the one that offers
  least resistance.
• Uncomplicate your designs.
• Program in the problem space, not in the solution
  space.
• Be sensitive to needs and circumstances.
• Computer Science as philosophy, doing good.

8
Armed with this set of tools....

• Develop systems that are used, for real.
• Motivate students and stimulate (simulate?)
  interest.
• Student creativity labor (i.e. free) generate a
  wealth of possibilities.
• Raise awareness of socially relevant problems.
• Improve quality of life for the handicapped.
• Responsibility and ethics .

9
Benefits

• Changed student behavior.
• Cheating disappeared.
• Projects focused incredible creative energy.
• Students saw themselves as individuals with the
  potential to give something to others.
• Students altered their own career objectives as
  result of these projects.
• Students who were at risk of dropping out or
  transferring schools stayed at UB to see these
  projects to completion.

10
Social Relevance Is Integrated Into the U.B.
Curriculum

• Embedded in Courses
• Programming for Majors Non-Majors
• Computer Literacy
• Throughout the Course
• Senior-level -- Software Engineering
• Computer Engineering Capstone Design Course
• Research

11
Social Relevance? Included in the Course

• Courses for Non-Majors Vignettes
• Voting Trends
• Propagation of Pollution in the Great Lakes
• Medication and Drug Interaction Prediction
• Phrase Selection in Augmentative Talkers for
  Those with Communicative Disorders
• Viruses as Adaptive State Machines
• Cause and Effect / Choice-making Therapies in
  Special Ed.
• Ethics and Responsibility - How computers are
  used matters
• Software Errors affect real people - Computers
  dont make mistakes, people do

12
Social Relevance? / Capstone Courses

• Two course sequence of Software Engineering
  Hardware/Software Integration
• Capstone design courses based on
  university-industry partnerships are more
  successful than stand-alone university courses.
• Socially-relevant courses common in
  social-sciences, healthcare, education
• Why not combine the benefit of university-industry
  partnerships with the ability of
  socially-relevance to make a societal difference?
• Why shouldnt CSE be able to make a difference in
  the world and thus better engage students in
  learning?

13
Social Relevance?Software Engineering
Lectures
Project
Design Methodologies
Customer Relations
Client Requirements
Incremental Methods
Legal and Ethical
Design
Software Errors
Managing Risk
Human Factors
Project Plan
Project Management
Tao of Engineering
Simulation
Testing Metrics
Maintenance
14
Social Relevance?Hardware/Software Integration
Lectures
Project
HW/SW Interfaces
Requirements
Safety
State Machines
Prototyping
Project Management
Configuration Mgmt
Integration Thread
Parts Selection
Case Studies
Final System
Development
Execution
15
Research
16
Research Participants

• A research community outreach group consisting
  of faculty and students from Computer Science and
  Engineering and other disciplines dedicated to
  applying technology to solve socially relevant
  problems.
• Faculty (CSE, AI, Comm. Disorders, Linguistics)
• Michael Buckley Dr. Jeffrey
  Higginbotham
• Dr. Kris Schindler Kim Adams
• Helene Kershner Dr. Michelle Gregory
• Dr. Carl Alphonce
• Dr. David Pierce

17
Socially-Relevant Projects Partnering with Local
Organizations

• Off campus
• Western NY (immediate access)
• Not-for-Profit -- Organizations that serve the
  Handicapped Disabled Community
• The Not-for-Profits serve two-ways
• As clients
• As teachers
• We visit them, they visit us, everyone benefits

18
Our Clients Customers
19
Our First Client ElderWood Senior Care at
Oakwood, Williamsville, NY

• Motivation
• David, a 43-year old stroke patient who has been
  speech impaired for over 20 years.
• Previously communicated using a letter board
• Technology solutions were not adequate for David
• Result
• The UB Talker was developed for David by students
• VESID (Vocational and Educational Services for
  Individuals with Disabilities) has purchased a
  tablet PC for David.
• David will be working with our research group as
  a consultant.

20
UB Talker

• Goal - Give those people without the ability to
  speak, a voice.
• Targeted Population - Handicapped/disabled who
  have limited motor skills and have difficulty
  with voice communication
• Phase I - Adult Talker
• Phase II - Child Talker
• Phase III - for the visually impaired, and for
  the severely handicapped (no motor skills)

21
UB Talker

• Allows for the creation of sentences, phrases and
  speeches that can be spoken with the push of a
  button.
• Phrase Word Completion - Based on frequency of
  use, last-used
• Content Prediction - Time and Context - Based on
  time of day, day of week, season
• Auto-Scan

22
UB Talker
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Main Menu
I want food
27
Keyboard (remember, real keyboard not present on
touch screen PC)
28
(No Transcript)
29
Story mode the ability to pre-enter paragraphs
30
Configurability
31
An Alternative Design
32
(No Transcript)
33
(No Transcript)
34
UB Talker in the News

• USA Today Magazine http//www.findarticles.com/c
  f_dls/m1272/2701_132/109085072/p1/article.jhtml
• Raleigh NC Channel 14 News http//rdu.news14.com
  /content/headlines/?ArID38493SecID2
• WPVI News in Philadelphia http//abclocal.go.com/
  wpvi/health/HK_October2003.html
• Russian Science News http//www.nauka.lucksite.co
  m/archive/2003/news_0109.html
• NY State Office of Science and Technology News
  http//www.nystar.state.ny.us/nl/archives2003/west
  A10-03.htm

35
The Center for Handicapped Children,
Williamsville, NY

• Center Based Program CHC LEARNING CENTER provides
  educational and therapeutic programming for
  pre-school and school age children (infant to 21
  years of age).
• Home Based Program Certified special teachers and
  licensed therapists provide services in the home
  for infants and young children unable to attend
  the center based program.
• Support Services Occupational, Speech and
  Language, and Physical Therapies are available to
  all students at CHC LEARNING CENTER.         
• Philosophy CHC LEARNING CENTER recognizes the
  need for support for the families of children
  with disabilities and the benefits to child and
  family when services are truly comprehensive and
  integrated.

36
DISCO

• Disabled Interactive Sensory Coordination
  Opportunities
• Interface multiple inputs (switches) to yield
  multiple outputs (lights and sound) through the
  use of a computer to create a cost-effective
  learning tool for disabled handicapped
  students.
• This tool accommodates a variety of needs beyond
  the hand and eye
• The use of lights, fog, music, video, etc. to
  create soothing and entertaining stimuli

37
DISCO

• Simple choice-making and the realization of
  cause-and-effect are a central goal of early
  education for the physically handicapped and
  developmentally delayed, especially children.
• The relationship between physical action and
  ultimate effect is not necessarily natural.
• The ability to select between alternatives of
  increasing number is part of a students
  Individual Education Program (IEP) from early-on.
  

38
DISCO

• The mechanism used usually presents large colored
  switches which respond with different noises or
  spoken words
• first a single switch,
• then two, then four,
• until sufficient cause-and-effect is established
  to graduate the student to a more powerful
  augmentative communications device such as a
  touch-screen talker.

39
DISCOs Goals

• Build a system that can create sensory experience
  for handicapped children
• Make the experience completely easily
  programmable by teachers and therapists
• Generate statistics from the saved log files in
  each session to show progress
• Ability to suggest sequences based on past
  statistics
• Engage University students in coursework that
  makes a difference

40
hardware setup
41
Interfacing hardware to a Tablet PC

• from Measurement
• Computing

42
..to AC power

• from StarMagic

43
to Lights and Sound
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Program Screen
52
In real use identifying progress on a client IEP

• Grades progress in choice making, cause and
  effect
• Categorizing the lessons verbal cues, visual
  cues, hand over hand?
• Looking for specific trends
• Showing graphically how certain statistics vary
  from session to session

53
Examples of Useful Statistics

• Each profile will contain a log file of every
  session that Disco was run
• Things we can calculate from these files
• Number of sequences passed, done or repeated
• Number of inputs repeated, or changed
• Pattern recognition
• Percent change of any of the above from session
  to session

54
Sample Progress Chart
55
some stories

• Ben
• telephone
• Jackie

56
(No Transcript)
57
Contact InformationComputer Science
EngineeringUniversity at Buffalo

• Helene Kershner, MS
• [phone] ext. 118
• kershner_at_cse.buffalo.edu
• Carl Alphonce, Ph.D.
• [phone] ext. 115
• Alphonce_at_cse.buffalo.edu

• Michael Buckley, MSEE
• [phone] ext. 156
• mikeb_at_cse.buffalo.edu
• Kris Schindler, Ph.D.
• [phone] ext. 135
• kds_at_cse.buffalo.edu