Integrating Testing into the Curriculum

1

Integrating Testing into the Curriculum Arsenic
in Small Doses
SIGCSE 2001, Charlotte, NC USA February 21-25,
2001 Edward L. Jones CIS Department Florida AM
University Tallahassee, FL USA
2
Outline

• Motivation Driving Principles
• Experience-Based Testing Framework
• Pick Your Poison
• Elective Software Testing Course
• Opportunistic Insertion (suggestions)
• Automated Program Grading
• The Software Test Lab (infrastructure)
• Conclusions / Future Work

3
Motivation

• 50 software cost related to testing
• Test practitioners for industry, research
• Testing complements development skills
• Approach Salt core courses with small doses
• Adding another course is not an option
• Small dose does not change taste of core courses
• Cumulative effects are beneficial

4
Guiding Principles

• Experience-based
• Practice what you preach -- testing in action
• Each course provides testing experiences
• Diverse experiences across courses
• Students participate in integration of testing
  into curriculum

5
The SPRAE Framework Essential Elements
Specification the basis for testing
Premeditation a systematic process
Repeatability tester independence
Accountability documented process, results
Economy cost-effective practices
6
Test Life Cycle
Specification
Test Plan
Test Cases
Test Script, Data, Driver
Test Results, Log
Defect Data Problem Reports
7
Elective Course -- Lessons

• 80/20 Practice/Theory
• Advantages
• In-depth, continuous concept treatment
• Preparation for future advanced work
• Deficiencies
• Not a mainstream course available to all
• Students compartmentalize course content
• Lessons
• Why Johnny cant test? Programming skills!
• Testing complements other software skills

8
Opportunistic Insertion into Existing Courses

• Risk of diluting course content
• Colleague asked to adjust course content
• Incremental approach
• One new test-salted experience in each course
• One new course each term
• Support colleague doing insertion

9
Simple Things to Try

• Grade another students program and justify grade
  -- in writing.
• (Group) Develop and discuss test cases before
  writing program.
• Treasure Hunt. Find seeded errors, document
  process followed, give evidence of fix.
• Blind testing. Write specification from
  executable.
• Develop and sell certified components to be used
  in subsequent assignments.

10
Testing in Action Automated Program Grading
Benefits
Overhead
Implement Grader
Grade Programs
Grading Log
11
Automated Program Grading Results Observations

• Student shock outrage at exactness
• Student behavior modification tester mindset
• Extra work on teacher
• Specification must be better than average!
• Must follow the software testing lifecycle
• Cost amortized via similar assignment styles
• Is grader too strict for CS1?
• Selling colleagues on the idea!

12
The Software TestLab

• Environment for training students
• An evolving laboratory
• Tools tutorials
• Staff (faculty, hand-picked students)
• Test problem/solution test bed
• Students participate in TestLab evolution
• Feedback on lab resources
• Create/Refine TestLab artifacts
• Workforce -- lab/grading assistants
• Technology insertion support services

13
TestLab -- The Big Picture
Corporate Sponsors

• Marketing
• Interns
• New Hires

Support ()
Software TestLab
Curriculum
Dissemination
TestLab Students
14
Training Model for TestLab Students

• Manage student skill development
• Set clear achievement goals
• Key Practices x Competency Levels
• Certify levels of progression
• Establish recognition program
• Certified students mentor/assist

15
Key Practices

• Practitioner - perform defined test.
• Builder - construct test machinery
• Designer - design test cases.
• Analyst - determine test needs, strategy.
• Inspector - verify correct process, results.
• Environmentalist - maintain test environment.
• Specialist - perform entire test life cycle.

16
Test Specialist I -- Competencies
17
Conclusions

• Testing should complement, not compete
• Numerous opportunities to inject testing
  experience into the student experience
• Little things may go a long way
• Support system (TestLab) needed
• Integration will take time

18
On-Going Future Work

• Evolve TestLab Training Model
• Select Courses for Doses
• Expand Support Automated Grading
• Disseminate Results
• Educational study
• Web site, conference publications

19
Questions?
Questions?
Questions?
20
Thank You