Software Testing in Small Teams

1
Software Testing inSmall Teams

• Don R. Gilman, P.E.
• Energy Systems Lab TEES
• Advanced Gaming Systems Inc

2
The Presenters

• Don
• Professionally programming/managing IT
  applications since 1984
• Contributed to over one million units of various
  computer games
• Licensed Professional Software Engineer (one of
  about 35 in the entire US)
• Member of the Software Engineering Task Force for
  the Texas PE Board
• Manager, Software Engineering Group Energy
  Systems Laboratory TEES

3
Presentation Overview

• What is testing?
• What are the levels of testing?
• Types of testing
• Who tests?
• Case Studies
• Unit Testing
• eCalc Automated Testing
• A Computer Game
• Summary
• Questions

4
What IS Testing?

• IEEE The process of operating a system or
  component under specification conditions,
  observing and recording the results, and making
  an evaluation of some aspect of the system or
  component.
• Boehm Verification and Validation
• Verification are we building it right?
• Validation are we building the right thing?

5
Testing is NOT

• Code inspections
• Design reviews
• Configuration management
• Bug tracking
• These are, along with testing, are part of
  Software Quality Assurance (SQA). Together these
  improve the quality of the product

6
Types of Testing

• Scope
• Unit
• Module
• Integration
• Interface
• Coverage
• Purpose
• Functional
• Stress

• Execution
• Manual
• Automated
• Phases/Triggers
• Nightly Builds
• Defect Removal
• Change Order Completion
• Alpha
• Beta

7
Who Tests?

• Developers
• Unit testing
• QA Team
• Module testing, automated testing
• Software
• Automated testing
• Designers
• Engineers, Analysts
• Users
• Unpleasant but true

8
Who some thoughts

• All New Members
• To train
• To get fresh test paths
• In Industry not a good model
• In Gaming a great way to recruit
• Release/QA Folks
• Typically Engineers or Accountants
• CompSci typically make lousy testers

9
Case Study Unit TestingWhat is Unit Testing?

• Lowest Level of Testing
• Programmer Safety Net
• Instant Feedback
• Documentation

10
Case Study Unit Testing Good Unit Tests

• Tests functionality not edge cases
• GOOD 100 points damage to demon death
• BAD referencing null demon throws exception
• Capture the meaning behind the code in tests

11
Case Study Unit Testing xUnit Tool

• Unit testing tools in many languages
• .NET NUnit
• Java junit
• C CppUnit

NUnit
12
Case Study e2Calc
13
Case Study e2Calc
14
Case Study e2Calc
15
Case StudyGames
16
Case Study Games

• Manual Testing
• Coder
• Core team
• Public Alpha/Beta Testing
• Sometimes process andfocus areas listed on
  websites
• Tools
• Command lines for auto logging
• Screen dumps
• Saved games
• Debug compiles
• Game vs game with new log analysis tools

17
Case Study Games
18
Case Study Games
19
Case Study Games
20
Summary

• Writing automated tests is one of the most
  difficult programming assignments
• Design to Test
• Let the computer do the dirty work
• Not everyone is wired to do this only the
  successful succeed.

21
Resources

• Software Development Magazine (www.sdmagazine.com)
  
• CrossTalk (www.stsc.hill.af.mil/crosstalk)
• Better Software (www.stickyminds.com)
• The Rational Edge (www.rational.com)
• SEI (www.seir.sei.cmu.edu)
• TDD, an Introduction (www.agiledata.org/essays/td
  d.htmlFigure1StepsOfTDD)
• Object Mentor TDD (www.objectmentor.com/writeUp
  s/TestDrivenDevelopment)

22
SE Books weve read that matter

• How to Win Friends and Influence People, Carnegie
• Software Requirements, Wiegers
• Agile Project Management, Jim Highsmith
• Planning Extreme Programming, Beck/Fowler
• Agile Project Management w Scrum Schwaber
• Mythical Man Month, Brooks
• Test Driven Development By Example, Kent Beck
• MS Solutions Framework Papers
• Decline and Fall of the American Programmer,
  Yourdan
• The E-Myth Revisited, The E-Myth Manager, Gerber

23
Questions?