?????? All about quality !!

1
??????All about quality !!
2
Testing

• Testing is not important in school homework
• Testing is not important in research work to
  produce experimental statistics for publishing
  paper
• Testing is not important in a research prototype
• Testing, however, is very important for a
  commercial products.

3
Why testing?

• Lets look at some well-known software defects
• ??????????????????,??????????????? (no stress
  testing)
• ???????????
• 2000-2005- ?????????-5?????????????????15?????????
  ?
• 2003 -????????????????5000?????,3???
• 2000 ????????,4??? (??????)
• 1997 ????,225??? (????????)
• 1995 ???????????159???(??????)

4
Statistics

• 2002- ?????????-????????595?????? (0.6 GDP)
• ??????
• ?????????????
• ?????????????

5
Quality Control and Quality Assurance

• Software is a product.
• Good quality product requires ????quality control
  (QC) and quality assurance (QA) ????

6
Manufacturing in other discipline

• ?????????
• ????(quality)?????????
• ????????????????????????100????100???????
• QC (Quality Control) ????
• ??????????????????????
• ????????????????
• ???????????(process improvement)?????,????????????
  ?

7
How to improve quality?

• ????????????????,?????????????????????????????????
  ???????????????????????????,???????
• Ex. ????????????????
• Question
• ?????????????????,?????????????

8
Assembly line in manufacturing
9
???

• ???????????????????????
• ???????????
• ?????????????????????????????????????
• ??????????.

10
Question?

• Is coding a design process or a manufacturing
  process
• Is Code a design or a product?
• Again, the analog breaks
• More questions
• If coding is not a design process, we should be
  able to hire many ????? to control our quality
• If coding is a design process, bad news, errors
  are inevitable because programmers are always
  doing complicated design jobs, not simple and
  easy job.

11
Facts

• Now, you should know why software has so many
  bugs or notorious for being flawed

12
The QC story
?????
software
??????
QC ????
13
A Joke

• ???????????????,
• ???????????,
• ?????????????
• ??????,???????????,?????????,?????????????,???????
  ????????
• ?????????,????K??????,???????????,?????????????,??
  ?????????????,???????
• ?????,???????????????,??????????????,????????????
  ?????????????????

• ?????????,
• ????????????????????
• ???????????????,
• ??
• ????????????,???????????????????
• ???????????,??????????????,?????????,?????????????
  ??????????,???????????????

14

• ???????,
• ?????????,
• ??????????????
• ?????????
• (??????????????
• 21????????????????
• ????????)?
• ????????,
• ????????????????
• ?????????,
• ???????????????????
• ??????,
• ?????????????,
• (????????????),

??????????????, ???????????????? ???????????! ??
???????, ????????????????, ???????????-- ?????????
????, ???????, ??????????????......? ???????, ???
????, ???????, ????????, ??????????????????, ?????
?????????????......?
15
Software Quality

• ???????,??????????????,,??????,??????????
• ???? programmers
• This is why testing is so important in matured
  software industry.

16
??????,?????????
17
How do you test a DVD player
18
How do you test a software
19
The complexity of software testing

• ???????????? (including user interface, network
  interface, file interfaces, etc. )
• ?????????????????
• Correct execution paths(?????)
• Incorrect execution paths (??????)
• ??????????????????????,??,??,??????????

20
SDLC (Software Development Life Cycle)

• ???? Requirement analysis (marketing research)
• ?????? Software function/performance
  specification
• ????? Analysis and Design (QA ??????????)
• ??,??PL,?? platform, database, network protocols.
• ????????,???????,??,??
• ????????? Coding and unit testing (by
  programmers)
• ???? Integration testing (by programmers/QAs)
• Alpha testing (by QAs)
• Most software functions and features are
  basically completed
• All functions are tested, no functions will be
  added beyond this point
• Serious flaws (high severity) are solved and
  addressed (show stopper)
• Beta testing (by Beta users)
• sub-serious bugs are all fixed
• Test plan has been completely executed
• Bug discovering rate is lower than bug fixing
  rate
• ???? Release
• Bug discovering rate is lower than bug fixing
  rate for a long period of time.
• The version after fixing bugs has been
  regressively tested (regression test)

21
Common software defects

• 70 occurs in design and hard to correct
• Software specifications are not precise
• Software is complicated
• Coding errors (20)
• Function changes which affects other components
• 3rd party software is flawed

22
? Early detection of errors
Error found Mar 13
Error found Feb 13
Error occurs Jan 13
Work (person-days)
10
20
Time
Work proceeds at (say) 10 person- days per month
10 person-days of work has been done assuming
the error is not there. Now this must be redone.
If error found this late, 20 person-days must be
redone
23
What to test for a QA (?????)

• ??? (completeness)
• ???????????,??????????????
• ??? (correctness)
• ??? (reliability)
• ??? (compatibility)
• ??(efficiency)
• ???? (usability)
• ??? (portability)
• ????? (scalability)
• ??? (testability)

24
??

• ????????????
• ???????????????????,????????????????
• ??????????,???????
• ???????,?????????,????????????
• ??????????????

25
??????

• ?????????
• Big software company has their testing team
• Small software companies which do not have
  resources to test can outsource to 3rd party
  testing company
• ?????????
• Big software company like ????Trend Micro has QA
  team
• Many software companies use programmers as
  testers a sad fact!

26
Company Organization
Developers
Marketings
QAs (testing teams)
27
Management Hierarchy
Director of development
Director of QA
Developer manager
Developer manager
Test manager
Test automation Team manager
Test facilities manager
programmers
Test automation programmers
Test database programmers
programmers
testers
28
Important reasons to make QA team and developer
team equal and separated

• ????????? programmers ?????(????)???
• ? programmer ????,???bugs ???????????,???????????
• ?????????bug ,?????????,????????????
• ??bug??,??????????,????????,???????
• ??????????
• QA ????????? programmer ?????
• QA ???????????? bug,????,QA??????
• QA ??? developer team ??,???????????????????

29
Marketing
????
Developer
QA (testers)
30
????V??
????
??????
????
??????????
???????
????
??????????
??????????
???????
????
???????
????
????
31
Smoke Tests

• A smoke test is a subset of the test cases that
  is typically representative of the overall test
  plan.
• Smoke tests are good for verifying proper
  deployment or other non invasive changes.
• They are also useful for verifying a build is
  ready to send to test.
• Smoke tests are not substitute for actual
  functional testing.

32
Load testing

• To exercise the system to the maximum load that
  is specified in specs.
• The goal is to test whether the system meet the
  requirement

33
(No Transcript)
34
Stress testing

• The goal of stress testing is to exercise a
  system beyond the load in specs to see what it
  can happen.
• require considerable cost and efforts.
• often require you to implement a system to test
  the system.
• Load testing is necessary but stress testing is
  optional.
• E.g., a online game server may limit the users to
  prevent system from crash

35
Lets back to the reality

• Most vendors do not possess a separate QA team
• They can fake test plan and report
• So? What can we do with the vendor?
• Ask for a bug report system (issue tracking
  system)
• Examine the test cases and sampling
• Smoke test
• Extended test
• Critical test cases
• Watch the bug report history

36

• If you are hired to build a dog house,
• what ability you should
• Have?

37
Different Scale of Software Development

• If you are hired to build a Taipei 101
• What skills and talent do you need to have?

38
Usability Test
39
A Common story

• ?????????????????
• ????????, ????,?????
• ??????????
• ?????????????
• ??????, ?????????????????
• ????????????
• ?????????????????????????,???????
• ??????????
• ?????,??????????????????????,????,?????????,?????
  ?????
• ???????? usability test

40
Some facts

• Most programmers do not possess the knowledge in
  UI design
• Programmers often think UI is not a difficult
  part, which is wrong in practice, especially in
  commercial products
• Change UI design (e.g., making UI more friendly)
  often means writing a lot more code, which is not
  expected.
• Programming tasks are often spec-oriented, not
  user oriented.

41
(No Transcript)
42
The failed story
43
Why Microsoft lost its markets?

• UI
• UI
• UI
• UI
• ..

44
Goals of usability tests

• Performance -- How much time, and how many steps,
  are required for people to complete basic tasks?
  (For example, find something to buy, create a new
  account, and order the item.)
• Accuracy -- How many mistakes did people make?
  (And were they fatal or recoverable with the
  right information?)
• Recall -- How much does the person remember
  afterwards or after periods of non-use?
• Emotional response -- How does the person feel
  about the tasks completed? Is the person
  confident, stressed? Would the user recommend
  this system to a friend?

45
Some usability test method

• Think aloud protocols
• Users are asked to say whatever they are looking
  at, thinking, doing, and feeling, as they go
  about their task.
• Observers at such a test are asked to objectively
  take notes of everything that users say, without
  attempting to interpret their actions and words.
• Test sessions are often audio and video taped so
  that developers can go back and refer to what
  participants did, and how they reacted. The
  purpose of this method is to make explicit what
  is implicitly present in subjects who are able to
  perform a specific task

46
Eye tracking
47
Eye tracking pattern
48
Investigation room
49
Observations

• Most ?????? I have used are difficult to use,
  learn, and remember
• Poor exception handling
• Function-oriented is a common

50
Source Code Quality
51
?????

• ?????????????
• ???????
• ????????
• ??????????????????

????,??????????????,????????????? (an order of
magnitude, 101) - SACKMAN, EROKSON, and GRANT
52
Yes

• ??????????????
• ?????
• ??????????
• ?????????????????????
• ????????????
• ?????????????????

53
AND?

• ????????????(???????????)
• ??????????????
• ??????????????
• ????????

54
Typical symptoms of bad code

• Difficult to understand
• Difficult to change (changing at one place
  results in changes in many other places)
• Hardwire the solutions (for example, instead of
  coming out a nice algorithms and data structures,
  many if is used)
• Source code is tightly coupled because the
  excessive use of global variables.
• Poor design that is difficult to extend.

55
Reasons for writing bad code

• CS curriculum
• In CS major, main focuses are problem solving
  such as algorithms, techniques (networking,
  multimedia).
• writing high quality code is not encouraged or
  viewed as an important topic among students

We teach students how to assemble gears to solve
a problem but do not educate them how to produce
quality gears and measure quality of the gears
56
What is a good design?
57
Comfortable?
58
The truths are

• Clean and simple interface, replaceability,
  modularity, composablity
• Their importance is not obvious in Software
  development, particularly if you never has chance
  to maintain a software

59
Coupling (??)

• Most software engineering methods (e.g., OO)
  focus on create beautiful design so that
• Change one thing does not involving changing
  many other things
• Replace/substitute one thing does not involving
  modifying many other things
• Extend one thing does not involving rewriting
  many other things
• High coupling is bad
• We pursuit low coupling
• OO is a technique to decouple the system
  architecture
• The concept of coupling will be introduced in the
  future.

Remember this word
60
Again

• Why writing low-coupled source code is not as
  easy as other engineering?
• Software is invisible and source code is not
  geometric related that you can judge from your
  eyes
• The dependency and coupling are not obvious and
  easy to identify. Consider a poor programmer
  writing thousand lines of code for you.
• Software behaviors are dynamic

61
Reasons for writing bad code

• Experienced programmers may write bad code
• Tight schedule and limited budget
• Demo-oriented programming
• Not enough time to do design
• They accustomed themselves to write bad code if
  no adequate software engineering processes has
  been established in their working environment

62
Motivation

• What is a bad code smell?
• No formal definitions and descriptions have been
  established so far
• Sometimes, it is not easy to explain why a piece
  of code is bad to beginners
• This means bad is not measurable quantitatively
  
• Understand what is bad can lead to a better
  understanding of what is good. For example, why
  using some design pattern is good and what can
  be improved?

63
Spaghetti Code
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
Lasagna code

• Lasagna code is used to describe software that
  has a simple, understandable, and layered
  structure. Lasagna code, although structured, is
  unfortunately monolithic and not easy to modify.
  An attempt to change one layer conceptually
  simple, is often very difficult in actual
  practice.

68
(No Transcript)
69
Ravioli code

• In ravioli code, each of the components, or
  objects, is a package containing some meat or
  other nourishment for the system any component
  can be modified or replaced without significantly
  affecting other components.
• The ideal software structure is one having
  components that are small and loosely coupled
  this ideal structure is called ravioli code.

70
(No Transcript)
71

• Now, lets look at some spaghetti code and give a
  critics

72
Top 100 signs that I may be writing spaghetti code

• I have no idea where this constant is defined
• I have printf stmts littered throughout my code.
• There is an error but it isn't handled, and I
  can't find it to figure out what's wrong.
• Passing variables directly to 635 functions and
  using 'global' on 453 functions
• I have to modify the same function 10 times
  because I have not moved it into a
  function...yet.
• I find that rewriting is quicker and easier than
  modifying the original design.
• You have a library of small functions that you
  wrote yourself, and will cut'n'paste them into
  almost every project you work on... Whether
  they're needed or not...
• Now lets continue and win your credits

73
Top 100 signs that I may be writing spaghetti
code- (CONT)

1. Every time you want to change a variable you have
   to use grep to find it )
2. you have so many "for" loops you've declared a
   variable for each letter in the alphabet.
3. there are enough "if" statements to make you
   scroll to the right.
4. You see comments in the code that read something
   like Your Name Don't ask me how or why this
   works, I know it looks overly complex and it
   appears there IS a simpler solution, but this
   works, it fixed a bug from the 'simpler
   code'...just leave it be...don't touch it, I've
   hired a voodoo witch doctor to place a curse on
   anybody who alters this code...trust me, just
   leave it alone!
5. you know you're digging through a bowl of the
   stuff when, as a regular tool, you're using the
   windoz's search feature to find functions,
   variables and other bits of code.

74
Top 100 signs that I may be writing spaghetti
code- (CONT)

1. having to "peel" your way through while/for loops
   and a chunks of IF and ELSE statements because
   the original author (probably in all his haste)
   couldn't be bothered to use brackets. The code
   works but not always for the right reasons.
2. use every imaginable variation of the word
   "foobar" in variable names
3. You sit down, open your program, and wonder where
   to begin with debugging... (ooh boy, I know the
   feeling of that!)
4. You wrap it up and realize that there are 20
   funtions in the include files which are never
   called, although at some point, they were all
   required.
5. Changing database is (e.g MS SQL to MySQL)
   requires the rewrite of SQL statments.
6. You realize that you are changing global
   variables with functions inside include files
7. You create a separate class for every single
   function.
8. You have no idea what a CRC Card is ...

75
Again, lets back to reality

• Source code quality can be manifested
• When you ask vendor to do some change, it takes a
  lot of time to complete it.
• And, it gets worse and worse..
• Until the original programmer leave
• They dont want to earn the maintenance contract
• Someday, you prepare new money for a new vendor
  to rework (transferring the data is a basic
  requirement)
• There is definitely no vendor dare to promise a
  task to repairs other vendors system

76
My advises!

• This is the most difficult part in software
  engineering! Little can be done about it.
• See it as a power to put more investment on
  software (The principle for FORD manufacturing)
• Like hardware, software cannot last forever
  (especially the small companies that produces MIS
  systems)

77
Test Driven Development

• For those of you who are programming in your work