Excellence Around the Globe Test Factory Services

1
Excellence Around the GlobeTest Factory Services

• Andreas Golze
• Global Practice Director HP CI

2
Agenda

• Why care about quality?
• Professional testing in a factory?
• How do I to get there?
• Why HP?
• Who has done it?

3
Why care about quality?
4
Why care about quality?
If all that can happen is a project delay

• What is the impact of a project delay?
• Project impact
• A four week delay for a 75 person project 1.3M
  in cost
• 75 people x 900/person/day x 20 days 1.3M
• Business impact
• In September 2004, HP struggled with SAP
  consolidation project resulting in a three week
  delay for their supply-chain software.

• Results HP's ordering system became chaotic
• 400M loss in Q3 revenue
• Damaged reputation with customers
• Three executives were dismissed
• HP's shares dropped 15 on the news

5
Success of IT projects
Source CHAOS Report 2004, Standish Group
International, Inc.
6
Understanding defects
Introduction of defects

• Majority of defects are introduced during the
  requirements and design phase!
• However majority of defects are actually detected
  during user acceptance testing and in production!

Detection of defects
Source NIST 2002 RTI Project 7007.011
6
November 1, 2006
HP Confidential
7
Assessing cost of defects
This industry average is used as a baseline to
calculate the cost savings
Industry References 3 B. Boehm and V. Basili,
"Software Defect Reduction Top 10 List," IEEE
Computer, IEEE Computer Society, Vol. 34, No. 1,
January 2001, pp. 135-137.
8
Establish the software test lifecycle
HP Quality Model (HPQM)

• Prioritize requirements by business impact/risk
• Test with a business process framework

Test
Design
Implement
Deploy
Business requirement
Plan
Requirements verification
Test
Test
Traditionalcost
Planned Cost
9
Profesional testing in a factory?
10
Introducing the HP Quality Model
Customer success roadmap defined
Level 4 Quality Expert
Level 3 Quality Savvy
Level 2 Quality Conscious
Level 1 Quality Initialising
Centre of Excellence
Level 0 Quality Agnostic
Service Utility
Centralized technology personnel QA processes
govern testing and quality initiatives QA is
thought leader and has enterprise influence
Product Utility
Centralized technology Centralized service
bureau for QA personnel Centralized methodology
and best practices for QA procedures
Centralised technology, licenses, admin and
support QA process in place, but mainly in
individual projects
Project-based technology, processes, personnel
The majority of testing, if not all, is still
manual
No documented QA processes or recognition of the
need for them Testing, if done at all, is manual
Increasing Maturity
11.50 12.00 10m 1 of 2
11
Testing in the Lifecycle - today
IT Supplier
Requirements
Business Analyst
Design
Unit Test
Implement
Design
Business User
Test Execution
Test Environment
Production
12
Professional Testing
IT Supplier
Requirements
Business Analyst
Design
Implement
Unit Test
Design
Business User
Quality Factory
AcceptanceCriteria
ManualTest
Test environment
Test execution
Test definition
Test dataspecification
Test automation
Production
Roll Out
13
Test Factory Service Catalog (example)
14
Test Factory Services Catalog (2)
15
Roles and Responsibilities
Application Developer
Onsite
Off site
Configuration Manager
Hand over to
Test Manager
Review requirement Define entry criteria Sign off
Ramp up resource Setup test environment
Application Owner
Test Engineer
Executive testing
Plan and design test
Test Service Manager
Test Lead
Manage testing project
Test Architect
Move to production
Testing Center/ Service
Application Operator
16
How to get there?
17
Building a Test Factory
1-2 days
EnvisioningWorkshop

• Present HP as solution Partner
• Agreement on milestones for engagement

10-20 days
Build blueprint for first test factory line

• Solution Blueprint Business Case

30-100 days
Set up first factory line and run one cycle
Pilot installation
gt100 days
Application QualityRoll-out Service
Development
18
Implementation Model
Set strategy
Pilot implementation -Set up first factory line-
Add more lines
Operation

• Project 2
• Functional test
• Risk assessment
• Requirement verification
• Test case development
• Test data development
• Test Script Development
• Test execution
• Test evaluation
• Performance test
• Scenarion definition
• Scripting
• Test execution
• Test evaluation

• Project 2
• Functional test
• Risk assessment
• Requirement verification
• Test case development
• Test data development
• Test Script Development
• Test execution
• Test evaluation
• Performance test
• Scenarion definition
• Scripting
• Test execution
• Test evaluation

• Project 1
• Functional test
• Risk assessment
• Requirement verification
• Test case development
• Test data development
• Test Script Development
• Test execution
• Test evaluation
• Performance test
• Scenarion definition
• Scripting
• Test execution Test evaluation

• Strategy service
• Investigate potential
• Specify process changes
• Draw up milestone plan

• Project n
• Functional test
• Risk assessment
• Requirement verification
• Test case development
• Test data development
• Test Script Development
• Test execution
• Test evaluation
• Performance test
• Scenario definition
• Scripting
• Test execution
• Test evaluation

• Project n
• Functional test
• Risk assessment
• Requirement verification
• Test case development
• Test data development
• Test Script Development
• Test execution
• Test evaluation
• Performance test
• Scenario definition
• Scripting
• Test execution
• Test evaluation

Outsourced Activities
18
22 November 2009
HPPrivate Dont Copy or Distribute
19
HP Test Factory Outline Service
Roadmap Development
20
Sample Timeline
Q308
Q408
Q109
Q209
Q309
Q409
Phase 1 Preparation
Phase 2 Setup the first line
Phase 3 Operate the first line
Phase 4 Evaluate the results and adjust
Phase 5 Setup the next lines
21
Why HP?
22
HP Quality Services tri-dimensional approach
aligns business with IT

• Improve performance
• Reduce backlog
• Ensure quality

22
11/22/2009
HP CONFIDENTIAL
23
Blueprint - Based on real world experience
24
Qualification

• Learning and Development Program (Grow_at_hp)
• Certified Tester Foundation Level
• Certified Tester Advanced Level (Test Management)
• Certified Professional for Requirements
  Engineering
• Certification Program

25
Who has done it?
26
One of the worlds largest mobile companies
SWB validation improves performance
IT improvements
Business Outcomes
26
2009?11?22????
27
Leading entertainment and communications company,
based in Canada
Automated testing of customer billing system
improves efficiency
IT improvements
Business Outcomes
27
28
(No Transcript)