Tools for Quality Code

1
Tools for Quality Code

• Nigel Watson, Microsoft Australia
• Sean Salisbury, Compuware Corp

2
Agenda

• Testing so what?
• Testing in Visual Studio Team System
• Extending VSTS Compuware DevPartner
• Summary

3
Projects and Testing

• Often an expensive afterthought
• Strategies for minimising impact

30
25
20
Relative Cost To Fix Bugs...
15
10
5
Coding
Integration
Post-Release
Requirements
Beta Test
4
Problems...

• It is expensive to find and fix bugs that get
  past daily development practices
  
• Potential security flaws need to be caught early
• It is hard to diagnose errors at runtime
• Why does an application run slowly?
• Individual Developers and Testers need to know if
  they are on track
  
• Test and development are often out of synch
• Final test phase for shipping is often ad-hoc
• How much testing is enough?

5
Defense In Depth

• Microsoft uses a 'defense in depth' strategy
• Unit testing
• Code reviews
• Frequent builds
• Catch bugs early
• Static checks
• Runtime checks

6
Testing in VSTS
7
Testing in VSTS
8
Test-Driven Development

• Integrate testing into the development process
• Tests define what code will do
• Tests come from specifications
• Write code to pass tests
• Don't write code that doesn't contribute to
  passing a test...

Code
9
VSTS Unit Testing

• Integrated into VS
• Automatic generation of test classes
• Comprehensive test management
• Code coverage testing

TestMethod() public void GetValueTest()
double d myObject.getValue()
if (d alue")
10
Unit Testing
11
Code Reviews

• For the Visual Studio 7.0 product cycle
• 86 of bugs occurred in reviewed code
• 60 of all bugs were coding errors
• Static analysis helps catch bugs
• Source code analysis
• PREfast for C and C
• FxCop for .NET

12
PREFast

• Static analysis for C/C code
• Managed and unmanaged C
• Catches common bugs
• Buffer overruns, uninitialized memory
• Memory leaks, null pointer dereference
• Reported as compiler warnings
• Display path to problem
• Use pragma to turn off

13
FxCop

• Static analysis for .NET assemblies
• Not just C
• Uses design guidelines
• (including many in the .NET Class Design
  Guidelines)
  
• Customizable
• Which checks to include
• Whether to report as error or warning
• Create custom rules

14
Static code analysis
15
Integrating Dev and Test

• Tests are just another form of source code
• Stored in source code control
• Versioned with the product
• Test Complete
• Test writing is scheduled along with development
  work
  
• Tracked by work items
• Testers are notified when bugs are fixed

16
VSTS Test Types

• Unit Tests
• Test class methods
• Web Tests
• Record and playback interactions
• Load Tests
• Simulate multiple users
• Manual Tests
• Provide scripts for manual tasks
• Third-party Tests
• Integrated into VSTS

17
Application Quality

• Best Practices and Tools

Sean Salisbury Senior Regional Tech Specialist C
ompuware Corporation
sean.salisbury_at_compuware.com
18
Compuware Corporation

• 30 years of delivering proven IT value to the
  worlds largest businesses
  
• 8,500 employees
• 84 offices in 16 countries
• Rock-solid financially
• 1 billion in revenue
• 700 million in cash
• Our commitment Real VALUE and ROI for every
  single dollar you spend with us.

19
Application Life Cycle
20
Microsoft and Compuware
Integrated development and test automation tools
Rich process management Detailed and relevant pro
ject information
21
Quality Begins in Development

• Addressing application quality needs including
  security during development is most
  cost-effective
  
• Profile performance and optimize memory
  utilization
  
• Carry out in-depth unit testing
• Monitor code coverage
• Seamlessly capture defect information

22
Compuware DevPartner Studioenhance and extend
Visual Studio

• Native and Managed Code Analysis
• Local and Remote Data Collection
• Performance Analysis
• .NET Memory Analysis
• Code Coverage Analysis
• Distributed Application Analysis
• VB, VB.NET, ASP.Net and C Source Code Review
  with 600 Rules
  
• C/C Memory Error Thread Deadlock Detection

23
Example Profiling
24
Code Analysis

• 600 Rules enhance problem resolution
• Supports VS6/2002/2003/2005
• Accelerates learning curves
• Improves code quality and maintainability
• Supports Visual Basic, VB.NET, C, ASP.Net

25
(No Transcript)
26
Memory Analysis

• Optimize Local or Remote Memory Use
• View allocations/deallocations over time get an
  overall feel for memory use
  
• Identify Objects That
• Consume a lot of memory
• Create a lot of temporary objects
• Stay around longer than they need to, including
  leaks
  
• Compare Runs- Did Code Changes Help?
• Tune Garbage Collection

27
Memory Analysis at Run Time
RAM usage
Real-Time trace of memory usage
Time
System Allocations
Your Code
28
Memory Analysis
Many Different Data Views with Details Available
29
Automatic Error Detection

• Memory/Resource/ Interface Leaks
• API Errors
• Threading Issues
• Event Debugging
• C/C/VC

30
(No Transcript)
31
Thread Deadlock Detection

• Locate Actual or Potential Thread Deadlocks or
  Other Synchronization Issues
  
• Deadlock 2 or more code paths running at the
  same time, contending for the same resource(s)
  
• Benefits
• Thread deadlock are difficult to detect
  automating detection is very useful

32
Performance Profiling

• Pinpoint bottlenecks across app Tiers/Versions
• Optimize application performance
• Increase usability

33
(No Transcript)
34
Compare Performance Runs
35
Code Coverage

• Quickly identify untested code across tiers
  VS6/02/03/05
  
• Ensure test coverage during unit testing
• More reliable components and applications

36
(No Transcript)
37
Distributed Analysis
38
Whats New.
39
IT Challenges

• Identifying what errors can occur when
• Tools lacking for error simulation and analysis
• Errors corrupt the debugging environment
• Impossible to trace error handling execution
• Difficult to create repeatable tests
• Time-consuming, manual process

40
Quality of Error Handlers

• Applications are comprised of between 10 and 30
  error handling code
  
• Its difficult/impossible to safely and
  repeatably simulate errors to test error-handling
  code
  
• The result
• Overall code coverage during test is lower than
  it could be
  
• Undetected bugs are passed into the production
  environment

41
What If You Could

• Quickly determine what errors could occur at any
  point in your application?
  
• Ensure you have error handlers in place to cope
• Simulate errors safely and efficiently?
• With no impact on the OS, .NET framework or any
  other running application
  
• Observe and debug your error handlers
• Build reusable fault test libraries?
• Create repeatable tests that are reusable by
  development QA

42
DevPartner Fault Simulator
Developer Insight What errors can occur at what p
oint in the code Integrated with Visual Studio de
bugging features to monitor error handling
execution Break at fault occurrence
43
DevPartner Fault Simulator
Error handling validation Simulate Environmental
and .NET Framework faults Simple method of select
ion of errors to validate, with user defined
conditions Reusable Fault Sets for repeat and QA
testing VS 2003/05 IDE integrated, standalone and
command line operation
44
DevPartner Fault Simulator
Results analysis Simulate Stack tracing error d
etails Go to source linking for detailed analy
sis Live view and summary of fault execution S
aved Results files for later review
45
DevPartner Fault Simulator

• Demonstration

46
Securing ASP.Net Applications
47
Security Vulnerability

• Today over 70 of attacks against a companys
  network come at the Application Layer, not the
  Network or System Layer
  
• John Pescatore, Gartner chief security analyst
• The responsibility for application security is
  shifting to the development organization
  
• How do they address this aspect of application
  quality?
  
• How do they gain the skills they need to assess
  and correct security vulnerabilities?

48
What If You Could

• Quickly locate security vulnerabilities in your
  application during development?
  
• Minimize the cost and mean-time-to-repair
• Improve the quality/reliability of your
  application
  
• Have a wealth of security expertise and advice at
  your fingertips?
  
• Have the information you need, when you need it

49
DevPartner SecurityChecker

• A vulnerability assessment scanner that locates
  security vulnerabilities in ASP.NET (C or
  VB.NET)
  
• Locates complex hard-to-find security problems
• Organizes results by priority and category
• Pinpoints vulnerabilities to the line of source
  code
  
• Explains why it is an issue
• Suggests steps to repair each vulnerability
• Provides links to additional technical information

50
DevPartner SecurityChecker
51
Security Issues by Analyzer
52
DevPartner SecurityChecker

• Demonstration

53
Quality Continues in Testing

• Automate functional testing and validation
• Manage test plans and execution
• Comparison of complex data results
• Seamlessly capture defect information
• Simulate application under load
• Simulate load conditions 000,000s of users
• Determine application scalability
• Compuware QACenter Enterprise Wide
• Compuware Vantage - Network and Server monitoring

54
Microsoft Compuware

• Tools to
• Improve application reliability performance
• Increase team productivity
• Lower costs
• Deliver better applications to the market faster

55
Summary

• Appreciated the importance of testing to the
  development process
  
• Had a quick look at some of the testing tools in
  Visual Studio Team System
  
• Sean showed us how Compuware DevPartner Studio
  uses the integration capabilities of Visual
  Studio to extend the power of the IDE

56
2004 Microsoft Corporation. All rights
reserved. This presentation is for informational
purposes only. Microsoft makes no warranties,
express or implied, in this summary.