Program Comprehension through Dynamic Analysis Visualization, evaluation, and a survey

1 / 40

About This Presentation

Title:

Program Comprehension through Dynamic Analysis Visualization, evaluation, and a survey

Description:

explain the program, its structure, its behavior, its effects on its operation ... Activity: what is being performed/contributed? e.g., architecture reconstruction. 8 ... –

Number of Views:44

Avg rating:3.0/5.0

Slides: 41

Provided by: Mar546

Category:

more less

Transcript and Presenter's Notes

Title: Program Comprehension through Dynamic Analysis Visualization, evaluation, and a survey

1
Program Comprehension through Dynamic
AnalysisVisualization, evaluation, and a survey

Bas Cornelissen (et al.)
Delft University of Technology
IPA Herfstdagen, Nunspeet, The Netherlands
November 26, 2008

1
2
Context

Software maintenance
e.g., feature requests, debugging
requires understanding of the program at hand
up to 70 of effort spent on comprehension
process
? Support program comprehension

3
Definitions

Program Comprehension
A person understands a program when he or she is
able to
explain the program, its structure, its behavior,
its effects on its operation context, and its
relationships to its application domain
in terms that are qualitatively different from
the tokens used to construct the source code of
the program.

4
Definitions (contd)

Dynamic analysis
The analysis of the properties of a running
software system

Unknown system
e.g., open source

Advantages
preciseness
goal-oriented
Limitations
incompleteness
scenario-dependence
scalability issues

Instrumentation
e.g., using AspectJ
Scenario
Execution
(too) much data
5
Outline

Literature survey
Visualization I UML sequence diagrams
Comparing reduction techniques
Visualization II Extravis
Current work Human factor
Concluding remarks

Literature survey

7
Why a literature survey?

Numerous papers and subfields
last decade many papers annually
Need for a broad overview
keep track of current and past developments
identify future directions
Existing surveys (4) do not suffice
scopes restricted
approaches not systematic
collective outcomes difficult to structure

8
Characterizing the literature

Four facets
Activity what is being performed/contributed?
e.g., architecture reconstruction

Target to which languages/platforms is the
approach applicable?
e.g., web applications

Method which methods are used in conducting the
activity?
e.g., formal concept analysis

Evaluation how is the approach validated?
e.g., industrial study

9
Attribute framework
10
Characterization
Etc.
11
Attribute frequencies
12
Survey results

Least common activities
surveys, architecture reconstruction

Least common target systems
multithreaded, distributed, legacy, web

Least common evaluations
industrial studies, controlled experiments,
comparisons

Visualization I Sequence Diagrams

14
UML sequence diagrams

Goal
visualize testcase executions as sequence
diagrams
provides insight in functionalities
accurate, up-to-date documentation
Method
instrument system and testsuite
execute testsuite
abstract from irrelevant details
visualize as sequence diagrams

15
Evaluation

JPacman
Small program for educational purposes
3 KLOC
25 classes
Task
Change requests
addition of undo functionality
addition of multi-level functionality

16
Evaluation (contd)

Checkstyle
code validation tool
57 KLOC
275 classes
Task
Addition of a new check
which types of checks exist?
what is the difference in terms of implementation?

17
Results

Sequence diagrams are easily readable
intuitive due to chronological ordering
Sequence diagrams aid in program comprehension
supports maintenance tasks
Proper reductions/abstractions are difficult
reduce 10,000 events to 100 events, but at what
cost?

18
Results (contd)

Reduction techniques issues
which one is best?
which are most likely to lead to significant
reductions?
which are the fastest?
which actually abstract from irrelevant details?

Comparing reduction techniques

20
Trace reduction techniques

Input 1 large execution trace
up to millions of events
Input 2 maximum output size
e.g., 100 for visualiz. through UML sequence
diagrams
Output reduced trace
was reduction successful?
how fast was the reduction performed?
has relevant data been preserved?

21
Example technique

Stack depth limitation metrics-based filtering
requires two passes

discard events above maximum depth
determine maximum depth
determine depth frequencies
Trace
Trace

0 28,450
13,902
58,444
29,933
10,004
...

200,000 events
42,352 events
gt depth 1
maximum output size (threshold)
50,000 events
22
How can we compare the techniques?

Use
common context
common evaluation criteria
common test set
? Ensures fair comparison

23
Approach

Assessment methodology
Context
Criteria
Metrics
Test set
Application
Interpretation

need for high level knowledge reduction success
rate performance info preservation output
size time spent preservation per type five
open source systems, one industrial apply
reductions using thresholds 1,000 thru
1,000,000 compare side-by-side
24
Techniques under assessment