Grokking Software Architecture

1
Grokking Software Architecture
2008 Working Conference on Reverse Engineering

• Richard C. Holt
• Software Architecture Group (SWAG)
• School of Computer Science, University of
  Waterloo, Canada

2
Retrospective
1998
2008
Ten years ago. WCRE most influential paper.
Structural Manipulations of Software
Architecture using Tarski Relational Algebra
Today. Retrospective. Grokking Software
Architecture
17 papers in WCRE
3
Grokking Software Architecture
Grokking
Software architecture
4
Overview of Talk 4 Parts

• Part 1. 1998 paper Hopes claims
• Part 2. Software Architecture
• Part 3. Formalizing Boxology
• Part 4. ROP Relation-Oriented Programming
  Grok-Like Languages

5
Part 1. 1998 paper Hopes claims

• Represent software architecture as a typed graph
• Graphs with colors of edges nodes
• Manipulate visualize these architectural graphs
• Manipulations can be specified algebraically ---
  and automatically executed

In brief Formalize architectural diagrams and
reap the benefits arising from the corresponding
mathematics.
6
Top View of As-Built Software Architecture
(250KLOC System)
7
View of One Subsystem of the 250 KLOC System
8
CS 746G Topics in Software ArchitectureUniversity
of Waterloo

• CS746 in Winter 1998 Linux (Operating System)
• CS746 in Winter 1999 Apache (Web Server)
• CS746 in Winter 2000 Mozilla (Web Browser)
• CS746 in Winter 2001 Eazel Nautilus (File
  Manager)
• CS798 in Winter 2002 Postgres et al (Data Base)
• CS746 in Winter 2003 EMACS et al (Editor)
• CS746 in Winter 2004 Gnumeric (Spreadsheet)
• CS746 in Fall 2004 Mozilla (Web Browser -- again)
  
• CS746 in Fall 2005 Open Office (Open Source
  Office Suite)
• CS746 in Fall 2006 Asterisk (Open Phone Switch)
• CS746 in Fall 2008 MySQL

9
Process of View Creation
Source code
Clustering
Parser
Facts extracted from code
Hierarchic decomposition
Grok Fact manipulator
Architectural diagram
Layouter
Browser
10
Transformations to do Hiding
Graph G
d
T
e
a
V
f
b
Graph I hideExt(G, S)
Graph H hide(hide(G,T),V)
11
Lifting Calls Up to File Level
call is a procedure call fileCall is a file level
call
File
File
fileCall
main.c
start.h
funcDef
funcDcl
startup
main
call
Procedure body
Procedure header
fileCall funcDef o call o inv funcDcl
12
Part 2. Software Architecture Boxology Approach

• Software architecture
• What is it?
• State of practice
• How is it represented
• Keep It simple
• Models tools
• Views of architecture
• Extracting As-Built architecture

13
Software ArchitectureWhat is it?

• Confusion. I have a sneaking suspicion that
  architecture is one of the most overused and
  least understood terms in professional software
  development circles. Gorton
• Consensus. Architecture captures system
  structure in terms of components parts and how
  they interact. Gorton

14
Software Architecture State of the Practice

• Its common for there to be little or no
  documentation covering the architecture in many
  projects. Gorton
• I'm hopeless when it comes to documentation.
  Torvalds
• The architecture that actually predominates in
  practice is the big ball of mud Foote et al

15
Software as Spaghetti
Foote et al
16
Software Architecture How is it Represented in
Practice?

• predominant tools used for architecture
  documentation are Microsoft Word, Visio and Power
  Point Gorton
• Whats needed Concepts, notations and tools
  that are
• easy to use and
• help us produce useful, understandable
  documentation

17
KISS Keep it Simple Stupid
Any fool can make things bigger, more complex,
and more violent. It takes a touch of genius -
and a lot of courage - to move in the opposite
direction. Einstein
Make everything as simple as possible, but not
simpler. Einstein
18
Models and Tools for Software Architecture

• UML has, for better or (many would say) worse,
  become the industry standard ADL Architecture
  Design Language Shaw
• UML lacks, however, a robust suite of tools for
  analysis, consistency checking Shaw

19
UML Component Diagram Box and Arrow Diagram
Gorton
20
Views of Software Architecture Kruchten
End user
As-Built View
Users View
Programmers software managers
Scenarios
Deployment View
Concurrency View
System Engineer
Integrator
21
Extracting the As-Built Architecture from the Code

• Reverse engineering is the process of analyzing
  a subject system to create representations of the
  system at a higher level of abstraction.
  Chikofsky
• Relational approach.
• Parse the code to produce relations, e.g
• (call, P, Q) means proc P calls Q
• Manipulate edges into as-built architecture

22
Boxology as a Central ADL (Architectural Design
Language)

• The most widely used design notation for
  software architecture is informal block and
  arrow diagrams. Gorton

23
Cross Fertilization!! Rev Eng, S/W Arch,
Relational Approach

• Reverse engineering
• Architecture extraction
• As-Built view Code is king
• Traceability
• Software architecture
• Need for representation tools
• Simplicity utility
• Relational approach
• Boxology
• Formalization --- Tarski algebra

24
Part 3. Formalizing Boxology

• Boxology is the Representation of an organized
  structure as a graph of labeled nodes (boxes)
  and connections between them (as lines or
  arrows). Wikipedia
• Toward boxology preliminary classification of
  architectural styles Shaw

25
Example Typed Graph
r
r
C
C
a
b
a
b
I
I
z
w
E
U
C
C
C
E
C
C
U
x
y
v
v
w
x
y
z
U
U

• C (r,a), (r,b), (a,v), (a,w) (a,x), (b,y),
  (b,z)
• I (a,b)
• E (b,y)
• U (v,w), (x,y)

26
Boxology is Just Scribbling?

• Box arrow diagrams
• Are just scribbles? No
• Formalized by typed graphs
• Visualized as (nested) boxes arrows
• Manipulated by Tarski algebra etc.
• Exchanged as
• Triples (RSF), extended to TA, or GXL or

27
Boxology has Semantics? Yes

• Compare to BNF
• Semantics by informal attachment to productions
• Compare to Codds relational approach
• Semantics by interpretation of tables.
• Semantics by attributes descriptions
• Separation of concerns
• Structure then semantics
• Use box/arrow diagrams as underlying formalism
  for software architecture (Mini-MOF?)

28
Adding Algebra to Boxology

• Tables then Codd relational algebra
• N-ary relations
• Boxes/arrows then Tarski relational algebra
• Binary relations

29
Example Typed Graph
r
r
C
C
a
b
a
b
I
I
z
w
E
U
C
C
C
E
C
C
U
x
y
v
v
w
x
y
z
U
U

• C (r,a), (r,b), (a,v), (a,w) (a,x), (b,y),
  (b,z)
• I (a,b)
• E (b,y)
• U (v,w), (x,y)

30
Tarski Algebraic Operators
Union I E (a,b), (b,y) Intersection E C
(b,y) Difference C - E (r,a), (r,b),
(a,v), (a,w), (a,x), (b,z) Inverse inv E
(y,b) Composition I o E (a,y) Identity id
(r,r), (a, a), (b,b), (w,w) Transitive
Cl. C (r,a), (r, b), (r,v), (r,w), (r,x),
(r,y), (r,z), (a,v), (a,w), (a,x), (b,y),
(b,z) Reflex. T.C. C ID C
31
TA Schemas for Box and Arrow Diagrams
ref
proc
var
call

• A Schema in TA
• Determines
• Types of boxes
• Types of edges
• Allowed connectivity between edges
• Supports inheritance in schemas
• Also attributes (strings) on boxes on edges

instance
instance
instance
instance
p
q
x
y
call
ref
Malton WCRE 2005
32
Why Formalize Boxology??Cause it Makes Our Life
Better

• Clear understanding clear specification
• What does RSF meaning?
• Meaning is independent of implementation
• Clarifies deeper concepts, e.g., expressiveness
• Generality
• Progress in reverse engineering
• Progress in software architecture
• Not just scribbling

33
Part 4. ROP Relation-Oriented Programming
Grok-Like Languages

• A paradigm shift

34
Example Mickey Eats Swiss Cheese

• Mickey . eat
• Swiss
• Roquefort
• eat . Mickey
• Garfield
• Fluffy
• eat o eat
• (Garfield Swiss)
• (Garfield Roquefort)
• (Fluffy Swiss)
• (Fluffy Roquefort)
• eat
• ,,,

The eat relation
Garfield
Fluffy
Nancy
Mickey
Roquefort
Swiss
35
Example ROP/Grok ProgramIs relation R a tree?
How you would program this test
36
Grok Program Is R a Tree?
Pseudo code
if R has no loops R has one root R
has only single parents then put R is a tree
Assume each node is a source or target of the
contain C relation
37
Grok Program Is R a Tree?
Pseudo code
Grok code
if ( R ID ) 0
if R has no loops
Does transitive closure of R have any self-loops?
Yes
R
R
R
R
a
d
b
c
38
Grok Program Is R a Tree?
Pseudo code
Grok code
if ( R ID ) 0 (dom R - rng
R) 1
if R has no loops R has one root
Does R have exactly one source? Yes
a
dom
b
c
g
f
d
e
rng
39
Grok Program Is R a Tree?
Pseudo code
Grok code
if ( R ID ) 0 (dom R - rng
R) 1 ((R o inv R) - ID) ! 0
if R has no loops R has one root R
has only single parents
R o inv R
b
d
inv R
Does my child have another parent? Yes
R
c
a
40
Grok Program Is R a Tree?
Pseudo code
Grok code
if ( R ID ) 0 (dom R - rng
R) 1 ((R o inv R) - ID) ! 0 then
put R is a tree
if R has no loops R has one root R
has only single parents then put R is a tree
Moral Relational progamming is not like low
level (Java level) programming. Loops typically
disappear.
41
Notation Does it Matter?
By relieving the brain of all unnecessary work, a
good notation sets it free to concentrate on more
advanced problems, and, in effect, increases the
mental power of the race. Alfred North
Whitehead
42
Wins Losses Using Tarski Algebra

• Wins
• Good for computing new edges, for finding
  properties of edges, eg, nodes in loops, leaves,
  etc.
• Losses
• Not good for locating patterns involving several
  nodes, e.g., find complete connected sub-graphs

43
Notation Grok (Tarski) vs. Crocopat
My parents (P) children (C) are my (reflexive)
siblings (S)
y
P
C
C
P
x
z
S
S
S P o C
S(x,z) EX(y, P(x,y) C(y,z))
Grok
Crocopat
Should Crocopat add Tarski operators??
44
Characterizing Grok-Like Languages

• Relational
• Useful for software analysis
• Expressiveness
• How powerful can a query be?
• Codd algebra and Crocopat are more powerful.
• How well can a query meet our needs? How
  writeable? How readable?
• Performance of implementation
• Can hold large graphs?
• Fast enough to manipulate large graphs?

45
Performance of Grok-Like Languages

• Size speed OK for --- Grok Crocopat
• All memory resident, no disk access
• Hundreds of thousands of edges
• Modeling million-line systems
• Most operations not more than a few seconds
• Crocopat scales up a bit more for transitive
  closure
• House keeping, e.g., time to read files, is
  critical
• Need to test on 64-bit implementations

46
Data Structures for Binary Relations

• Tables One for each type of relation DBMS
• Single table of triples Grok
• Linked lists
• Pointers and nodes Lsedit, JGrok (caches
  sorted lists)
• BDD Binary Decision Diagram Relview, Crocopat
• Memory efficient storage of binary relations
• Works well with dense graphs
• Proven useful RelView, Crocopat
• Surprising (to me) BDD efficient for transitive
  closure

47
Grok-Like Languages
Discussion of Grok-Like Languages
PS Paul Klints relational language ...
48
Progress Using Grok-Like Languages

1. Enforce architecture rules. Holt 96, Feijs 98,
   Knodel 08
2. Lift dependency edges. Holt 98, Feijs 1998
3. Find design pattern instances. Consens 98, Beyer
   02
4. Find violations of patterns. Guo 99
5. Find anti-patterns. vanEmden 02, Feijs 98
6. Change impact analysis. Feijs 98
7. Specify extraction from syntax. Lin 08
8. Find source of dependency. Fahmy 01, Feijs 98
9. Locate uses of protocols. Wu 01
10. Type inference using transitive closure.
    vanDeursen 99

49
Conclusions
Grokking Software Architecture
50
Conclusions

• Typed graphs nicely formalize various software
  structures
• Software architecture can benefit from a ROP
  approach
• Tarski algebra, added to boxology, is elegant
• Does not handle multi-node patterns
• Grok-like (ROP) languages are elegant and
  sufficiently efficient
• ROP is high level, is faster, more reliable, more
  flexible
• Lots of
• Work done so far
• Room for more work