Software%20Architecture%20Evaluation%20Methodologies

1
Software Architecture Evaluation Methodologies

• Presented By
• Anthony Register

2

• Software architecture (SWA)
• - The fundamental organization of a system
  embodied in its components, their relationships
  to each other, and to the environment, and the
  principles guiding its design and evolution as
  defined by IEEE

3

• SWA Quality determined by quality attributes such
  as
• Flexibility
• Usability
• Performance
• Maintenance

4
Evaluation methods

• SAAM Scenario-Based Architecture Analysis
  Method
• ALMA Architecture Level Modifiability Analysis
• ATAM Architecture Trade-off Analysis Method
• PASA Performance Assessment of Software
  Architecture

5
Assumption

• - Several of the most common SWA evaluation
  methods are similar in the techniques applied,
  but have evolved to serve different purposes or
  viewpoints for evaluating the quality of a
  specific SWA.

6
Scenario Based

• Documented in OO design and user interface as a
  technique to solicit requirements
• Used for comparing SWA options
• Simulate system events or changes
• Determine impact on SWA
• Result in brief description of a functional
  requirement that eventually may be implemented
  into the system at design time or in the future

7
Scenario Based

• Benefits
• Break down and clearly describe the architectural
  description and analysis points
• Catalyst for generating more questions

8
Types of Scenarios

• Direct requires no change to system
  architecture
• Indirect requires change to system architecture

9
Scenario Example
10
Scenario-Based Architecture Analysis
http//www.sei.cmu.edu/pub/documents/00.reports/pd
f/00tr004.pdfsearch22ATAM2022

• 5 steps
• Describe the candidate architecture
• Develop scenarios
• Perform scenario evaluations
• Reveal scenario interaction
• Overall SWA evaluation

11
SAAM (Steps 1 2)

• Describe the candidate architecture
• Use a common syntactic and semantic notation
  that all designers and stakeholders can easily
  understand.
• Develop scenarios
• Direct and indirect

12
SAAM (Steps 3 4)

• Perform scenario evaluations
• Evaluate quality attributes (flexibility,
  usability, performance, maintenance, etc.)
• Reveal scenario interaction
• Indirect scenario affects need to be at a minimum
  or the component may be a risk to the SWA
• Scenario analysis indicates that many indirect
  scenarios affect the same component, then too
  much architectural level coupling may be indicated

13
SAAM (Step 5)

• Overall SWA evaluation
• Weighted ranking or scoring performed for the
  overall SWA evaluation.
• A scoring technique using an appropriate
  weighting analysis can be applied to track the
  affects of the indirect scenarios on the SWA for
  future decision making

14
SAAM
Benefits Disadvantages
Detecting issues early Lacking in tool support for managing large amounts of data for system and scenario descriptions
SWA documentation through the documentation of scenarios, outcomes, and affects on the system architecture Lacks architectural metrics for assisting designers when making decisions between alternative designs
Improved understanding of possible SWA issues
15
SAAM Strengths

• Evaluate or compare future and existing systems,
  such as using the low cost method (tabular
  calculations and comparisons) for measuring
  change when choosing between SWA designs
• Evaluation result enables the designers and users
  to focus on the details of the SWA and not be
  distracted from the less important areas
• Provides a guided approach for evaluation and
  provides for open dialogue between the
  stakeholders of the system under evaluation

16
Architecture Level Modifiability Analysis (ALMA)

• Focuses on modifiability of the SWA
• Risk assessments
• Distinguishes multiple analysis goals
• Explicit assumptions
• Provides repeatable techniques for performing the
  steps
• Predicting future maintenance costs
• Evaluating the flexibility of a system at an
  architectural level
• Main quality inputs are the SWA specification and
  quality requirements

http//www.sei.cmu.edu/pub/documents/00.reports/pd
f/00tr004.pdfsearch22ATAM2022
17
ALMA Process Steps

• 5 Steps
• Set the goal and determine the focus of the
  analysis
• Create a description of the SWA
• Create scenarios from the functional requirements
• Evaluate the effects of the scenarios
• Analyze the results of the independent scenario
  evaluations

18
ALMA (Steps 1 2)

• Goal Setting
• Maintenance cost prediction, risk assessment, and
  SWA selection
• Description of SWA
• SWA information in order to derive an
  architectural description of the system
• Activity evaluates the decomposition of the
  system components and evaluates the relationships
  between the components

19
ALMA (Steps 3 4)

• Create scenarios from functional requirements
• Determine the impact of change to the SWA
• Providing a metric for analysis performed in the
  next step for evaluating the different scenarios
• Evaluate effects of scenarios
• Collection of information to be used in SWA
  impact analysis
• Identify components affected by the change,
  determine what the effects were, and determine
  the ripple effects on relational components

20
ALMA (Step 5)

• Analyze the results of the independent
• scenario evaluations
• Interpretation of the results needs to align with
  the goal set forth for the evaluation
• Goals
• Maintenance cost prediction, the scenarios should
  cover future events of the system
• Risk assessment, the scenarios should provide
  complex change events in order to determine and
  interpret the effects of the changes

21
ALMA

• Benefits
• Identification of SWA risks
• Measurement of the amount of effort required for
  changes
• Deciding between available SWA options
• Reduction in the number of scenarios and a
  process that provides guidance as to when to stop
  generating scenarios
• All change categories explicitly considered
• New change scenarios do not affect the
  classification structure

22
Architecture Trade-off Analysis Method (ATAM)
http//www.sei.cmu.edu/architecture/ata_method.htm
l
Evaluates a SWA for quality attribute target
goals, but focuses more on the trade-offs between
the quality attributes.
23
ATAM

• Main inputs
• business goals
• software specifications
• SWA description
• Main Outputs
• list of scenario sensitivity points
• trade-off points
• risks
• different approaches to the SWA
• utility tree
• quality attribute questions with the responses

24
ATAM Process Steps

• The 4 phases are
• Presentation
• Investigation and Analysis
• Testing
• Reporting

25
ATAM (Phase 1 2)

• Presentation
• Activities
• Presenting the ATAM
• Presenting the business requirements
• Presenting the architecture
• Investigation and analysis
• Activities
• Identify the architectural approaches suggested
  by the architect before they are analyzed
• Create an attribute utility tree
• Analyze the architectural approaches

26
ATAM Utility Tree
http//www.sei.cmu.edu/architecture/ata_method.htm
l

• An analytical method that provides a top-down
  approach for decomposing the quality attributes
  as designated by the ATAM goals.

27
ATAM (Phase 3 4)

• Testing
• Activities
• Brainstorming scenarios
• Analyzing the architectural approaches
• Utilizes the designated high priority scenarios
  to be used for test cases
• The goal is targeted to identify any hidden
  architectural approaches, risks, sensitivity
  points, and tradeoff points
• Reporting
• Activities
• Present the results
• Presented to the stakeholders in the form of a
  final analysis report

28
ATAM

• Benefits
• The quality attributed requirements are clarified
• Provides for improved SWA documentation that can
  be used in the foundation of future SWA decisions
• Promote communication between the stakeholders,
  such as customers, architects, and testers.
• Identify system risks early in the solution
  life-cycle

29
Performance Assessment of Software Architecture
(PASA)
Comparison of Scenario-Based Software
Architecture Evaluation Methods

• Evaluates performance issues for SWA systems
• Goal of PASA utilizes performance based scenarios
  
• Unique the documentation extracted comes from
  the developers and source code (since can be
  performed during development cycle)
• Only includes interaction with the development
  team which is different from the other techniques

30
PASA Process

• Starts by setting goals, identifying required
  information, understanding stakeholder
  expectations, and describing the method of
  approach
• Selects key performance scenarios as elicited
  from the developers
• Focuses on the architectural style or patterns
  used
• The output of the process is a presentation of
  the results.

31
Common Goals and Activities of all Methods

• Common Goal evaluate and predict the quality
  attributes as applied to a SWA evaluation
  analysis
• Common Activities
• Evaluating
• Planning and preparation
• Explanation of SWA approaches
• Elicitation of quality sensitive scenarios
• Analysis of SWA options
• Interpretation and presentation of the evaluation
  results for final SWA decision making

32
Methods Overall

• SAAM attempts to identify the potential risks to
  SWA and assess the modifiability.
• ALMA attempts to predict the modifiability based
  on risk assessment, support costs, and SWA
  comparisons.
• ATAM analyzes the sensitivity and trade-off
  points to determine what may prevent realizing
  the best combination of quality attributes for a
  given SWA.
• PASA evaluates the performance risks.

33
Differences

• Participants
• SAAM and ATAM involve the architects, designers,
  and the end users
• ALMA only mostly includes the architect designer
• PASA only includes the developers
• SAAM and ATAM are the only two methodologies that
  are close to success when providing details as to
  the costs associated with a SWA evaluation or
  resource requirements
• ATAM is one of the few if not the only evaluation
  method that provides sufficient process steps.
  The other evaluation methods provide descriptions
  of the required activities, but do not provide
  enough granular detail.
• All of the methods are influenced by
  non-technical issues, such as stakeholder
  interests and political factors, but the ATAM
  process is the only method that provides
  instructions for detailed guidelines and
  techniques to manage the social issues

34
Original Assumption

• The most common SWA evaluation techniques are
  similar in the methodology applied, but have
  evolved to serve different purposes or
  perspectives for evaluating the quality of a
  specific SWA.

35
Final Assumption

• Slightly reversed in truth as it has been
  determined that the methodologies applied are
  different and unique in their approaches, but
  they all serve the common goal for determining
  the best SWA option for a given set of business
  scenarios
• No single evaluation technique provides a clearly
  decisive result by specifying a single SWA
  selection out of multiple available SWA options

36
Conclusion

• The various techniques of SAAM, ALMA, ATAM,
  and PASA use different approaches and steps in
  the quest for selecting the best SWA solution.
  Ultimately the common goal or purpose of all of
  the techniques is the selection of the SWA that
  provides maximum usability and satisfies all of
  the quality attributes as defined by the business
  goals.

37
Questions and Discussion