ZEN Tool Architecture

1
(No Transcript)
2
Today

• Who we are
• What we did
• How we did it
• What we learned
• Moving forward

3
The Usual Suspects

• Team Zen
• Marc Novakouski
• Session Mwamufiya
• Sajjad Mustehsan
• Somakala Jagannathan
• Lung-San (Allen) Hsu

• Mentors
• Felix Bachmann
• Philip Bianco
• Client
• Grace Lewis

4
What We Did
5
In The Beginning

• Service Migration and Reuse Technique (SMART)
• Technique to analyze the effort, risk and cost of
  migration of legacy components to a target
  Service-Oriented Architecture (SOA) environment
• Three Elements
• Interview Process
• Analysis Process
• Evolution Process

6
This Doesnt Work

• Entire SMART process is manual
• Interview process
• Dynamic interview environment makes it difficult
  to record data in an organized way
• Manual consolidation post-interview
• Risk analysis reports
• Overload of data
• History tracking and trend application difficult

7
What to do?

• Goal To improve the efficiency of the SMART
  process
• Streamline SMART engagements
• Efficient data collection
• Automated interview guide (SMIG) navigation
• Automated data consolidation and report
  generation
• Develop a centralized server and
  interviewer-operated client software

8
Lets see it in action!
9
Lets see it in action!

• Zen Client

10
Lets see it in action!

• Installation

11
Lets see it in action!

• Bring up the Client

12
Lets see it in action!

• Download Engagement

13
Lets see it in action!

• Record some data

14
Lets see it in action!

• Done for the day

15
Lets see it in action!

• Ooops! Password Recovery

16
Lets see it in action!

• Upload Data

17
Lets see it in action!

• Client Reporting

18
Lets see it in action!

• Update Engagement

19
Lets see it in action!

• Zen Server

20
Lets see it in action!

• Server Reporting

21
Lets see it in action!

• SMIG Evolution

22
Lets see it in action!

• The Cycle is Complete

23
How we did it
24
Zen Development MethodZDMTM

• ACDM
• Architecture Development
• Experiments
• XP
• Pair Programming
• Test-Driven Development
• Continuous Integration
• Prototyping as Design

25
Zen Development MethodZDMTM

• Implementation Process
• Design
• Design Review
• Code
• Verification
• Validation
• Code Review

26
Architecture Centricity

• Fall
• Architecture-based Requirement Elicitation
• Architecture Training
• Spring
• Architecture-based Experiments
• Architecture Design Documentation
• Architecture Reviews
• Summer
• Architecture-based Detailed Design
• Architecture Validation
• Architecture Refinement

27
Architecture Centricity

• Quality Attributes
• Security
• Performance
• Usability
• Extendibility/Maintainability
• Business Drivers
• Improve Efficiency of SMART process
• Zero Cost development
• Transition to Industry

28
What We Learned
29
Dont Reinvent the Wheel

• Open Source COTS products

30
Dont Reinvent the Wheel

• Development Environment Eclipse

31
Dont Reinvent the Wheel

• Eclipse

• Client Framework
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status

32
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status

• Tracking
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking

33
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking

• Server Infrastructure
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files

34
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files

• Quality Assurance
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• Apache Logging
• ECLEmma

35
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• Apache Logging
• ECLEmma

• Database Management
• MySQL
• HSQLDB
• Table Normalization
• JDBC
• IBATIS with Spring Framework

36
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• Subclipse
• SVN
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• ECLEmma
• Apache Logging

• Server Infrastructure
• JBOSS
• XMLRPC over HTTPS
• STRUTS 2
• AJAX

37
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• ECLEmma
• Apache Logging

• Security
• Java Cryptography Extension (JCE)
• BouncyCastle Cryptographic Provider
• JBOSS
• XMLRPC over HTTPS
• STRUTS 2
• AJAX

38
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• ECLEmma
• Apache Logging

• Reporting
• BIRT
• PDF Export
• DOC Export
• Java Cryptography Extension (JCE)
• BouncyCastle Cryptographic Provider
• JBOSS
• XMLRPC over HTTPS
• STRUTS 2
• AJAX

39
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• ECLEmma
• Apache Logging

• Documentation
• MediaWiki
• eUML 2
• Omondo
• BIRT
• PDF Export
• DOC Export
• Java Cryptography Extension (JCE)
• BouncyCastle Cryptographic Provider
• JBOSS
• XMLRPC over HTTPS
• STRUTS 2
• AJAX

40
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• ECLEmma
• Apache Logging

• Templates
• Excel CSV Parser
• MediaWiki
• eUML 2
• Omondo
• BIRT
• PDF Export
• DOC Export
• Java Cryptography Extension (JCE)
• BouncyCastle Cryptographic Provider
• JBOSS
• XMLRPC over HTTPS
• STRUTS 2
• AJAX

41
Dont Reinvent the Wheel

• Eclipse
• Eclipse RCP
• Eclipse PDE Build
• ECLEmma Status
• ZenTrack
• MS Project
• MS Excel
• Lighthouse Online Tracking
• SVN
• Subclipse
• BLAT
• Windows Task Manager
• Batch Files
• PMD
• JUNIT
• PDE Overnight Builds
• ANT
• ECLEmma
• Apache Logging

• Table Normalization
• JDBC
• IBATIS with Spring Framework
• Excel CSV Parser
• MediaWiki
• eUML 2
• Omondo
• BIRT
• PDF Export
• DOC Export
• Java Cryptography Extension (JCE)
• BouncyCastle Cryptographic Provider
• JBOSS
• XMLRPC over HTTPS
• STRUTS 2
• AJAX

42
Architectural Mismatch

• High Avoidance
• MySQL
• BIRT
• Eclipse RCP Framework
• Major issue
• Data Access Layer

43
Re-Planning

• Architectural Holes
• Components AND Connectors
• Rework
• Reuse
• Authentication
• Server Pages
• DAOs
• Prototyping

44
Re-Planning

• Re-Planning Process
• Examine Current Tasks
• Examine Free Hours
• Match Tasks to Estimators

45
Accurate Estimation

• Estimation Process
• 2 Person Wideband Delphi
• Milestone Based
• Corrections based on Experiment Data
• 3rd person added where consensus not reached

46
Accurate Estimation

• Overall Estimation Accuracy
• Initial Client 80.5 (Over)
• Re-estimates 92.7 (Over)
• Initial Server 68.7 (Under)
• Re-estimates 80.3 (Under)

47
Accurate Estimation

• Overall EVA

48
Accurate Estimation

• Milestone Averages
• 38 met on time
• 18 met early
• 60 met late

49
Moving Forward
50
Fall Plan

• Client Requirements
• 4 Enhancements
• 9 Open Issues
• Report Training
• Data Documentation Migration
• 6 hours/week in September
• As Needed afterwards

51
Fall Plan

• Mentors/Program Requirements
• SEI Technical Report on Architecture-Centric
  Development
• Plan
• Abstract
• Outline
• 2 Drafts
• Final Report

52
Fall Plan

• Distance Collaboration
• 3/5 team members off campus
• Email/Skype/Wiki
• Lots of Practice!

53
Do you seek enlightenment?
54
Original Plan
55
Primary Re-plan
56
Final Plan
57
Week 1 Status
58
Week 2 Status
59
Week 3 Status
60
Week 4 Status
61
Week 5 Status
62
Week 6 Status
63
Week 7 Status
64
Week 8 Status
65
Week 9 Status
66
Week 10 Status
67
Week 11 Status
68
EVA Prior to Re-Plan
69
Lines Of Code-Client

• Total Unit Test LOC 5250

70
Lines Of Code-Server
71
Lines Of Code-Common

• Total Unit Test LOC 7100

72
Lines Of Code per Engineer
73
Estimates vs Actuals
74
PMD Statistics
75
Unit Test Code Coverage
76
Nightly Build

• Success Rate 89

77
Average Hours

• Planning 3.5 hours/week
• Proposals lt1 hour/week/person
• Architecture 2 hours/week
• Integration 13 hours/week
• Dev Environment 2 hours/week
• Meetings 3 hours/week/person
• Overhead 2-6 hours/week/person

78
Database Selection Criteria

• Embedded vs External vs Flat File
• Maintainability
• Usability

79
Report Engine Selection Criteria

• JasperReports
• Pros
• This is the most popular open source reporting
  tool (based on the number of downloads). It
  supports reporting functionality and exports to a
  variety of formats like PDF, HTML, XLS, CSV and
  XML.
• Cons
• Web reporting functionality does not seem to be
  straightforward. The idea is to export to HTML
  (or XML) and then use that for rendering.
• BIRT
• Pros
• This is a reporting tool provided by Eclipse. It
  supports reporting functionality and exports to
  formats like PDF and HTML.
• Web reporting is straightforward. Same code can
  be used on the client side (within an
  application) and on server side (to render on a
  browser)
• Easy to integrate in Eclipse RCP framework
• Cons
• Export to CSV is not straightforward.
• OpenReports
• Pros
• This is a web reporting tool. It supports
  reporting functionality and exports to formats
  like PDF, HTML, CSV, XLS, RTF, and Image
• It interacts with other reporting engines like
  BIRT and Jasper to provide one unified reporting
  framework.
• Cons
• Integrating OpenReports into a standalone
  application is not straightforward.

80
Risks

• 14 Initial Risks documented
• 13 Mitigated
• 1 Became a problem

81
Development Process (1)

• Design
• Develop acceptance tests from use case scenarios
  for the component/module and confirm these
  acceptance tests with the client.
• Develop a class diagram for the module
• Specify API's that will be exposed
• Artifacts acceptance test cases, class diagrams,
  and API document
• Design review
• Review the class diagrams to ensure conformity
  with architecture
• Review the interfaces exposed, and document
  component dependencies
• Artifacts review comments and developer response
  (conformed to, or justification why not) will be
  stored in a issues tracking tool.
• Code
• Develop unit tests, and ensure that the code will
  be at least 60 covered.
• Implement the module's required responsibilities
  based on the Architecture document.
• Fix all errors.
• Artifacts code files with comments

82
Development Process (2)

• Verification
• Run the unit tests (ensure that all tests pass).
• Run PMD using the following rulesets and fix all
  non-trivial warnings. Rulesets basic, clone,
  codesize, design, imports, strictexception,
  strings and sunsecure.
• Artifacts PMD test reports
• Validation
• Run the acceptance tests (ensure that all
  acceptance criteria are met).
• Artifacts Acceptance test reports
• Code review
• The review process is defined here.
• Artifacts review comments and developer response
  (conformed to, or justification why not) will be
  stored in a issues tracking tool.