Approaches to Application Modernization

1
Approaches to Application Modernization

• Don Denoncourt
• dondenoncourt_at_gmail.com

2
Outline

• What is App Modernization and Why Bother?
• Modernizing Coders
• Code Moves to Data, Modernizing the Database
• Gradational versus Global Approach
• Fear Factor
• Grokking RPG
• Save the Program, Save the Platform

3

• "Modernization is our jobs. Its what IT does. We
  improve access to information for the businesses
  we work for."
• Trevor Perry

4
Traits of a Modern App

• It has maintainable and testable code
• It is modular and component-based
• It provides a rich user interface
• It uses the Model-View-Controller (MVC) design
  pattern
• Separate presentation, flow control, and business
  logic
• It is responsive to new business requirements

5
Why Bother?

• Modernizing applications is costly
• Not modernizing is more costly
• "60 to 80 percent of all software development is
  enhancements to or maintenance of existing
  projects."
• Gartner Group report
• Amounts to a 10 increase in legacy code per year

6
Escalating Costs

• Effort required to enhance and maintain legacy
  apps escalates with time
• Fact
• Code is read far more often than it is written.
• So
• "What sense does it make to bloat unreadable code
  at a rate of 10 percent per year?"
• Reducing maintenance costs may be the biggest
  reason to modernize

7
Modular Applications

• Allow you to quickly respond to business
  requirements
• Improves interoperability
• Other applications
• Languages
• Technologies
• Including SOA and REST

8
Modernizing Coders
9
Three Contexts of Modernization

• User Interface
• Code
• Coders

10
Modern UI

• If you're not rewriting your entire app in a new
  language
• Need to modularize RPG
• before adding a Web front-end
• What about HATS and WebFacing?
• WDHT is the Botox of UI modernization
• Quick and easy
• But behind that puffed-up face
• There will always be the same old person

11
Application Make-over

• To make over a person (or an application)?
• You need to start from the inside
• That means
• Attitude
• Knowledge
• Organization

12
Attitude, Knowledge, and Organization

• Attitude should come from personal fulfillment
• But it can also be coerced from effective
  management
• Knowledge is acquired from experience or
  education
• Organization
• micro-level
• when restructuring a program
• macro-level
• when planning an attack for modernization.

13
Training

• Formal education
• Most common strategy
• But the best training is mentoring
• Conferences and formal classes
• are worthwhile
• But mentoring virtually guarantees
• New techniques can transform from knowledge to
  functioning business code

14
Mentor

• If you dont have internal expertise
• Get a consultant
• one with mentoring skills
• To substantially decrease costs
• Consider
• WebEx, GoToMeeting, and alternative web
  conferencing tools
• Consultant may be off-site and part-time
• Maximize your training budget's ROI
• With 2 hours mentoring a day

15
Master Programming Toolsets

• Promote use of a modern development tools
• WDSc/RDi
• Eclipse, NetBeans, Visual Studio
• A good IDE provides tools that are necessary in
  the modernization process.

16
Grokking the Code with WDSc
17
Application Diagram
18
Search and Replace
19
Outline View
20
Integrated Help
21
Streamlined edit-compile-debug
22
Code Moves to Data
23
Modernize Your Database

• 20-year-old RPG
• Code manages what should be done with db
• Table relationship management
• Foreign key constraints.
• Refactoring that code out
• Reduces requirement for code

24
Does your database really need modernization?

• If your files are defined with DDS
• If you are not using constraints and triggers
• If you are planning to implement Business
  Intelligence
• If you are planning to provide a web front-end to
  your data
• If new developers need to look at code to
  understand your database
• Then. Yes. You really need to modernize your
  database.

25
Significant Performance Improvements

• DDL-defined table page size much larger that
  DDS-defined files
• DDL page size 64K
• DDS page size 8-32K
• Encoded Vector Indexes (EVI)?
• SQL query engine (SQE) runs below the MI
• Versus the Classic Query Engine (CQE)

http//systeminetwork.com/article/performance-comp
arison-dds-defined-files-and-sql-defined-files
26
New Column/Field Capabilities

• BLOBs and CLOBs
• Constraints
• User-Defined Data types (UDT)?
• Formula columns
• Encryption and decryption
• Complex view definitions (logical files)?
• Expressions
• Aggregation
• Complex selection

27
New Column/Field Capabilities

• Long table and view names
• 128 characters
• Identity and RowId columns
• Sequences
• Auto-increment of keys
• Column-level triggers
• Column-level privileges (authorities)

28
Object-relation Mapping Tools

• Generators for ORM engines
• Such as
• Java Persistence API
• Hibernate
• OJB
• Work best with a well-crafted database
• And standard SQL DDL constructs

29
Object Traversal

• Enabled with 1 database operation via joins
• Versus a read for each row/record
• Java
• event.getVenue().getAddress().getStreet()
• Grails
• event.venue.addresss.street

30
More reasons to modernize

•  SQL is a widely used standard.
• Openness
• more and better options for third-party tools
• Books and training readily available
• Performance
• IBM is investing money on improving database
  access through SQL, not elsewhere.
• Availability of skills
• Easier find Java and SQL skills over RPG and DDS
  knowledge.
• Functionality
• Some new functions require SQL.

31
Code moves to data

• Business rules
• As much as possible
• belong in DDL
• DDL-based Business Rules
• Constraints
• Primary and foreign keys
• Column-level contraints
• Triggers
• Operations to perform on
• Add, update, or delete operations.

32
Benefits of DDL Business Rules

• Removes duplicate code
• Decreasing complexity
• Improving maintainability
• Self-describing
• Enforces database integrity
• Rules are always followed
• Whether .Net, PHP, or Java web front-end
  applications
• Or other RPG

33
SQL's Data Definition Language (DDL)5-minute
tour
34
create table item ( itemid varchar(10)
not null, productid varchar(10) not null,
listprice decimal(10,2) not null, unitcost
decimal(10,2) not null, supplier int
not null, status varchar(2) not null,
attr1 varchar(80) not null, attr2 varchar(80)
not null, attr3 varchar(80) not null,
attr4 varchar(80) not null, attr5 varchar(80)
not null, constraint pk_item primary key
(itemid), constraint fk_item_1 foreign
key (productid)? references product
(productid), constraint fk_item_2 foreign
key (supplier)? references supplier
(suppid)? ) create index itemProd on item
(productid)

• PetStore DB
• Used in the ubiquitous sample web application

35
create table product ( productid varchar(10)
not null, category varchar(10) not null,
name varchar(80) not null, descn varchar(255)
not null, constraint pk_product primary key
(productid), constraint fk_product_1
foreign key (category)? references
category (catid)? ) create table category (
catid varchar(10) not null, name varchar(80)
not null, descn varchar(255) not null,
constraint pk_category primary key (catid)? )
create table supplier ( suppid int not null,
name varchar(80) not null, status
varchar(2) not null, addr1 varchar(80) not
null, addr2 varchar(80) not null, city
varchar(80) not null, state varchar(80) not
null, zip varchar(5) not null, phone
varchar(80) not null, constraint pk_supplier
primary key (suppid)? )
36
PetStore on Squirrel SQL Client
37
PetStore Database onWDSc's Data Perspective
38
Field-level Constraints

• CREATE TABLE orders (
• CustNum CHAR(4),
• OrderNum CHAR(4),
• Quantity INTEGER CHECK (Quantity BETWEEN 1 AND
  50)
• )

39
On Delete and On Update

• CREATE TABLE CORPDATA.EMPLOYEE
• (EMPNO CHAR(6) NOT NULL PRIMARY KEY,
• FIRSTNME VARCHAR(12) NOT NULL, MIDINIT
  CHAR(1) NOT NULL,
• LASTNAME VARCHAR(15) NOT NULL,
• WORKDEPT CHAR(3)
• CONSTRAINT WORKDEPT_EXISTS
• REFERENCES CORPDATA.DEPARTMENT
  (DEPTNO)?
• ON DELETE SET NULL ON UPDATE RESTRICT,
• PHONENO CHAR(4), HIREDATE DATE,
• JOB CHAR(8), EDLEVEL SMALLINT NOT
  NULL,
• SEX CHAR(1), BIRTHDATE DATE,
• SALARY DECIMAL(9,2), BONUS
  DECIMAL(9,2), COMM DECIMAL(9,2),
• CONSTRAINT UNIQUE_LNAME_IN_DEPT UNIQUE
  (WORKDEPT, LASTNAME)?
• )

40
Database Reverse Engineering Tooling

• Navigator
• WDSc
• System i System API
• Third-party tools

41
Navigator has a suite of SQL tools
Note IBM is phasing out database development
with Navigator
42
WDSc's Database Perspective
43
Generate DDLon a PF called ALLTYPES
44
System i API

• Generate Data Definition Language (QSQGNDDL)?
• Carsten Fleming wrote an iSeries command
• GENSQLDDL DBOBJ(yourFile)
• DBLIB(QSYS)
• TYPE(PF)
• SRCFILE(QGPL/QDDLSRC)
• MBR(yourFile)
• MBROPT(REPLACE)NAMING(SQL)        

http//systeminetwork.com/article/apis-example-rev
erse-engineering-database-files-and-objects-sql-dd
l-statements
45
Gradational versus Global Approach
46
Two Strategies Globally or Gradually

• Global modernization
• Management backs the project with bucks
• Bring in consultants
• Purchase a modernization tool-set
• such as Databoroughs X-Analysis.
• Gradational modernization
• Refactor code on an ongoing basis during normal
  business development

47
Global Realities

• Management often will not back such a
  concentrated effort
• Even though, in the long run,
• It is the most cost effective
• There are two alternatives to this
• Do nothing
• And let your code swell beyond recognition
• Improve code with the gradational approach.

48
Gradational Philosophy

• "Regard every bug fix or enhancement as an
  opportunity to improve code."

49
Gradational Realities

• Consider a change request
• Estimated as a 10- to 20-hour project
• Quick-and-dirty status-quo
• Opportunity to modernize
• First time realities
• Potentially take 80140 hours
• Time will appreciably decrease on each new request

50
Gradational Benefits

• Soon you will be more receptive change requests
• Along with
• Increased adaptability
• Greater interoperability
• Reduced code bloat

51
Fear Factor
52
Fear Factor

• Strategy a bit too radical?
• Scared to make far sweeping changes to a program?
• Frightened because you cant deduce the impact of
  those changes?

53
Fear is Why Legacy Code Bloats

• Instead of correcting existing code
• We add
• Global variables
• Subroutines
• Or copy the program and modify the new on
• We do this so we know our new code wont affect
  old code

54
We Need Confidence

• Conviction that changes will work in production
• "The only way to be confident about code changes
  is automated testing."
• "If you cant figure out how to test code, dont
  bother writing it. "

55
Refactoring

• Well known software engineering practice
• Changing code to simplify or improve readability
  without changing the result.
• Automated unit testing ensures
• Code modifications cannot cause the application
  to fail

56
RPG Unit Test Example

• /free
• // test encryption
• encrypted IMAXgetIFSFileAsBase64('/rxs/co
  ntract/irm/dontdelete.xml')
• assertEquals(IMAXgetIFSFile('/rxs/contract/
  irm/dontdelete.xml')
• IMAXdecryptBase64(encrypted))
  
• // test uploadFiles web service with 2
  good files, and 2 bogus recipients
• paths(1) '/rxs/contract/irm/US01060912352
  11t.TXT'
• paths(2) '/rxs/contract/irm/duplicate.txt
  '
• recipients(1) 'Sam'
• recipients(2) 'Will'
• uploadRsp IMAXuploadFiles('TRACM01908''r
  pdp01'
• paths
  recipients 'RXS test request')
• assertEquals('500' char(uploadRsp.statusC
  ode))
• kar uploadRsp.statusMsg
• assertEquals('Internal System Error -
  Invalid usersSAM,WILL' kar)

57
Gradational 4 Steps

• Write a wide range of unit tests over existing
  code
• Run them before any modifications are made.
• Make modernization changes
• Without adding bug fixes or enhancements
• Rerun the tests.
• Add tests for the change request
• Run the tests
• which will fail
• Make the change
• Iteratively rerun tests and tweak the code
• Until the tests succeed
• And consider fear no longer a factor.

58
Confidence with Source Control

• Source control make you bold with code changes
• Without the ability to revert to a prior version
  of code
• Coders are reluctant to make extensive changes
• Use a source-control product
• Don't rely on manual methods.
• Use open-source CVS and Subversion tools.
• or one of the various commercial products

59
Grokking RPG
60
Grokking the Code

• When a change request comes in
• Don't hack the change
• Grok the code and modernize
• Grok means
• "Understand profoundly and intuitively"

61
Grokking RPG

• Rip the program apart with both your brain and
  WDSc tooling until you fully comprehend it.
• If you can't Grok RPG because of a poorly named
  variable or routine
• Rename it to something more meaningful
• Run unit tests to prove that your refactoring
  didnt break the code
• Discover business entities
• Those not already in record formats
• Define external structures for them.
• Refactor the code to use the external entity
  structure
• Recompile and retest

62
Separating UI from Biz Logic

• First step to modernizing
• Separating business logic from display gives
  three advantages
• You can create automated tests
• You can reuse the modules in other applications
• You can more easily interface with web
  technologies

63
Implement the Change Request

• With UI and Biz logic separate
• And perhaps readability improvements
• Now ready to implement the change request

64
Code Change Process

• If change request requires modification of a
  subroutine
• Convert it to a sub-procedure
• Remove global variables in the subprocedure
• May force you to add parameters to the
  subprocedure
• But enables unit testing
• Note that file parameters are available for RPG
  with V6R1
• File I/O was the last vestige of requirements for
  global variables in modern RPG

65
Code Change Process continued

• With global variables removed
• The subprocedure can be placed in its own source
  member
• And becomes a module
• Create unit tests for the subprocedure
• How did we used to test subroutines?
• We didnt, because we couldnt.
• Subroutines werent modular
• so automated atomic testing was not an option
• With subprocedures
• Unit testing should be a requirement

66
Save the Program, Save the Platform
67
Code Inventory and Empirical Study

• Do an inventory of your applications
• What are the assets
• Liabilities
• Do an empirical study
• What programs give you the most trouble?
• Or require the most frequent changes
• Modernize those first

68
Getting Started

• It costs less to modernize than to
• Put your coding heads in the sand
• And ignore code bloat
• Deal with the fear factor
• Mitigate risk with unit testing and source
  control
• Get a mentor
• Promote the use of WDSc and improved tooling
• Start code reviews

69
RPG is One of IBM i's Greatest Strengths

• Every RPG you rejuvenate strengthens the platform

70
systeminetwork.com Articles

• "Grokking Software Development" (August 2007,
  article ID 20979 at SystemiNetwork.com),
• iUnit from SystemiNetwork.com/code (select April
  2007).
• Instructions for iUnit
• "Developing Reusable RPG APIs" (February 2006,
  article ID 20395)
• "Testing RPG-based Stored Procedures with RPG"
  (April 2007, article ID 20866).
• "RPG Source Control for Free" (August 2005,
  article ID 20205),