jBASE Product Roadmap for the future

1
jBASE Product Roadmap for the future

• Martin Bailey Clive A Ketteridge

2
Agenda

• Where are we now
• Release 3
• Release 4
• jBASE Web Builder
• Procedures
• What's being constructed?
• jBASE Web Builder 4.0
• Whats being researched ?
• Flexible Binaries
• Warmstart Recovery

3
Where are we (Release 3) ?

• Release 3 currently in maintenance
• 3.4.6 due end of this year
• Release 3 life began in 1997 and will come to an
  end ???

4
Where are we (Release 4) ?

• jBASE Release 4 is probably the most substantial
  engineering project undertaken in the MV industry
  in the last decade
• It represents a significant investment in the
  future of the jBASE product
• It was a huge venture
• Large product
• Virgin territory

5
Where are we (Release 4) ?

• Such an undertaking is bound to take some
  bedding in
• Stability of Release 4 is vitally important for
  all concerned
• We will focus on Release 4 stability until it is
  achieved.
• It will be our base for a decade or more to come

6
Where are we (jBASE Web Builder) ?

• jBASE Web Builder 3.4 in maintenance
• jWB 3.4.3 just released
• Native Java code
• Java API
• Cascading StyleSheet (CSS) support
• N-tier Servlet architecture
• Development has started on jWB 4.0

7
Where are we (Procedures) ?

• We learned lessons from the Release 4 project
• Development under much tighter control
• Closely monitored and managed
• ISO standards
• Means output will be more predictable
• QA has been and will continue to be enhanced
  significantly

8
Whats next ?

• Focus on stability of Release 4
• jBASE Web Builder
• XML / XSLT delivery (Under Construction)
• jBASE - Research
• Flexible Binaries (White paper stage)
• Warmstart Recovery (White paper stage)

9
Flexible binaries

• BASIC is currently pre-compiled to C and
  subsequently compiled to operating system code
• Problems with this approach
• Developers must have every machine available to
  build
• Multiple distributions of applications
• C compiler required for every platform
• Difficult to implement other interim languages
• Alternative approach is usually P-code or
  intermediate code
• Inefficient

10
Flexible binaries

• Why is P-code or intermediate code inefficient ?
• Consider the statement
• PRINT a b
• Typically when converted to P code, would look
  something like this
• PUSH A
• PUSH B
• PUSH R0
• ADD POP POP POP PUSH
• PRINT POP

11
Flexible binaries

• Many instruction cycles lost in stack operations
• Flexible Binaries would use a P-code structure
  with no stack operations
• Very closely tied to underlying library call
• PRINT a b
• Maps to PRINT_VIB(ADD_BBB(R0, A, B)
• ADD
• R0
• A
• B
• PRINT
• R0

12
Flexible binaries

• Flexible Binary P-code Much lighter than
  traditional intermediate code
• Particularly on more complex operations
• No stack handling
• This code produced by using BASIC command
• Same on all platforms
• Still not runnable
• Another step required
• Analogous to CATALOG
• Called Finalize

13
Flexible binaries

• Finalizing options
• To finalized flexible binary code
• Different endian formats
• Different character sets (EBCDIC /ASCII)
• Requires a virtual machine
• Estimated 10 performance penalty
• To C code
• Like we have now
• To Java code
• To dot Net
• JIT compiler / Interpreted BASIC

14
Flexible binaries

• Summary of benefits
• No need for every platform in the development
  shop
• Good news if you want to ship on expensive
  platforms (iSeries, zSeries etc)
• No need for a C compiler
• Much easier distribution of applications
• One binary for all platforms
• Finalized onto target machine

15
Warmstart recovery

• Currently, jBASE can be configured to be
  recoverable following a crash
• Using Transaction Journaling
• After a crash reload from last backup and
  replay transaction log
• Can take a long time
• Currently, jBASE files need to be resized
  frequently
• Must be done when no user activity

16
Warmstart recovery

• Firstly new file type
• To allow dynamic sizing of files
• Data and indexing of item ID will be separated
• B-tree index will be created holding
• Hash
• Compartment
• In this way, compartment is no longer dependant
  on hash value of item id
• Compartments may be added on the fly
• Performance will not degrade substantially as the
  file grows

17
Warmstart recovery

• New file type will be able to scale gt 2GB
• Will also have configurable flushing
• As jPlus j3s
• No security
• No corruption
• No loss of data
• In addition changes to TJ
• Ability to set up short rotating log
• On system restart
• Go back twenty minutes roll forward committed
  transactions

18
Scenarios currently on j4s
Consider a transaction that writes to each ofthe
three files (A, B C)
Updates held in memory until TRANSEND statement
issued then written to the transaction journal
which is flushed to disk
A
A(x)
B
B(y)
x y z
C
C(z)
Transaction journal
Transaction journalA(x),B(y),C(z)
Updates then applied to the files themselves.
Should the system crash During the writing of
these updates we have a problem.
19
Scenarios currently on j4s

• If the system was right in the middle of
  performing one of the writes we could have a
  corrupted file
• Restore from backup
• Replay the journal
• If the system didn't get as far as making one or
  more of the writes we have an incomplete
  transaction
• Replay the journal
• From how far back ?
• The risk can be reduced if we switch from j4
  files to jPlus files
• Remove corruption possibility
• At the expense of performance (more flushing)

20
Scenarios currently on jPluses
Consider a transaction that writes to each ofthe
three files (A, B C)
Updates held in memory until TRANSEND statement
issued then written to the transaction journal
which is flushed to disk
A
A(x)
B
B(y)
x y z
C
C(z)
Transaction journal
Transaction journalA(x),B(y),C(z)
Updates then applied to the files themselves,
followed by a flush. We May end up with an
incomplete transaction but no corruption.
21
Future scenario with Warmstart

• With warmstart, the only flushing that needs to
  happen is to the transaction journal
• Improves performance
• No compromise on security
• When a system comes up after a system crash
• Crash detected
• Roll back to predetermined point in the journal
• Far enough back to cover any updates not yet
  flushed
• Probably 15 20 minutes
• Re-apply all completed transactions where
  necessary
• Much faster recovery time

22
Future scenario with Warmstart
Consider a transaction that writes to each ofthe
three files (A, B C)
Updates held in memory until TRANSEND statement
issued then written to the transaction journal
which is flushed to disk
A
A(x)
B
B(y)
x y z
C
C(z)
Transaction journal
Transaction journalA(x),B(y),C(z)
Updates then applied to the files themselves. We
don't care about flushing, as warmstart will
sort it out afterwards.
23
jWB Current logical view
sysclasses
parms
HTTP request inHTML response out
Processes each object andproduces HTML. This can
be recursive where objectincludes other objects
(egtables / menus).
Sysclasses cached at startupof queue / entry
point. Parmsgives us the routine name
sysqueryin
syscreate
Object build routines(syso and syse)
sysqichk
Reads in page andevaluates each embeddedobject
calls build routine
Session credentials checkedand session data
loaded
sysobjects
syssessions
24
jWB Current logical view
sysclasses
parms
Need the ability toOutput XML for XSLT
Processes each object andproduces HTML. This can
be recursive where objectincludes other objects
(egtables / menus).
Sysclasses cached at startupof queue / entry
point. Parmsgives us the routine name
sysqueryin
syscreate
Object build routines(syso and syse)
sysqichk
Reads in page andevaluates each embeddedobject
calls build routine
Session credentials checkedand session data
loaded
sysobjects
syssessions
25
jWB Current logical view
sysclasses
parms
Need the ability toOutput XML for XSLT
Processes each object andproduces HTML. This can
be recursive where objectincludes other objects
(egtables / menus).
Sysclasses cached at startupof queue / entry
point. Parmsgives us the routine name
sysqueryin
syscreate
Object build routines(syso and syse)
sysqichk
Reads in page andevaluates each embeddedobject
calls build routine
Session credentials checkedand session data
loaded
sysobjects
syssessions
Need to store data in XML format
26
Rationale

• Outputting XML allows translation by external
  XSLT stylesheet
• More modular
• More mainstream
• More scalable
• XML storage means that object store is readable
• More open
• More mainstream
• Web Builder becomes an XML assembly engine for an
  XSLT translation process
• This process could be an EJB / Servlet on another
  machine

27
Design constraints

• Performance must remain acceptable
• J2EE server must not be an absolute requirement
• Optional extra
• Backwards compatibility must be maintained
• Ease of use must be maintained
• Install
• Upgrade
• Configuration
• Maintenance

28
Approach

• Current page build process

Web builder evaluates every objectthat makes up
the page and fires offthe appropriate build
routine.
Page
Table
Label
The build routine reads in the objectrecord and
examines each property.
Textbox
Button
As each property is examined, the appropriate
HTML is written to theoutput stream
29
Approach

• Intended page build process

There will be a new constructionphase called
stylesheet build
Page
Table
This will examine every object andall properties
evaluating whetherthe HTML to be produced is
likelyto be volatile or remain static.
Label
Textbox
Button
Static content is written directly outto the
stylesheet. Dynamic contentresults in a rule
being written out.
30
Approach

• Intended page build process

At the same time, jWB will updatethe page record
to tell the constructionengine what data to
include in the XML response.
Page
Table
Label
Textbox
Until the stylesheet build processhappens again,
this is the only areaof the page record that jWB
will read when rendering the page
Button
Volatile DataPointers
31
Typical XSLT stylesheet
lt?xml version1.0?gtltxslstylesheet
xmlnsxslhttp//www.w3.org/1999/XSL/Transformgtltx
sloutput methodhtml /gtltxsltemplate
match/gt lt!doctype html public "-//w3c//dtd
html 3.2//en"gt lthtmlgtltheadgt ltmeta
http-equiv"Pragma" content"no-cache"gt ltmeta
http-equiv"Content-Type" content"text/html
charset"gt lttitlegtlt/titlegt lt/headgt ltbodygt
ltform methodpost action"development.dll"gt
ltinput typehidden namesessionid
value_at_sessionid/gt ltinput typehidden
namefiller value""gt ltinput typehidden
nametrigger value""gt ltinput typehidden
namedata value""gt ltinput typetext
namefld_1_1gt ltxslattribute name"value"gt
ltxslvalue-of select//sysdata_at_value1/gt
lt/xslattributegt lt/inputgt lt/formgtlt/bodygt
lt/htmlgtlt/xsltemplategt
32
Typical XML output

• All jWB has to do is examine the Volatile Data
  Pointers in the page
• Produce XML output accordingly
• Much less CPU intensive
• Much less file I/O

ltresponse sessionid03666165244844124329736061143
9000969gt ltsysdata value1gtwidgetlt/sysdatagtlt/r
esponsegt
33
Transformation process

• Transformation can happen in a number of
  different places
• By the jWB engine after XML construction
• By a java bean on another machine
• By the servlet prior to delivery
• On the browser itself
• After transformation, wherever it occurs, the
  HTML should be identical to what would be
  produced by previous versions of jWB

34
Intended architecture
HTTPRequests
jBC
Java
J Q Server
XSLTTransformation(EJ) Bean
XML Payload
Servlet
jWB
Servlet Engine
Optional MQ Series
OptionalJ2EEServer
jBASE
jBASEStorage
HTTP Request
XMLStorage
HTMLResponses