Data flows, serial structures, data interchange standards and XML

1
Lecture 6

• Data flows, serial structures, data interchange
  standards and XML

2
(No Transcript)
3
Fixed Format file

• 01 70Centre 1
• 020720
• 020740
• ..
• 01 99Centre to Blackboy Hill 2
• 020930

4
Fixed format file definition

• There are two record types 01 and 02. 01 records
  define routes, 02 records define departures on
  the preceding route.
• 01 Format
• 1-2 record number 01
• 3-6 route number
• 7-27 destination
• 28 bay number ( 1-5)
• 02 Format
• 1-2 record number 02
• 3-4 departure time hours (24 hour clock local
  time)
• 5-6 departure time minutes
• See the atco-cif file definition

5
META-DATA data about the data.

• File structure
• Validation rules
• Origination and authorisation
• Version

6
Formal syntax definitions

• Enable a process to read the syntax definition,
  then check a file conforms to the syntax is
  well-formed
• Enable programmers to define transformations of
  the file into other formats (for XML using XSLT)
• Communicates a file definition to writers and
  readers of the file
• Defines the implicit data model

7
Comma Separated Variable (CSV) Files

• 01,70,Centre,1
• 02,0720
• 02,0740
• ..
• 01,99,Centre to Blackboy Hill, 2
• 02,0930

8
Existing File formats

• Files from legacy COBOL systems (coded as fixed
  format records, with no attached meta-data)
• BACS files for bank standard published by the
  Association for Payment Clearing Services
• Image file formats (JPEG,GIF)
• Sound files (MP3)
• Archived, Compressed files (ZIP)
• HTML?

9
HTML
lttable border1gtlttrgtltthgtRoutelt/thgtltthgtTimelt/thgtltth
gtDestinationlt/thgtltthgtBaylt/thgtlt/trgtlttrgtlttdgtlta
href'http//www.firstcityline.co.uk/tt/070mf.htm
'gt70lt/agtlt/tdgtlttdgt152400lt/tdgtlttdgtCentrelt/tdgtlttdgt3
lt/tdgtlt/trgt lttrgtlttdgtlta href'http//www.firstcityli
ne.co.uk/tt/070mf.htm 'gt70lt/agtlt/tdgtlttdgt153900lt/t
dgtlttdgtCentrelt/tdgtlttdgt3lt/tdgtlt/trgt
10
XML

• ltroutegt
• ltroutenogt99lt/routenogt
• ltdestinationgtCentre via Blackboy
  Hilllt/destinationgt
• ltbaygt2lt/baygt
• ltdeparturesgt
• ltdtimegt700lt/dtimegt
• ltdtimegt720lt/dtimegt
• lt/departuresgt
• lt/routegt

11
Cascading Style Sheet

• routeno display block colorred
  font-size400
• destination display block border-styledouble
  border-widththick color pinkfont-size200
• bay display none
• dtime display table-cell

12
Jackson Structure Diagram
13
Extract from TransXchange doc
14
BNF syntax specifiation

• file route
• route routeno destination bay departures
• departures departure
• routeno 2 char 3
• destination 5 char20
• bay digit
• departure dtime
• dtime digit digit '' digit digit
• digit '0' '1' '2' '3' '4' '5' '6'
  '7' '8' '9'
• char letter digit
• letter A B C D E F G
  H I J K L M N O
  P Q R S T U V W
  X Y Z

15
Generated random data

• 01,E60,ETET NHNXETE,2
• 02,5939
• 02,1943
• 02,8355
• 02,5027
• 02,7069
• 02,8766
• 02,4821
• 02,9326
• 02,9911
• 02,3620
• 02,1323

16
XML DTD for the schedule

• lt!DOCTYPE route
• lt!ELEMENT route (routeno,destination,bay,departu
  res)gt
• lt!ELEMENT routeno (PCDATA)gt
• lt!ELEMENT destination (PCDATA)gt
• lt!ELEMENT bay (PCDATA)gt
• lt!ELEMENT departures (dtime) gt
• lt!ELEMENT dtime (PCDATA) gt
• gt

17
XSchema definition

• lt?xml version"1.0"?gt
• ltxselement name"route"gt
• ltxscomplexTypegt
• ltxssequencegt
• ltxselement name"routeNo" type"xsstring"/gt
• ltxselement name"destination"
  type"xsstring"/gt
• ltxselement name"bay" type"xsstring"/gt
• ltxselement name"departures"gt
• ltxscomplexTypegt
• ltxssequencegt
• ltxselement name"dtime"
  maxOccurs"unbounded" type"xstime"/gt
• lt/xssequencegt
• lt/xscomplexTypegt
• lt/xselementgt
• lt/xssequencegt
• lt/xscomplexTypegt
• lt/xselementgt
• lt/xsschemagt

18
Beyond Syntax

• Is this file authorized by the service provider?
• How are authorized providers set up within the
  consolidator?
• Which side of the interaction is responsible for
  initiating the transfer
• the consolidator ( pull)
• or the service provider (push)
• Where are pushed files stored before processing?
• If there is an error in the file, how is this
  fact notified to the provider?
• What else?

19
Exercises

• Take a look at the BNF sentence generator
• Modify my syntax to add the weblink
• Look at the atco-cif file spec
• Draw the JSD
• Draw the underlying ER model
• Look at the TransXchange XSchema