Dynamic Analysis and Testing of Ajax User Interfaces

1
Dynamic Analysis and Testing of Ajax User
Interfaces

• Ali Mesbah, Arie van Deursen

Software Engineering Research Group Delft
University of Technology, The Netherlands
2
Classical Web Applications
HTTP
Client Browser
GET
Server
HTML

• Shortcomings of the Multi-page style
• Low lever of user interactivity
• Redundant data transfer
• High user-perceived latency
• Client is passive

3
Ajax

• Asynchronous JavaScript and XML
• Asynchronous server communication through
  XMLHttpRequest
• Web standards-based (HTML, CSS, XML)
• Dynamic display through DOM
• JavaScript binds everything together
• Features
• No full page refresh!
• Single-page web interface
• Google Suggest, Gmail, Yahoo! Mail

4
Ajax-based Web Applications

• Client-side execution
• State changes navigation
• Dynamic DOM
• Delta-communication
• Clickables Elements having event-listeners
  attached to them

lta href"javascriptOpenNewsPage()"gt lta href""
onClick"OpenNewsPage()"gt ltdiv
onClick"OpenNewsPage()"gt lta href"news.html"
class"news"gt ltinput type"submit"
class"news"/gt ltdiv class"news"gt (".news").clic
k(function() ("content").load("news.php")
)
5
Research Question

• How can we automatically analyze (test) Ajax web
  applications?

6
Current Approaches

• General Approach extract link, send a request to
  the server and analyze the response. i.e., HTML
  (based on the Multi-page style)
• Capture/Replay demands a substantial amount of
  manual effort
• Static Analysis misses the complex run-time
  behavior
• Dynamic Analysis promising but detecting the
  various doorways to different dynamic states and
  providing proper interface mechanisms for input
  values is challenging.

7
A Method for Deriving Ajax States

• Conduct dynamic analysis of the user interface to
  find Clickable elements and state changes
• Reverse-engineer a State-flow Graph to abstract
  the navigation paths and state changes

8
User Interface State Changes

• In classic web applications, each UI state is
  represented by a URL and the corresponding web
  page
• In Ajax, it is the internal change of the DOM
  tree on the (single-page) user interface that
  represents a UI state change
• To adopt a generic solution, we define a UI state
  change as
• a change on the DOM tree caused either by
  server-side state changes propagated to the
  client, or client-side events handled by the Ajax
  engine

9
State-flow Graph
10
Inferring the State Machine (Crawljax)
11
Detecting Clickables

• Find a set of Candidate Elements (CC)
• Expose each to en event (onclick, onmouseover, )
• Analyze the effect on the DOM
• We distinguish between 3 ways of obtaining CCs
• Full Auto Scan HTML tagname (div, span, a, )
  with attribute constraints, e.g.,
  divclassmenuitem
• Annotation element attribute crawljaxtrue, or
  false
• A DSL Crawling Ajax Specification Language (CASL)

12
Detecting States

• Input DOM Trees before and after the event
• Compare DOM Trees
• As is
• Textual extract textual content and compare
• Structural extract structural content and
  compare
• Meta-model reverse-engineer the Schema for each
  DOM, then compare the schemas
• Comparison method
• Edit Distance, Levenshtein method with a
  similarity threshold

13
Processing Document Tree Deltas

• Upon every new state entry
• Compute the differences between the previous
  document tree and the current one
• We use an enhanced Diff algorithm to find the
  delta updates
• Delta updates may be due, for example, to a
  server request call that injects new elements
  into the DOM
• Find new candidate elements in the delta updates,
  and examine them in a recursive depth-first manner

14
Navigating the States

• Problem going Back in the browser
• Solutions
• Register the state explicitly and use Back
• Click through from the initial state
• Use element IDs not always present, not always
  persistent
• Use XPath Expressions much more reliable, not
  100 water-proof

15
Data Entry Points

• Detecting DOM forms
• For each state change, extract all the forms
• For each form, calculate a hashcode based on the
  form properties
• Check if form is already present in the database
• If not, store the form completely in the database
  with the hashcode as its ID
• If present, use the corresponding custom values
  in the database (if present) to fill the form in
  the browser and submit
• Upon submission, the resulting state change is
  recursively analyzed.

16
Full Auto Scan

• For each state
• Retrieve all candidate clickables
• Fire event on each candidate element
• Compare DOM trees Edit Distance. Is DOM changed?
• Make a new State and add to the State-flow graph
• Find the XPath Expression of the Clickable
  Element
• Add the clickable as an edge to the SFG between
  state before and after the click
• Analyze forms
• Do all this recursively

17
Testing Ajax States

• With access to different dynamic DOM states we
  can check the user interface against different
  constraints
• We use invariants as oracle
• Invariants on the DOM tree
• Invariants between DOM states, and
• Application-specific invariants

18
Generic DOM Invariants

• Generic DOM Invariants
• Validated DOM
• No error messages in DOM
• Accessible State
• Secure State
• Discoverable links
• No Dead Clickables
• Consistent Back-Button

19
Application-specific Invariants

• Examples
• //case one warn about collapsable divs within
  expandable items
• String xpathCase1 "//LIcontains(_at_class
  ,expandable)/DIVcontains(_at_class
  ,collapsable)"
• //case two warn about collapsable items within
  expandable items
• String xpathCase2 "//LIcontains(_at_class
  ,expandable)/UL/LIcontains(_at_class
  ,collapsable)"

20
Testing Ajax Paths

• While running the crawler to derive the state
  machine can be considered as a first full test
  pass, the state machine itself can be further
  used for testing purposes.
• We generate test cases from the state machine
• K shortest paths from Index to sinks (nodes with
  no outgoing edges)
• Cycles included only once, All-transitions
  coverage
• Transform each path found into a JUnit test case

21
Testing Ajax Paths

• _at_Test
• public void testcase1 ()
• browser.goToUrl (url)
• /Element-info SPAN classexpandable-hitarea /
• browser .fireEvent(new Eventable (new
  Identification("xpath",
• "//DIV1/SPAN4"), "onclick"))
• Comp.assertEquals(oracle.getState ("S_1").
  getDom(),
• browser.getDom ())
• /Element-info DIV classhitarea idmenuitem2 /
• browser.fireEvent(new Eventable(new
  Identification("xpath",
• "//P2/ DIV2"), "onmouseover"))

22
Testing Ajax Paths

• Generated test suite can be used in several ways
• It can be run as is on the current version of the
  Ajax application, but for instance with a
  different browser to find browser
  incompatibilities
• It can be applied to altered versions of the Ajax
  application to support regression testing
  (against a Gold-standard)
• The typical use of the derived test suite will be
  to take apart specific generated test cases, and
  augment them with application-specific
  assertions, capturing specific fault-sensitive
  click trails.

23
Tool Implementation

• Crawljax crawling through dynamic states
• ATUSA plugin-based framework on top of Crawljax
• preCrawling
• inCrawling
• postCrawling
• Java 1.6
• Watij (IE), XULRunner (Mozilla)
• Numerous libraries
• Java Plugin Framework
• Open source

24
Tool Implementation
25
Empirical Evaluation

• 4 case studies (3 open source), 1 industrial
• TUDU, FreakTask, PetStore, CoachJeZelf
• Instrumented the server side code with Clover
• Instrumented the client-side JavaScript code with
  JSCoverage
• Fault seeding and code coverage
• Code Coverage 73 server-side, 70 client-side
• More than 180 Violations per case
• 80 fault detection

26
Concluding Remarks

• We have proposed a method for testing Ajax
  applications automatically through
• A crawler that can detect clickables and forms
  and analyze state changes
• A testing framework that can check the
  application behavior through generic invariants
  as well as application-specific invariants
• A test suite generation from the inferred state
  machine.
• Tools are available for download
• http//spci.st.ewi.tudelft.nl/content/softwar
  e

27
Publications

• Ali Mesbah, Engin Bozdag, and Arie van Deursen
  (2008). Crawling Ajax by Inferring User Interface
  State Changes. In Proceedings of the 8th
  International Conference on Web Engineering
  (ICWE'08), IEEE Computer Society.
• Ali Mesbah and Arie van Deursen (2009).
  Invariant-Based Automatic Testing of Ajax User
  Interfaces. In Proceedings of the 31st
  International Conference on Software Engineering
  (ICSE'09), IEEE Computer Society.