Supporting ModelBased Validation at Runtime

1
Supporting Model-Based Validation at Run-time

• Insup Lee and Oleg Sokolsky
• Department of Computer and Information Science
• University of Pennsylvania

2
Software Development Process

• Requirements capture and analysis
• Informal to formal
• Consistency and completeness
• Assumptions and interfaces between system
  components
• Application-specific properties
• Design specifications and analysis
• Formal modeling notations
• Analysis techniques (simulation, model checking,
  equivalence checking, testing, etc.)
• Abstractions
• Implementation
• Manual/automatic code generation (refinement)
• Validation
• Testing
• Model extraction and verification
• Run-time monitoring and checking

Requirements
Design specification
Implementation
3
Model-based validation at run-time

• Run-time verification is to assure the compliance
  of an execution of a system with its formal
  requirements.
• This is achieved by monitoring the current
  execution and checking properties.
• The monitoring and checking (MaC) system can
• detect incorrect execution of an application
• predict error and steer computation
• collect statistics of actual execution (e.g., QoS
  validation)

4
MaC Methodology
System Spec
Requirements Spec
Formal verification
Design
5
The MaC Languages

• Meta Event Definition Language(MEDL)
• Describes the safety requirements of the system,
  in terms of conditions that must always be true,
  and alarms (events) that must never be raised.
• Target program implementation independent.
• Primitive Event Definition Language (PEDL)
• Maps the low-level state information of the
  system to high-level events.
• Provides primitives to refer to values of
  variables and to certain points in the execution
  of the program.
• PEDL is defined so that events can be recognized
  in time linear to the size of the PEDL
  specification
• Depends on target program implementation

• Run-time state
• control locations
• object state
• local variables

• Abstract state
• events
• conditions
• auxiliary variables

PEDL
MEDL
6
Design of the MaC Languages
start(position100)
end(position100)
raiseGate

position 100
10010
10030
10015
Time

• Must be able to reason about both time instants
  and information that holds for a duration of time
  in a program execution.
• Events and conditions are a natural division,
  which is also found in other formalisms such as
  SCR.
• Conditions, which are true or false for a finite
  duration of time (e.g., is variable x gt5?)
• Events, which are either present or absent at
  some instant of time (e.g., is control right now
  at the end of method f?).
• Need temporal operators combining events and
  conditions in order to reason about traces.

7
Logical Foundation

• Conditions interpreted over 3 values true,
  false and undefined.
• ., .) pairs a couple of events to define an
  interval.
• start and end define the events corresponding to
  the instant when conditions change their value.

8
Meta Event Definition Language (MEDL)
ReqSpec ltspec_namegt / Import section /
import event ltegt import condition ltcgt
/Auxiliary variable / var int ltaux_vgt
/Event and condition / event ltegt ...
condition ltcgt ... /Property and violation
/ property ltcgt ... alarm ltegt ...
/Auxiliary variable update/ ltegt -gt
ltaux_v'gt ... End

• Expresses requirements using the events and
  conditions, sent by event recognizer.
• Expresses the subset of safety properties.
• Describes the safety requirements of the system,
  in terms of conditions that must always be true,
  and alarms (events) that must never be raised.
• property safeRRC IC -gt GD
• alarm violation start (!safeRRC)
• Auxilliary variables may be used to store
  history.
• endIC-gt num_train_pass
  num_train_pass
  1

9
Property checking

• A MEDL specification can be seen as an automaton
  with auxiliary store running on a stream of
  events provided by the event recognizer

aux. variables
10
PEDL for Java
MonScr ltspec_namegt / Export section /
export event ltegt export condition ltcgt /
Overhead reduction / timestamp
valueabstract deltaabstract
multithread / Monitored entities /
monobj ltvargt monmeth ltmethgt / Event and
condition/ event ltegt ... condition ltcgt
... End

• The language maps the low-level state information
  of the system to high-level events and conditions
  used in describing the requirements.
• Provides primitives to refer to
• primitive variables
• beginnings/endings of methods
• Primitive conditions are constructed from
• boolean-valued expressions over the monitored
  variables
• Ex condition IC (position 100)
• Primitive events are constructed from
• update(x)
• startM(f)/endM(f)
• Ex event raiseGate startM(Gate.gu())

11
Java-MaC Prototype
Requirement Specification
Program (Java source code)
Requirements (MEDL)
Monitoring Script (PEDL)
Program (Java byte code)
MEDL Compiler
PEDL Compiler
Instrumentation Information
Filter Generator (JTREK)
Static Phase
Instrumented Target program
Filter
(Compiled PEDL) Event Recognizer
(Compiled MEDL) Run-time Checker
Dynamic Phase
12
PEDL for Java (cont.)

• Events can have two attributes - time and value
• time(e) gives the time of the last occurrence of
  event e
• used for expressing temporal properties
• value(e,i) gives the ith value in the tuple of
  values of e
• value of update(var) a tuple containing the
  current value of var
• value of startM(f) a tuple containing
  parameters of the method f
• value of endM(f) a tuple containing parameters
  and a return value of the method f

13
Specifications for Stock Clients
MonScr StockClient export event startPgm,
periodStart, conFail,
queryResend, oldDataUsed monmeth void
Client.main(String) monmeth void
Client.run() monmeth void Client.failConnecti
on(ConnectTry) monmeth Object
Client.retryGetData(int) monmeth Object
Client.processOldData() event startPgm
startM(Client.main(String)) event
periodStart startM(Client.run()) event
conFail startM(Client.failConnection(ConnectTry)
) event queryResend startM(Client.retryGetD
ata(int)) event oldDataUsed
startM(Client.processOldData()) End
ReqSpec StockClient import event startPgm,
periodStart, conFail,
queryResend, oldDataUsed var long
periodTime var long lastPeriodStart var
int numRetried var int numConFail
alarm violatedPeriod end((perioidTime gt 900)
(periodTime lt 1100)) alarm
wrongFT oldDataUsed when (
(numRetries lt 4) (numConFail lt 3))
startPgm -gt periodTime 1000
lastPeriodStart time(startPgm) -1000
numRetries 0 numConFail 0
periodStart -gt numREtries 0
numConFail 0 periodTime
time(periodStart)-lastPeriodStart
lastPeriodStart time(periodStart) ... End
14
Extensions for Embedded Systems

• MaC based on hybrid system models
• Distributed MaC
• Steering-based adaptation
• Model-based profiling

15
Monitoring of hybrid systems

• Embedded systems often contain continuously
  evolving as well as discrete components
• Monitoring of continuous entities introduces
  additional challenges
• Monitoring of discrete systems is event-driven,
  which is not enough for continuous changes
• Continuous monitoring, in practice, involves
  sampling
• Sampling rates for each continuous entity can be
  obtained from a hybrid model of the embedded
  system
• Deriving properties of embedded systems
• Use a hybrid model as an oracle, driven by the
  observations

16
Distributed MaC

• Run-time monitoring and checking of distributed
  systems
• Local monitors and checkers
• How to decompose a global property into locally
  checkable properties, i.e.,
• Given the set of nodes, N1,, Nm and a global
  property Pg, find locally checkable properties,
  P1 ,, Pm such that if Ni detects the violation
  of Pi, then Pg is violated.
• Tradeoff between amount and frequency of
  communication and timely detection of property
  violation

17
Steering-based Adaptation
18
Steering-based adaptation

• Steering provides feedback from the monitor to
  the system
• When can a system be effectively steered?
• the system is designed for run-time adjustments
• e.g., Simplex architecture
• the system naturally offers a degree of tolerance
  
• e.g., routing protocols flush buffers when
  performance decreases
• Need timing guarantee over feedback loop
• Real-time Java

19
Model-based Profiling

• Collect statistics during monitored executions
• Over multiple runs, such statistics will provide
  dependability and performance measures
• Possible uses
• Collect coverage
• Successful runs serve as tests
• Enhance synthetic test suites
• Infer dynamic interfaces (e.g. interface
  automata)
• Record input and output patterns of successful
  executions
• Helps in reusing the component in different
  contexts

20
Summary

• The MaC architecture provides a lightweight
  formal methodology for assuring of the correct
  execution of a target program at run-time
• Rigorous analysis, Flexible, Automatic
• Identified several possible extensions to MaC
• Reduction of monitoring overhead
• Port the MaC architecture to platforms other than
  Java
• http//www.cis.upenn.edu/rtg/mac

21
Q A