NeurOnline Studio: An Informative Tutorial

1
NeurOn-line StudioAn Informative Tutorial

• Nicholas Stepenosky
• Patrick Giordano

2
Outline

• Introduction
• Definitions
• Importing and Managing Data
• Visualization and Labeling
• Preprocessing
• Backpropagation Networks
• Radial Basis Function Networks
• Example
• Discussion/Conclusion

3
A snapshot into the Mind
4
Introduction

• NeurOn-Line (NOL) Studio is a graphical,
  object-oriented software product for building
  neural network applications.
• Using NeurOn-Line Studio, you can model dynamic,
  nonlinear phenomena that are difficult to
  describe by analytical models, using historical
  data stored in databases, process data
  historians, or text files.
• Typical applications include quality assurance,
  sensor validation, diagnosis, and process
  modeling.

5
Introduction

• You dont have to be an expert in neural networks
  or statistics to use NOL.
• You simply load your data, graphically select the
  portions youd like to use for model development,
  and let NOL Studio do the rest.
• Advanced users can make use of additional,
  powerful options for data analysis and custom
  model building.

6
Introduction

• Designed to handle large, messy data sets
• Typically produced from industrial operations.
• Many defects, such as missing and bad data,
  incompatible formats, and combine together
  different production runs into large files.
• Limited only by memory capacity,
• NOL Studio can handle data sets consisting of
  100,000 data points with 100variables.

7
Introduction

• Two types of models are supported predictive
  models and optimization models.
• Predictive models are used for creating virtual
  analyzers (software sensors), fault detection,
  sensor validation, and forecasting.
• Optimization models are used for determining the
  best operational settings for a process, to
  minimize an objective function you define.

8
Introduction

• Predictive models can be of the five types
• Predictive model
• Backpropagation Net model
• Autoassociative Net model
• Radial Basis Function Net model
• Rho Net model
• The last four models are identical to the models
  in classic NeurOn-Line.

9
Introduction

• Once a tentative model is built, you can use a
  variety of powerful analysis tools to test and
  validate the fit.
• For example, applying the model to new data, that
  was not used in the training process. Or, you can
  plot response surfaces to analyze the
  input-output relationships learned by the model.

10
Introduction

• Some of the key features of NeurOn-Line Studio
• Data Importing
• Accepts a wide variety of ASCII text file formats
• Able to combine multiple files covering different
  data ranges
• Search-and-replace capability
• No explicit size limitation on data sets
• Data Preprocessing
• Interactive graphical data labeling
• User-defined label categories
• Projection plots for outlier identification
• User-defined mathematical formulas (transforms)

11
Introduction

• Modeling
• Steady-state and dynamic models
• Automatic selection of relevant inputs, time
  delays, and feedbacks
• Automatic determination of network architecture
  that optimizes future prediction accuracy
• Efficient training
• Validation and Simulation
• Predicted versus actual plots
• Response surface plots
• Input sensitivity analysis
• Optimization

12
Definitions

• Data series
• A data series represents a set of measurements on
  certain variables. Each row of a data series
  represents measurements taken at a certain time.
  Each row has a unique time stamp. Data series can
  be combined by appending or time-merging.
  Appending adds more rows to a data series.
  Time-merging creates a new data series by placing
  the variables in two or more data series under
  the same set of time stamps.
• File Formats
• A file format is a description of the layout of
  an ASCII text file, used to import data into NOL
  Studio. Whenever you import a text file, a file
  format is automatically created for that file.

13
Definitions

• Labels
• Labels are used to mark the raw data, to indicate
  regions of special interest. Examples of label
  categories are outlier, transient, steady state,
  product transition, or cut.
• Preprocessors
• A preprocessor defines the pretreatment of data,
  before it enters the neural network model. Each
  preprocessor contains two parts a filter and an
  optional list of formulas. The filter defines
  which parts of the raw data you want to use in
  training a model. The filter is based on the
  labels you apply to the raw data. A simple filter
  might be all data excluding data labeled cut.
  The formula list allows you to perform
  mathematical transformations on the filtered
  data, to fill in missing values, smooth noisy
  signals, calculate ratios, etc.

14
Definitions

• Predictive Models
• Models are generated by the training process. You
  designate input and output variables, optional
  time delays, and other training parameters. You
  can generate as many models as you wish, and
  compare their performance. Predictive models are
  used for creating virtual analyzers (software
  sensors), fault detection, sensor validation, and
  forecasting.
• Backpropagation Nets
• BPNs are generated by the training process. You
  designate input and output variables, model
  architecture, and training parameters. You can
  generate as many models as you want and compare
  their performance. BPNs are useful for creating
  virtual analyzers (software sensors), fault
  detection, sensor validation, and forecasting

15
Definitions

• Radial Basis Function Nets
• RBFNs are generated by the training process. You
  designate input and output variables, model
  architecture, and training parameters. You can
  generate as many models as you want and compare
  their performance. RBFNs are useful for fault
  detection, pattern recognition, and forecasting.
• Simulations
• Simulations are used to show the response of a
  model to user-defined inputs. Like other objects,
  simulations are automatically stored as part of
  your project, to allow you to return to
  scenarios, or apply the same scenarios to
  different models.

16
Importing Data

• One of the biggest obstacles in modeling
  processes with neural nets is the ability to
  manage large sets of data easily and efficiently
• Two types of data series that NeurOn-line will
  use
• Time-Based
• Row-Based

17
Importing Data

• We will be interested in Time-Based data since
  everything will be monitored real-time and be
  sent with time stamps
• Each row represents observations or measurements
  at a certain time. The time for each row is
  referred to as the timestamp for the row.

18
Importing Data

• There are two types of file formats that are
  recognized
• An ASCII file format, which allows you to import
  or append an ASCII file that follows a standard
  formatting convention.
• A BINARY file format, which is used for saving
  and loading data after it was already imported
  into NOL Studio.
• Side note
• Files in the predefined ASCII format have the .ds
  extension. Files in the predefined BINARY format
  have the .bds extension.

19
Importing Data
Data Series stored as a text file, with Time and
Var established by the user while importing the
data

• The import data series provides the user with a
  prompting user interface to walk through the
  necessary tasks of importing new data
• This is where the user will specify where the
  time series is vs.. the collected data. It will
  also allow the user to view there selections

20
Importing Data

• Since G2 is our primary focus for system wide
  health management it would be important that
  NeurOn-line would be able to interface and
  collect data from it
• G2 Gateway supports two-way communication between
  dynamic external processes and G2 applications.
  Through a G2 Gateway bridge to an external
  system, you can quickly obtain real-time data
  that a G2 application needs to make intelligent
  control decisions in a time-critical processing
  environment.

21
Importing Data

• G2Gateway bridges enable G2 KBs to communicate
  with a wide variety of externalsystem, such as
• Database management systems (DBMSs)
• Programmable logic controllers (PLCs)
• Supervisory control and data-acquisition
  (SCADA) systems
• Distributed control systems (DCSs)
• C/C programs, Non-G2 operator consoles or
  displays
• External simulation software

22
Importing Data

• NOL Studio can also communicate across
  Intranet/Internet if it is based on the TCP/IP
  protocol
• Simple wizard allows quick access to data over
  the Intranet/Internet

23
Importing Data Viewing the Data Series

• Once you have imported data successfully into NOL
  Studio, you can begin to examine the properties
  of your data. You do this by accessing the
  properties table for the data series, and then
  drilling down to view individual variables.

24
Exporting Data

• NOL Studio provides a facility to export raw data
  series as well as processed data into data files.
  You can save a data series into a BINARY file or
  an ASCII file, the two predefined formats
  supported by the NOL Studio.

25
Importing Data

• Among many more features you can append new data
  to existing raw data files as well as
• Remove unneeded data
• Rename series
• Saving incoming data in different formats
• Append data to existing data file on the fly

26
Visualization and Labeling

• NOL Studio allows you visualize data in many
  different views.
• Each view presents a different aspect of your
  data, and helps you gain additional insight into
  the underlying process.
• Helps locate anomalies so you can remove them
  before training a model.
• Spreadsheet view
• This view allows you to view a data series in a
  tabular, column/row format.
• Line chart view
• This view allows you to plot one or more
  variables versus time or row index.

27
Visualization and Labeling

• X-Y scatter chart view
• This view allows you to plot one variable versus
  another variable, with time implicit. The number
  of rows of both variables viewed must be of equal
  length.
• Projection chart view
• This view depicts a projection of selected
  variables from a single data series, using
  Principal Component Analysis (PCA). Projection
  plots are powerful ways to examine the
  multivariate distribution of your data.
• Histogram view
• This view depicts a bar chart showing the
  distribution of a specified variable.

28
Visualization and Labeling

• You access any of these views either from the NOL
  Studio View menu or the toolbar.
• All of these views are read-only in that you
  cannot modify the data contained within the view.
  
• The views are also interactive, allowing you to
  select and label data using mouse gestures.

29
Visualization and Labeling

• Why do we want to label the data
• At this point, you have loaded data into NOL
  Studio, and you have used various graphical views
  to examine the data. During this process, you may
  have noticed some flaws in your data outliers,
  shutdown periods, operational transients,
  changeovers, and the like. It is necessary to cut
  out the bad or inapplicable portions of the data,
  to get a clean data set suitable for training.

30
Visualization and Labeling

• There are different ways depending on how you
  wish to view the data that you can label it
• As a spreadsheet
• ?

31
Visualization and Labeling

• or graphically

32
Visualization and Labeling
As well as in a X-Y scatter plot
Its as simple as click-and-go Just highlight the
data that you would like to label and then
determine what value for the label and your done.
This way you can train your network with exactly
the data you need
33
Preprocessing

• A preprocessor is a tool that processes a subset
  of the raw data used to build models.
• The Create New Preprocessor wizard guides you
  through the necessary steps.
• Choose Object gt New gt Preprocessor
• Name the preprocessor
• Select the data
• Select variables
• Queries labels to include and exclude
• Select the new preprocessor to program formulas
  needed
• Range of mathematical function from simple
  function to complex neural network operations.

34
Preprocessing

• Select the new preprocessor to program formulas
  needed
• Range of mathematical function from simple
  function to complex neural network operations.
• Data is labeled as a new variable after being
  preprocessed.
• To define a formula, you specify the output
  variables, a function, and input arguments.
  Functions use prefix notation, so, if you want to
  multiply two variables, you express this as
• output Multiply(input1, input2)
• where output, and input1 and input2 are names
  of variables.
• Functions can be nested, so an input argument can
  be another function with its own input arguments.
  An example of a nested function is
• output Divide(Multiply(input1, input2), 2.0)

35
Backpropagation Networks

• The Backpropagation Net, or BPN, is another type
  of predictive model - a feed-forward, layered
  network.
• Each node in a layer is connected to all other
  nodes in the layer before it and in the layer
  after it.
• Similar to predictive models, you can start
  building BPN models after importing data,
  labeling and filtering data by using a
  preprocessor, and creating formulas that
  condition the data in the same preprocessor.
• You need to manually select the architecture of a
  Backpropagation Network model before training.

36
Backpropagation Networks

• One NOL Studio project can contain any number of
  BPN models.
• This allows you to train models with different
  architectures for the same problem, then to
  compare models, using the validation tools until
  you are completely satisfied with the performance
  of your model or models.
• You can then save your best model or models.
• To create a BPN model, you follow the steps in
  the modeling wizard.
• Choose Object gt New gt Backpropagation Net

37
Backpropagation Networks

• The wizard guides you through these steps to
  create a model
• Name the model.
• Select whether to use old model parameters.
• Specify the preprocessor for the model.
• Specify the output data series to be used in the
  model.
• Classify the variables as input, output, or
  unused.
• Specify time delays, if any, for the model
  inputs,
• Automatically select inputs and delays.
• Specify the model architecture.

38
Backpropagation Networks

• Control over parameters just like an MLP
• Number of layers, number of nodes, transforms
  (linear sigmoidal), training methods, iterations
• Can terminate or continue training as one sees
  fit
• The error is displayed in a graph
• General properties show information about the
  network
• Model ratings of Good, OK, and Need
  Improvement
• Statistics - show how well the model fits the
  training data set
• Model structure
• To deploy a backpropagation network, you should
  save the weights of the backpropagation network
  to a text file

39
Radial Basis Functions Networks

• The Radial Basis Function Network, or RBFN, is a
  3-layer, feed-forward network, whose middle layer
  uses a multi-variate Gaussian function. It is
  especially useful for classification problems.
  The RBFN is best for choosing which class out of
  many classes an item belongs to.

40
Radial Basis Functions Networks

• Once the input output structure is done next is
  to specify the internal architecture of the RBFN
  model. A RBFN model contains exactly three
  layers. The number of nodes in the first layer is
  the same as the number of input variables.
• The number of nodes of the last layer is the
  same as the number of output variables. The
  middle or hidden layer can have any number of
  nodes.

41
Radial Basis Functions Networks Training the
network

• If you are fitting the network to a function,
  choose Regular K-Means Clustering.
• If you are solving a classification problem,
  choose Class-Separate K-Means Clustering.

42
Radial Basis Functions Networks Performing
operations on the model

• In this dialog, you can show the prediction of
  any output versus the training target values of
  that output, as a line chart (shown to the
  right), or as an x-y (scatter) chart.
• You can also save the model so that it can be
  implemented later in simulation

43
Radial Basis Functions Networks Performing
simulations on the model

• Simulations in NOL Studio are another way to
  validate a model. Simulations allow you to
  specify some data to input to a model, inspect
  the output generated from that data, and then
  save the results.
• The simulate window becomes available after the
  network has been trained
• Then all that is needed is for the testing values
  to be presented to the network and then you can
  review the simulation results

44
Example

• Lets go to the Lab.
• We will be looking at a prepared set of time
  dependent samples so that we can go through the
  software.
• A quick Reference tutorial will be provided that
  will aid in working through the problem as well
  as act as a guide in the future for setting up
  the basic procedures inside NeurOn-line

45
Discussion/Conclusion

• NOL Studio is a powerful tool for many data
  processes from visualization to entire neural
  networks.
• NOL can be used as an alternative to MATLAB for
  neural network and predictive applications.
• The wizards are an excellent touch that guides
  the user every step of the way.
• You dont need to be an expert to use this
  application.

46
Assignment

• Show us that you went through the tutorial and
  understood the fundamentals
• Print the statistics of the raw data along with
  its plot
• Export the simulation output and input as binary
  files