Train Monitoring System Version 3

1
Machine Vision Analysis of the Energy Efficiency
of Intermodal Freight Trains Sibley Site Update
Chris Barkan and Narendra Ahuja Co-Principal
InvestigatorsJohn M. Hart - Senior Research
Engineer, Riley Edwards - Lecturer Graduate
Students Tristan Rickett, Avinash
Kumar Undergraduate Students Ruben Zhao,
Sujay Bhobe, Chun Yang, Suchithra
Gopalakrishnan, Phil Hyma, Mike Wnek, Beckman
Institute for Advanced Science and
Technology Railroad Engineering Program,
University of Illinois
2
Wayside Machine Vision System
Image Acquisition System
Data Analysis System
Machine Vision Algorithms
To BNSF
IM Train
3
Sibley Site Equipment Layout
4
Machine Vision Site Equipment
CCD Video Camera
Halogen Lighting

• Camera Enclosure on Tower to House CCD Camera
• Bungalow Containing Computers and Train Detection
  Circuitry
• 30ft Tower for Set of Halogen Lights and Wireless
  Antennas

Machine Vision Processing Equipment
5
Train Detection System
Presence Detectors
Wheel Detectors
Loop Detectors
Daylight Sensor
Programmable Logic Controller
Train Status Monitor
Artificial Lighting
Machine Vision Computer
6
Machine Vision Site Equipment

• 19" Equipment Rack In Bungalow

CCD Video Camera
Machine Vision Computer
Halogen Lighting
Machine Vision Processing Equipment
Train Detection Sensors
Train Status Monitor
7
Site Activation Scenario

• System in Wait State and Continuously Monitoring
  Train Detectors
• Train Approaches Machine Vision Site (West/East
  Bound)
• Locomotive Triggers (East/West) Presence Detector
• Lights are Turned On If Photocell Does Not Detect
  Daylight
• Camera Turns On and Focuses Region of Interest on
  Target
• Exposure Adjustment is Made Based on Target Only
• Camera Region of Interest is Returned to Entire
  Scene
• Locomotive Triggers (East/West) Wheel Detector
• Video Recording is Started to Capture Background
  Appearance
• Train Passes by Camera and is Recorded Against
  Background
• Video is Taken at 30f/s and Buffered to Memory
• Video Recording is Ended and Video Frames are
  Stored
• If Lighting was Used, Lights are Turned Off
• System Returns to Wait State for Next Train
  Arrival

8
Details of Activation Scenario

• System in Wait State Continuously Monitoring
  Train Detectors
• The Train Status Monitor (TSM) checks the state
  of the detectors
• Uses data acquisition board inside the pc
  connected to all detectors through the
  Programmable Logic Controller (PLC)
• West Presence Detector
• West Wheel Detector
• West Loop Detector
• East Loop Detector
• East Wheel Detector
• East Presence Detector
• Controlled by a custom program - DIControl
• Frequency Rate of monitoring is admustable
  (currently 1/10 sec)
• When not looking at detectors, control of
  processor is returned to OS

9
Details of Activation Scenario

• Train Approaches Machine Vision Site (West/East
  Bound)
• Locomotive Triggers (East/West) Presence Detector
• Presence detector pulse is received by the PLC
• Presence detector pulse is also captured by the
  Train Status Monitor

10
Details of Activation Scenario

• Lights are Turned On If Photocell Does Not Detect
  Daylight
• PLC enables the power to the lights
• If the photocell is detecting light, it inhibits
  the power signal to the lights
• Lights now staggered to more evenly distribute
  lighting (initial config)

11
Details of Activation Scenario

• Camera Turns On and Focuses Region of Interest on
  Target
• Camera is started by custom software
  pgrAperture
• Because video frames of the background are needed
  prior to the train,
• the camera must adjust exposure without the
  presence of the train
• The target is designed to reflect light similarly
  to the side of the train
• The camera view is then restricted the region of
  the target only

12
Details of Activation Scenario

• Exposure Adjustment is Made Based on Target Only
• The camera parameters are allowed to adjust to
  the lighting on the target,
• with the exception of the shutter speed
• The shutter speed is set to a value (2ms)
  determined experimentally
• Pevents image motion blur due to the moving
  train (normal camera below)
• Camera Region of Interest is Returned to Entire
  Scene
• Before train reaches wheel detector

13
Details of Activation Scenario

• Locomotive Triggers (East/West) Wheel Detector
• Wheel detector pulse is captured by the Train
  Status Monitor
• Wheel detector is placed 75ft from camera to
  start recording prior to train
• Video Recording is Started to Capture Background
  Appearance
• These frames are used to create a model of the
  background by the TMS

14
Details of Activation Scenario

• Train Passes by Camera and Is Recorded Against
  Background
• With exposure set by target, train should not
  appear dark even if background is bright

15
Details of Activation Scenario

• Video is Taken at 30f/s and Buffered to Memory
• To continuous capture 30f/s, frames are buffered
  before converting to video
• Video Recording is Ended and Video Frames are
  Stored
• Videos are stored in multiple 1Gbyte segments for
  OS requirements
• If Lighting was Used, Lights are Turned Off
• System Returns to Wait State for Next Train
  Arrival

16
Now Testing System Automation
Video Acquisition
Video Storage
Train Monitoring Sys
Gap Measurements
Load Identification
AEI Reader Data
Train Panorama
Train Scoring System
Mini-Umler Database
Train Score
Loading Evaluation
17
Demo In Computer Vision and Robotics Lab of
Duplicate Image Acquisition ComputerAdjusting
to Ambient Lighting Conditions and Recording Video
18
Camera Line of Site
Viewing Volume
Inter-modal Train
Camera
19
Train Monitoring System

• Input A video of an intermodal freight train
• Output Length of gaps between the load
• Improve aerodynamic efficiency of the train
• Large savings on fuel costs

20
Input Train Video
21
Challenge 1

• Varying outdoor imaging conditions

22
Challenge 2

• Different Types of Containers

23
Challenge 3

• Computations involved need to be fast to handle
  railroad traffic.
• 1 day has 20-30 trains on both sites
• 1 train is completely captured in approx 5000
  frames
• 1 frame is 640x480 pixels
• Need to process all frames

24
Method Step 1

• Estimate initial train velocity in pixel
  shifts/frame

Image shifts by v pixels
x v
x
25
Method Step 1
1. Select a square window and calculate
normalized cross correlation with the static
background C_background
x
26
Method Step 1
2. Select another window at location x v in the
previous frame
27
Method Step 1
3. Calculate Normalized Cross Correlation between
these two windows as C_previous
28
Method Step 1
4. Similarly calculate normalized cross
correlation between current frame and next frame
as C_next
29
Method Step 1
Calculate Foreground Cost (C_previous C_next
C_background)/4
30
Method Step 1

• Extract foreground region from a stripe at the
  center of each train frame

Background
Foreground
31
Method Step 1

• Repeat for consecutive frames

32
Method Step 2

• Juxtapose stripes from consecutive frames to
  generate panorama

33
Method Step 2

• Post process panorama to remove background near
  edges

34
Method Step 3

• Classify each container into 3 different types

Double Stacks of two different kinds
Single Stack
Trailer
35
Method Step 4

• Obtain gap lengths and histogram for analysis

36
Results

• Tested on 110 train videos with 3 different types
  of containers
• 573 Type 1 (Double Stack) containers
• 515 Type 2 (Trailers) containers
• 10 Type 3 (Single Stack)containers
• Gap detection is accurate to approx 1 ft error
• Confusion matrix for load type detection

Type 1
Type 2
Type 3
573 0 0
5 515 0
0 0 10
37
Data Analysis System

• Tristan Rickett

38
Outline

• Train Resistance
• Train Scoring System
• Description
• Inputs AEI, TMS, UMLER
• AEI
• Current Setup at LPC and Sibley
• Using Available AEI Data for Sibley Videos
• Data Transfer
• Matching TMS file with AEI data
• Future Work

39
Train Resistance

• Train Resistance considers the effects of inertia
  that tend to keep a body at rest and the effects
  of friction that cause it to lose momentum once
  moving
• The general equation for train resistance is the
  following R AW BV CV2
• A Journal Resistance
• B Flange Resistance
• C Aerodynamic Resistance

40
Sources of Aerodynamic Drag

• Gap lengths
• Varying heights
• Rough surface
• Drag area of the lead locomotive
• Lack of streamlining

41
Current practice in intermodal freight train
loading

• Slot Utilization is metric used to measure the
  percentage of the spaces (a.k.a. slots) on
  intermodal cars that are used for loads
• However, this metric does not account for the
  size of the slot and the size of the load

8 loads / 10 slots 80 Slot Utilization
10 loads / 10 slots 100 Slot Utilization
Y.C. Lai 2008
42
Slot Efficiency Methodology

• Slot Efficiency comparison of the difference
  between the actual and ideal loading
  configuration
• This metric is similar to slot utilization except
  that it also considers the energy efficiency of
  the load-slot combination

43
Train Scoring System (TSS)

• The purpose of the train scoring system is to
  evaluate an intermodal trains loading efficiency
  and provide an aerodynamic coefficient to
  estimate fuel consumption
• The results from the TSS can aid terminal
  managers in creating more fuel-efficient trains

44
Flow of TSS
45
TSS Inputs

• Mini-UMLER Database has the database with all the
  railcars and their equipment
• Gap-length files contain the trains loadings and
  the gap lengths
• AEI (Automatic Equipment Identification) data
  provides a list of the trains equipment and axle
  timestamps

46
Mini-UMLER Database

• The information contained in the database
  includes the following
• Car Initial (e.g. DTTX)
• Car Number (e.g. 749452)
• Cars Outside length in feet (e.g. 270 ft)
• Car Type (e.g. S)
• Car Attribute 1 (e.g. 1)
• Car Attribute 2 (e.g. 6)
• Car Attribute 3 (e.g. 2)

47
Progress Made

• Improved how the code produces the output
• It is now embeddable so that it can run from
  inside another program
• Formatted a newer UMLER database
• Integrated TSS with the proposed system automation

48
AEI Data Collection at LPC

• The TSS was originally programmed to use AEI that
  had axle timestamp values like the PRT AEI reader
  at LPC
• At Sibley, we have begun collecting videos since
  last December but the problem is that the AEI
  data does not have timestamps

49
Addressing Present AEI Data Acquisition

• If the hot-box data is available, it would be
  worth calculating our own timestamps using this
  available data
• With the new AEI reader for the Sibley site, it
  is recommended that it provides axle timestamps

50
Determination of Axle Timestamps

• Using kinematics equations and some assumptions,
  we can determine the timestamps.
• Using di vi x ti 0.5aiti2
• Distance, di, is provided in the AEI data
• Assume velocity is around 20 to 25 mph (or 29.3
  to 36.7 ft/sec)
• No acceleration

51
Determination of Axle Timestamps

• Having an acceleration of zero cancels out half
  of the equation allowing di / vi ti .
• Because axle timestamp values are cumulative, the
  final equation will be
• ti ti 1 di / vi

52
Using a Wheel Detector for Timestamps

• Use one wheel detector already installed at the
  site to measure axle timestamp values.
• System would be triggered by one of presence
  detectors
• The difficulty is finding a place in the
  automation where the AEI data can be combined

53
Matching the Scoring Data with CAD Data

• All videos and AEI are named according to the
  date and time of when they were captured
• With the date of the scored train, it can also be
  attached to computer-aided dispatching data so
  terminal managers can review the efficiency of
  their trains loaded at their yard

54
Matching Data
55
Future Work

• Use the Aerodynamic Subroutine Version 4
• This version is capable of inputting double
  stacks with differing top and bottom load lengths
• Evaluate intermodal trains with trailers
• Improve output file
• Integrate AEI data to the automated system
• Use latest UMLER database
• Request an AEI reader for the Sibley site that
  has the capability of measuring timestamps
• Add capability of sorting AEI and TMS data
• Only scores IM trains and not other train types
  like manifest or grain trains

56
Acknowledgements

• Special Thanks to
• BNSF
• Paul Gabler, Hank Lees, Josh McBain, Larry
  Milhon, Cory Pasta, and Mark Stehly
• LJN and Associates
• Leonard Nettles and Kevin Clarke

57
Interdisciplinary Team Members

• From previous presentation.

58
(No Transcript)