Frameworks

1
Frameworks

• IKAROS
• http//www.lucs.lu.se/Ikaros/
• http//www.ikaros-project.org/
• YARP
• http//yarp0.sourceforge.net/
• http//yarp0.sourceforge.net/specs/dox/
• Libraries
• Intel Open CV
• http//www.intel.com/technology/computing/opencv/i
  ndex.htm
• http//opencvlibrary.sourceforge.net/
• Ffmpeg
• http//ffmpeg.mplayerhq.hu

2
IKAROS

• An event-driven framework
• Each module has a tick method which is called
  once per input visual/audio data frame
• Modules use vectors of real numbers as inputs
  outputs
• Inputs can be taken from sources including visual
  or audio (e.g., Quicktime video files)
• Inputs and outputs are named by the modules when
  the modules are initialized
• Modules connected together in networks
• Network (XML format) files used to specify how
  modules connect together using input output
  names of modules
• Browser interface used for visualization

3
IKAROS examples

• In this section on Frameworks, the IKAROS modules
  are from IKAROS/Modules in
• http//www.cprince.com/PubRes/EpiRob06/examples.tg
  z

4
Example Network File

• lt?xml version"1.0"?gt
• lt!-- Simple example network file in IKAROS --gt
• ltnetworkgt
• ltmodulesgt
• ltmodule
• class "SimpleExample_Input"
• name "input"
• /gt
• lt/modulesgt
• ltconnectionsgt
• lt/connectionsgt
• ltviewsgt
• lt/viewsgt
• lt/networkgt

SimpleExample/SimpleExample.xml
5
Quick IKAROS How To

• IKAROS is run from the command line
• For the details of running IKAROS, incorporating
  a new module, and compiling the module, see
• http//asip.lucs.lu.se/IKAROS/Documentation/HowToW
  riteAModule/index.html

6
IKAROS Module Definitions

• Each module is a C class and typically has the
  following methods
• Module SimpleExample_InputCreate(char name,
  Parameter p)
• SimpleExample_InputSimpleExample_Input(char
  name, Parameter p)
• void SimpleExample_InputSetSizes()
• void SimpleExample_InputInit()
• SimpleExample_InputSimpleExample_Input()
• void SimpleExample_InputTick()

7
Simple ExampleInput Module

• Purpose
• Generates 2D matrix input data of particular
  width and height per tick
• Three output types Random, sequential, default
• XML and C code files
• SimpleExample/SimpleExample1.xml
• SimpleExample/SimpleExample_Input.cc
• SimpleExample/SimpleExample_Input.h
• Running code
• 1) In the SimpleExample directory, type from the
  Terminal
• IKAROS -W8000 SimpleExample1.xml
• 2) Then, run a compatible web browser (e.g.,
  Firefox), and type the URL
• http//127.0.0.18000 (then, click on View 0)

8
SimpleExample1.xml

• ltnetworkgt
• ltmodulesgt
• ltmodule
• class "SimpleExample_Input"
• name "input"
• width "100"
• height "100"
• output-type "sequential"
• /gt
• lt/modulesgt
• ltconnectionsgt
• lt/connectionsgt
• ltviewsgt
• lt!-- Due to a peculiarity of the way the
  IKAROS grayscale conversion
• works (see WebUI.cc), if there is no
  variation in the image,
• the grayscale display turns all black. My
  workaround for this is
• to just use the RGB type of output.

9
SimpleExample1.xmlBrowser Output
10
Simple ExampleProcessing Module

• Purpose
• Illustrate two modules one connected to the
  output of the other
• On each tick, SimpleExample_Processing copies
  input to output, except for border which is
  assigned pixel value 0 on the output
• XML files and C code
• IKAROS/SimpleExample/SimpleExample2.xml
• SimpleExample/SimpleExample_Processing.cc
• SimpleExample/SimpleExample_Processing.h
• Running example
• 1) In the SimpleExample directory, type from the
  Terminal
• IKAROS -W8000 SimpleExample2.xml
• 2) Then, run a compatible web browser (e.g.,
  Firefox), and type the URL
• http//127.0.0.18000

11
SimpleExample2.xml

• lt?xml version"1.0"?gt
• lt!-- Simple example network file in IKAROS --gt
• ltnetworkgt
• ltmodulesgt
• ltmodule
• class "SimpleExample_Input"
• name "input"
• width "100"
• height "100"
• output-type "sequential"
• /gt
• ltmodule
• class "SimpleExample_Processing"
• name "processing"
• border-size "5"
• /gt
• lt/modulesgt

12
SimpleExample2.xmlBrowser Output
13
IKAROS Audio/Video Files

• Quicktime
• Apple supported API and file format for audio and
  video files
• IKAROS Quicktime input and output modules
• InputQTMovie, InputQTAudio, OutputQTAudioVisual
• Pixels represented as real numbers
• That is, to have a module read image frames from
  another module, such as the InputQTMovie, the
  module reads those frames as matrixes of pixels
  represented as floats
• Range 0.0 to 1.0
• Audio samples are also represented as real
  numbers
• Range -1.0 to 1.0

Mac OS X only
14
IKAROS ExampleVisualExample

• Purpose
• Demonstrate reading from a Quicktime video file
• Draws a horizontal line in an output frame to
  demonstrate access to the image frames
• Running example
• 1) In the VisualExample directory, type from the
  Terminal
• IKAROS -W8000 VisualExample.xml
• 2) Then, run a compatible web browser (e.g.,
  Firefox), and type the URL
• http//127.0.0.18000

Mac OS X only
15
VisualExample
16
IKAROS ExampleAudioVisualExample

• Purpose
• Read visual frames and corresponding audio data
  from a video file
• Run the example
• 1) On the AudioVisualExample directory, type from
  the Terminal
• IKAROS -W8000 AudioVisualExample.xml
• 2) Then, run a compatible web browser (e.g.,
  Firefox), and type the URL
• http//127.0.0.18000

Mac OS X only
17
(No Transcript)
18
Yarp

• A framework that allows communication between
  separate processes, separate computers, and
  between devices
• Provides a name server to allow for port
  registration, to connect processes
• The framework installs on Windows, Mac OS and
  Linux

19
A small test of your installation

• Yarp provides a few command line programs for
  devices, the name server, and testing
• First, a yarp name server must be running
• In a separate command line window, type
• yarp server
• Then, in three separate windows, do the following

20
Window Action
1 yarp write /a
2 yarp read /b
3 yarp connect /write /read
You should now be able to type in the write
window and see the text in the read window
21
Examples

• In each of the following Yarp examples, a name
  server must be running
• In most of the examples, you also need to build
  the example program using the following commands
• cmake .
• make

22
Yarp Simple Text Example Yarp/TextExample

• Purpose
• This example provides C program code
  demonstrating the text send/receive just
  demonstrated with the yarp command
• Running example
• Start the sender and receiver, in separate
  command line terminals
• ./send --name /out
• ./receive --name /in
• Next, connect them together
• yarp connect /out /in
• You should now be able to type text in the send
  terminal, and it should be received in the
  receive terminal

23
Yarp Image ExampleYarp/SimpleExample

• Purpose
• C code example of a program that sends images
  on a port
• Images have a circle shape moving from left to
  right
• Running example
• First start send program
• ./send --name /out
• Then run an image viewer
• yayarpview --name /in (Mac)
• yarpview --name /in (Windows/Linux)
• Then connect send program to viewer
• yarp connect /out /in

24
Yarp Image ExampleYarp/SimpleExample2

• Purpose
• Same send program code as in the last example,
  plus a receive program that reads images from a
  port, and adds a horizontal red line, moving from
  top to bottom
• receive program also outputs to a new port
• ports are named ltnamegt/in and ltnamegt/out where
  ltnamegt is the base name you supply when starting
  the program
• Running example
• First start send receive program in their own
  terminals
• ./send --name /out
• ./receive --name /receive
• Then run image viewer
• yayarpview --name /view (Mac)
• yarpview --name /view (Windows/Linux)
• Then connect send to receive program, receive
  to viewer
• yarp connect /out /receive/in
• yarp connect /receive/in /view

25
Yarp Visual Example Yarp/VisualDisplay

• Purpose
• Reads from an AVI format video file using a yarp
  OpenCV framegrabber, and writes this to a port
  (/avi)
• Display those visual frames using a yarp viewer
• Running example
• First, use the script startavi to start reading
  the avi file
• You will have to edit the name of the file in the
  startavi script
• Then run image viewer
• yayarpview --name /view (Mac)
• yarpview --name /view (Windows/Linux)
• Then connect AVI output to viewer
• yarp connect /avi /view

26
Yarp Visual ExampleYarp/VisualExample

• Purpose
• Reads from an AVI format video file using a yarp
  OpenCV framegrabber, and writes this to a port
  (/avi)
• Same receive program code as in the 2nd to last
  example is used to read that data
• ports are named ltnamegt/in and ltnamegt/out where
  ltnamegt is the base name you supply when starting
  the program
• Running example
• First, use the script startavi to start reading
  the avi file
• You will have to edit the name of the file in the
  startavi script
• Then, start the receive program in its own
  terminal
• ./receive --name /receive
• Then run image viewer
• yayarpview --name /view (Mac)
• yarpview --name /view (Windows/Linux)
• Then connect AVI output to receive program,
  receive to viewer
• yarp connect /avi /receive/in
• yarp connect /receive/in /view