FPMS TCTS

1
FPMS - TCTS

• Circuit Theory And Signal Processing Lab
• Avenue Copernic 1, B-7000 Mons
• http//www.tcts.fpms.ac.be

2

• TCTS as a teaching unit
• 4 academics
• 2 technicians
• courses
• Circuit Theory
• Switching Theory
• (digital) Signal Processing
• Filtering Theory
• Information Theory
• Biomedical Signal Processing Imaging
• Computers Architectures
• Automated Manufacturing Systems
• Embedded Systems

3

• TCTS as a teaching unit
• 4 academics
• 2 technicians
• courses
• Circuit Theory
• Switching Theory
• (digital) Signal Processing
• Filtering Theory
• Information Theory
• Biomedical Signal Processing Imaging
• Computers Architectures
• Automated Manufacturing Systems
• Embedded Systems

Important contribution in 3  certificates 
(thematic modules)  Multimedia ,
 Bio-engineering   Embedded systems 
4

• TCTS as a research unit
• 25 researchers
• 20 projects

5

• TCTS partnership with MULTITEL asbl
• 15 admins
• 65 engineers

6

• TCTS research tracks
• Speech processing
• Speech Synthesis (TTS)
• Speech Recognition (ASR)
• Voice quality assessment
• Audio processing
• Image processing
• Biomedical data processing
• Hardware for signal processing

7

• TCTS research ongoing projects
• CINEMA (RW/INIATIVES)Communications immersives
  pour la narration étendue par les mélanges
  audiovisuels
• COST277 (EU/COST)Non linear speech processing
• COST278 (EU/COST)Spoken language interaction in
  telecommunication
• DOMINI (RW/FIRST-EUROPE)Multimodal access and
  managment of digital medical records
• DREAMS (RW/WALEO-BIOMEDICAL)Development of
  automatic method for polysomnogram (sleep
  recording) diagnosis
• F3M (RW/RESEAU-I)Multimodal solutions for using
  wearable computers in wireless environments
• GRA2PHO (RW/FIRST-EUROPE)Methods for multiple
  grapheme-to-phoneme transcription (required for
  ASR)
• ICC (RW/WIST-IT)Multimodal interface for
  automatic interpretation of natural sketch and
  plan
• iMED (RW/FIRST-EUROPE)Detection of embolism from
  tomography-based images

8

• TCTS research ongoing projects (contd)
• MAIS (RW/WIST-IT)Low-cost, low-consumption smart
  card for distant reading (windshield embedded)
• MERCATOR (RW/WALEO-BIOMEDICAL)Image
  coregistration for video-enhanced surgery
• MOUSTIC (RW/FIRST-SPINOFF)New frameworks for
  broadcasting traffic information (radio and GSM
  networks)
• MUSIQUE (RW/FIRST-SPINOFF)Fault detection in
  production process by multiple signal analysis
• PIST (RW/WIST-IT)Positionning in transport based
  on heterogeneous sensors
• PSOC-ASR (RW/FIRST-EUROPE)Speech recognition on
  Programmable-System-On-a-Chip (dedicated
  hardware)
• STOP (RW/EUREKA)Speech dynamics and voice
  quality analysis for improved speech synthesis
• SIMILAR (EU/FP6-NoE)Detection of embolism from
  tomography-based images
• SYPOLE (RW/INITIATIVES)Portable device for
  text/image recognition and speech synthesis to
  blind people

9

• TCTS resources for IRMA project
• Long research record in ASR
• COST249 (EU/COST, 1994-2000)Continuous speech
  recognition over the telephone network
• COST278 (EU/COST, 2001- )Spoken language
  interaction in telecommunication
• HIRMANET (EU/FP6-NoE, 1993-1995)Hidden Markov
  models and neural networks for ASR
• SPRACH (EU/ESPRIT, 1995-1998) Improvement of HMM
  / ANN speech recognition algorithms
• THISL (EU/ESPRIT, 1997-2000)Thematic indexing of
  spoken language
• RESPITE (EU/ESPRIT, 1998-2001)Recognition of
  speech by partial information techniques
• DIVINES (EU/FP6-STREP, 2004-2007)Diagnostic and
  intrinsic variabilities in natural speech

collaboration with MULTITEL
10

• TCTS resources for IRMA project
• Long research record in ASR
• COST249 (EU/COST, 1994-2000)Continuous speech
  recognition over the telephone network
• COST278 (EU/COST, 2001- )Spoken language
  interaction in telecommunication
• HIRMANET (EU/FP6-NoE, 1993-1995)Hidden Markov
  models and neural networks for ASR
• SPRACH (EU/ESPRIT, 1995-1998) Improvement of HMM
  / ANN speech recognition algorithms
• THISL (EU/ESPRIT, 1997-2000)Thematic indexing of
  spoken language
• RESPITE (EU/ESPRIT, 1998-2001)Recognition of
  speech by partial information techniques
• DIVINES (EU/FP6-STREP, 2004-2007)Diagnostic and
  intrinsic variabilities in natural speech

strong expertise in speaker-independent robust
ASR for small to medium vocabulary
collaboration with MULTITEL
11

• TCTS resources for IRMA project
• Past research in speaker recognition
• ARTHUR (RW, 2000-2003)Telecommunications for
  emergency services
• COST250 (EU/COST, 1995-2000)Speaker recognition
  in telephony
• NIST evaluation (USA/DARPA, 2000)Speaker
  recognition track

collaboration with MULTITEL
12

• TCTS resources for IRMA project
• Past research in speaker recognition
• ARTHUR (RW, 2000-2003)Telecommunications for
  emergency services
• COST250 (EU/COST, 1995-2000)Speaker recognition
  in telephony
• NIST evaluation (USA/DARPA, 2000)Speaker
  recognition track

expertise in speaker identification and speaker
verification
collaboration with MULTITEL
13

• TCTS resources for IRMA project
• Long development experience
• MODIVOC (RW/INITIATIVES, 2002-2004) Distributed
  speech recognition
• ICC, DOMINI, PSOC-ASR, F3M, SIMILAR
• ALLADIN (EU/FP6-STREP, 2004-2007)Natural
  language based decision support in
  neuro-rehabilitation
• MAJORCAL (RW/EUREKA, 2003-2005)Customer
  relationship management call center
• LASCOT (RW/EUREKA, 2003-2005)Large scale
  collaborative decision support technology
• WEARIT_at_WORK (EU/FP6-IP, 2004-2009)Empowering
  the mobile worker by wearable computing

collaboration with MULTITEL
14

• TCTS resources for IRMA project
• Long development experience
• MODIVOC (RW/INITIATIVES, 2002-2004) Distributed
  speech recognition
• ICC, DOMINI, PSOC-ASR, F3M, SIMILAR
• ALLADIN (EU/FP6-STREP, 2004-2007)Natural
  language based decision support in
  neuro-rehabilitation
• MAJORCAL (RW/EUREKA, 2003-2005)Customer
  relationship management call center
• LASCOT (RW/EUREKA, 2003-2005)Large scale
  collaborative decision support technology
• WEARIT_at_WORK (EU/FP6-IP, 2004-2009)Empowering
  the mobile worker by wearable computing

strong expertise in software developpement
collaboration with MULTITEL
15

• TCTS contributions in IRMA project
• Automatic speech recognition for
• Indexation of spoken material
• Capture of spoken input in retrieval interface
• Speech-based biometrics
• Speaker-based document segmentation
• User verification

16

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval an important problem
• ConferencesICASSP2004Topics in Speech
  Understanding Systems
• ICSLP2004Spoken and Natural Language
  Understanding (I and II)Spoken Language
  Identification, Translation and Retrieval (I and
  II)
• LREC2004Spoken Indexation and Retrieval
• ICASSP2005Spoken Language Understanding and
  Dialog
• INTERSPEECH2005Spoken Language Extraction and
  Retrieval (I and II)Spoken Language
  Understanding (I and II)

17

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval an important problem
• EvaluationsNIST (DARPA)Broadcast news
  transcription and indexation in English
• ESTER (AFGP)Broadcast news transcription and
  indexation in French
• ProjectsTHISL www.dcs.shef.ac.uk/spandh/projects/
  thisl/Thematic indexing of spoken language
• AMI www.amiproject.orgMulti-modal interaction
  in the contexte of meetings
• M4 www.m4project.org Structuring, browsing and
  querying of an archive of automatically analysed
  meetings

18

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval an important problem
• PRODUCTSAURIX Speech Recognition
  Systemwww.aurix.com/product/index.htm
• NEXIDIA Speech Recognition Systemwww.nexidia.com
  /product.asp
• SCANSOFT Dragon AudioMiningwww.scansoft.com/audi
  omining
• BBN AudioIndexerwww.bbn.com/For_Government_Custo
  mers/Data_Indexing_and_Mining/Audio_Indexer.htmlA
  RTSUMwww.artsum.netAutonomy -
  Viragewww.blinkx.tv

19

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval an important problem
• Open Software for LVCSRSphinxcmusphinx.sourcefor
  ge.net/html/cmusphinx.php
• HTKhtk.eng.cam.ac.uk
• ISIPwww.cavs.msstate.edu/hse/ies/projects/speech
  /software/
• STRUTwww.tcts.fpms.ac.be/asr/

20

• TCTS contributions in IRMA project
• STRUT (Speech Training and Recognition
  Toolkit)www.tcts.fpms.ac.be/asr/

• - plate-forme logicielle pour le prototypage
  rapide de systèmes de reconnaissance
  automatique de parole
• - flexibilité de la modélisation acoustique
  (dépendant ou non du locuteur, modèles de mots
  ou de phonèmes, etc)
• - flexibilité de la modélisation grammaticale
  (parole continue ou mots isolés, lexique
  jusquà 10000 mots, grammaire détats finis, etc)
  
• - analyses acoustiques robustes (bruit,
  réverbération, microphone, etc)
• - détection de parole, mesures de confiance, etc
• - disponibilité dans la plupart des langues
  européennes et laméricain
• développement rapide de nouvelles langues /
  lexiques / grammaires
• EAR STRUT-based SDK

21

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval where do we operate ?

Lexicon
Grammar
Feature Extraction
Acoustic Model
Decoder
Words(or N-best, lattices)
Acoustic vectors
Local classification(eg, phonemes)
Speech signal
I cost ltltltR cost gtgtgt
I cost ltltR cost gtgt
I cost gtgtR cost ltlt
I cost gtgtgtR cost ltltlt
Out-Of-Vocbulary (OOV) words ?
22

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval influence of relaxing ?

Actual content
Retrieval
Indexing
Missed !
23

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval influence of relaxing ?

Indexing
Retrieval
Indexing with relaxing method
Caught !
24

• TCTS contributions in IRMA project
• ASR for spoken document indexation and
  retrieval influence of relaxing ?

Indexing
Retrieval
Retrieval with query expansion
Caught !
25

• TCTS contributions in IRMA project
• ASR for speech input in multimodal interface
  similar to SIMILAR
• Speech source estimation for
• Speech-based biometrics (eg, speaker
  verification)
• Speech/Non-speech segmentation
• Speaker segmentation (eg, id, gender, age)