Lehlohonolo Mohasi

1
A bilingual corpus-based TTS system as a foreign
language learning tool

• Lehlohonolo Mohasi
• Supervisors Prof. Mqhele E. Dlodlo
• Prof. Mbulungeni Madiba
• Co-Supervisor Prof. Daniel Mashao

2
Outline

• Current Situation
• Problem
• Solution
• Evaluation
• Conclusion
• Future Work

2
3
Current Situation
I dont think so. Andiphilile neze.
Ana u ea sekolong tsatsing lee?
3
4
Current Situation

• Language Acquisition
• How babies learn to talk understand language
• Exposure to spoken language
• Reading writing learnt through instruction
• Second/Foreign Language Learning
• Not easily acquired
• Need voice exposure as well instruction
• Different instructional methods

4
5
Problem

• Not enough voice exposure for L2 or foreign
  language learners.
• There might be exposure but there is also a need
  for ability to read and write for various
  reasons.
• Use of correct grammar pronunciation in written
  and spoken languages.

5
6
Problem Research Question

• How can technology be used as an effective
  second/foreign language learning tool?

7
Solution

• Human Language Technologies/Information
  Communication Technology
• Proposed technology -gt Text-to-speech (TTS) as a
  second/foreign language learning tool
• What is TTS?
• Automatic conversion of written text into speech.

7
8
Solution - TTS
DSP Concatenation-based Synthesis Waveform
Generation

• NLP
• Text Analysis
• LTS Rules
• Prosody Generation

Speech
Text
8
9
Solution - TTS

• TTS Applications
• Telecommunications
• Visually-impaired
• Deaf and Vocally Handicapped
• Educational Applications
• What value is TTS?
• Easy voice access mode to all
• Useful for the blind and illiterate
• Easy way of improving language listening
  speaking skills
• Helps with correct pronunciation

9
10
Solution Language TTS

• So how do we make sense of text-to-speech
  technology implementation in foreign language
  learning?

10
11
Solution Language TTS
12
Solution - Methodology

• Scope
• Use of 3 SA official languages English, Sesotho
  isiXhosa.
• Multiconcord Mconsal
• Festival Speech Synthesis Engine
• Parallel corpora selection from 1994 to date
• 1 million words

13
Solution - Methodology

• Phase 1 Creating corpora
• Creating a parallel corpora database
• Tools
• BITS (Bilingual Internet Text Search)
• Strand
• Scanner
• Multiconcord
• MinMark

14
Solution - Methodology

• Phase 2 TTS implementation
• TTS database taken from parallel corpora
• Limited domain synthesis
• Tools
• Festival Speech Synthesis Engine
• Phase 3 Web interface
• Integration of Phases 1 2
• Tools
• PHP
• mySQL

15
Evaluation

• Acoustic measurements voice talent selection
• Fundamental frequency, F0
• RMS energy for voiced and unvoiced speech
• Long-term spectra
• Vowel formants and bandwidths
• Target costs
• Concatenation costs
• Functional testing - TTS-system assessment on
  performance for communicative purpose
  (intelligibility measure)

15
16
Evaluation

• MOS (Mean Opinion Score) - general speech quality
  evaluation
• Pleasantness Pronunciation
• Articulation Understandability
• Sound quality Intelligibility
• Listening effort

17
Evaluation

• Groups of interest
• UCT staff (n75)
• UCT students (n75)
• Medical (Rehabilitation Doctors)
• Law
• Other streams
• Exposure to voice access non-exposure
• Monitor over time
• Results obtained from assessment exams
• Use age as one variable

17
18
Conclusion

• Text-to-speech technology is a convenient and
  support tool - possible solution.
• With an increasing recognition for demographics -
  need for engaged learners to take full
  responsibility for their language learning.
• There is also a need for teachers who are not
  only language experts but who are also trained in
  the use of technology and who can facilitate
  foreign language learning.

18
19
Future Work

• Multilingual Speech to Speech
• Speech to speech takes speech processing a step
  further towards a voice-only computer interface.
• Incorporates ASR TTS
• Enables computing devices to first understand
  spoken language, then analyze the utterance
  through natural language processing, and finally
  formulate and utter a response through
  synthesized speech.

19
20
Thank You
QUESTIONS??
SUGGESTIONS??
20