VoIP on the iPhone: Imagine the Possibilities

1
VoIP on the iPhone Imagine the Possibilities Jan
Linden, VP of Engineering
2
VoIP on the iPhoneOutline

• Market trends
• Technology trends
• Mobile VoIP challenges
• Solutions and recommendations

3
Evolution of IP Voice andVideo Processing
Solutions Designed Specifically for VoIP
PSTN Solutions Adapted for VoIP
4
Mobile Market Trends

• Fixed Mobile Convergence
• Mobile phone increasingly perceived as a computer
• Advanced operating systems such as Apple OSX are
  making this a reality
• Users expect to access the same applications
  available across multiple platforms
• Increased accessibility and affordability
• Network operators are introducing new pricing
  models to encourage uptake of data driven
  services and applications
• IP networks enable more cost effective transport
• Handset prices declining
• iPhone 3G half the price of 1st Generation

5
Mobile Technology Trends

• Migration from legacy cellular to next-gen
  networks
• Allows for faster data connection
• Growth of Smartphone adoption
• iPhone and other dual-mode phones provide greater
  flexibility in connecting to available networks
• All-in-one devices able to run multiple
  applications efficiently
• Consumers can place calls and receive data
  simultaneously
• Development of 3rd Party Applications
• Operating systems, like the Apple OS X, more
  conducive to advanced 3rd party applications
• VoIP over mobile in high demand

6
The iPhone as a VoIP Platform

• iPhone most VoIP friendly phone on market
• Open APIs for easy development
• Sufficient CPU resources
• Support of wideband codecs
• Intuitive UI makes for simple navigation between
  applications
• Demand for 3rd party apps is higher than other
  phones
• but strengths present unique challenges
• Running too many applications can drain resources
• Usage scenarios can negatively impact voice
  quality

With the right voice processing software,
developing VoIP applications is a cinch!
7
Technology Challenges Network Connections

• IP Networks
• Latency
• Jitter
• Packet loss
• Wireless connections
• Low signal strength
• Interference from users in same or neighboring
  frequencies
• Available bandwidth varies
• WiFi access point bottlenecks

8
Technology Challenges

• Device Limitations
• Limited processing power
• Battery life
• Data intensive applications can cause jitter
• Mobile Environments
• Tend to have excessive background noise
• Hands Free operation enhances acoustic echo

9
Impact of IP Networks
Delay
Packet Loss
Network Jitter
10
Impact of IP Networks Latency

• Major effect is stepping on each others talk
• Usage scenario affects annoyance factor higher
  delay can be tolerated for mobile devices
• Long delays make echo more annoying

11
Sources of Latency

• Codec
• Capture
• Playout
• Network delay
• Jitter buffer
• OS interaction
• Transcoding

12
Impact of Latency on Voice Quality
Data from ITU-T G.114

• ITU-T (G.114) recommends
• Less than 150 ms one-way delay for most
  applications (up to 400 ms acceptable in special
  cases)
• Users have become accustomed to longer delays
• Still, low delay very important for high quality

13
Impact of IP Networks Packet Loss and Jitter

• Packet Loss
• Occurs due to flushed buffers in network nodes
• Same effect if packets are too late to be used
• Smooth concealment necessary
• Network Jitter
• Transmission time differs for each packet
• Jitter buffer necessary to ensure continuous
  playout
• Trade-off between delay and quality

14
Technology Challenges Mobile Environments

• Acoustic Echo
• Speakerphone operation common for iPhones
• Micropohone and speaker close
• High delay in VoIP makes echo more prominent

• Background Noise
• Environments inherently noisy
• Usage scenarios can make differentiating between
  speech and noise difficult

15
Technology Challenges Devices

• Limited Resources
• Size limits processing power
• Battery life also limited
• VoIP Friendliness
• Many smartphones limit access to soundcard
• Device buffers add latency
• Clock drift
• Too much of a Good Thing
• Running too many applications can drain resources
• Multiple applications accessing network can
  result in jitter

16
VoIP Design Considerations
Device Considerations
17
Mobile VoIP Design Considerations
Environment Background Noise, Room Acoustics,
etc.
Speech Codec
Coping with Network Degredation
Power Consumption
Echo Cancellation
Hardware Issues (Processor, OS, Acoustics, etc.)
Additional Voice Processing Components
18
Speech Codec

• Many conflicting parameters affect choice of
  codec
• Determines upper limit of quality
• Support of several codecs necessary
• Interoperability
• Usage scenario
• IPR issues a significant concern

Input Signal Robutness
19
Audio Bandwidth Effect on Quality

• Better than PSTN quality is achievable in VoIP
• Utilizing full 0 4 kHz band in narrowband
• Wideband coding offers more natural and crispier
  voice

20
iPhone Specific Challenges

• Applications cannot run in the background
• Audio HAL APIs (regular APIs) not available on
  iPhone. (As on Mac.) Very iPhone specific
  solution had to be designed
• Must consider change of audio route (e.g. headset
  in/out) and handle this
• Wideband audio not available via built-in mic  

21
Recommendations

• IP Networks
• No need for dedicated bandwidth if solution
  includes high performance
• Bandwidth management
• Robust codec framework
• Low latency achieved by efficient jitter buffer
• Audio
• High quality audio is crucial
• Users expect the same quality they experience on
  the PC
• Optimized Solutions
• Efficiency for ARM processors on iPhones
• Reduce CPU constraint and increase battery life

22
Recommendations

• Adaptive Jitter Buffer and PLC
• Manage up to 30 packet loss
• Adapt to network busts and excessive mobile
  jitter
• Codecs Designed for IP Networks
• Work in tandem with jitter buffer to overcome
  packet loss
• Can deliver Wideband experience
• Robust and bit-rate adaptive codecs necessary
• Efficient use of the total available bandwidth
• GIPS iLBC already in the iPhone, GIPS iSAC and
  other wideband codecs provide even better user
  experience
• Aggressive Voice Quality Enhancement
• Detect and cancel background noise
• Suppress acoustic echo

23
Recommendations

• VoiceEngine Mobile for iPhone
• Comprehensive package of voice processing
  components
• Handles delay, jitter, packet loss and mobile
  environments
• Optimized for iPhone and Smartphones
• Efficient solution reduces CPU demands
• Flexible and easy to integrate