Title: Media Services in CINEMA
1Media Services in CINEMA
- Presented by Kundan Singh
- Joint work with Henning Schulzrinne, Jonathan
Lennox, Xiaotao Wu, Wenyu Jiang, Sankaran
Narayanan, - Ajay Nambi and Ali Khwaja
- Computer Science Department, Columbia University
- April 25, 2003
2Purpose
N
- What are media services?
- What is CINEMA?
- What services are provided by CINEMA?
- Conferencing
- Streaming media
- Unified messaging
- Interactive voice response
- How can we use CINEMA?
- The discussion on services and specific CINEMA
implementation components are intermixed.
Not relevant or not implemented On-going
work Yes, Implemented
N
O
Y
3IP telephony services (PSTN)
O
- Call routing services pre-call, one party
- speed dial
- call forwarding
- follow me
- call filtering/blocking (in/out)
- do not disturb
- distinctive ringing
- Call handling features
- autoanswer
- Multi-party features
- call waiting
- call transfer (blind, consultative)
- conference call
- call park
- call pickup
- music on hold
- call monitoring
IP-telephony cant win by just providing what
PSTN has
4IP telephony services (Internet)
O
- Presence-enabled calls
- place call only if callee is available
- Presence-enabled conferencing
- call conference participants when all are online
and not busy - IM conference alerts
- receive IM when someone joins a conference
- Unified messaging
- receive email, IM alert for new voicemails
5Where the services reside?
O
Double ringing sound when boss calls
Enter your authentication PIN for billing
Use finger for locating user
B2BUA
Endpoint
Make call when boss is online
Proxy/registrar
Endpoint
Forward to office phone during day, and home
phone during evening
- Common gateway interface (CGI)
- Call processing language (CPL)
- SIP servlet
- Language for End System Services (LESS)
6Bricks in media applications
N
Interactive voice response
Internet Telephony
Internet Radio/TV
Messaging and Presence
Unified messaging
Video conferencing
SIP
SAP
RSVP
RTCP
Media G.711 MPEG
RTP
RTSP
Quality of service
Media transport
Signaling
Program Call routing
Voice XML
DTMF
Mixing
Speech/ text
SDP
7Audio Packet Transfer
N
- Digitization (e.g., sampling at 8kHz, 16 bits per
sample, i.e, 128 kb/s or 320 bytes per 20 ms) - Real-time compression/encoding (e.g., G.729A at 8
kb/s) - Transport to remote IP address and port number
over UDP (Why not TCP?) - Processing on receiver side is the reverse
8Problems with UDP
N
Sender
1
2
3
4
5
6
7
(a)
(b)
1
2
3
5
7
6
timeline
Receiver
- Unreliable UDP
- Packet loss
- Out-of-order (very rarely)
- Jitter (delay variation)
9Receive Buffer
Y
- Receive buffer to absorb jitter
- Tradeoff in buffer size
- Adaptive delay adjustment
- Sequence number to detect packet loss Just
ignore the loss!
Sender
1
2
3
4
5
6
7
8
9
0
1
2
3
4
1
2
3
5
7
6
8
9
0
2
3
4
Receiver
1
2
7
1
2
5
7
8
9
0
2
3
1
2
3
5
6
7
8
9
0
2
10Real-time Transport Protocol
Y
IP header
Sequence number
Payload type
CC
M
V
P
X
UDP header
Timestamp (proportional to sampling time)
RTP Header
Source identifier (SSrc)
Encoded Audio
Optional contributors list (CSrc)
msg
RTP media transport RTCP QoS feedback
sendto(, msg, ) recvfrom(, msg, )
11Streaming media
Y
- Messages
- RTSP vs HTTP
- UDP, Stateful, Control, Timed
- Applications
- Recording, Playback, Live media, Conference
12Columbia rtspd
Y
- Existing
- Playback/recording of ?-law (Sun .au files) and
raw - Unicast address
- Multi-threaded
- Portable (Unix/win32)
- Works with Apple QuickTime client
- UDP TCP
- IPv4 IPv6
- Planned
- Multicast
- MPEG video
- Mp3, wav audio
- Media over TCP
- Live media
13RTP-based Conference
N
?-law
Mixer
Transcoder
?-law
?
?
G.729
G.729
?-law
?-law
14Why do we need signaling?
Y
- What is the IP address of Alices host?
- What audio/video encoding can it support?
REGISTER as alicegt128.59.19.194
Call alice
128.59.19.194
Alices host 128.59.19.194
bobs host
home.com
- Session Initiation Protocol
- Email-like address alice_at_home.com
- DNS
- Proxy, redirect
15Session description
Y
INVITE alice_at_home.com I can support ?-law and
G.729 Send me audio at 202.16.49.276780
Alice
Bob
128.59.19.194
202.16.49.27
OK I can support ?-law Send me audio at
128.59.19.1948000
RTP
To port 8000
RTP
To port 6780
16Columbia sipc
Y
17Emergency service
O
- Calling
- E911, location
- Assistance
- Audio/video, remote-help
- Notification
- Earthquake, fire
18Presence/event notification
Y
office.com
bob_at_office.com
Presence server
PUA
PA
REGISTER
alice_at_home.com
SUBSCRIBE
NOTIFY
PUA
registrar
19Columbia IM and presence
Y
20Endpoint call control
O
- Language for End System Services (LESS) for
endpoint service creation - Direct user interaction, direct media control
- Handle converged information, e.g., call,
presence, email - Example when buddy is online, make a call
- ltless name"online_call"
- require"generic presence ui"gt
- ltnotification status"online" priority"0.5"gt
- ltaddress-switch field"origin"gt
- ltaddress isalice_at_office.com"gt
- ltcall /gt
- ltalert soundring.au"
- text"Calling " /gt
- lt/addressgt
- lt/address-switchgt
- lt/notificationgt
- lt/lessgt
21Network call control
Y
if (defined ENVSIP_FROM ENVSIP_FROM
/sipboss_at_mycompany.com/) foreach reg
(get_regs()) print
"CGI-PROXY-REQUEST reg SIP/2.0\n" print
"Priority urgent\n\n"
22Call transfer
O
B
A
- REFER
- Blind/ consultation/ attended
C
active call
REFER C Referred-By B
INVITE C Referred-By B
BYE A
active call
23B2BUA and third-party call control
O
- Back-to-back UA
- Incoming call triggers outgoing call
- Services
- Calling card
- Anonymizer
B
A
C
INVITE
OK (SDP1)
ACK
INVITE (SDP1)
OK (SDP2)
ACK
INVITE (SDP2)
OK
ACK
24Voicemail
Y
Endpoint redirects
- Problems in PSTN
- Design alternatives
- Issues
Redirect after 10s
alice_at_vmail.yahoo.com
Proxy controls
Voicemail acts like a phone
vmail.pl
SIP_FROM SIP_TO stdin
CGI-PROXY-REQUEST stdout
If no response
accept after 15s
25Columbia sipum
Y
- Goals
- Universal access
- Scalability
- Provider independent
- Why SIP and RTSP?
- Reuse existing infrastructure and tools
26Columbia sipum
Y
- Retrieval
- Web interface
- rtsp//server/alice /inbox/1677.au,
- sipalice-1677-retrieve_at_server
- press 1 to listen
- Configuration
- Folders
- Options
- Email
27VoiceXML
Y
Gateway
sipvxml
PSTN
Telephone
IVR platform
- Voice and telephony functions
- (ASR, TTS, DTMF)
- Service logic (application specific)
28VoiceXML contd.
Y
ltformgt ltfield nameidgt ltpromptgt
Your ID, please. lt/promptgt lt/fieldgt
ltblockgt ltsubmit nexturl/gt
lt/blockgt lt/formgt
ltform actionurlgt Enter your Id ltinput
nameidgt ltinput typesubmitgt lt/formgt
Telephony, speech synthesis or audio output, user
input and grammar, program flow, variable and
properties, error handling,
29DTMF
Y
PSTN
Internet
Telephone
SIP/PSTN gateway
Internet user
Audio
D
SIP INFO
D
RTP/ RFC2833
30Columbia sipvxml
Y
PSTN
Internet
Telephone
SIP/PSTN gateway
- Unified messaging access
- Email by phone
- Event notification and scheduling
- Audio volume control for conference
- Advanced conference control
TTS, ASR, DTMF, XML, HTTP, RTSP
sipvxml
SIP phone
Web server
Email.tcl
31Email phone
Y
Email by phone
Inbox
Email
procmail
important mails
Internet
Email to IM
- Login
- Email formatting
- Listen, reply, delete, compose, forward
- Navigation -next, previous, jump
SIP
- Email formatting
- SIP based
- Text-to-speech
SIP
VoiceXML browser
TTS
SIP
HTTP
Internet
Email servlet
JSP
Inbox
DB
Email to phone
32IM voice call
Y
- Who can initiate?
- authentication, billing,
- Feedback
- to voice user
- to IM user
- initial IM greeting
- Talk-spurt detection
- Speech recognition
SIP MESSAGE
TTS ASR
SIP INVITE
RTP
33Unified messaging
O
- The gaps among different media (audio, video,
text), devices (PC, phone) and means of
communications (Email, SIP, IM) disappear for
messaging
34Notification
O
- Calendar
- Events
- Conferences
Schedule from a browser
SIP call 2129397063
IM kundan_at_
Web server
Calendar.cgi
Email Kns10_at_
at 600pm
35Phone announcement server
Y
Text or audio
Destinations
- Input
- Range93970??
- List A, B, C
- Example
- Announcement
- Emergency
- Issues
- Voicemail
- Failure detection
TTS
PAS
SIP
. . .
36RTSP TTS ASR
O
TTS ASR
Media server
SETUP rtsp//server/tts.cgi ?textHowareyou.
SETUP rtsp//server/asr.cgi
PLAY
RECORD
RTP
RTP
SET_PARAMETER TextI am fine, thank you.
37Audio conference models (non-multicast)
N
Topology star
full-mesh
ad-hoc
No central point of failure
Advantages
Heterogeneous simple clients
Typically only three party conferences
Complex endpoints Complex signaling
Disadvantages
External server with high bandwidth link
38Centralized conferencing
O
- Conference as URL
- staffmeet_at_office.com
- On the fly conferences
- my.adhoc_at_server
- Basic task join/leave
- Dial in, Refer dial in
- Dial out, Refer dial out
INVITE
INVITE
REFER
INVITE
server
REFER
39Columbia sipconf
Y
- G.711, GSM, DVI, G.722 mixing (decode-mix-encode)
- Video replication IM text VNC screen sharing
40Columbia sipconf
Y
41Conference control
O
- Access control
- Floor control
- Notification
- State
- Participant active, departed, booted, failed
- dialog state, floor state
- Media
- sending received-by-all, muted
- receiving receiving-all, anchor-only
- ltconference-info version"0" state"full"
entitymeeting_at_example.com"gt - ltuser uri"sipalice_at_home.com"
display-nameAlice"gt - ltstatusgtactivelt/statusgt
- ltmedia-statusgt
- ltmedia-stream media-type"audio"/gt
- lt/media-statusgt
- lt/usergt
- ltuser uri"sipbob_at_office.com"
display-nameBob Wilson"gt - ltstatusgtdepartedlt/statusgt
42Conference VoiceXML
Y
1. INVITE sipvxml
2. Call accepted
3. Enter your four digit PIN
4. Entered 4-6-8-3
5. Authenticate user, 4683gtAlice
6. Enter the conference identifier
7. Entered 2-3-
8. Permission to join, 23gtmeet
sipvxml
9. REFER meet_at_conference
10.Terminate the old call
Caller
11.INVITE meet_at_conference
Call transfer vs bridged mode
43New conference applications
Y
The ease flexibility of sipvxml enables us to
build custom telephonic applications to suit our
needs. e.g., Volume Check Application
1. INVITE sipvxml
2. Menu 1. Vol Check 2. Mic Check
3. User enters 2
sipvxml
4. User speaks out a voice sample
5. Voice sample is analyzed
6. SipVXML Vol level too high/low/
Caller
7. User adjusts the vol level.
sipconf
8. User now joins conference.
44Conferencing
Y
- Automatic volume adjustments
- Automatic load balancing
- Delay adjustments
- Conference recording
- Local or RTSP
Recording in a media server
45Ncast MPEG distribution
Y
INVITE sipxyz_at_cs.columbia.edu
sip.cgi?sipxyz_at_cs.columbia.edu
46Y
sipjohn_at_cs.columbia.edu
INVITE sipjohn_at_cs.columbia.edu
My owners SIP address is sipjohn_at_cs.columbia.edu
Help!!! (invoke sipc to call sipjohn_at_cs.columbia.
edu)
47Summary of media services
Y
- Streaming media
- Playback, recording, live-media
- Unified messaging
- Voicemail, video/IM integration, ...
- Conferencing
- Mixing, load-balancing
- Interactive voice response
- Tele-banking, VoiceXML
48CINEMAColumbia InterNet Extensible Multimedia
Architecture
Y
Distributed component architecture
User-agent (client) applications
CINEMA
Application and proxy servers
Libraries and APIs
Management and control
49Columbia SIP user agent sipc
Y
- Media
- Audio, video, text, white board
- Screen sharing
- Shared web browsing
- Advanced
- Presence, instant messaging
- Programmable call handling
- Conference control (on-going)
- Emergency notification and handling (on-going)
- Device control
- Language for End System Services (on-going)
50Columbia SIP servers (CINEMA)
Y
Single machine
Telephone switch
rtspd media server
Local/long distance 1-212-5551212
sipconf Conference server
Quicktime
RTSP
PSTN
RTSP clients
Department PBX
sipum Unified messaging
sipd Proxy, redirect, Registrar server
Internal Telephone Extn 7040
713x
SQL database
SIP/PSTN Gateway
Web based configuration
SNMP (Network Management)
H.323
siph323 SIP-H.323 translator
Video encoder
NetMeeting
51Libraries and APIs
Y
CPL, CGI, Servlet
Parsing, SIP, SDP, RTP, mySQL interface, SNMP
interface, Portability stubs, etc.
52Status of CINEMA services
53Whats next?from multimedia communications to
collaboration
- Synchronous communications
- Conferencing, IM
- Asynchronous communications
- Voicemails, message board, file sharing
- Ubiquitous computing
- i-button, ID-card
- Service creation
- User friendly, end-point/network
kns10_at_cs.columbia.edu http//www.cs.columbia.edu/
kns10/ http//www.cs.columbia.edu/IRT/cinema
54Media Services in CINEMA
55SIP beyond telephony
- SUBSCRIBE door_at_alice.home.net
- NOTIFY alice_at_work.com
- DO light_at_alice.home.net
- INVITE camera_at_alice.home.net
56PSTN interworking
sipbob_at_home.com
1 212 9397063
Telephone network
Telephone subscriber
SIP/PSTN gateway
SIP server
IP endpoint
- Translating audio (PCMU/PCMA)
- Translating signaling (PRI/T1,ISUP)
- Overlap signaling
- Advanced features in SIP are lost in PSTN
- Translating identifiers (phone number)
- Determining transition points
57PSTN to IP
- Gateway knows the SIP server
- ltsip7063_at_conductor.cs.columbia.eduuserphonegt
- ENUM
- DNS
- 1 212 9397042 gt 2.4.0.7.9.3.9.2.1.2.1.e164.arpa
gt siphgs_at_cs.columbia.edu - Suitable for relatively static contacts
58IP to PSTN
- Static mapping
- 1-212-939xxxx gt _at_itgw1.cs.columbia.edu
- ITGW information is dynamic
- Overlapping networks
- Multiple providers
- Load balancing
- TRIP
- Route advertisement
- Can be implemented in outbound proxy
- Suitable for current hierarchical network
- 1 _at_service.mci.com at 4/min
- 1212 _at_nyc.gw.com at 1/min
- 1212939 _at_itgw1.columbia.edu free
59SIP-CGI
- Programming language independent
- Maintains state via an opaque token
- For SIP proxies and endpoints
- call routing
- controlling forking
- call rejection
- call modification (Priority, Call-Info)
- RFC 3050
- Upload via web or REGISTER
60SIP CGI contd.
- Block _at_anonymous.com
- if (defined ENVSIP_FROM
- ENVSIP_FROM "sip_at_anonymous.com")
-
- print "SIP/2.0 600 I can't talk right
now\n\n" -
- Make calls from boss as urgent
- if (defined ENVSIP_FROM
- ENVSIP_FROM /sipboss_at_mycompany.com/)
-
- foreach reg (get_regs())
-
- print "CGI-PROXY-REQUEST reg SIP/2.0\n"
- print "Priority urgent\n\n"
-
61CPL
- XML-based interpreted
- Control
- Restricted to branching and subroutines
- no variables, no loops
- structured tree describing actions performed on
call setup event - Event
- incoming and outgoing events
- Switch
- address, language, time, priority
- Action
- Proxy, redirect, reject (busy)
- Graphical tools
- Protocol-independent
- SIP, H.323
- Extensions
- CPL for presence
- Logging
- Email
62CPL contd.
- lt?xml version"1.0" ?gt
- lt!DOCTYPE cpl PUBLIC "-//IETF//DTD RFC2824 CPL
1.0//EN" "cpl.dtd"gt -
- ltcplgt
- ltincominggt
- lttime-switchgt
- lttime dtstart"20020830T140000Z"
- dtend"20020830T150000Z"gt
- ltreject status"reject" reasonBusy" /gt
- lt/timegt
- lt/time-switchgt
- lt/incominggt
- lt/cplgt
63SIP servlet
- Java-based
- Runs in SIP server
- Receives SIP objects and processes them
- public class RejectServlet extends
SipServletAdapter - public boolean doInvite(SipRequest req)
- SipResponse res req.createResponse()
- res.setStatus(603)
- res.send()
- return true
-