ORTC Walk Through IETF 88

1
ORCA Object-RTC API Community Group
ORTC Walk Through IETF 88
Robin Raymond, Hookflash Bernard Aboba,
Microsoft Adalberto Foresti, Microsoft Open
Technologies, Inc.
2
Meeting Purpose

• The purpose of this meeting is explain
• Why a community has come together to work on an
  alternative API
• High level features of the ORTC API
• How to use the API to implement a simple video
  chat demo
• How you can get involved with developing the spec
  and code
• Introduce open source effort / call to action
• There will be time for refreshments and QA the
  last 30 minutes
• Session is recorded and streamed to the
  web(sponsored by Hookflash)

3
Introduction to ORTC

• ORTC Object Realtime Communications
• A next-generation JavaScript API for browser,
  mobile, and cloud realtime audio / video
  scenarios
• ORTC is an alternative API to the W3C WebRTC 1.0
  draft API
• It is being developed in the ORCA W3C Community
  Group

4
The ORCA W3C Community Group

• W3C Community Groups are forums open to all
  without fee
• Simple IPR policy royalty-free commitments to
  ones own contributions
• Not on standards track
• Mature CG specs can get broader IPR commitments,
  transition to standards track
• Dont have to be W3C member!

5
The ORCA W3C Community Group

• Object RTC API Community Group,
• Formed in August 2013Chair and editor Robin
  Raymond
• Goal to develop WebRTC API that exposes an object
  API rather than blobs of SDP
• Community group founded by JS developers coming
  from both SIP/other protocol signaling
  backgrounds, node.js and WebRTC enthusiasts.

6
This is not a community group meeting

• Goal here is to explain mission and progress,
  recruit new members
• To suggest API changes, join the Community Group
  we cant accept suggestions from those who have
  not signed the Contributor License Agreement

7
Special Thanks to

• ORCA Community Group asked Microsoft Open
  Technologies if they would implement ORTC as demo
  / validation of concept.
• Special thanks for Microsoft Open Technologies,
  Inc. for implementing demo / validation of
  concept.
• (and the refreshments!)

8
WebRTC Needs

• Mobile SDKs enabling development of mobile apps
  protocol-compatible with WebRTC.
• Analytics. Diagnostics and monitoring for
  carrier scale operation.
• Cloud. Easy setup and management of WebRTC
  systems.
• Tools. Enabling connection of WebRTC clients to
  legacy infrastructure (e.g. node.js)
• APIs. Enabling management of STUN/TURN
  credentials, audio/video mixers, SIP proxies,
  XMPP connection managers, etc.
• Extensibility Enabling addition of functionality
  beyond the core (e.g. mobile-friendly video
  codecs such as H.264/SVC)).
• A next generation JavaScript API compatible with
  the above.

9
What are ORTC potential uses?

• Browser code (simplified RTC control)
• Mobile development via ortclib SDKunder
  development https//github.com/openpeer/ortclib
• Server code via Node.jsnew services or legacy
  inter-op solutions call to contributeunder
  development https//github.com/openpeer/ortc-node
• Open Source projects are under way!

10
Write ORTC apps using existing WebRTC 1.0 Browsers

• An WebRTC 1.0 shim will be made available to
  maintain backwards compatibility
• If you like the WebRTC 1.0 API, you can still use
  the 1.0 API!

11
ORTC Migration Strategy
1.0 Strategy
2.0 Future ?
Web App
Web App
SDP / SIP (shim)
ORTC (shim)
ORTC
Node.JS
Browsers
Mobile
WebRTC 1.0 (SDP)
Node.JS
Browsers (see demo)
Mobile (ortclib)
Chrome / Firefox / etc
12
Isnt ORTC just CU-RTCWeb?

• No
• Both are object models but at different levels
• While CU-RTCWeb inspired ORTC, they are are
  different.
• Issues with CU-RTCWeb
• too powerful/difficult to use
• exposing too much socket level control (i.e.
  security risks)
• couldnt shim CU-RTCWeb

13
Where will ORTC be supported by vendors?

• Support is via the shim over WebRTC 1.0 for now
  for browsers.
• Direct ORTC API source code is being developed.
• Microsoft Open Tech developed a shim plugin for
  IE (but not a product announcement).

14
openpeer/ortcGitHub Specification Repo

• https//github.com/openpeer/ortc
• Original home of the ORTC API discussion
• Spec discussion spun off to CG, OSS code for ORTC
  stays in GitHub
• Why do we need the GitHub repo?
• Pull requests discussed on public-orca mailing
  listhttp//www.w3.org/community/orca/

15
openpeer/ortclibGitHub Mobility Repo

• https//github.com/openpeer/ortclib
• ORTC mobile implementation
• Maintained by Hookflash
• Exposes C WebRTC library as an ORTC API
• To be wrapped in iOS, Java (for Android) and
  JavaScript hybrid

16
openpeer/ortc-nodeGitHub Node.JS Repo

• https//github.com/openpeer/ortc-node
• ORTC Node.JS implementation
• Call to action seeking contributors
• What should implementation look like?
• What are native code integration points?

17
Will contributing to ortcbreak protocol
compatibility with web browser?

• No
• Wire level compatibility is maintained
• Maintaining wire compatibility is a tenet of ORTC
  implementations with WebRTC enabled browsers
• We encourage expansion of features, not removal
  of basic wire support.
• as much as they are compatible with each other

18
Motivation for ORTC Design
19
Traditional Browser Rendering Engine
Resource
JS
Description Format (HTML/CSS)
Render Engine
Output
20
This is not how we program HTML on the web
JS asks browser Give me webpage that meets this
criteria
1
Browser generates some HTML (gives to JavaScript)
2
Modify browser provided HTML
3
4
Set HTML
21
WebRTC 1.0 Capture / Render Engine
Cloud
JS
JS
Description Format (SDP)
Description Format (SDP)
Resource
Resource
Capture/Render Engine
Capture/Render Engine
Input/Output
Input/Output
22
ORTC Capture / Render Engine
Cloud
JS
JS
Capture/Render Engine
Capture/Render Engine
Input/Output
Input/Output
23
ORTC Simple Use Case
cloud
signaling
signaling
Browser
Browser
Application
Application
RTCConnection
RTCConnection
RTP / DATA / DTLS / ICE
24
Questions?
25
ORTC API
26
ORTC Simple Use Case
cloud
signaling
signaling
Browser
Browser
Application
Application
RTCConnection
RTCConnection
RTP / DATA / DTLS / ICE
27
ORTC - Object Relationships
remote
RTCIceCandidate
RTCIceCandidate
RTCConnectionOptions
RTCConnection
RTCSocket
local
remote
local
MediaStream
RTCStream
RTCConnectionInfo
RTCIceCandidate
RTCStream
MediaStream
RTCIceConnectionInfo
RTCConnectionInfo
RTCIceConnectionInfo
RTCTrack
MediaStreamTrack
RTCIceCandidate
RTCTrack
MediaStreamTrack
RTCTrack
MediaStreamTrack
RTCDtlsConnectionInfo
RTCTrack
MediaStreamTrack
RTCDtlsConnectionInfo
28
RTCConnection
Constructor(RTCConnectionOptions
options) interface RTCConnection EventTarget
readonly attribute RTCSocket
socket readonly attribute
RTCConnectionState state
attribute RTCConnectionRole role
readonly attribute RTCConnectionInfo local
attribute RTCConnectionInfo
remote void
addRemoteCandidate(RTCCandidateDescription
candidate) void
connect() void
gather() (RTCSTream? or RTCTrack?)
send((MediaStream or MediaStreamTrack or RTCTrack
or RTCStream) streamOrTrack) (RTCStream? or
RTCTrack?) receive((RTCStream or RTCTrack)
streamOrTrack) sequenceltRTCStreamgt
sendStreams() sequenceltRTCTrackgt
sendTracks() sequenceltRTCStreamgt
receiveStreams() sequenceltRTCTrackgt
receiveTracks() void
close () attribute EventHandler
oncandidate attribute
EventHandler onendofcandidates
attribute EventHandler
onactivecandidate attribute
EventHandler onstatechanged
attribute EventHandler
onnetworkchange attribute
EventHandler onunknowntrack
29
RTCConnection (p1)

• RTCSocket (attached to one network socket)
• States
• new connection not started
• searching searching for connectivity
• halted no connectivity found
• connected found connectivity (ready)
• closed connection is finished (one-time-use)

30
RTCConnection (p2)

• role (controlling vs controlled)
• Local and remote information
• ICE credentials (usernameFrag / password)
• DTLS fingerprint (generated or discovered)
• ICE Candidates local and remote (for
  connectivity checks)

31
RTCConnection (p3)

• Gathers local ICE candidates
• Performs ICE connectivity checks
• Causes sending / receiving of media tracks RTP
  data over the nominated ICE connection

32
RTCConnection (p4)

• RTCStreams and RTCTracks (sending or receiving)
• Contains mapping of on-the-wire RTC/RTP/RTCP/etc
  information to media streams / media stream
  tracks)
• Contains respective MediaStream or
  MediaStreamTrack objects.

33
RTCConnection connect

• Set remote RTCIceConnectionInfo ICE
  usernameFrag and password
• Call connect to cause candidate search
• Will causes state changes from new,
  searching, haulted, connected

34
RTCConnection gather

• Find new candidates
• - Restarts network topology ICE candidate
  gathering
• Does not affect connection state by itself
• Start gathering candidates before connect

35
RTCConnection send

• Send MediaStream, MediaStreamTrack,
  RTCStream, RTCTrack
• Convenience to auto-map MediaStream and
  MediaStreamTrack into RTP layer constructs
• Manipulate RTP mapping by first creating
  RTPStream / RTCTrack from MediaStream and
  MediaStreamTrack

36
RTCConnection receive

• Send RTCStream, RTCTrack
• Setup mapping for MediaStream and
  MediaStreamTrack for RTP layer constructs
• Manipulate mapping by constructing RTPStream /
  RTCTrack objects

37
RTCConnection close

• Shuts down RTCConnection object
• Object goes to closed state

38
RTCConnection events

• oncandidate Fires for local candidate (can be
  cancelled)
• onendofcandidates Fires when all local
  candidates are discovered
• onactivecandidate Fires when localremote
  candidate connection changes
• onstatechanged Fires as connection state
  changes
• onnetworkchange Fires when network topology
  change has been detected
• (auto gather called but can be cancelled)
• onunknowntrack Fires when track discovered
  which has no mapping

39
RTCSocket
interface RTCSocket
40
RTCSocket

• No exposed attributes, members or events
• Represents one physical local socket (as well as
  virtualized paired firewall/relay socket)
• Re-use the same RTCSocket in another
  RTCConnection to send over same physical local
  socket

41
RTCStream
Constructor, Constructor(MediaStream) interface
RTCTrack attribute
MediaStream source
attribute DOMString msid
sequenceltRTCTrackgt tracks()
void start()
void stop()
void remove()
42
RTCStream

• References single MediaStream
• Collection of RTCTracks
• Change collection then call RTCConnection
  send/receive again
• Simplifies MediaStream usage in RTCConnection
• Convenience start / stop methods
• remove method to remove from RTCConnection

43
RTCTrack
Constructor, Constructor(MediaStreamTrack) inte
rface RTCTrack attribute
MediaStreamTrack source
attribute DOMString
id attribute DOMString
kind
attribute unsigned long
ssrc attribute
sequenceltRTCCodecgt codecs
attribute sequenceltRTCMediaAttributesgt
mediaAttributes attribute
Object
rtpExtHeaders void
start() void
stop() void
remove()
44
RTCTrack

• References single MediaStreamTrack
• Mapping of RTP/RTCP/etc wire information such as
• ssrc
• codecs
• media attributes (e.g. max width / height)
• Change properties then call RTCConnection send
  / receive again
• Convenience start / stop methods
• remove method to remove from RTCConnection

45
RTCConnectionInfo
dictionary RTCConnectionInfo
attribute RTCIceConnectionInfo ice
readonly attribute RTCDtlsConnectionInfo
dtls
RTCIceConnectionInfo
dictionary RTCIceConnectionInfo DOMString
usernameFrag DOMString
password
RTCDtlsConnectionInfo / CertificateFingerprint
dictionary RTCDtlsConnectionInfo attribute
CertificateFingerprint fingerprint dictiona
ry CertificateFingerprint getter
ArrayBuffer (DOMString hashFunction)
46
RTCIceCandidateInfo
dictionary RTCIceCandidateInfo DOMString
foundation int
component DOMString
transport int
priority
DOMString
connectionAddress int
connectionPort DOMString
type DOMString
relAddress int
relPort
47
Object Realtime Communications API (ORTC)

• Next generation WebRTC API, under development in
  the W3C ORCA community group
• http//www.w3.org/community/orca/
• Rationale http//tools.ietf.org/html/draft-raymon
  d-rtcweb-webrtc-js-obj-api-rationale
• Specification
• http//openpeer.github.io/ortc/
• Github repository
• ORTC API drafts and shim https//github.com/openp
  eer/ortc
• ortclib https//github.com/openpeer/ortclib
• Demos https//github.com/openpeer/webrtc-demo

48
To Learn More

• ORCA Community Group http//www.w3.org/community/
  orca/Demo and discussion tomorrow to follow