WebRTC Browser Client

Overview

By leveraging the new WebRTC live-streaming Extensions, it is now possible to stream any Omniverse application to web browsers.

Using the provided front-end source code and sample application, you can even build your own interactive experiences. This can be used to create a variety of services:

  • Offering remote workflows

  • Using Omniverse application on phones, tablets or other devices

  • Having one-to-many collaborative sessions, where attendees can request control of the presentation

  • And more!

Built something you’d like to share or showcase? We’d love to hear about it!

Omniverse WebRTC live-streaming

Enabling the Extension

  1. From the menu bar, navigate to Window > Extensions

  2. Using the search field of the Extension Manager, search for “WebRTC Client” Extension

  3. Install and enable the omni.services.streamclient.webrtc Extension

Note

Please note that it is not recommended to auto-load this Extension, as it will consume resources which may lead to sub-optimal experiences when not actively used.

User Manual

Streaming over Local Network

Using the WebRTC live-stream Extension is a straightforward process to access a host located on the same network as clients:

  1. Enable the omni.services.streamclient.webrtc Extension on Omniverse applications (Kit, USD Composer, Isaac Sim, etc.)

  2. Find the IP address of the machine hosting the Omniverse application on your local network

  3. Navigate to the stream page in your favorite web browser:

    • For Kit: http://<server IP address>:8011/streaming/webrtc-client?server=<server IP address>

    • For USD Composer: http://<server IP address>:8111/streaming/webrtc-client?server=<server IP address>

    • For Isaac Sim: http://<server IP address>:8211/streaming/webrtc-client?server=<server IP address>

    • For Kaolin: http://<server IP address>:8311/streaming/webrtc-client?server=<server IP address>

  4. Click the “play” button to start streaming your Omniverse application

From there, you can control your Omniverse application as if you were using it natively using mouse and keyboard inputs.

Note

If you have multiple Omniverse applications running on the same machine, or if the server is unable to use port 8011 because it is already in use, an alternative port will be generated and displayed in the Console.

In such cases, use the provided port to access the stream page using the following pattern: http://<server IP address>:<kit port>/streaming/webrtc-client?server=<server IP address>

Streaming across Networks

In the case where the host may be located on a different network than clients, or when attempting to connect to a host deployed on the infrastructure of a Cloud-based provider, additional configuration may be required.

WebRTC communication requires that all parties be able to exchange information about the capabilities they have, and the type of data they wish to exchange. When hosts and clients are located in different network, they may not be able to reach each other using a known public IP due to network or security restrictions put in place by firewalls, or corporate policies.

In these cases, WebRTC relies on NAT traversal techniques known as STUN and TURN, which both rely on external, Internet-facing infrastructure to assist parties in performing a handshake:

  • STUN servers are relatively simple components which only assist parties in obtaining their public IP addresses when addressing others, thus making it possible for a host or client to advertise its IP for others to reach it. The WebRTC Extension ships by default with public STUN servers offered by Google, which avoids the requirement of having to deploy and maintain custom infrastructure for Users.

  • TURN servers require additional infrastructure in case communication between parties cannot be established using STUN servers, and act as relay in the transmission of data between parties. Due to the nature of TURN servers which act as third-party brokers in the relay of data, Users may be required to either deploy their own TURN solution (we recommend using coturn), or to use the services of a communication platform provider.

Both custom STUN and TURN servers can be used by the WebRTC Extension, by supplying settings informing the extension about the server with which to communicate when attempting to connect parties:

./kit apps/omni.create.kit \
    --enable omni.services.streamclient.webrtc \
    --/exts/omni.services.streamclient.webrtc/ice_servers/1/urls/0="turn:turn.example.com:3478?transport=udp" \
    --/exts/omni.services.streamclient.webrtc/ice_servers/1/urls/1="turn:turn.example.com:3478?transport=tcp" \
    --/exts/omni.services.streamclient.webrtc/ice_servers/1/username="user" \
    --/exts/omni.services.streamclient.webrtc/ice_servers/1/credential="secret" \
    --no-window \
    --allow-root

The schema for the settings to supply STUN and TURN servers follows this structure:

  • urls (required): List of URLs for the STUN or TURN server (including the stun:, turn: or turns: prefix).

  • username (optional): Username to use to authenticate against the server, in the case of a TURN server.

  • credential (optional): Credential to use to authenticate against the server, in the case of a TURN server.

Should the host be located on the infrastructure of a third-party Cloud provider, it may additionally be required that the following inbound ports be opened:

  • TCP: 47995 to 48012

  • TCP: 49000 to 49007

  • TCP: 49100

  • UDP: 47995 to 48012

  • UDP: 49000 to 49007

Current Limitations

  • For a best user experience, it is recommended that only a single User at a time be provided with mouse or keyboard control while others remain in spectator mode.

  • Support for gamepads, touch inputs, or AR/VR devices may be limited.

  • Copy/paste support is not yet available.