Python Environment

This document will cover:

Details about how running standalone Python scripts works.
A short list of interesting/useful standalone Python scripts to try.
How Jupyter Notebook support works.
How Visual Studio Code support works.

Details: How `python.sh` works

Note

On windows use python.bat instead of python.sh
The details of how python.sh works below are similar to how python.bat works

This script first defines the location of the apps folder so the contained .kit files can be located at runtime.

# Get path to the script
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# The apps directory is relative to where the script lives
export EXP_PATH=$SCRIPT_DIR/apps

Then we source the Omniverse Isaac Sim Python environment so all extension interfaces can be loaded correctly.

source ${SCRIPT_DIR}/setup_python_env.sh

The setup_python_env.sh script update/defined the following environment variables:

ISAAC_PATH: Path to the main isaac folder
PYTHONPATH: Paths to each extensions Python interfaces
LD_LIBRARY_PATH: Paths to binary interfaces required to find symbols at runtime
CARB_APP_PATH: path to the core Omniverse kit executable

Finally, we execute the Python interpreter that is packaged with Omniverse:

python_exe=${PYTHONEXE:-"${SCRIPT_DIR}/kit/python/bin/python3"}
...
$python_exe $@

SimulationApp

The SimulationApp Class provides convenience functions to manage the lifetime of a Omniverse Isaac Sim application.

Usage Example:

The following code provides a usage example for how SimulationApp can be used to create an app, step forward in time and then exit.

Note

Any Omniverse level imports must occur after the class is instantiated. Because APIs are provided by the extension/runtime plugin system, it must be loaded before they will be available to import.

Important

When running headless:

Set "headless": True in the config when initializing SimulationApp
Any calls that create/open a matplotlib window need to be commented out

from isaacsim import SimulationApp

# Simple example showing how to start and stop the helper
simulation_app = SimulationApp({"headless": True})

### Perform any omniverse imports here after the helper loads ###

simulation_app.update()  # Render a single frame
simulation_app.close()  # Cleanup application

Details: How SimulationApp works

The sample code shown below can be executed via:

./python.sh standalone_examples/api/omni.kit.app/app_framework.py

There are three steps to loading an Omniverse application in Python. The first is to get the carbonite framework:

import carb
self.framework = carb.get_framework()
self.framework.load_plugins(
        loaded_file_wildcards=["omni.kit.app.plugin"],
        search_paths=[os.path.abspath(f'{os.environ["CARB_APP_PATH"]}/plugins')],
    )

Here CARB_APP_PATH was previously defined by running using python.sh.

We then set up the startup arguments for the application:

# Inject experience config:
sys.argv.insert(1, f'{os.environ["EXP_PATH"]}/omni.isaac.sim.python.kit')

# Add paths to extensions
sys.argv.append(f"--ext-folder")
sys.argv.append(f'{os.path.abspath(os.environ["ISAAC_PATH"])}/exts')
# Run headless
sys.argv.append("--no-window")

# Set some settings
sys.argv.append("--/app/asyncRendering=False")

And then start the application:

self.app.startup("Isaac-Sim", app_root, sys.argv)

Shutting down a running application is done by calling shutdown and then unloading the framework:

self.app.shutdown()
self.framework.unload_all_plugins()

Enabling additional extensions

There are two methods for adding additional extensions:

Add them to the [dependencies] section of apps/omni.isaac.sim.python.kit:

[dependencies]
# Enable the layers and stage windows in the UI
"omni.kit.window.stage" = {}
"omni.kit.widget.layers" = {}

Enable them from Python directly:

# Start the omniverse application
simulation_app = SimulationApp(CONFIG)
from omni.isaac.core.utils.extensions import enable_extension
# Enable the layers and stage windows in the UI
enable_extension("omni.kit.widget.stage")
enable_extension("omni.kit.widget.layers")
simulation_app.update()

Standalone Example Scripts:

Time Stepping

This sample shows how to start an Omniverse Kit Python app and then create callbacks which get called each rendering frame and each physics timestep. It also shows the different ways to step physics and rendering.

The sample can be executed by running the following:

./python.sh standalone_examples/api/omni.isaac.core/time_stepping.py

Load USD Stage

This sample demonstrates how to load a USD stage and start simulating it.

The sample can be executed by running the following, specify usd_path to a location on your nucleus server:

./python.sh standalone_examples/api/omni.isaac.kit/load_stage.py --usd_path /Isaac/Environments/Simple_Room/simple_room.usd

URDF Import

This sample demonstrates how to use the URDF Python API, configure its physics and then simulate it for a fixed number of frames.

The sample can be executed by running the following:

./python.sh standalone_examples/api/omni.importer.urdf/urdf_import.py

Change Resolution

This sample demonstrates how to change the resolution of the viewport at runtime.

The sample can be executed by running the following:

./python.sh standalone_examples/api/omni.isaac.kit/change_resolution.py

Convert Assets to USD

This sample demonstrates how to batch convert OBJ/STL/FBX assets to USD.

To execute it with sample data, run the following:

./python.sh standalone_examples/api/omni.kit.asset_converter/asset_usd_converter.py --folders standalone_examples/data/cube standalone_examples/data/torus

The input folders containing OBJ/STL/FBX assets are specified as argument and it will output in terminal the path to converted USD files.

Converting folder standalone_examples/data/cube...
---Added standalone_examples/data/cube_converted/cube_fbx.usd

Converting folder standalone_examples/data/torus...
---Added standalone_examples/data/torus_converted/torus_stl.usd

This sample leverages Python APIs from the Asset Importer extension.

The details about the import options can be found here.

Livestream

This sample demonstrates how to enable livestreaming when running in native Python.

See the livestream documentation for more information on running the client.

The sample supports the Omniverse Streaming Client backend.

./python.sh standalone_examples/api/omni.isaac.kit/livestream.py

Note

Running livestream.py will not have all of the default isaac sim extensions enabled. See enabling additional extensions for more information.

Jupyter Notebooks

In order for Isaac Sim to work inside of a Jupyter Notebook we provide a custom Jupyter kernel that is installed when you run:

./jupyter_notebook.sh

Note

Jupyter notebook support is linux only currently

The kernel.json itself is fairly simple:

{
    "argv": ["AUTOMATICALLY_REPLACED", "-m", "ipykernel_launcher", "-f", "{connection_file}"],
    "display_name": "Isaac Sim Python 3",
    "language": "python",
    "env": {
        "ISAAC_JUPYTER_KERNEL": "1"
    },
    "metadata": {
        "debugger": true
    }
}

The important part is that AUTOMATICALLY_REPLACED gets replaced by jupyter_notebook.sh with the absolute path to the python executable that is located in the kit/python directory at runtime. Once the variable is replaced, the kernel is installed and the notebook is started. There is an extra variable ISAAC_JUPYTER_KERNEL that is used inside of Isaac Sim to setup for notebook usage properly.

Because notebooks require asyncio support, and Isaac Sim itself uses asyncio internally, we automatically execute the following two lines when loading the isaacsim module (or the omni.isaac.kit extension) which provides the SimulationApp class:

import nest_asyncio
nest_asyncio.apply()

This ensures that asyncio calls can be nested inside of the Jupyter Notebook properly.

Visual Studio Code (VSCode) Support

The Isaac Sim package provides a .vscode workspace with a pre-configured environment that provides the following three files:

.vscode/launch.json
.vscode/settings.json
.vscode/tasks.json

launch.json

This file provides three different configurations that can be executed using the Run & Debug section in VSCode.

Python: Current File: Debug the currently open standalone Python file, should not be used with extension examples/code.
Python: Attach: Attach to a running Isaac Sim application for debugging purposes, most useful when running an interactive GUI application. See Attaching the Debugger to a Running App for usage information.
(Linux) isaac-sim Run the main Isaac Sim application with an attached debugger.

settings.json

This file sets the default Python executable that comes with Isaac Sim:

"python.pythonPath": "${workspaceFolder}/kit/python/bin/python3",

As well as a configuration for "python.analysis.extraPaths" which by default includes all of the extensions that are provided by default. You can add additional paths here if needed.

tasks.json

This is a helper file that contains a task used to automatically setup the Python environment when using the Python: Current File option in Run & Debug.

"tasks": [
        {
            "label": "setup_python_env",
            "type": "shell",
            "linux": {
                "command": "source ${workspaceFolder}/setup_python_env.sh && printenv >${workspaceFolder}/.vscode/.standalone_examples.env"
            }
        }
    ]

Once executed, the task generates the .standalone_examples.env file used by VSCode to launch the Python debug process.

Python Environment

Details: How python.sh works

SimulationApp

Usage Example:

Details: How SimulationApp works

Enabling additional extensions

Standalone Example Scripts:

Time Stepping

Load USD Stage

URDF Import

Change Resolution

Convert Assets to USD

Livestream

Jupyter Notebooks

Visual Studio Code (VSCode) Support

launch.json

settings.json

tasks.json

Details: How `python.sh` works