Physics
Physics Simulation Fundamentals
On a high-level, simulations with Omniverse™ Physics work as follows:
The USD Physics schema of robot and environment assets are parsed and corresponding objects are created in the PhysX SDK backend.
Then, for each discrete-time step of the simulation, Physics advances the PhysX SDK objects given their current state and additional inputs such as, for example, control-policy torques.
The updated state is written back to USD by default, where the state can be further processed by the user, a reinforcement-learning policy, or other extensions such as the Omniverse RTX Renderer.
Omniverse™ Physics propagates runtime changes to physics parameters in USD to the PhysX SDK objects.
To setup robot simulations, take the following general steps:
Import the robot and environment assets, see the following docs and tutorials.
Setup physics properties on the assets, such as collision geometries, masses, and joint properties, see below.
Configure global simulation parameters such as time-step size and solver settings, see below and Simulation Management.
Analyze and debug simulation issues with UI debug visualization, runtime visualization options in the Physics Debug Window debug visualization, or the Omniverse Visual Debugger.
Further Simulation Resources
Omniverse™ Physics core documentation and programming guide
In Explore Physics demos you can learn about physics features and how to set them up from Python sample code.
USD Physics Schemas and PhysX SDK-engine-specific Physx Schemas
Explore further Omniverse simulation extensions.
Physics of Assets
The physics properties of assets are all well-defined using USD Physics Schemas and Physx Schemas . The documentation of Physics properties and how to access them in code is defined in C++, but you can follow these guidelines to find the equivalent calls in Python. For example, where generic names are used to represent an arbitrary API, the general usage is:
from pxr import Usd, UsdGeom, UsdPhysics, PhysxSchema
import omni.usd
stage = omni.usd.get_context().get_stage()
prim = stage.GetPrimAtPath("/Path/To/Prim")
physics_api_prim = UsdPhysics.SomePhysicsAPI(prim)
physx_api_prim = PhysxSchema.AnotherPhysxAPI(prim)
# Check if the API is Applied, if not, Apply it.
if not physics_api_prim:
physics_api_prim = UsdPhysics.SomePhysicsAPI.Apply(prim)
physics_attr = physics_api_prim.GetSomePhysicsAttr()
physx_attr = physx_api_prim.GetPhysxAttr()
# Check if Attribute is authored, otherwise create it
if not physics_attr:
physics_attr = physics_api_prim.CreateSomePhysicsAttr(1.0)
print(physics_attr.Get())
physics_attr.Set(10.0)
In some cases, you may need to have additional parameters when casting the Prim to a given API, for example, Joint State does require the joint type (“Prismatic”, or “Angular”, for instance). In these cases the C++ signature will contain a “TfToken” type. Replace it with a basic string and it should work in Python.
If you need to know the attribute name of some physics attribute you see on the UI, Hover over the attribute in the properties panel, and it will show its name in the tooltip. The attribute name standard is schema_name:attribute_name, so for example something like physics:velocity on a rigid body means it’s using the Physics Rigid Body API and the attribute name is velocity, so the corresponding attribute getter would be UsdPhysics.RigidBodyAPI(prim).GetVelocityAttr().
Simulation Timeline
Simulation time differs from real-time. Depending on system configuration and the size of the simulated environment, each time step may be computed faster or slower than the time it’s simulating, resulting in a warped speed if results are presented sequentially (often, physics simulation in Isaac Sim is faster than real-time). To mitigate this, Isaac Sim is configured by default with a limiter to match real-time speed.
Moreover, the simulation may run at a faster pace than rendering, meaning there may be more than one simulation time-step occurring in the background for every rendered frame. In the simplified example below, the simulation is set to run at 120 time steps per second, while rendering is set to 60 frames per second, resulting in two physics steps per rendered frame:
Note
The physics step time doesn’t necessarily coincide with system time (from the simulation start). In cases where the simulation can run faster than real-time, it’s possible to run an accelerated version of the simulation in a timeline without rendering or frame-rate blocking.
Ideally, simulation and rendering would match or be multiples of each other, but when this isn’t the case, each rendered frame may contain an uneven number of simulation timesteps. For example, simulation set to 100 steps per second, rendering set to 30 frames per second, resulting in most render updates having 3 simulation steps but occasionally 4 in a frame.
There are three event streams on the timeline (among a few others, but these are notably the most relevant for Isaac Sim). You can subscribe directly to Simulation Events or to Frame update events, either pre or post-rendering. OmniGraph nodes are typically updated on a pre-render event, but there are ways to set them to update on different events, such as every physics step.
Configuring Frame Rate
The simulation rendering frame rate can be configured by adjusting the current stage metadata. In the Layer tab, select the Root Layer, and in the properties panel modify the Timecodes per second property.
Configuring Simulation Timesteps
Simulation steps per second are determined in the Physics Scene. If there’s no Physics scene in your stage, it uses the default value, which is 60 steps per second.
To add a Simulation Scene element:
Click on Create > Physics > Simulation Scene.
Select the Simulation scene.
In the Properties panel check the element Simulation Steps per Second.
For more details on other parameters in the Physics Scene, refer to Simulation Management.
Simulation Components
Prims in a USD stage do not have physics enabled by default, but you may add simulation properties through the UI or using Python scripts. The following creates an example scene to which elements are added as we progress through the basic physics object types.
To begin, create a new scene and add a ground plane to it: File > New, then Create > Physics > Ground Plane.
Rigid Body
This is the most basic element. Adding rigid body dynamics enables an element to be subject to gravitational acceleration and other external forces.
1, Add a container to use as our Rigid Body: Create > Xform.
2. Move it up to Z=10 in the properties panel.
3. To make it a rigid body, right-click on it in the stage, then Add > Physics > Rigid Body.
Verify that the Xform is now be a rigid body, although you may not see much because it has no visual meshes.
You can fix that by nesting a Cube in it:
Create > Mesh > Cube, and drag it into the Xform.
Ensure the cube’s Translate is set to [0,0,0.5].
After you’ve completed the same setup as the screenshot above, hit play and see what happens:
Review the following:
Notice how the Z position gets updated as the object falls - this is because we are highlighting the rigid body directly. Try again selecting the cube, and you’ll notice that it doesn’t change.
The cube falls straight through the ground. We need to let the simulation know it needs to collide with other objects.
Colliders
To make our rigid body collide, you must indicate to the simulation that you want it to. For that, there’s the Collider API.
Select the Cube prim, and click on the Add Button > Physics > Collider.
Run the simulation again and verify that the rigid body stops at the ground.
Colliders can also be added to non-movable objects. Let’s experiment:
Create a new cube and place it at Z=3.0.
Then change its scale to [2,2,0.01] to create a 2x2 meter platform.
Add the collider to it just like before, without adding the Rigid body.
Play the simulation again, and verify that this is the result:
Raise the Xform position to Z=80.
Play the simulation again.
With this example, you are solving some of the common issues of physics simulation. Because time is discretized, if objects move too fast, during one time-step the object is above the platform, and in the next it has completely passed through it, with no collision captured. This doesn’t occur with the ground plane because it implements a “force field” that pushes penetrated objects towards the ground surface.
To remedy this, enable an option in the physics scene called Enable CCD (Continuous Collision Detection). CCD sweeps the object from one pose to the next. This option must also be enabled in the rigid body itself:
Select the Xform.
In the properties panel, enable CCD under the rigid body properties.
There are other ways to solve this issue, but for this scenario, this is the most effective.
Remember that collision has nothing to do with what you see on screen. For instance, you could hide the cube and the collider would behave the same, or you could add another cube or a sphere under Xform and it would have no effect unless you apply the Collision API to it.
Many object colliders are made using a composition of multiple mesh elements, giving it its shape and behavior. They work as a single rigid body even if they are physically separated on the stage, as long as they are all children of a rigid body.
Try adding and removing colliders to this rigid body or adding more rigid bodies to this scene and see how they behave.
Convex Hull
This next experiment with colliders removes the platform you added before and returns our Xform to Z=10.
Add a Torus mesh in the place of the platform at
Z=3.0and scale it to [5.0, 5.0, 5.0].Add > Physics > Rigid Body With Colliders Preset.
Run the simulation.
The Cube sits on top of the torus hole because the default approximation for mesh geometry is a convex hull. This is an approximation that the simulation engine can process efficiently, i.e. they are a good choice for performant simulations. We will review more complex, and therefore more computationally expensive approximations below.
To see the collision shape in use:
Click on the eye icon on the top-left side of the Viewport.
Show by type > Physics > Colliders > Selected.
Verify that green lines appear on the Torus.
This is a debug view of the collision shape.
You can also view a solid display of the colliders by opening the Physics debug menu: 1. Window > Simulation > Debug. 2. In the debug window, scroll to “Collision Mesh Debug Visualization”. 3. Check “Solid Mesh Collision Visualization”. 4 Verify that when you select the torus, its shape displays solidly.
|
|
Convex Decomposition
At a small expense, the torus collider can have the hole by a composition of convex shapes. This composition can be:
manually created by adding multiple shapes
computed with Physics Convex Decomposition
Select the Torus.
In the properties panel, scroll down to the Collision section, and select Convex Decomposition from the drop-down.
By opening the Advanced tab, you can adjust the parameters until you find a decomposition to your satisfaction.
Note
Fewer convex hulls typically results in higher performance.
In the Simulation Debug tab, you can also increase the Explode View distance to split the collider shapes and better understand how the composition is made.
The Collider drop-down contains more options to explore, like Bounding Cube and Sphere - the cheapest collisions possible, and a mode “Sphere Approximation”, which is similar to Convex decomposition but directly uses a group of spheres instead of conforming meshes.
Note
While triangle mesh and mesh simplification are not supported by rigid bodies and fall back to convex hull, it is possible to use a triangle mesh geometry directly on a rigid body by adding a signed-distance field to it; select SDF Mesh in the approximation drop-down to do so.
For more details on Rigid Bodies and Colliders, check Rigid Body Simulation and Collision Settings.
Contact and Rest Offset
In the Collider Advanced tab, there are two more parameters that can be important tuning parameters when there are collision issues, in particular with small and thin objects.
The Rest Offset can be tuned to inflate or shrink the collision geometry set; it can be useful to adjust in cases where the visual mesh is larger or smaller than the collision geometry so that the collision locations are consistent with the visual representation.
The Contact Offset dictates how far from the collision geometry, irrespective of Rest Offset, the simulation engine starts generating contact constraints. The tradeoff for tuning the contact offset is performance vs. collision fidelity: A larger Contact Offset results in many contact constraints being generated which is more computationally expensive; a smaller offset can result in issues with contacts being detected too late, and symptoms include jittering or missed contacts or even tunneling (see notes on CCD above).
Contacts and Friction
Besides making sure that object do not interpenetrate, collisions can transfer or dissipate energy as modeled by restitution and friction.
The parameters for the contact model are available in Physics materials. To create a Physics material:
Go to Create > Physics > Physics Material.
Select Rigid Body Material.
Physics materials are typically assigned to Collider Geometry but behave analogous to USD render materials otherwise; see Physics Materials for a full explanation of USD material resolution logic. For example, you may assign different materials to different collision geometry of a rigid body, or you may assign a material to the rigid body prim and configure it to override any materials set on the collider children.
To assign a physics material:
Select the collider prim.
Scroll to the Collider settings.
In Physics Materials on Selected Models, select the desired material. The list only allows picking materials that have physics properties.
Note that you may also add a physics material to a render material with “Add” > Physics > Rigid Body Material and assign the material in the render material section; the physics properties will be picked up.
Compliant Contacts
You may configure the rigid material to produce compliant (i.e. spring-damper) contact dynamics in the Advanced tab. This may be useful for approximating deformable bodies with rigid bodies.
Combine Modes
Because contacts are an interaction between two bodies, each contact parameter is not enough to describe how this interaction plays out. Just like in the real world, one surface material property may dominate the interaction or they may seamlessly combine into an average value. To replicate that, friction, restitution, and compliant-contact damping have a configurable combine mode field. Because both sides of the contact have this combine mode, the precedence of the combine mode matters:
The lower in the drop-down, the lower the priority of a mode in a combine mismatch resolution; so average < min < multiply < max.
For example, if Collider A has a friction combine mode average while Collider B has min, their interaction resolves as the minimum friction between the two. If a body C with combine mode max contacts A and B, the friction between A and C are resolved with max, as well as B and C.
Joints
Robots are typically composed of multiple jointed rigid bodies. Joints create constraints between two bodies. In the following, you use a Revolute Joint, but the steps are similar for other joint types, see a list in Joints.
You must configure the relative pose of the joint frames for each body to be jointed. Find more details, in particular the local scaling aspect of joint frames in the Joint Frames Section.
Note that when creating a joint through the UI, the joint’s frames are set to match the pose of the second rigid body selected for the creation.
Now create a joint as follows:
Select first the Xform rigid body, and then the Torus rigid body.
Go to Create > Physics > Joints > (Joint Type).
For this tutorial, use the Revolute Joint type. Because the Torus was selected second, the joint is at its center.
You will notice a circle on-screen, representing the origin and range of motion for the joint. If you start the simulation now, the Torus and Cube fall together. When the torus hits the ground, the cube stops moving. It’s in a stable position, but if you nudge it, it moves down in a circular pattern. Interact with the cube by pressing shift and left-clicking the cube.
Check the properties panel and review the following attributes:
Body 0: /World/XForm
Body 1: /World/Torus
These are the Poses relative to the bodies. You will notice that Position 0 is Z=-7.0.
Position 0:
[0, 0, -7.0]Rotation 0:
[0, 0, 0.0]Position 1:
[0, 0, 0.0]Rotation 1:
[0, 0, 0.0]
Note
When setting up joints that are part of an articulation, make sure that Body 0 will be the parent of Body 1 in the articulation-tree hierarchy. This way, joint-related quantities like link incoming joint forces or joint drive targets have a one-to-one correspondence in the PhysX SDK and USD.
Joint Axis
A revolute joint provides one degree of freedom and you may choose what axis of the joint frames is free. By default, the X axis is selected. You can change that in Properties, under the Revolute Joint section.
Joint Limits
The joint limits determine how far the joint can move from its original position. By default, when a joint is created, it comes without limits. With the joint selected, scroll down in the Properties panel and modify the Lower Limit and Upper Limit under the Revolute Joint section. Remember that USD uses degrees, not radians to represent angles.
Adding a Joint Drive
You may control the position and velocity of the degree of freedom that the joint added using a Joint Drive. You can do that by clicking the Add Button > Physics > Angular Drive. For details on configuring a joint drive, refer to Tuning Joint Drive Gains.
Articulation
An articulation is an optimized simulation structure for jointed bodies that provides superior performance, fidelity, and features for robotics. There are some limitations regarding topology (loop-closing) and joint support, which you can learn about in Articulations.
Stepping an OmniGraph with Physics
To guarantee one graph step per physics step at the moment it happens, you must use a modified version of an OmniGraph.
Create a new Action Graph through Create > Visual Scripting > Action Graph.
Select the created graph on the stage and in the Raw USD Properties section, in the pipeline stage, select PipelineStageOnDemand.
On the Action Graph window, search for On Physics Step. Drag and Drop it on your OmniGraph.
Continue your OmniGraph as usual.
Simulation Residuals
The physics simulation provides a metric to check how well it converged to a solution, i.e. how well it resolved constraints. To check for this result there is another API that can be applied to a few physics elements.
To check the Residuals:
Click on the selected physics element.
“Add” > Physics > Residual Reporting.
Verify that you can see the Residual plot over time on the Simulation Data Visualizer: (eye icon on viewport) > Show by Type > Physics > Simulation Data Visualizer.
The types of Physics Objects that report residuals are Simulation Scenes, the Articulation Roots, and Joints.
Physics Static Collision Extension
The Physics Static Collision Extension Extension is used to visualize collision meshes. Use this Utility extension to add static collision APIs to an entire Stage. The extension can also be used to remove all physics related APIs for testing purposes.
This extension is enabled by default. If it is ever disabled, it can be re-enabled from the Extension Manager by searching for omni.isaac.physics_utilities.
To access this Extension, go to the top menu bar and click Isaac Utils > Physics Utils.
Note
Dynamic objects are currently not supported.
User Interface
The User Interface provides options to add or clear static collision on selected static objects.
Configuration Options
Apply to children: Recursively create collision on all selected children; otherwise, create collision for just the selected object.
Visible only: Ensure the prim is visible before creating collision. (Ignores hidden prims)
Collision Type: Type of collision approximation to use
Apply Static: Applies collision to the current selection.
Remove Collision API: Clears the collision from the current selection.
Remove All Physics APIs: Remove all Physics-related APIs (including collision) from the current selection.
Enable Visualization
To visualize collision in any viewport:
Select: the
eye icon.Select: Show by type.
Select: Physics Mesh.
Check: All.
Note
Enable visualization after collision APIs have been applied or removed. Otherwise there will be a loss in performance while the extension traverses the desired subtree.
Omniverse™ Physics and PhysX SDK Limitations
This section provides a table of known limitations with the PhysX engine and workarounds.
Feature |
Description of Limitation |
Recommended Workaround |
IsaacSim GPU RL Perf Impact |
Perf Impact Reason |
|---|---|---|---|---|
Custom Geometry GPU pipeline contact reports |
Custom geometry contact data is not made available via the tensor API GPU contact information getters. |
Use GPU native collision approximation like convex hull, SDF tri-mesh, base geoms like sphere/box/capsule. Disable custom geometry cylinders and cones in physics stage settings. |
Yes |
Contacts are generated by CPU custom geometry callbacks |
Custom Geometry collision against GPU Features |
Custom geometry does not collide with GPU features (particles and deformable bodies), and may have poor collision quality against SDF tri-mesh colliders. |
Use GPU native collision approximation like convex hull, SDF tri-mesh, base geoms like sphere/box/capsule. Disable custom geometry cylinders and cones in physics stage settings. |
Yes |
Contacts are generated by CPU custom geometry callbacks |
Particles and deformable body contact reports |
Particles and deformable body do not support contact reports |
NA |
No |
|
Simulation-resume determinacy |
Replaying a simulation from an in-contact simulation state saved out in the middle of a simulation run can be nondeterministic. The PhysX SDK is using internal contact state that can persist over multiple simulation steps and that cannot currently be serialized and recovered from USD data. |
Restart simulations from the beginning to achieve determinism. |
No |
NA |
Conveyor Belts / Kinematics with nonzero velocity |
|
Do not use the conveyor belt feature with particles, deformable bodies, or SDF tri-mesh collision geometry. Use rigid bodies with non-SDF collision geometry instead. |
Yes |
The conveyor belt feature will trigger a CPU code path, so it is best to avoid for maximum GPU pipeline performance. |
Isosurface |
Isosurface may be leaking memory |
Do not use isosurface feature if memory leaks are an issue. It is a render-only feature and does not affect underlying fluid simulation. |
Yes |
Memory leak will lead to out of memory (OOM) |
Deformable Bodies and Particles static friction |
Static friction is not supported. |
NA |
No |
|
Deformable Bodies and Particle Friction Combine Mode |
Friction combine mode is not supported. Any interaction is using the dynamic friction set on the particle/deformable actor. |
NA |
No |
|
Particles simulation |
Particles can roll off flat collision surfaces that are perpendicular to gravity due to solver ghost forces. |
NA |
No |
|
Articulation Tendons |
The simulation fidelity and behavior can be inadequate. Articulation joint incoming force reported will be excessively high when using nonzero TGS velocity iterations. |
Fixed tendons: Use the Mimic Joint feature. Spatial tendons: Apply external forces to the links to mimic the spatial tendons. Force sensing: Use TGS with zero velocity iterations. |
No |
|
GPU Convex Hull Vertex/Face Limit |
For performance/memory footprint, the SDK limits GPU-compatible convex hulls to 64 vertices and faces. This can lead to poor approximation quality to the asset collider tri-mesh. |
Use convex decomposition or SDF tri-meshes to capture details better. Be aware that a convex decomposition with the same set of vertices as a single convex hull may not produce the exact same behavior because contact detection runs on each convex independently. |
Yes |
Switching to convex decomposition or SDF will have a simulation perf impact due to higher computational cost. |
Spherical Articulation Joints on links with nonidentity center-of-mass transform (cmasslocalpose) |
Joint limits and drives for spherical articulation joints may not respond correctly in certain joint state ranges if the articulation joint is setup on a link with nonidentity mass frame setup (cmasslocalpose) |
If possible, transform asset such that prim and mass frame coincide such that an identity transform can be used for the mass frame. |
No |
|
TGS Velocity Iterations |
|
|
No |
|
D6 Joint Drive |
D6 Joint Drive does not behave exactly as expected when the TGS solver is employed. |
If there are drive behavior issues then use PGS instead. |
No |
|
D6 Joint Drive |
D6 Joint Drive does not work well with a combination of TGS and velocity iterations. |
When TGS is employed it is recommended to focus computational effort on position iterations and to have zero velocity iterations. |
No |
|
Articulation Link Force Sensors |
The force sensors are deprecated and will be removed in a future version. They reported incorrect/implausible values for many use cases. |
Use contact force reports or link incoming joint force instead. |
No |
|
Articulation Joint Solver Forces |
The joint solver force reporting is deprecated and will be removed in a future version. The reported forces were incorrect/implausible. |
Use link incoming joint force reporting instead. |
No |
|
Particle Cloth |
Particle cloth will be deprecated when replaced with surface deformable bodies when they are ready. We don’t have an ETA for this yet. |
Do not use particle cloth. |
No |
|
Articulation Loop-closing using D6 Joints |
We have seen issues with loop-closures using D6 Joints where the simulation goes unstable, mainly using GPU simulation. |
Try increasing simulation time steps per second on the scene (i.e. decrease the simulation time step), try increasing articulation solver iterations, and try PGS and TGS solvers. |
No |
|
Articulation joint friction |
There are reports of differing effective joint friction between PGS and TGS solver. The friction model may not be suitable for all applications,see details in the API documentation. |
A more common velocity-proportional dynamic friction model can be implemented using a joint drive with zero target velocity and a suitable damping parameter. |
No |