Lula Robot Description and XRDF Editor
Learning Objectives
This tutorial shows how to use the Robot Description Editor UI tool to generate a configuration file that supplements the information available about a robot in its URDF. Currently two motion generation packages leverage the Robot Description Editor to specify necessary configuration information: cuMotion and Lula.
This tutorial describes the motivation for needing specific config files for Lula and cuMotion algorithms, and goes over the minimal set of data that needs to be written into a robot description file for each available Lula algorithm.
This tutorial then shows how to use the Robot Description Editor UI tool to automatically write the appropriate information into (or edit) a robot_description.yaml file for Lula or an XRDF file for cuMotion.
The Robot Description Editor is used on a stage that already has an Articulation on it. To follow along with the steps in the tutorial exactly, it is best to open a single asset by reference. I.e. drag and drop a USD file onto an empty stage rather than clicking on the USD file to open it directly.
What is in a Robot Description File?
A Robot Description File is the main configuration file that is required along with the robot URDF to use all Lula algorithms. Creating a robot_description.yaml file is the first and most time-consuming step that a user must take when hoping to use Lula algorithms on a new robot.
Defining the Robot C-Space: Active and Fixed Joints
A key aspect of a Robot Description File is defining the robot c-space. For example, suppose we have a 7 DOF robot manipulator such as the Franka arm with an attached 2 DOF gripper. In the robot URDF file, there are a total of 9 non-fixed joints that could be considered controllable. However, the set of Lula algorithms (RMPflow, Lula RRT, Lula Trajectory Generator, etc.) are designed to move the robot into position but not to control the end effector. In a typical use-case, we might use RmpFlow to move the robot end effector into position above a block and then separately open and close the gripper.
A Robot Description File must distinguish each joint as either an “Active Joint” or a “Fixed Joint”. Anything marked as an “Active Joint” will be directly controlled, while anything marked as a “Fixed Joint” will be assumed to be fixed from the perspective of Lula algorithms. In the case of using RmpFlow on the Franka robot, the seven joints in the Franka’s arm are marked as “Active Joints”, and the gripper joints are marked as “Fixed Joints”.
In the Robot Description Editor, positions must be selected for both active and fixed joints. The positions of “Active Joints” are taken to be default positions. When RmpFlow is not given any target, it will move the robot towards the default position. And when it is given a target, it will use the default positions of the “Active Joints” to resolve null-space behavior; i.e. there are many ways for a 7 DOF robot to reach a single target, and RmpFlow will be biased towards a c-space position that is close to the default position.
There is no way of telling RmpFlow that the “Fixed Joints” are in any other position than the position written into the Robot Description File, and as such it is important to choose a reasonable value for the positions of fixed joints. In the Franka example, the gripper joint positions are given a fixed value corresponding to the gripper being open, as this best facilitates RmpFlow avoiding collisions between the gripper and obstacles no matter the gripper state (when closed, the gripper fingers are inside the convex hull of an open gripper).
Collision Spheres
Lula algorithms use a custom configuration in order to perform efficient collision avoidance. For a given robot, a set of collision spheres must be defined that roughly cover the surface of the robot. Lula algorithms will not allow any collision sphere defined in the Robot Description File to intersect any obstacle in the USD world. The Robot Description Editor provides multiple tools that allow the user to quickly define a complete set of collision spheres for any robot.
What is the difference between a Robot Description File and an XRDF file?
An XRDF file is the main configuration file that is required by cuMotion for a specific robot, and it contains a superset of the data in a Lula Robot Description File. The Robot Description Editor can be used to generate an XRDF file that contains the minimal data required to start using cuMotion. The use of the Robot Description Editor need not change in any way when configuring a robot for use with cuMotion versus Lula. In the future, Lula will fully support XRDF files and deprecate Robot Description Files.
As of Isaac Sim 4.0.0, the Robot Description Editor was modified to support `XRDF` files, and some parts of this tutorial reference UI components that have changed.
What Information is Required for Each Lula Algorithm?
Different Lula algorithms require different levels of completion of the Robot Description File. Every algorithm requires the user to appropriately choose active and fixed joints. However, collision spheres are only necessary to configure when using algorithms that perform collision avoidance with external obstacles. For example, the Lula Kinematics Solver is purely kinematic, and it does not interact with the outside world. As such, the collision sphere representation may be ommitted from the Robot Description File. RMPflow can function without any collision spheres being defined, but it will not be able to avoid obstacles.
Using the Robot Description Editor
This section of the tutorial includes brief text descriptions of the different panels in the Robot Description Editor UI tool, and a more detailed walkthrough is presented in video form to capture the interactive nature of the extension. If the generated robot description file appears to be causing undesirable results, it will be worth the time to watch the tutorial videos thoroughly to understand each field.
The Robot Description Editor is not compatible with Instanceable Assets, but a Robot Description File generated for an asset that was later converted to an instanceable asset will still work on the instanceable asset.
Getting Started
The Robot Description Editor can be found from the tool bar under Isaac Utils -> Lula Robot Description Editor. To get started, open the USD file of your chosen robot and click the “Play Button” on the left-hand side.
In the Selection Panel, once a robot is on the stage and the stage is playing, a drop-down menu will populate where your robot can be selected. Select the prim path of your robot Articulation from the “Select Articulation” field. Once this is done, another drop-down labeled “Select Link” will populate with the names of each link in our robot. This will be needed later as we use the tool.
We have done everything we need to do to start making our Robot Description File. Other panels will populate with robot-specific information, and we can move on to the Command Panel.
Command Panel
As of Isaac Sim 4.0.0, Command Panel was renamed to Set Joint Properties, and fields were added to each joint for jerk and acceleration limits.
Once the robot Articulation has been selected from the “Select Articulation” menu, the Command Panel will expand and populate. The Command Panel requires the user to supply critical information for the Robot Description File to be properly generated. Watch the included video or carefully read Defining the Robot C-Space: Active and Fixed Joints.
In the Command Panel select a “Joint Position” and a “Joint Status” for each joint in the robot Articulation. Keep in mind the following:
Joints are marked as “Active Joints” if and only if the user intends for that joint to be directly controlled by Lula algorithms. Typically this involves marking each joint in the robot arm as active while leaving the joints in the manipulator attached to the arm as “Fixed Joints”. At least one joint must be marked as an “Active Joint”.
The joint positions of “Fixed Joints” can matter, depending on the use-case and are worth some thought. The positions of “Fixed Joints” will be assumed by Lula to be truly fixed; i.e. there is no way override the positions at runtime.
The positions of “Active Joints” are considered to be the default configuration of the robot. This default configuration is used by a subset of Lula algorithms, with the main case being RmpFlow. A default configuration should be chosen that is in front of the robot (along the +X axis by convention in Isaac Sim) and is not near any joint limits.
Adding Collision Spheres
Collision spheres are added to the robot one link at a time. The user may select the link of interest from the “Select Link” field of the Selection Panel. The Link Sphere Editor panel contains functions that are within the scope of the selected link such as adding spheres, scaling spheres, and clearing spheres only within the link. The Editor Tools panel contains functions that are outside the scope of the selected link such as “Undo” and “Redo” buttons, changing the color of collision spheres, and toggling the visibility of the robot.
When spheres are added to a link, they are added to the USD stage as a prim that is nested under the selected link. The user may click on and modify any sphere by moving it around on the stage or changing its radius. The position of a sphere relative to the origin of the link that contains it is written as a fixed value into the Robot Description File.
There are three main ways to add a sphere to a link:
Add Sphere: Add a single sphere with a specified relative translation from the origin of the link. This translation can be easily changed after creation by modifying the sphere prim.
Connect Spheres: Select two spheres that have already been created under a link and connect them with a specified number of spheres in between. The locations and sizes of the connecting spheres are interpolated to best fill the volume of the cone-section defined by the two spheres being connected.
Generate Spheres: Select a mesh that defines the volume of the link, and automatically generate a set of N spheres that best fill the volume of the mesh. When a number of generated spheres is specified, a preview of the generated spheres will automatically appear, which can be finalized by clicking the “Generate Spheres” button. Any visible robot must will at least one mesh defining its link. When there are more than one mesh, it is best to try each of them to figure out the minimal set of spheres that can be generated for good coverage. It is typically better to “Connect Spheres” by hand for links with simple cylindrical shapes.
Exporting Configuration Files
Lula Robot Description File
After completing the Command Panel and creating a collision sphere representation of the robot, the Robot Description File may be exported under Export To File -> Export to Lula Robot Description File. A file path to your local machine must be selected with a file name ending in .yaml. The “Save” button will become enabled when a valid file path has been typed.
XRDF File
After completing the Command Panel and creating a collision sphere representation of the robot, an XRDF file can be generated under Export to File -> Export to cuMotion XRDF. The file path must end in .yaml or .xrdf. The “Save” button will become enabled when a valid file path has been typed. When exporting an XRDF file, the Robot Description Editor has the following behavior:
Create a single collision group that is used for both collision and self_collision that uses the spheres created in the editor.
Under self_collision, set each link to ignore both its parent, and other links that have the same parent.
Do not write Tool Frames
Do not write Modifiers.
Because XRDF files can contain more information than is represented in the Robot Description Editor, it is possible to merge the data in the Robot Description Editor with an existing XRDF file. By selecting a file path to an XRDF file that already exists, an option will appear to Merge With Existing XRDF. When merging with an existing XRDF file, the Robot Description Editor has the following behavior:
Copy Tool Frames from the existing file.
Copy Modifiers from the existing file.
Copy self_collision -> ignore from the existing file if self_collision -> geometry matches collision -> geometry.
Copy collision spheres from the existing file for any frames that were not represented in the Robot Description Editor.
Importing Configuration Files
Lula Robot Description File
A pre-existing Robot Description File may be imported into the editor under Import From File -> Import Lula Robot Description File. Importing will overwrite all information in the Robot Description Editor.
XRDF File
A pre-existing XRDF file may be imported under Import From File -> Import XRDF File. The Robot Description Editor imports XRDF files with the following behavior:
The format version is assumed to be compatible with version 1.0.
Only the collision group spheres are imported.
Modifiers are not used.
Tool Frames are not used.
The self_collision group is not used.
Importing will overwrite all information in the Robot Description Editor.
Summary
This tutorial shows how to use the Lula Robot Description Editor to efficiently generate a Lula Robot Description File. This covers most or all of the configuration information required for different Lula algorithms!
The Robot Description Editor also supports XRDF files for use with cuMotion.
Further Learning
Now that we’ve seen how to generate Robot Description Files, it’s time to get our robot moving around with Lula algorithms. See: