Surface Gripper Extension

About

The Surface Gripper Extension Extension is used to create a suction-cup gripper behavior for an end-effector. It works by creating a Joint between two bodies when the parent body is close enough to the child at the gripper point of contact.

This extension is enabled by default. If it is ever disabled, it can be re-enabled from the Extension Manager by searching for omni.isaac.surface_gripper.

API Documentation

See the API Documentation <../../../../py/isaacsim/source/extensions/omni.isaac.surface_gripper/docs/index.html> for usage information.

Tutorials & Examples

The following example showcases how to best use this extension:

Examples

  • Surface Gripper Example: Isaac Examples > Manipulation > Surface Gripper

Surface Gripper Example

To run the Example:

  1. Go to the top menu bar and click Isaac Examples > Manipulation > Surface Gripper

  2. Press the Load Scene button.

  3. Press the PLAY button to begin simulating.

  4. Toggle the Gripper State button to CLOSE to attach the cone to the cube. The cone will turn RED on successful attachment.

  5. Adjust the Gripper Force and Gripper Speed sliders to change the force intensity at the Cone and press the respective APPLY`. Excessive forces will break the gripper attachment, and the cone will go back to GREEN when the surface gripper is open.

../../_images/ext_isaac_surface_gripper.webp

Surface Gripper Properties

Property

Description

Parent Path
The rigid body that will contain the gripper

Offset
local Offset transform from the origin of the parent. The Gripper will attempt to close on the X direction of the offset pose.

Grip Threshold
Distance from the gripper point that will accept closing contact

Force Limit
Force that the gripper can hold before it will break the constraint

Torque Limit
Torque that the gripper can withstand before it will break the constraint

Bend Angle
Angle that the surface gripper can bend when a load is applied

Stiffness
Bend Stiffness

Damping
Bend Damping

Retry Close
When flag is on, the gripper will attempt to close until it is sucessful in grabbing an object, otherwise it will attempt once and stop.

Omniverse OmniGraph Node

The Surface Gripper extension provides a UI implementation through Omniverse OmniGraph. To use it, select the rigid body prim to attach the surface gripper, and click on Create > Isaac > Grippers > Surface Gripper. This will create the surface gripper action graph.

  1. Use the SurfaceGripperOffset prim to define the gripper placement

  2. Change the Surface Gripper properties in the SurfaceGripperNode on the properties panel.

Note

The body selected must have the Rigid Body API applied to it, otherwise it will result on an error and the gripper won’t be created on simulation.

Tutorial: Creating a Surface Gripper Using OmniGraph

Let’s begin by Creating a new, empty stage, and add a ground plane to it with on File > New, then Create > Physics > Ground Plane.

Next, we will create two cubes to serve as our rigid bodies: Create > Mesh > Cube. Move one of the cubes above the other, Select both, and on the Properties pane click the + Button, and add Physics > Rigid Body with Colliders Preset.

Now, select the box on top, and go to Create > Isaac > End Effectors > Surface Gripper. This will generate an Omniverse OmniGraph that you can use to control the surface Gripper. Inside the Omniverse OmniGraph, you will find three Prims: impulse_monitor, SurfaceGripperNode, and SurfaceGripperOffset.

../../_images/ext_isaac_surface_gripper_og_1.png

The impulse_monitor is a trigger event that gets called whenever there’s an update to the surface gripper condition (e.g the grip is broken). SurfaceGripperNode is the Node that is used to configure the gripper, using the parameters above. Finally, SurfaceGripperOffset is the placement of the surface gripper on the rigid body it is attached to. The gripper will always attempt to close in the offset’s X direction. In our example we want it to be facing down, so we rotate the offset by 90 degrees on the Y axis.

Note

To view the appropriate offset direction, change the gizmo display to show local coordinates by either clicking in the coordinates button , or pressing W. In local Coordinates mode, it will have an orange core transform_ico

Now, the gripper needs to be placed outside of the body’s Collision box, otherwise it won’t be able to detect the connecting bodies, and instead it will attempt to connect to itself. If that happens, a warning message will appear, showing by how much you must offset the current position so you clear out of the collision. While it will still work if you don’t do so, it will incur in additional compute time to find the collision border of the current object and move the grip point out of it at every closing attempt.

To control the gripper, you need to connect an upstream node to the Open and Close Triggers. The simplest way would be to connect an Impulse event - the same type of node that gets automatically added to the onStep. Open the SurfaceGripperActionGraph that was added inside the cube by selecting it, then right click and “Open Graph”. To add the node, search on the left bar for Impulse, or select the impulse from On Step and duplicate it with Ctrl-D, and connect it to the Close, and a second one to the Open triggers.

../../_images/ext_isaac_surface_gripper_og_2.png

To actuate With the simulation running, when you select the Impulse event, on the properties panel, select the State EnableImpulse and check it. It will blink and return to unselected. At this point the surface gripper will be closed on the other cube. If you have not moved the gripper position to the surface of the cube, you will receive the following message:

[Warning] [omni.isaac.surface_gripper.python] Surface Gripper is inside the parent Rigid body collider. please move it forward in the X offset direction by 0.500997 to avoid wasted computation. Move the offset by the given distance and it will be on the surface of the object.

../../_images/ext_isaac_surface_gripper_og_3.png

Note

Due to a known issue, when reusing the UI to Enable the Impulse, you’ll have to click on it three times to check, uncheck and check again without leaving the selection. This is not necessary if interacting with the node in code.

To interact with the Omnigraph through code, first get the prim at the node path, then set the enable impulse to 1, and call request_compute.

 1from pxr import Usd, UsdGeom
 2import omni.usd
 3import omni.graph as og
 4stage = omni.usd.get_context().get_stage()
 5
 6
 7ogn = og.core.get_node_by_path("/World/Cube_01/SurfaceGripperActionGraph/impulse_monitor_01") #Get the Impulse node connected to Close
 8
 9attr = ogn.get_attribute("state:enableImpulse") #Get the Attribute to enable the impuse
10attr.set(1) #Set it to "True"
11ogn.request_compute() # Trigger the node execution that will send an impulse to the surface Gripper.

Creating a Surface Gripper fully on code

This section describes how to implement a surface gripper completely from code. In this mode, no USD attribute is created to represent the surface gripper, and it exists only in the physics simulation. The code below was extracted from the Surface Gripper example code.

Surface Gripper Properties

Surface Gripper properties are defined within the _create_scenario function. It is during this step that you define what will be the surface gripper parent rigid body, and what is the offset to the parent’s origin. The offset is in the parents frame of reference. Once the properties are defined, you create a Surface Gripper object, and initialize it with the defined properties. All control for the gripper then is done through the gripper object. Let’s take a closer look at the different parameters of the Gripper Properties:

 1async def _create_scenario(self, task):
 2    # [...]
 3
 4    # Create the Gripper Properties
 5    self.sgp = Surface_Gripper_Properties()
 6    self.sgp.d6JointPath = ""
 7    self.sgp.parentPath = "/GripperCone"                # Set the Cone as the parent object
 8    self.sgp.offset = dc.Transform()                    # Offset the transform to the base of the Cone
 9    self.sgp.offset.p.x = 0
10    self.sgp.offset.p.z = -0.1001                       # Offset the transform to the base of the Cone
11    self.sgp.offset.r = [0.7171, 0, 0.7171, 0]          # Rotate to point gripper in Z direction
12    self.sgp.gripThreshold = 0.02                       # Set attachment to within 2cm of the base of the Cone
13    self.sgp.forceLimit = 1.0e2                         # Set other limits of the gripper ...
14    self.sgp.torqueLimit = 1.0e3
15    self.sgp.bendAngle = np.pi / 4
16    self.sgp.stiffness = 1.0e4
17    self.sgp.damping = 1.0e3
18    self.sgp.retryClose = False                         # If set to True, surface gripper will keep trying to close until it picks up an object
19
20
21
22
23    self.surface_gripper = Surface_Gripper(self._dc)    # Create the Gripper
24    self.surface_gripper.initialize(self.sgp)           # Initialize the Gripper with these Properties
25
26    # [...]

Update Gripper State

The Surface Gripper is updated on every simulation step in the _on_simulation_step function. This is mandatory to capture the event of the surface gripper breaking contact, so you can trigger the appropriate response on your code.

In the example we use it to update the gripper status, and then change the cone color to RED if it’s closed or GREEN if it’s open.

 1def _on_simulation_step(self, step):
 2
 3    # [...]
 4
 5    self.surface_gripper.update()                       # On every sim step, update the gripper status
 6    if self.surface_gripper.is_closed():                # Assign color to Cone based on current state
 7        self.coneGeom.GetDisplayColorAttr().Set([self.color_closed])
 8    else:
 9        self.coneGeom.GetDisplayColorAttr().Set([self.color_open])
10
11    # [...]

Controlling the Gripper

The Gripper State is controlled by the on_toggle_gripper_button_clicked function connected to the extension’s UI. It toggles the Gripper State between OPEN and CLOSE.

1def _on_toggle_gripper_button_clicked(self, button):    # When receiving a button press
2    if self._editor.is_playing():
3        if self.surface_gripper.is_closed():            # Toggle Open or Close
4            self.surface_gripper.open()
5        else:
6            self.surface_gripper.close()