Agent Control

Overview

Omni.Replicator.Agent (ORA) supports controlling two different agents:

human characters
Isaac Nova Carter autonomous mobile robots

The behaviors of the agents can be randomized by ORA.

This section discusses supported commands for each agent and the commands for randomization.

The ORA UI provides command editors under the agent setting panel, so you can edit the commands for each agent and control the behaviors for the agents in the simulation. Additionally, the ORA UI allows you to inject commands into a running simulation to control the agents during runtime.

Character

The Character Behavior control is based on the Omni.Anim.People extension. The GoTo, Idle, and LookAround commands can be randomized. Characters can also Sit and Queue, but these commands require a manual input. For a more detailed guide, refer to the Omni.Anim.People documentation.

GoTo

GoTo moves the character to a location. It can be followed by a single point or a sequence of points, where the last point specifies the ending rotation that the character must have upon reaching its destination. If you would prefer to not set a rotation, you can use “_” for the ending rotation.

When a sequence of points is given, the character crosses every point. If Navmesh Navigation and Dynamic Obstacle Avoidance is on, then the character tries to avoid static and dynamic obstacles.

Example: female_adult_police_03 GoTo 10 10 0 90

Idle

Idle makes the character stand still. It takes a duration value in seconds and performs the action for that duration.

Example: female_adult_police_03 Idle 10

LookAround

LookAround makes the character stand in the same spot, while moving its head from left to right. It takes a duration value in seconds and performs the action for that duration.

Example: female_adult_police_03 LookAround 10

Robot

Robot behavior control is supported by the Omni.Anim.People and Omni.Isaac.Wheeled_Robots extensions. The GoTo and Idle commands can be randomized by ORA. The iw.hub robot supports the LiftUp and LiftDown commands, which both require manual input.

GoTo

The GoTo command moves the robot from its current location to a target location. The command expects the X Y Z world coordinate of the target location.

Example: Nova_Carter_01 GoTo 10.02 5.9 0

Idle

The Idle command makes the robot stay where it is for a given duration in seconds. It can be combined with the GoTo command to make the robot go to a position, stay there for a while, and then go to another position.

Example: Nova_Carter_01 Idle 5

LiftUp/LiftDown

The iw.hub robot also supports the LiftUp and LiftDown commands, which move the loader on the iw.hub up or down by 4 cm.

Example: iw_hub_01 LiftUp

Command Randomization Control

ORA allows you to add custom commands with custom animations, as well as customize the randomization of these commands. To add new commands, refer to Custom Command. For every new command added, ORA automatically adds an entry to the Command Randomization transition matrix.

You can control the possibilities of command randomization by defining the command transition matrix. The command transition matrix determines the possibility of transitioning to a state from a state. Each row of the transition matrix represents, under the current command (row head), what the possibility of going to the next command is.

To do so, go to the Command Randomization tab under the Character Settings panel. You will see the current transition matrix. Once the transition matrix is edited, it must be save in order to take effect.

For example, in the following transition matrix, a GoTo command has a 50% chance of being followed by an Idle command and 50% chance of being followed by a LookAround command. A LookAround command has a 70% chance of being followed by a GoTo command and a 30% chance of being followed by Idle.

                  GoTo        Idle  LookAround
      GoTo           0         0.5         0.5
      Idle         0.6           0         0.4
LookAround         0.7         0.3           0

Note

The sum of probabilities for each row must be equal to 1.

Command Injection

ORA allows you to inject commands into a running simulation to control the agents in real time. This enables you to adjust the simulation during runtime to create emergent events or simulate a particular scenario.

Command injection works similar to regular commands. The format is agent_name command params and there is one panel for all the agents. This feature only works while the simulation is running. To inject commands, type the commands in the Inject Control UI panel, which is the last panel in the ORA UI, and then click the Inject button.

When a command is injected, the agent stops whatever command it is currently executing and starts the injected command immediately.

Note

If a character is performing the Sit command and a new command is injected, the character stands up first and then executes the injected command.
If a character is in a queue when an Idle command is injected, the character appears to stay in the queue but the Queue command is already interrupted.

Below is an image of the command injection UI. It is injecting a GoTo 1 1 0 _ command to Character_01 and an Idle 10 command to Nova_Carter_02.