Omniverse Radar Extension
Introduction
The Radar Sensor extension consists of RTX Sensor based models and omnigraph nodes for various post processing functions (ex: to transcode radar data in vendor specific formats)
The currently supported models are:
DmatApproxRadar (lower performance needs)
WpmDmatApproxRadar (higher fidelity)
Each of these Models has its own parameterization and at least one post processing omnigraph definition available that defines the nodes of the processing pipeline for the model with some default parameterization.
There can be multiple post processing omnigraphs for the same radar model, each can have its own parameter values and/or different nodes.
Under each model you can find details on parameterization.
Plugins
DmatApproxRadarPlugin
Introduction
The name is short for “Detection Matrix Approximation” radar model plugin. and it’s a model that emulates the behavior of real commercial radars by directly approximating the detection matrix or data cube that would be generated in these radars
The model aims to do this approximation using the Specs of the sensor normally provided in an OEM technical product specification. Since typically there is a huge number of parameters in a typical radar pipeline and often the processing stages in such a pipeline are OEM IP that can’t be shared, then approximating such pipelines with their specs can be advantageous.
This approximation aims to:
be real time.
be physically accurate to the extent needed for ADAS functions (ex: localization, fusion .. etc). it is not targeted for sensor validation or to be 100% matching to the real physical sensor.
capture important system-level effects (ex: merging detections at long range, resolution degradation with angle, cross talk between spatially different objects, multiple ghost detections .. etc.).
The output corresponds to simulating the radar as a black box, and producing raw radar detections.
Plugin Name
omni.sensors.nv.radar.dmatapprox.plugin
Model Parameters
The model supports 2 different scans (a near and far scan), each can have different parameterization. the full model params are listed in the table below
Parameter |
Description |
---|---|
NearScanMaxRangeM |
near scan max range in meters |
NearScanMaxVelMpsSequence |
near scan radial velocity sequences in meters per second |
NearScanMaxAzAngRad |
near scan max azimuth angle in radians (symmetry over bore sight implied) |
NearScanMaxElevAngRad |
near scan max elevation angle in radians (symmetry over bore sight implied) |
NearScanRangeSepM |
near scan range separation in meters |
NearScanVelSepMps |
near scan velocity separation in meters per second |
NearScanBoreAzSepRad |
near scan azimuth separation at bore sight in radians |
NearScanBoreElevSepRad |
near scan elevation separation at bore sight in radians |
NearScanElevMode |
near scan elevation mode (0 no elevation, 1 positive only, 2 full elevation) |
FarScanMaxRangeM |
far scan max range in meters |
FarScanMaxVelMpsSequence |
far scan radial velocity sequences in meters per second |
FarScanMaxAzAngRad |
far scan max azimuth angle in radians (symmetry over bore sight implied) |
FarScanMaxElevAngRad |
far scan max elevation angle in radians (symmetry over bore sight implied) |
FarScanRangeSepM |
far scan range separation in meters |
FarScanVelSepMps |
far scan velocity separation in meters per second |
FarScanBoreAzSepRad |
far scan azimuth separation at bore sight in radians |
FarScanBoreElevSepRad |
far scan elevation separation at bore sight in radians |
FarScanElevMode |
far scan elevation mode (0 no elevation, 1 positive only, 2 full elevation) |
NumRaysPerDeg |
num rays per degree |
NumExtraThreads |
num extra threads used (only relevant if Algo = 0) |
FrameDurationNs |
frame duration in nanoseconds |
WpmDmatApproxRadarPlugin
Introduction
The name is short for “Wave Propagation Model Detection Matrix Approximation” radar model plugin. The model uses the Wave Propagation Model (WPM) to enhance the fidelity compared to DmatApproxRadarPlugin, while still maintaining real time capability. New features are:
multiple bounces
support for different materials and improved radiometry
antenna gain patterns
Plugin Name
omni.sensors.nv.radar.wpm_dmatapprox.plugin
Model Parameters
The model supports multiple scans (e.g. near and far scan), each can have different parameterization. the full model params are listed in the table below
Parameter |
Description |
---|---|
WaveLengthMm |
Wave length in millimeters. Set to 3.9. |
TraceTreeDepth |
Number of bounces |
FrameDurationNs |
Duration of a frame in nanoseconds. Should match the sensor rate. |
InstanceTimeOffsetUsec |
Time offset for instances in microseconds. |
CfarMode |
Mode of CFAR: ‘2D’ for range-doppler. ‘4D’ for range-doppler-azimuth-elevation. |
AntennaGainMode |
Mode for antenna gain: ‘COSINEFALLOFF’ for a cosine falloff, ‘CONSTANT’ for constant gain over FOV. |
ConfigReloadIntervalFrames |
Number of frames after which configuration is reloaded, 0 for never. Currently not supported. |
RaysPerDeg |
Number of rays per degree in the scan. (Part of ScanCfgs) |
TimeOffsetUsec |
Time offset for this scan configuration in microseconds. (Part of ScanCfgs) |
PowerFactor |
Total output power of radar. (Part of ScanCfgs) |
BinsFromSpec |
Boolean flag to determine if bins are derived from specifications. (Part of ScanCfgs) |
EnAngAliasing |
Enable angular aliasing. (Part of ScanCfgs) |
DetValFromBinIdx |
‘true’: detection value derived from bin index ‘false’: more accurate inside cell (Part of ScanCfgs) |
ElevMode |
‘FULL_EL’ for full elevation tracking. ‘POS_EL’ only positive. ‘NO_EL’ no elevation.(Part of ScanCfgs) |
MaxRangeM |
Maximum scanning range in meters. (Part of ScanCfgs) |
MaxAzAngDeg |
Maximum azimuth angle in degrees. FOV is 2x MaxAzAngDeg (Part of ScanCfgs) |
MaxElAngDeg |
Maximum elevation angle in degrees. FOV is 2x MaxElzAngDeg (Part of ScanCfgs) |
RangeResM |
Range resolution in meters. Only works if BinsFromSpec is true. (Part of ScanCfgs) |
VelResMps |
Velocity resolution in meters per second. Only works if BinsFromSpec is true. (Part of ScanCfgs) |
BoreAzResDeg |
Azimuth resolution at bore sight in degrees. Only works if BinsFromSpec is true. (Part of ScanCfgs) |
BoreElResDeg |
Elevation resolution at bore sight in degrees. Only works if BinsFromSpec is true. (Part of ScanCfgs) |
RBins |
Number of range bins. Only works if BinsFromSpec is false. (Part of ScanCfgs) |
VBins |
Number of velocity bins. Only works if BinsFromSpec is false. (Part of ScanCfgs) |
AzBins |
Number of azimuth bins. Only works if BinsFromSpec is false. (Part of ScanCfgs) |
ElBins |
Number of elevation bins. Only works if BinsFromSpec is false. (Part of ScanCfgs) |
CfarRnT |
CFAR number of range test cells. (Part of ScanCfgs) |
CfarRnG |
CFAR number of range guard cells. (Part of ScanCfgs) |
CfarVnT |
CFAR number of velocity test cells. (Part of ScanCfgs) |
CfarVnG |
CFAR number of velocity guard cells. (Part of ScanCfgs) |
CfarAznT |
CFAR number of azimuth test cells. (Part of ScanCfgs) |
CfarAznG |
CFAR number of azimuth guard cells. (Part of ScanCfgs) |
CfarElnT |
CFAR number of elevation test cells. (Part of ScanCfgs) |
CfarElnG |
CFAR number of elevation guard cells. (Part of ScanCfgs) |
CfarMinVal |
CFAR minimum value for detection. Cells below that value will be discarded. (Part of ScanCfgs) |
CfarOffset |
Multiplied with raw CFAR noise level. Set to 1 for no offset (Part of ScanCfgs) |
CfarNoiseMean |
Mean noise level used in CFAR. Set to 0 for no mean noise. (Part of ScanCfgs) |
CfarNoiseSDev |
Standard deviation of noise in CFAR. Set to 0 for no gaussian noise.(Part of ScanCfgs) |
MaxVelMpsSequence |
Sequence of maximum unambiguous velocities in meters per second. (Part of ScanCfgs) |
RcsTuningCoefficients |
0:min threshold dBsm, 1:multiplier for rcs sm, 2:noise standard deviation dBsm. (Part of ScanCfgs) |
AzimuthRadNoiseSDev |
Standard deviation of azimuth radial noise in radians at bore sight. (Part of ScanCfgs) |
RangeNoiseMean |
Mean noise added to the range measurements. (Part of ScanCfgs) |
RangeNoiseSDev |
Standard deviation of noise added to the range measurements. (Part of ScanCfgs) |
Available Parameterizations
The following parameterization to support specific radars already exists
Radar Model |
Params File name |
---|---|
Generic |
Example.json |
Generic High Res |
Example_highRes.json |
The attributes are specified through a sensor config JSON file, which is placed in “data/sensors/radar/<senosr_type>”, where <sensor_type> is either dmat_approx_radar or wpm_dmat_approx_radar. Users can create custom parameterization files but they have to be placed in the same directory as the default parameterization files. The parameterization name is then the file name. This JSON file can be read with the ProfileReader utility plugin. The desired sensor profile is set via the USDA attribute “custom token rtxsensor:modelConfig”.
Parameterizing the model
Parameterization is specified through a sensor config JSON files, which can be found in “data/radar”. The desired param profile is set via the USDA attribute “custom token rtxsensor:modelConfig”. The parameterization name “Example” has to match one of the Available Parameterizations
Radar Point Cloud
The Radar extension uses the GenericModelOutput as output format. See the documentation of the common extension for more information on the GenericModelOutput.
Omnigraph Nodes
Currently there is only one node in the Radar extension. A transcoder which encodes the Radar data into vendor-specific UDP packets and dumps them to file.
TranscoderRadar
Introduction
The Transcoder node encodes the radar data stream into vendor specific udp packets and dumps them to file. Currently, the node supports the following formats:
GENERIC
Omnigraph Node Name
omni.sensors.nv.radar.TranscoderRadar
Parameters & Attributes
Parameter Name |
Description |
Value Type |
Value Range |
Default Value |
Example |
---|---|---|---|---|---|
remotePort |
The port to address |
int |
10001 |
||
remoteIP |
Remote IP of UDP channel |
string |
239.0.0.1 |
||
interfaceIP |
Local interface to use |
string |
127.0.0.1 |
||
format |
data send format |
string |
|||
serviceID |
ServiceMethod IDs in SOME/IP Header. Only for NCD formats |
int |
41610 |
||
fileName |
Name for DW bin file |
string |
empty |
Example Radar Pipeline
def RenderProduct "RadarGeneric"
{
def "PostRender"
{
def OmniGraph "DSPipeline"
{
custom token evaluator:type = "push"
custom token flatCacheBacking = "StageWithoutHistory"
custom token pipelineStage = "pipelineStagePostRender"
def OmniGraphNode "RadarSource"
{
custom token node:type = "omni.graph.nodes.GpuInteropCudaEntry"
custom int node:typeVersion = 1
custom string inputs:sourceName = "RtxSensorCpu"
custom uint64 outputs:cudaMipmappedArray
custom uint64 outputs:format
custom uint outputs:height
custom uint outputs:mipCount
custom uint64 outputs:stream
custom uint outputs:width
custom double outputs:simTime
custom double outputs:hydraTime
}
def OmniGraphNode "RadarMetaSource"
{
custom token node:type = "omni.graph.nodes.GpuInteropCudaEntry"
custom int node:typeVersion = 1
custom string inputs:sourceName = "RtxSensorMetadata"
custom uint64 outputs:cudaMipmappedArray
custom uint64 outputs:format
custom uint outputs:height
custom uint outputs:mipCount
custom uint64 outputs:stream
custom uint outputs:width
custom double outputs:simTime
custom double outputs:hydraTime
}
def OmniGraphNode "Transcoder"
{
custom token node:type = "omni.drivesim.sensors.nv.radar.SinkPrepRadar"
custom int node:typeVersion = 1
custom uint64 inputs:cudaStream
custom uint64 inputs:src
custom uint64 inputs:meta
custom int inputs:remotePort = 0
custom uint64 outputs:cudaStream
custom uint64 outputs:syncData
custom uint64 outputs:taskingCounter
custom uint64 outputs:dst
custom uint64 outputs:bufferSize
custom string outputs:channelName
prepend uint64 inputs:src.connect = </Ecu/PostRender/DSPipeline/RadarSource.outputs:cudaMipmappedArray>
prepend uint64 inputs:meta.connect = </Ecu/PostRender/DSPipeline/RadarMetaSource.outputs:cudaMipmappedArray>
prepend uint64 inputs:cudaStream.connect = </Ecu/PostRender/DSPipeline/RadarSource.outputs:stream>
}
}
}
}
Notes:
in the above pipeline/graph definition there are only 2 nodes, a source node connected to a transcoder node that sends the sensor output as a UDP ethernet stream
all RtxSensor post processing graphs has to start with the source node of type “omni.graph.nodes.GpuInteropCudaEntry” 1. for now the usda action can only override the “remotePort” with “action:camera:port” .. in case other processing graph params needs to be overridden a new graph definition needs to be built and referenced in the action