Omniverse Lidar Extension
Introduction
The Lidar extension consists of RtxSensor based Lidar simulation plugins (currently only one), several plugins and different Omnigraph Nodes. The simulation plugin is a generic lidar model, which can be configured via a sensor profile json file.
LidarPointCloud
The Lidar extension uses the GenericModelOutput as output format. See the documentation of the common extension for more information on the GenericModelOutput.
LidarProfile
The lidar profile parameters affect the behavior of the lidar sensor. It consists of mandatory basic parameters and optional ones. Examples are the Example_Rotary and Example_Solid_State profiles.
Basic Parameters
scanType: The type of the lidar sensor (rotary for rotating and solidState for non-moving sensors)
nearRangeM/farRangeM: The min/max range of the lidar sensor in meters
rangeResolutionM: The resolution of the range in meters
rangeAccuracyM: The accuracy of the range in meters
wavelengthNm: The wavelength of the lidar sensor in nanometers
maxReturns: Number of max returns per emitter (e.g., 2 for dual return mode)
reportRateBaseHz: Number of times the firing pattern is applied per second, this is defined as a “tick”
numberOfEmitters: Number of emitters in the lidar sensor per scan
scanRateBaseHz: Number of full point cloud scans per second
numLines (Solid State): Number of rows per scan
numRaysPerLine (Solid State): Number of rays per line/row, array with numLines elements
- Firing Pattern Parameter (see Emitter State Parameters for detailed information):
emitters: Array of emitters if emitterStateCount is not defined
emitterStateCount: Number of states per emitter
emitterStates: Array of states for each emitter
Optional Parameters
avgPowerW: Average power of one lidar emitter pulse in watts (default: 0.002)
pulseTimeNs: Pulse time of one lidar emitter pulse in nanoseconds (default: 6)
validStartAzimuthDeg: Start azimuth of the valid range in degrees (default: 0)
validEndAzimuthDeg: End azimuth of the valid range in degrees (default: 360)
minDistBetweenEchos: Minimum distance between two echoes in meters (default: min(nearRangeM,0.4))
rangeOffset: Range offset in meters (default: 0) – useful for not hitting the prop
numberOfChannels: Number of channels/detectors in the lidar sensor (default: numberOfEmitters)
minReflectance: Minimum reflectance expected to see at minReflectanceRangeM (default: 0.1) – not the overall min reflectance
minReflectanceRange: Maximum range at which the minimum reflectance is expected to generate a valid return (default: 200)
intensityProcessing: Processing to get final intensity (supported right now: “normalization” and “correction”)
rayType: Type of ray in geometrical sense (e.g., “IDEALIZED”, “GAUSSIAN_BEAM”)
- Detector Parameters:
effectiveApertureSize: Effective aperture size of the detector in meters (default: 0.01)
quantumEfficiency: Quantum efficiency of the detector (default: 0.9)
pixelPitch: Pixel pitch of the detector (default: 40.0)
calibrationGain: Calibration gain of the detector (default: 40.0)
bitDepthResolution: Bit depth resolution of the detector (default: 65536)
- intensityMapping:
intensityMappingType: Type of intensity mapping (e.g., “LINEAR”, “NONLINEAR” – intensityMappingDecoding/Encoding)
intensityMappingDecoding: Decoding function values for intensity mapping
intensityScalePercent: Intensity scale in percent (default: 255)
- Range Parameter: For setting a per emitter range (nearRangeM,farRangeM will be used as default)
rangeCount: Number of ranges in array
ranges: Array of ranges for each emitter e.g., [{“min”: 0, “max”: 100}, {“min”: 3, “max”: 200}]
- Beam Parameter: Parameter of laser beam
beamWaistHorM: Horizontal beam waist in meters (default: 0.002)
beamWaistVerM: Vertical beam waist in meters (default: 0.002)
divergenceHorDeg: Horizontal divergence of the laser beam in degrees (default: calculated from beam waist)
divergenceVerDeg: Vertical divergence of the laser beam in degrees (default: calculated from beam waist)
focusDist: Focus distance of the laser beam in meters (default: 0.15)
Msquared: M squared value of the laser beam (default: 1.4)
Emitter State Parameters
The firing pattern, potentially, consists of multiple emitter states, whereas one emitter state is a full firing of all emitters. The minimum emitter parameters which have to be set are the following:
azimuthDeg: Azimuth of the emitter in degrees relative to the tick azimuth (see reportRateBaseHz)
elevationDeg: Elevation of the emitter in degrees
fireTimeNs: Time of the emitter firing in nanoseconds relative to the tick time (see reportRateBaseHz), should be inside the tick delta time
Additionally, the following optional parameters can be set:
channelId: Channel id of the emitter, to map specific emitter to channels (default: emitterId)
rangeId: Range id of the emitter, to map specific emitter to specified ranges (default: 0)
bank: Bank id of the emitter, to map specific emitter to banks/lines for solid state lidar (default: 0)
vertOffsetM: Vertical offset of the emitter in meters (default: 0)
horOffsetM: Horizontal offset of the emitter in meters (default: 0)
distanceCorrectionM: Distance correction of the emitter in meters (default: 0)
reportRateDiv: Report rate divisor of the emitter, to reduce the report rate of the emitter (default: 1)
focalSlope: Focal slope of the emitter, to correct the focal point of the emitter (default: 0)
focalDistM: Focal distance of the emitter in meters, to correct the focal point of the emitter (default: 0)
LidarPCConverter
The LidarPCConverter converts the RTXSensor Lidar data stream to an easily usable point cloud (GenericModelOutput format). The interface is defined in (include/omni/sensors/lidar/ILidarPCConverter.h).
Users can create objects that implement this interface by acquiring the ILidarPCConverterFactory (include/omni/sensors/lidar/ILidarPCConverterFactory.h) carbonite interface and using the createInstance() method.
Instantiation
To instantiate the LidarPCConverter the user needs to acquire the carbonite ILidarPCConverterFactory interface, and use the create instance method:
ILidarPCConverterPtr m_pcConverter = carb::getFramework()->acquireInterface<omni::sensors::lidar::ILidarPCConverterFactory>()->createInstance();
Initialization
The user has to initialize the converter, with the corresponding converter cfg (omni::sensors::lidar::LidarPCConverterCfg) like this:
m_pcConverter->init(cfg);
Conversion
The conversion of the buffer differs with the conversion buffer type (PACKETS, GENERIC_FILE (HDF5) or live sensor buffer). In the live sensor buffer the user calls the following function
m_pcConverter->convertBuffer(sensorBuffer, dataSize, numPoints, scanComplete, cudaStream);
Next he can get the point cloud:
LidarPointCloud pc = m_pcConverter->getPointCloud(cudaStream);
The user has to keep in mind, that in live CPU mode the getPointCloud function has to be called in a cudaLaunchHostFunc with the appropriate cudaStream, as every work is scheduled asynchronously.
LidarCorePlugin
Introduction
The Lidar core plugin is a generic Lidar simulation utilizing RtxSensor. It can be configured through sensor config file (json format) and it supports rotary as well as solid state lidars.
Plugin Name
omni.sensors.nv.lidar.lidar_core.plugin
Parameters & Attributes
The attributes are specified through a sensor config JSON file, which are placed in “data/sensors/lidar”. The desired sensor profile is set via the USDA attribute “custom token rtxsensor:modelConfig” (see Example in USDA for an example). This JSON file can be read with the ProfileReader utility plugin.
Available Parameterizations
The following parameterization to support specific Lidar already exists:
Lidar Type |
Params File name |
Model Config Entry |
Variant |
---|---|---|---|
Generic Rotary |
Example_Rotary.json |
“Example_Rotary” |
|
Generic Solid State |
Example_Solid_State.json |
“Example_Solid_State” |
Atmospheric Modeling
The Lidar model includes a mie scattering based atmospheric simulation model, which currently supports rain. It is planned to add fog, haze and snow at a later time.
Besides the wavelength dependency, the rain model has one basic parameter: the rain rate in mm/h (configurable as a carb::setting: “/app/sensors/nv/atmospherics/rainRate”). For instance, a rain rate of 0.25mm/h is light rain, whereas a rain rate of 25mm/h corresponds to a very heavy rain. Every other parameter for the mie scattering (e.g., rain drop size distribution) is derived from the rain rate.
Additionally, the rain model provides the possibility to simulate rain drop hits as false positive. The amount of false positives can be set via a rain drop hit threshold (configurable as a carb::setting: “/app/sensors/nv/atmospherics/rainDropHitThresh”). This threshold is related to the mie back scattering efficiency parameter and a distance dependent rain drop hit probability distribution.
Both parameter can be set via command line are dynamically via the script editor, as both parameter are defined as a carb::setting. In example you can start the simulation with a rain rate of 25mm/h if you’re adding “–/app/sensors/nv/atmospherics/rainRate=25.” to the startup command of kit. It is important to activate the material framework to get accurate readings from wet roads (point drops) and false positives due to back scattering (under the ground).
Omnigraph Nodes
One example of the current nodes in the Lidar extension is the TranscoderLidar. A dumping transcoder, which encodes the Lidar data into vendor specific udp packets and dumps them into a bin-file.
TranscoderLidar
Introduction
The Transcoder node encodes the Lidar data stream into vendor specific packets and records them via binary file writing.
Omnigraph Node Name
omni.sensors.nv.lidar.TranscoderLidar
Parameters & Attributes
Parameter Name |
Description |
Value Type |
Value Range |
Default Value |
Example |
---|---|---|---|---|---|
sensorProfileName |
Filename of sensor profile (excluding directory and .json) |
string |
“GENERIC” |
||
returnType |
Indicates the desired return (first,last,dual,etc.,) |
int |
0 |
||
runMode |
CPU = 0 or GPU = 1 |
int |
0 |
||
dumpPackets |
Dump packets as file |
bool |
false |
||
decoderPath |
Optional path to ndas decoder |
string |
“” |
||
fileName |
Filename of the dumped bin file |
string |
“” |
Python Bindings and Scripts
The Lidar extension offers a variety of python bindings and scripts for the Lidar. These can be used for easy post-processing of the simulated Lidar point cloud.
Python Bindings
The python bindings make it possible to use the Lidar utilities and specific classes in python code. For instance, the user can import the Lidar python bindings in the following way:
import omni.sensors.nv.lidar.bindings._lidar as lidar
Furthermore, there exist the LidarUtilities script, containing some helper functions for ease of use of the Lidar bindings. The script can be imported, e.g., the following way:
import omni.sensors.nv.lidar.scripts.LidarUtilities as lutil
There are binding definitions for all basic classes and structs of the Lidar and lidar Tools extension. See the following (incomplete) list of the most basic classes:
LidarProfile
Furthermore, there are bindings for the following utilities:
LidarPCConverter
The binding exists for the LidarPCConverter, which accumulates and converts Lidar sensor buffers to the LidarPointCloud. There is a mode to accumulate and convert udp packets to a Lidar point cloud, but not every sensor is supported. To get a LidarPCConverter object the user can call the following helper function of the LidarUtilities script
cfg = lutil.PCGeneratorConfig(bin_file) # or (bin_file,"",sensor_name) for reading a HDF5 file (only supported in linux for now)
converter = lutil.get_pc_converter(cfg)
For more options of the PCGeneratorConfig see the LidarUtilities script.
Currently, only host side conversion of the buffer is supported, but GPU accelerated conversion will be added in the near future. An example conversion is as follows
# convert buffer
# File is specified in the cfg
converter.convertBuffer()
# or this for a GenericModelOutput binary buffer
# converter.convertBuffer(buffer,data_size, num_points, scan_complete)
pc = converter.getPointCloud()