raisim::DepthCamera

final class · #include <raisim/sensors/DepthSensor.hpp>

Inherits: raisim::Sensor

Depth camera simulated by CPU ray casting against the collision geometry of the world.

Sensor frame convention: +x is the optical axis (forward), +y points left and +z points up. Images are stored row-major with index row * width + col; row 0 is the top of the image and column 0 its left side (as seen when looking along +x). Depth values are in meters, measured along the optical axis (+x), not along the ray.

The constructor selects MeasurementSource::MANUAL. Call setMeasurementSource(MeasurementSource::RAISIM) to let RaiSim compute the depth image at the sensor's update rate.

Nested classes

• struct DepthCameraProperties — Configuration of a DepthCamera. Call updateRayDirections() after modifying it in place.

Public types

Frame

enum Frame : unsigned

Coordinate-frame identifiers. Not used by the current DepthCamera API.

Enumerator	Value	Description
SENSOR_FRAME	0	Sensor frame.
ROOT_FRAME		Root-body frame of the articulated system.
WORLD_FRAME		World frame.

Constructors & destructor

DepthCamera

explicit DepthCamera(
    const DepthCameraProperties &prop,
    class ArticulatedSystem *as,
    const Vec<3> &pos,
    const Mat<3, 3> &rot)

Create a depth camera and precompute its ray directions. The measurement source is MANUAL.

Parameters

Parameter	Description
prop (in)	Camera properties (validated; invalid values cause a fatal error).
as (in)	Owning articulated system.
pos (in)	Sensor position relative to the body (link) it is defined on.
rot (in)	Sensor orientation relative to the body (link) it is defined on.

~DepthCamera

~DepthCamera() final = default

Public member functions

updateRayDirections

void updateRayDirections()

This method must be called after sensor properties are modified. It validates the properties, recomputes the ray directions and resizes the depth and point buffers to width * height.

serializeProp

server::BoundedWriter serializeProp(server::BoundedWriter data) const final

Used by the server. Do not use it if you don't know what it does. It serializes the sensor properties

Parameters

Parameter	Description
data (in)	where the property is written; throws server::SerializationError if it does not fit

Returns where the next data should be written

set3DPoints

void set3DPoints(const std::vector<raisim::Vec<3>> &data)

This method is only useful on the real robot (and you use raisim on the real robot). You can set the 3d point array manually

Parameters

Parameter	Description
data (in)	The 3d points, width * height entries in the image layout (world frame, to match get3DPoints()). A size mismatch is a fatal error.

getDepthArray

const std::vector<float> &getDepthArray() const

Depth image of width * height floats in meters along the optical axis, row-major with row 0 at the top. Pixels with no hit within [clipNear, clipFar] are NaN. The values are zero until the first update; check that getUpdateTimeStamp() is non-negative (negative means never updated).

Returns depthArray

std::vector<float> &getDepthArray()

Mutable access to the depth image (same layout as the const overload); use it to write MANUAL measurements.

Returns depthArray

setDepthArray

void setDepthArray(const std::vector<float> &depthIn)

Set the data manually. This can be useful on the real robot

Parameters

Parameter	Description
depthIn (in)	Depth values (meters along the optical axis), width * height entries in the image layout. A size mismatch is a fatal error.

applyNoise

void applyNoise()

Apply one exposure's noise to the depth image. Manual setters do not add noise. This does not refresh get3DPoints(); update() fills both buffers for CPU captures.

void applyNoise(float *depth, size_t size)

Apply noise to a custom image of exactly width * height optical depths in meters.

Parameters

Parameter	Description
depth (in, out)	Depth image in top-left row order; invalid depths stay invalid.
size (in)	Number of floats.

void applyNoise(float *depth, size_t size, bool topLeftOrigin)

Apply noise to top-left (true) or bottom-left (false) row storage, preserving the same seeded measurement at every physical pixel in either layout.

resetNoise

void resetNoise() noexcept

Restart the stream from the sensor seed, identity, owner id and World environment seed.

get3DPoints

const std::vector<raisim::Vec<3>, AlignedAllocator<raisim::Vec<3>, 32>> &get3DPoints() const

Hit points of the last RaiSim update (or the values given to set3DPoints()), one per pixel in the image layout. Pixels without a valid hit are NaN. Only RaiSim-sourced updates fill this buffer.

Returns 3D points in the world frame

getProperties

DepthCameraProperties &getProperties()

Get the depth camera properties. MUST call updateRayDirections() after modifying the sensor properties.

Returns the depth camera properties

const DepthCameraProperties &getProperties() const

Returns Immutable camera properties.

update

void update(class World &world) final

Ray-cast one depth image: refreshes the sensor pose from its attachment frame, then fills getDepthArray() and get3DPoints(). The owning body (the attachment frame's parent body) is excluded from the ray test. Called by RaiSim when the measurement source is RAISIM.

Parameters

Parameter	Description
world (in)	the world that the sensor is in

depthToPointCloud

void depthToPointCloud(
    const std::vector<float> &depthArray,
    std::vector<raisim::Vec<3>> &pointCloud,
    bool isInSensorFrame=false) const

Convert the depth values to 3D coordinates

Parameters

Parameter	Description
depthArray (in)	input depth array to convert (width * height entries in the image layout; a size mismatch is a fatal error)
pointCloud (out)	output point cloud, resized to width * height. Non-finite depths and rays with a nonpositive forward component give NaN points.
isInSensorFrame (in)	True to output points in the sensor frame, false for the world frame (uses the current pose of the attachment frame)

getPrecomputedRayDir

const auto &getPrecomputedRayDir() const

Returns Precomputed ray directions in the sensor frame, one per pixel in the image layout. Pinhole directions are not normalized (their x component is 1); fisheye directions are unit vectors.

Static member functions

getType static

static Type getType()

Returns Sensor::Type::DEPTH.

Inherited members

From raisim::Sensor: getArticulatedSystem, getFrameId, getFrameOrientation, getFramePosition, getFullName, getMeasurementSource, getName, getOrientation, getOriInSensorFrame, getPosInSensorFrame, getPosition, getSensorSetModel, getSerialNumber, getUpdateRate, getUpdateTimeStamp, isUpdateDue, lock, lockMutex, MeasurementSource, serializeMeasurements, setAttachmentFrame, setMeasurementSource, setSerialNumber, setUpdateRate, setUpdateTimeStamp, try_lock, Type, unlock, unlockMutex, updatePose