Skip to content

Changelog: v2.17 ​

v2.17.0 (unreleased) ​

RaiSim

  • World::joinObjects() connects two objects with a joint and replaces them with one articulated system that simulates the joint in generalized coordinates. Boxes, spheres, cylinders, capsules, file meshes, compounds and articulated systems can be joined in any combination, or joined to the world. Every body keeps its pose, momentum is conserved, and inertia (including run-time changes), geometry, collision filters, materials, joint properties, PD control, actuators, and the robot descriptions' pin, equality and mimic constraints are carried over. A joint that connects bodies that are already connected closes a loop and is enforced as an exact loop-closure constraint (ArticulatedSystem::getLoopJointNames()). World::detachJoint() removes any joint, including fixed joints and the fixed-base root joint, and splits the system when it falls apart. JoinOptions::childPrefix keeps the names of two copies of one robot apart. See Joining objects.

  • VisObject::sourceLink names the link that declared a visual, kept when fixed links are merged into their parent.

  • Exporting a system without URDF text names the system and the reason instead of saying it was created from a URDF file.

  • A loop-closure constraint whose directions are all held by the joints only warns for constraints of the robot description, not for joined loop joints.

  • Spherical-joint friction is up to 6% faster (clang builds) with bit-identical results.

  • Thrusters and grippers. A URDF <thruster> fixes a thruster to a link: the thrust set with ArticulatedSystem::setThrust() (held until it is set again, clamped to <limit lower upper>) pushes the link along the thruster's axis at its position every step, and an optional torque_coefficient turns the link with a propeller's reaction torque, so a quadrotor hovers and yaws with four thrusts. A URDF <gripper type="suction"> or <gripper type="magnetic"> puts a suction cup or an electromagnet on a link: while setGripperActive() has it on, it pulls the first surface in front of its pad within range toward the pad, and the link the other way. A cup seals on a surface that faces it within seal_angle and pulls with max_force; a magnet pulls only materials marked with World::setMagneticMaterial() (or <material><magnetic name="steel"/></material>), with max_force (1 - gap/range)^2. The contacts then hold the object against a load of up to the pull and, through friction, a sideways load of up to the friction coefficient times the pull; a static object anchors the robot instead. getGripperState() reports what a gripper holds. setThrusters() and setGrippers() replace them at run time; checkpoints, the world XML, World::joinObjects() and World::detachJoint() keep them with their commands. See Thrusters and Grippers.

  • Voxel maps. World::addVoxelMap() adds a static 3D occupancy map of cubic voxels that can hold overhangs, caves and rooms, and can be edited while the simulation runs (setVoxel(), setVoxels(), fillBox(), fillSphere(), setOccupancy(), clear()). Bodies of every shape collide with its exposed faces, edges and corners only, so they slide across the seams of a flat voxel surface as on a plane; bodies inside the solid are pushed out, and sleeping bodies near an edit wake. Ray tests, lidars and depth cameras hit the first occupied voxel exactly. VoxelMapResource shares one occupancy between maps and worlds, copied on the first edit, and checkpoints restore it. See Voxel Map.

  • The rayrai TCP viewer draws voxel maps as an instanced cube per visible voxel and rayrai windows as their surface; the server sends them with the new protocol feature PROTOCOL_FEATURE_VOXEL_MAPS (kProtocolFeatureVoxelMaps in rayrai), only after edits.

  • Contact detection between convex meshes and triangle meshes skips the triangles that cannot touch the hull, with unchanged contacts: a convex hull touching a dense 15,744-triangle mesh is up to 12x faster.

  • Scene collision queries. World::checkOverlap(), queryOverlap(), queryProximity() and sweepShape() test a sphere, capsule, box, cylinder or convex hull (CollisionQueryShape) against the colliders of the world, voxel maps included, without stepping it: whether it overlaps anything, every collider it overlaps, the closest collider within a distance with witness points and normal, and the first collider it hits when moved along a displacement. CollisionQueryFilter selects colliders by collision group and mask and can leave out one object or one of its bodies. World::meshClearance() and isMeshWithinClearance() measure how far a capsule (two end points and a radius) is from the rigid triangle-mesh surfaces of the scene. See Convex collision queries.

  • Shared resources. HeightMapResource, ArticulatedSystemResource and MeshResource load a heightmap, a robot description or a mesh once; World::addHeightMap(), addArticulatedSystem() and addMesh() create instances from them in any number of worlds, which share the geometry and own their state. See Shared resources across worlds.

  • Behaviour change: rayTest(), lidar scans and getRayCastBodies() called between steps see every rigid body where it is now, at the pose the last step left it or a setter such as ArticulatedSystem::setGeneralizedCoordinate() put it; before, colliders lagged one step behind and articulated poses set between steps were invisible. Rays cast during a step and the next step's contacts are unchanged.

  • Fixes from a full code review:

    • Changing gravity or materials (setGravity(), setMaterialPairProp(), setDefaultMaterial()) wakes sleeping bodies.
    • Removing a voxel map right after editing it no longer corrupts sleep tracking.
    • Restoring a snapshot of a compound keeps each child's material and collision filter.
    • The USD importer loads collision shapes under instanceable prims (Isaac instanceable assets) and honors physics-purpose material bindings. A body with an authored mass but no inertia gets its inertia and center of mass from its colliders.
    • A capsule lying across a box face gets two contacts, not one.
    • Nearly parallel cylinders touching at overlapping ends get contacts.
    • Cylinder-plane contacts keep the deepest points when the contact budget is small.
    • MJCF tendons load for models with several floating bases or unnamed joint chains.
    • World XML export rejects voxel maps instead of writing a file that cannot be loaded.
    • XML object_class primitives apply the class's inertia and center of mass.
    • A cloth visits each rigid collider once per step instead of every pair of bodies.
    • Split steps (integrate1()/integrate2()) apply a deformable's forces once and keep a kinematic particle's velocity.
    • Articulated systems in substepped (monolithic deformable) worlds use the substep duration for damping, PD, impacts and inverse dynamics.
    • getJointLimitViolations() no longer returns stale indices for a sleeping robot.
    • Granular rolling friction acts on the relative spin of touching grains and conserves angular momentum.
    • Independent tendons and legacy wires no longer cost quadratic time in their number.
    • .OBJ mesh files with uppercase extensions load.
    • Runtime-joint endpoints entering water get the full water-entry damping.
    • Automatic MBP broadphase bounds leave the ground plane out, so a ground no longer puts every body into one grid cell; 1,000 boxes on a ground go from 125,500 candidate pairs to 1,000.
    • Mesh-on-heightmap contacts check terrain planarity once per mesh triangle instead of rescanning the footprint for every terrain triangle.
    • The USD importer replaces changed mesh cache files atomically and leaves unchanged ones alone, so concurrent imports never read a half-written collision mesh.
    • RaisimServer:
      • A restored checkpoint's height-map heights reach the viewer.
      • addVisualHeightMap(name, const HeightMap*) draws the copy at its source's height.
      • On Windows, a failed socket setup is a fatal error, as on Linux and macOS, instead of leaving a spinning server thread, and reserved ports are skipped.
      • A viewer save of a world with a voxel map warns and keeps serving.
      • A pose request for an object that is gone no longer drops the viewer's connection. rayrai
  • Automatic diffuse lighting. A .rscene scene bakes up to two bounces of indirect diffuse light from its static geometry once its meshes load (automaticIrradiance, automaticIrradianceBounces). Probes are placed adaptively and interpolated with per-probe visibility, and bakes are cached by scene content. AdditionalLight::runtimeIrradianceRadius opts one fixture into a live diffuse bounce (runtimeIrradianceEnabled).

  • Ground-truth ambient occlusion (screenSpaceGtao), histogram HDR exposure metering (autoExposureHdrMetering), temporal upscaling (ViewerUpscaleFilter::Temporal), and reactive transparency for reprojected temporal AA (temporalAaReactiveTransparency).

  • Vulkan ray-query reflections for off-screen and glossy hits (ssrRayQueryFallback, ssrGlossyRayQueries), with textured and alpha-masked hits.

  • Integrated rectangular area lights (integratedAreaLights), local-light soft shadows (localLightSoftShadows), stable cascade extents (directionalShadowStableExtents), HDR probe lighting for instanced batches (instancedHdrLighting), and shadowed spot and area light scattering in fog (volumetricLocalLightStrength).

  • GPU LiDAR on voxel maps. measureSpinningLidarSingleDrawGPU scans voxel maps and returns the points of the CPU LiDAR ray by ray, including rays that start on a voxel face, run along an edge or pass through a corner. An edit uploads only the 16³-voxel bricks it changed, so a scan right after carving or filling costs about as much as any other. An Ouster OS1-64 turn over an 84 m × 84 m voxel terrain takes 0.41 to 0.46 ms on an RTX 2070 SUPER, about 22 times faster than on the CPU.

  • Point lights cast their own shadows in the compact PBR shader that desktop GPUs with 32 texture units use; the main light's shadow no longer darkens other lights there (which had hidden the missing point shadows).

  • Fixes: contact shadows with temporal upscaling no longer darken the frame; on 16-unit GPUs (macOS) contact shadows no longer drop other post effects; non-PBR materials sample local-light shadows correctly; a zero-scale or non-finite object no longer turns off hardware reflections; a second overlapping probe no longer dims instanced lighting; moving a live fixture during its capture no longer publishes the old position; the irradiance capture no longer overruns its readback for odd face sizes; the bake cache works for Windows profiles with non-ASCII names; the mouse no longer keeps turning the temporally upscaled viewer camera.

  • Frames are the same from run to run: opaque visuals at the same distance, such as the pieces of one .rscene model, no longer draw in an order that depends on which textures finished loading first, and world objects draw in World order instead of by memory address. Visuals at the same distance tie by name instead of by hash-map order, and a non-finite position no longer breaks the transparent sort (it was undefined behaviour).

  • Instanced batches, foliage and shadow casters whose instance data exceed the GPU's largest storage buffer binding (128 MiB on some drivers) skip GPU culling and filtering instead of overrunning it. The buffers that one frame's foliage filtering shares stay within the 128 MiB that every WebGPU device allows up to about 33 million filtered instances.

  • ExampleApp (rayrai/example_common.hpp) saves what a program renders when the environment asks, with no code in the program: one PNG (RAYRAI_EXAMPLE_CAPTURE=view.png), a numbered frame sequence with an optional looping camera sway, or the whole window with its panels. See Capturing an example. raisin::writeRgbaPng() (rayrai/PngWriter.hpp) writes RGBA images as PNG files.

  • Fixes from a full code review:

    • Shadows:
      • Directional shadow cascades follow the external camera that renders them.
      • Additional directional, spot and area lights' shadow maps are sampled, and so is the main point light's shadow cubemap.
      • Wide spotlights no longer light past their shadow frustum.
      • Shadow maps refresh when a caster turns in place (visuals and articulated links), not only when its bounds move.
    • Picking, frame reuse and culling:
      • Picking keeps the caller's split read and draw framebuffers.
      • Destroying a RayraiWindow no longer leaves its picking program current.
      • Editing a static primitive invalidates frame reuse.
      • The fallback occlusion test no longer hides a foliage group around the camera.
      • Long capsules are frustum-culled by their real bounds.
      • Hi-Z culling results are not reused by draws without occlusion.
      • Instanced batches of meshes larger than a unit cube are no longer culled while visible, and editing a mesh's vertices updates its culling bounds.
      • Blended submeshes of a visual with opaque parts draw in the transparent pass.
    • Lights and materials:
      • Imported lights no longer promote the viewer's fill light over the sun.
      • Core PBR honors the light's specular color.
      • Retuned textures refresh the sampler cache.
      • Nonuniform visual scaling keeps the normal-map basis aligned.
      • Instanced draws apply the material UV scale, offset and rotation to albedo and alpha cutouts in every pass.
      • Generated tangents keep their handedness across mirrored UV seams.
    • Explosions, water and loading:
      • Projectile tracers keep moving after the muzzle effect ends.
      • Moving or resetting a water body with flow re-simulates it.
      • A synchronous mesh-loading scope no longer leaves physics meshes invisible.
    • Cameras and post effects:
      • Depth-camera frustums respect the image-window offset.
      • Standalone lens and color effects run on non-HDR cameras.
      • External cameras clear depth whatever the caller's clear value.
      • Orthographic depth of field focuses at the focus distance, and in-focus pixels keep stylized effects.
      • FXAA keeps chromatic aberration.
    • Mesh import and asset loading:
      • Mirrored glTF nodes keep their front faces.
      • Edits to external glTF buffers invalidate the preprocess cache.
      • glTF MASK keeps zero alpha, and untextured glTF materials keep metallic and roughness.
      • Destroyed offscreen renderers release their asset-cache entries.
      • A scene reload never reuses an in-flight import of the old file.
      • Async instanced loading no longer duplicates LOD generation.
      • Async model loading no longer stalls when imports share textures or a large texture waits for decode budget.
      • Reloading a GLB reuses its embedded textures instead of uploading new copies.
    • Depth, segmentation and refraction:
      • Depth and segmentation draw billboards facing the camera.
      • Geometry refraction treats rectangular emitters and the main spotlight correctly, and near-clipped opaque geometry no longer hides glass.
      • Capped refraction fallback frames show the portable result.
      • Fully transparent cutout texels no longer write depth or segmentation when the alpha cutoff is 0.
      • Hybrid reflections of simple-material cutouts sample the same texel as the raster.
    • Streaming and foliage:
      • Streamed terrain has no seams between fine and coarse patches.
      • Foliage atlases are freed and keep sRGB decoding.
      • Interleaved tile bakes keep their instances.
      • Streamed foliage retries failed mesh loads.
      • The prepared foliage cache works across libstdc++ versions.
    • Water, probes and irradiance:
      • Water normals include the edge-fade slope.
      • Scene reflection probes capture the HDR background.
      • Automatic irradiance bakes no longer store runtime geometry under the scene's cache key.
    • GL state:
      • Readbacks and uploads (water bakes, GPU LiDAR, clouds, sky visibility) ignore the application's pixel pack and unpack state.
    • Miscellaneous:
      • Strided point clouds upload their edited range.
      • Weighted OIT no longer clips opaque HDR highlights.
      • Async physics meshes and URDF visual alpha select the right pass.
      • Static voxel-map edits update culling bounds.
      • A recycled native GL context handle no longer returns another context's textures.
      • Diagnostics JSON ignores the C++ locale.
    • TCP viewer:
      • The pose grabber holds only single bodies.
      • Child panels close correctly.
      • UTF-8 export folders work on Windows.
      • --screenshot exits with an error when the capture fails.
      • Resetting rendering settings keeps the TCP update rate.
      • Changing viewer settings or panes keeps the scene's lights, render settings and weather.
      • Spawn file extensions are checked as the server checks them.
      • Loopback discovery beacons reach every listener.
      • Arrow visuals and vertex-array meshes from the server are drawn.
      • A camera behind larger ones still gets sensor updates.
      • Output folder, video and session log paths longer than 511 bytes are kept whole.
  • The TCP viewer reads the camera of RaisimServer::setCameraPositionAndLookAt() in order on every compiler. Built with GCC or MSVC it could swap the x and z coordinates of the position and the target, putting the camera behind or under the scene. A source rule now rejects two reads from one stream in the arguments of one call.

Browser (WebAssembly)

  • Rayrai's GPU instance culling, foliage occlusion filtering and shadow-receiver filtering run in the browser, as they do with desktop OpenGL 4.3. The browser backend now supports buffer clears, image copies (including multisample depth), multisample textures and, where the GPU allows it, base instances in indirect draws, and the renderer checks for each of these features instead of the OpenGL version. On desktop, limiting Rayrai to the paths the browser had before takes the forest example from 13.8 to 25.3 ms per frame (RTX 5070).
  • Foliage scenes (Tank Survival, the forest and showcase examples, depth and segmentation cameras) no longer crash Chrome's GPU process with NVIDIA's 595 Linux driver, whose shader compiler failed on Rayrai's foliage dithering.
  • The segmentation camera runs in the browser: the WebGPU backend now supports its signed-integer target (GL_RGBA32I, glClearBufferiv, glUniform4i). Integer images also blit, clear through scissors and color masks, and convert between signed and unsigned values as in OpenGL.
  • Material reflections no longer stop the warehouse example in the browser when no depth pyramid or reflection history is available yet.
  • Light projector maps that no light uses are no longer checked in the browser. On GPUs that cannot filter 32-bit float textures (NVIDIA), a float texture left on such a map's unit stopped the viewer with "Rayrai texture requires unsupported filtering: additionalLightProjectorMap0", for example in voxel_map_ray_casting.

raisimPy

  • raisimpy.Joint, raisimpy.JoinOptions, World.joinObjects(), World.detachJoint() and ArticulatedSystem.getLoopJointNames(); examples/articulatedJoin.py carries and releases a tool and docks two carriages.
  • raisimpy.ThrusterDefinition, raisimpy.GripperDefinition, raisimpy.GripperState, the thruster and gripper methods of ArticulatedSystem, and World.setMagneticMaterial() / World.isMagneticMaterial().
  • raisimpy.VoxelMap, raisimpy.VoxelMapResource and World.addVoxelMap(), with keys and dense occupancy as numpy arrays.
  • raisimpy.CollisionQueryShape, CollisionQueryFilter, CollisionQueryHit and World.checkOverlap(), queryOverlap(), queryProximity(), sweepShape(), meshClearance() and isMeshWithinClearance(); raisimpy.HeightMapResource, ArticulatedSystemResource and MeshResource.

raisimGymTorch

  • With record_video: True and a raisimGymTorch built with rayrai, evaluation videos are recorded in the training process with rayrai and ffmpeg (policy_<iteration>.mp4 in the run's data directory), so no viewer has to be connected. start_video_recording() returns whether recording started and get_video_status() says why not (no OpenGL context, no ffmpeg on PATH); is_visualizer_connected() reports a connected viewer.
  • The rsg_anymal_terrain environment trains ANYmal on fractal heightmap terrain that all environments share through one HeightMapResource, with a height scan in the observation, and switches every environment to a new terrain every terrain/randomize_every_n iterations.

Examples and benchmarks

  • The articulated_join example builds a chain from boxes, picks up and releases a tool with a gantry, and docks two robots in one scene.
  • articulated_join_benchmark compares a pendulum chain built with joinObjects() (12.9 µs per step for eight 8-link chains) with the same chain held by runtime hinge joints (1350 µs), and times joining a tool to an ANYmal (133 µs) and detaching it (134 µs).
  • The drone_flight example: a quadrotor flown by four thrusters takes off from its pad, flies figure eights around two posts facing the way it flies, turned by the reaction torques of its propellers, and lands again. It has its own example page and gallery tile, and force_device_example_drone checks its flight.
  • The magnetic_wall_climbing example: a Go1 with magnetic feet modeled on KAIST's MARVEL (530 N of pull, 440 N of sole friction per foot) trots up and down a steel wall at 0.5 m/s, switching each magnet off 20 ms before its foot lifts and on again as it lands.
  • The rocket_hop example: a rocket (rsc/rocket/rocket.urdf) whose main engine is a thruster on a two-axis gimbal hops 20 m between two pads, steering by tilting the engine, holding its roll with four roll thrusters and landing on its legs.
  • The suction_cup_picking example: a Kinova arm with a vacuum suction cup (rsc/kinova_suction/kinova_suction.urdf) stacks four parcels and a can into a tower with solveIK() and takes it apart again.
  • thrusters_grippers benchmark: 64 quadrotors hovering with thrusters (about 20 ns per thruster per step over setting the same forces by hand), and 25 lifters with suction cups or magnets (about 130 ns per active gripper per step for its ray).
  • The voxel_map_ray_casting example scans a voxel map with a spinning lidar and a probe ray, and voxel_map_collision drops primitives and meshes onto voxel stairs, walls and rubble while carving craters under them.
  • Examples contain only their lesson. The options that served screenshots, recordings, benchmarks and self-checks were removed: --screenshot, --record, --frames, --benchmark and similar from rayrai_blue_wall_scene, rayrai_nested_glass, rayrai_offroad_buggy, rayrai_pbr_texture_maps, rayrai_segmentation_camera, rayrai_complete_showcase, rayrai_city, rayrai_warehouse, rayrai_tendons, rayrai_deformable_terrain and rayrai_inverse_kinematics (whose tour starts from the panel's Tour checkbox), and --headless from robotiq_gripper_mimic, robotiq_cloth_fold, articulated_join, runtime_joints and the tendon examples. RAYRAI_EXAMPLE_CAPTURE saves the view of any rayrai example (Capturing an example), and the engine's example tests check what the removed options used to check.
  • Benchmarks have an overview and are grouped by topic in the navigation. Voxel ray casting compares RaiSim's voxel ray cast on the CPU and rayrai's on the GPU with OpenVDB, NanoVDB, Bonxai and OctoMap: exact lidar rays 1.7 to 34 times faster on one CPU core, and 1.3 to 1.8 times faster than NanoVDB in CUDA, with charts made from the recorded results (docs/image_generators/voxel_ray_casting_charts.py).
  • Every benchmark chart (MuJoCo, Newton, linear elasticity, voxel ray casting) is drawn by one chart module (docs/image_generators/benchmark_chart_style.py) in one style, from the JSON files the pages link: RaiSim in blue, rayrai in orange, the compared libraries in gray. The linear elasticity benchmarks gained charts of cold solves against MFEM and scikit-fem, changing loads, multigrid against ICCG and meshing against fTetWild.
  • rayrai_lidar_benchmark --voxel-size=M times the GPU and CPU LiDAR over the same terrain as a voxel map of M voxels, also with a sphere carved or filled before every update.
  • voxel_lidar, voxel_contacts and voxel_edit benchmarks time lidar scans of the same terrain as a voxel map and as a heightmap, a pile of bodies resting on voxel rubble, and carving spheres out of a voxel map.

RaiSim — physics for robotics and AI.