What is raisimGymTorch?¶
raisimGymTorch is a gym environment example with raisim. A simple pytorch-based RL framework is provided as well but it should work well with any other RL frameworks. Instead of using raisimPy, pybind11 wraps a vectorized environment in C++ so that the parallelization happens in C++. This improves the speed tremendously.
(optional) virtualenv or anaconda
raisimGymTorch is designed such that you can collect tens of billions of state transitions with a single desktop machine. Such a number of state transitions is necessary to train for very difficult tasks. An example of a trained policy is shown below
About 160 billion time steps were used to train the above controller. raisimGymTorch can process about 500k time steps per second in the above environment (with 3950x) with an actuator network (which is as heavy as the physics simulation).
How to run the example¶
We provide an ANYmal locomotion example. In the raisimGymTorch directory,
python setup.py develop python raisimGymTorch/env/envs/rsg_anymal/runner.py
To visualize the policy, run raisimUnity as well.
It will show/record the performance of the policy every 200 iterations.
All recorded videos can be found in
How to debug¶
A pybind11 package (e.g., your environment) might be difficult to debug because you have to write it in C++ but you cannot run it as a normal executable. So we provide a debug app that wraps your environment and creates an executable. To build the debug app, build your environment with
python setup.py develop --Debug
Then, the debug executable is created next to your pybind11 package (
If you use CLion (which is recommended), open raisimGymTorch directory in CLion.
It will automatically add the debug app executable.
It provides a convenient gui for debugging.
You can run the debug app as
./debug_app_<environment name> <full path to rsc directory> <full path to the cfg file>
or add the arguments to CLion executable configuration then run.
In Windows, make sure that you are linking against the debug-build raisim. Visual Studio compiled executables will not work if it links against a library built with different compile flags.
How does it work?¶
RaiSimGymTorch wraps a c++ environment (i.e., ENVIRONMENT.hpp) as a python library using Pybind11.
When you call
python3 setup.py develop, all environments under
raisimGymTorch/raisimGymTorch/env/envs are compiled.
The compiled libraries are stored in
All the rest happens in Python.
You can import your environment from your python code.
For example, the anymal locomotion example can be imported as
from raisimGymTorch.env.bin import rsg_anymal
Your launch file (e.g.,
runner.py) can be customized for your need.
How to add a custom environment?¶
You can add your environment in
If you want to keep your source file somewhere else, then add a symlink to it in
An example environment can be found here
Code structure (if you are curious)¶
ENVIRONMENT class is where you define the dynamics, reward, termination condition and so on.
This class inherits from
RaisimGymEnv, which add basic functionalities to the environment such as
getObDim and so on.
RaisimGymEnv is not general enough for you, you can also make
ENVIRONMENT independent from
RaisimGymEnv is wrapped by
VectorizedEnvironment, which parallelizes the environment using openmp.
You can consider it similar to
VectorEnv in OpenAI Baselines but RaisimGym parallelization happens in C++, which makes it orders of magnitude faster.
raisim_gym.cpp is a Pybind11 wrapping of
It simply defines the interface functions.
RaisimGymVecEnv is a python class that wraps a python library created from