[Question] How do you control the action frequecy for a robot.

[mehulsuresh]

Hey,

I'm training a model on a robot hand in simulation and would like to run the trained policy on the real robot. How do I control the action frequency to slow down the rate at which actions are sent so i can train a policy better suited to run on the real robot. I am using the decimation parameter with the dt parameter set in the SimulationCfg. Based on my understanding from the documentation if i set the dt to 1/120 the physics will run at 120Hz and the decimation to 12 the action frequency will be 1/10 or 10 Hz. But this doesn't seem to be the case. I feel like i don't understand what the decimation parameter does exactly and how do i actually control the action frequency ? I am using a direct RL environment setup similar to the shadow hand environment.

[kellyguo11]

Hi there, the decimation parameter controls how many simulation steps to take for each environment step. Within the environment step, we make one call to _pre_physics_step() before taking any simulation steps, then go into a loop that iterates for decimation number of iterations. Inside the decimation loop, self._apply_action() and simulation step is called once per iteration. You can choose to implement _pre_physics_step() if you would only like the logic to run once per environment step.

[WangYCheng23]

I found that the action value serves merely as the setting point. To effectively deploy it on the real robot, we need to carefully adjust the stiffness (kp) and damping (kd). I met the same problem at #1077

[mehulsuresh]

I found that the action value serves merely as the setting point. To effectively deploy it on the real robot, we need to carefully adjust the stiffness (kp) and damping (kd). I met the same problem at #1077

This was also how I slowed down the hand. However, I’m confused about why increasing the decimation parameter causes the robot to move faster and lowers the frame rate—it doesn’t seem logical to me.

With a fixed simulation and physics time step, increasing the decimation should mean the robot needs fewer action updates, giving it more time to reach its position targets. So, in theory, this should lead to smoother, slower movements, and the frame rate should improve. But instead, I’m seeing the opposite. What am I missing here? It doesn’t make sense to me.

[WangYCheng23]

I found that the action value serves merely as the setting point. To effectively deploy it on the real robot, we need to carefully adjust the stiffness (kp) and damping (kd). I met the same problem at #1077

This was also how I slowed down the hand. However, I’m confused about why increasing the decimation parameter causes the robot to move faster and lowers the frame rate—it doesn’t seem logical to me.

With a fixed simulation and physics time step, increasing the decimation should mean the robot needs fewer action updates, giving it more time to reach its position targets. So, in theory, this should lead to smoother, slower movements, and the frame rate should improve. But instead, I’m seeing the opposite. What am I missing here? It doesn’t make sense to me.

In the manager_based_env.py code, the render rate appears to depend on self.cfg.sim.render_interval. In my setup, the render frame corresponds to the value of self.decimation (please check yours). Therefore, increasing the decimation parameter results in a lower frame rate.

# perform physics stepping
for _ in range(self.cfg.decimation):
    self._sim_step_counter += 1
    # set actions into buffers
    self.action_manager.apply_action()
    # set actions into simulator
    self.scene.write_data_to_sim()
    # simulate
    self.sim.step(render=False)
    # render between steps only if the GUI or an RTX sensor needs it
    # note: we assume the render interval to be the shortest accepted rendering interval.
    #    If a camera needs rendering at a faster frequency, this will lead to unexpected behavior.
    if self._sim_step_counter % self.cfg.sim.render_interval == 0 and is_rendering:
        self.sim.render()
    # update buffers at sim dt
    self.scene.update(dt=self.physics_dt)

Additionally, the perception that the robot is moving quickly is not due to its actual speed; rather, it is a result of the motor's delayed response, which creates the illusion of rapid movement.

In my experience, increasing self.decimation may hinder policy regression.

[WangYCheng23]

That's all I've observed; feel free to reach out if you'd like to discuss further.