[Question] What's the right way to use isaac lab for model-based planning?

[zxhuang97]

Hi Isaac lab team,

I'm currently working on a model-based approach for tasks in Isaac Lab, which requires using the env in isaac lab to evaluate the sampled actions. Specifically, the pipeline works as follows:

1. Planner sampled a bunch of action sequences
2. Reset the $env_{plan}$ to current state of $env_{test}$
3. Roll out $env_{plan}$ with the sampled actions, and record the rewards

I met two challenges when implementing this pipeline

• If I understand correctly, each process is only allowed to create one env instance, which is vectorized. And when stepping the env, all children envs will step simultaneously. In my case, I would like to separate $env_{test}$ from $env_{plan}$. One workaround I see is to run another process as a server for $env_{plan}$. It's feasible but might bring complications to debugging. Also, transferring the rollout query between processes might require moving the tensor between CPU and GPU. I would really appreciate it if you can shed some lights on the correct way to do this.

• My second question is regarding resetting $env_{plan}$. I read this from the documentation of Isaac Sim:

  Replaying a simulation from an in-contact simulation state saved out in the middle of a simulation run can be nondeterministic. The PhysX SDK is using internal contact state that can persist over multiple simulation steps and that cannot currently be serialized and recovered from USD data.

  It there a way to synchronize the internal contact state between $env_{test}$ and $env_{plan}$ ? From your experience, how big of a difference would it make, if I only copy the states like positions and velocities but not the contact states?

Thank you all so much for creating this wonderful framework and I appreciate your feedback.

[mpgussert]

Heya @zxhuang97

This is a great question and aligns with discussions we have been having regarding how to handle (truly) asynchronous environments. I have added the dev team label, and will notify the team for further discussion.

The second question is an Isaac-Sim question, so if you don't find what you need here, I would recommend asking about it here as well.