Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error implementing ppo_4x4grid.py code #205

Open
Kunal-Kumar-Sahoo opened this issue Jun 5, 2024 · 2 comments
Open

Error implementing ppo_4x4grid.py code #205

Kunal-Kumar-Sahoo opened this issue Jun 5, 2024 · 2 comments

Comments

@Kunal-Kumar-Sahoo
Copy link

I get the following error while executing the code given in experiments/ppo_4x4grid.py. The console output is as follows:

2024-06-05 15:05:36,177 INFO worker.py:1743 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265 
2024-06-05 15:05:36,577 INFO tune.py:622 -- [output] This will use the new output engine with verbosity 2. To disable the new output and use the legacy output engine, set the environment variable RAY_AIR_NEW_OUTPUT=0. For more information, please see https://github.com/ray-project/ray/issues/36949
/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/box.py:130: UserWarning: WARN: Box bound precision lowered by casting to float32
  gym.logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/utils/passive_env_checker.py:164: UserWarning: WARN: The obs returned by the `reset()` method was expecting numpy array dtype to be float32, actual type: float64
  logger.warn(
/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/utils/passive_env_checker.py:188: UserWarning: WARN: The obs returned by the `reset()` method is not within the observation space.
  logger.warn(f"{pre} is not within the observation space.")
╭────────────────────────────────────────────────────────╮
│ Configuration for experiment     PPO                   │
├────────────────────────────────────────────────────────┤
│ Search algorithm                 BasicVariantGenerator │
│ Scheduler                        FIFOScheduler         │
│ Number of trials                 1                     │
╰────────────────────────────────────────────────────────╯

View detailed results here: /home/rankit/ray_results/PPO
To visualize your results with TensorBoard, run: `tensorboard --logdir /tmp/ray/session_2024-06-05_15-05-34_652990_98780/artifacts/2024-06-05_15-05-36/PPO/driver_artifacts`

Trial status: 1 PENDING
Current time: 2024-06-05 15:05:36. Total running time: 0s
Logical resource usage: 5.0/32 CPUs, 0/1 GPUs (0.0/1.0 accelerator_type:G)
╭────────────────────────────────────╮
│ Trial name                status   │
├────────────────────────────────────┤
│ PPO_4x4grid_f6ab8_00000   PENDING  │
╰────────────────────────────────────╯
(RolloutWorker pid=101043)  Retrying in 1 seconds
(RolloutWorker pid=101043) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 18ms, vehicles TOT 0 ACT 0 BUF 0)                     
(RolloutWorker pid=101043) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101043, ip=10.158.21.84, actor_id=2e663024f15594f45f8c23ae01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x73a20e032d00>)
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 407, in __init__
(RolloutWorker pid=101043)     self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=101043)   File "decentralized_ppo.py", line 29, in <lambda>
(RolloutWorker pid=101043)     lambda _: ParallelPettingZooEnv(
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 198, in __init__
(RolloutWorker pid=101043)     self.observation_space = gym.spaces.Dict(self.par_env.observation_space)
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/dict.py", line 84, in __init__
(RolloutWorker pid=101043)     assert isinstance(
(RolloutWorker pid=101043) AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'>
2024-06-05 15:05:41,606 ERROR tune_controller.py:1332 -- Trial task failed for trial PPO_4x4grid_f6ab8_00000
Traceback (most recent call last):
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/air/execution/_internal/event_manager.py", line 110, in resolve_future
    result = ray.get(future)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/worker.py", line 2667, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/worker.py", line 866, in get_objects
    raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 229, in _setup
    self.add_workers(
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 682, in add_workers
    raise result.get()
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/utils/actor_manager.py", line 497, in _fetch_result
    result = ray.get(r)
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101040, ip=10.158.21.84, actor_id=d3525b9ea91a53ffeae781f501000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x74d31ffd2ca0>)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 407, in __init__
    self.env = env_creator(copy.deepcopy(self.env_context))
  File "decentralized_ppo.py", line 29, in <lambda>
    lambda _: ParallelPettingZooEnv(
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 198, in __init__
    self.observation_space = gym.spaces.Dict(self.par_env.observation_space)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/dict.py", line 84, in __init__
    assert isinstance(
AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'>

During handling of the above exception, another exception occurred:

ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/algorithms/algorithm.py", line 533, in __init__
    super().__init__(
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/tune/trainable/trainable.py", line 161, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/algorithms/algorithm.py", line 631, in setup
    self.workers = WorkerSet(
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 181, in __init__
    raise e.args[0].args[2]
AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'>

Trial PPO_4x4grid_f6ab8_00000 errored after 0 iterations at 2024-06-05 15:05:41. Total running time: 4s
Error file: /tmp/ray/session_2024-06-05_15-05-34_652990_98780/artifacts/2024-06-05_15-05-36/PPO/driver_artifacts/PPO_4x4grid_f6ab8_00000_0_2024-06-05_15-05-36/error.txt
2024-06-05 15:05:41,608 WARNING experiment_state.py:205 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.
You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.
You can suppress this error by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0).
2024-06-05 15:05:41,608 INFO tune.py:1016 -- Wrote the latest version of all result files and experiment state to '/home/rankit/ray_results/PPO' in 0.0017s.

Trial status: 1 ERROR
Current time: 2024-06-05 15:05:41. Total running time: 4s
Logical resource usage: 0/32 CPUs, 0/1 GPUs (0.0/1.0 accelerator_type:G)
╭────────────────────────────────────╮
│ Trial name                status   │
├────────────────────────────────────┤
│ PPO_4x4grid_f6ab8_00000   ERROR    │
╰────────────────────────────────────╯

Number of errored trials: 1
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Trial name                  # failures   error file                                                                                                                                                   │
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ PPO_4x4grid_f6ab8_00000              1   /tmp/ray/session_2024-06-05_15-05-34_652990_98780/artifacts/2024-06-05_15-05-36/PPO/driver_artifacts/PPO_4x4grid_f6ab8_00000_0_2024-06-05_15-05-36/error.txt │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Traceback (most recent call last):
  File "decentralized_ppo.py", line 62, in <module>
    tune.run(
  File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/tune/tune.py", line 1042, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_4x4grid_f6ab8_00000])
(PPO pid=100933) 2024-06-05 15:05:41,597        ERROR actor_manager.py:517 -- Ray error, taking actor 1 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101040, ip=10.158.21.84, actor_id=d3525b9ea91a53ffeae781f501000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x74d31ffd2ca0>)
(PPO pid=100933) 2024-06-05 15:05:41,598        ERROR actor_manager.py:517 -- Ray error, taking actor 2 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101041, ip=10.158.21.84, actor_id=ba1dbac0fd1f53db94adb0fa01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x73e99a674be0>)
(PPO pid=100933) 2024-06-05 15:05:41,598        ERROR actor_manager.py:517 -- Ray error, taking actor 3 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101042, ip=10.158.21.84, actor_id=62441173fc95b5d7f98b65c401000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x727dee6acbe0>)
(PPO pid=100933) 2024-06-05 15:05:41,598        ERROR actor_manager.py:517 -- Ray error, taking actor 4 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101043, ip=10.158.21.84, actor_id=2e663024f15594f45f8c23ae01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x73a20e032d00>)
(PPO pid=100933) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
(PPO pid=100933)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 229, in _setup
(PPO pid=100933)     self.add_workers(
(PPO pid=100933)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 682, in add_workers
(PPO pid=100933)     raise result.get()
(PPO pid=100933)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/utils/actor_manager.py", line 497, in _fetch_result
(PPO pid=100933)     result = ray.get(r)
(PPO pid=100933) ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101040, ip=10.158.21.84, actor_id=d3525b9ea91a53ffeae781f501000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x74d31ffd2ca0>)
(PPO pid=100933) 
(PPO pid=100933) During handling of the above exception, another exception occurred:
(PPO pid=100933) 
(PPO pid=100933) ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
(PPO pid=100933)     super().__init__(
(PPO pid=100933)     self.setup(copy.deepcopy(self.config))
(PPO pid=100933)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/algorithms/algorithm.py", line 631, in setup
(PPO pid=100933)     self.workers = WorkerSet(
(PPO pid=100933)     raise e.args[0].args[2]
(RolloutWorker pid=101043) Exception ignored in: <function SumoEnvironment.__del__ at 0x73a20db78280>
(RolloutWorker pid=101043) Traceback (most recent call last):
(RolloutWorker pid=101043)   File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 460, in __del__
(RolloutWorker pid=101043)     self.close()
(RolloutWorker pid=101043)   File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 450, in close
(RolloutWorker pid=101043)     traci.close()
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/main.py", line 262, in close
(RolloutWorker pid=101043)     connection.check().close(wait)
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 396, in close
(RolloutWorker pid=101043)     self._sendCmd(tc.CMD_CLOSE, None, None)
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 231, in _sendCmd
(RolloutWorker pid=101043)     return self._sendExact()
(RolloutWorker pid=101043)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 137, in _sendExact
(RolloutWorker pid=101043)     raise FatalTraCIError("Connection closed by SUMO.")
(RolloutWorker pid=101043) traci.exceptions.FatalTraCIError: Connection closed by SUMO.
(RolloutWorker pid=101042)  Retrying in 1 seconds [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/ray-logging.html#log-deduplication for more options.)
(RolloutWorker pid=101042) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 17ms, vehicles TOT 0 ACT 0 BUF 0)                      [repeated 3x across cluster]
(RolloutWorker pid=101042) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101042, ip=10.158.21.84, actor_id=62441173fc95b5d7f98b65c401000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x727dee6acbe0>) [repeated 3x across cluster]
(RolloutWorker pid=101042)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/dict.py", line 84, in __init__ [repeated 27x across cluster]
(RolloutWorker pid=101042)     self.env = env_creator(copy.deepcopy(self.env_context)) [repeated 8x across cluster]
(RolloutWorker pid=101042)   File "decentralized_ppo.py", line 29, in <lambda> [repeated 8x across cluster]
(RolloutWorker pid=101042)     lambda _: ParallelPettingZooEnv( [repeated 8x across cluster]
(RolloutWorker pid=101042)     self.observation_space = gym.spaces.Dict(self.par_env.observation_space) [repeated 8x across cluster]
(RolloutWorker pid=101042)     assert isinstance( [repeated 8x across cluster]
(RolloutWorker pid=101042) AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'> [repeated 9x across cluster]
(RolloutWorker pid=101042) Exception ignored in: <function SumoEnvironment.__del__ at 0x727dee1de280> [repeated 3x across cluster]
(RolloutWorker pid=101042) Traceback (most recent call last): [repeated 3x across cluster]
(RolloutWorker pid=101042)   File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 460, in __del__ [repeated 3x across cluster]
(RolloutWorker pid=101042)     self.close() [repeated 3x across cluster]
(RolloutWorker pid=101042)   File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 450, in close [repeated 3x across cluster]
(RolloutWorker pid=101042)     traci.close() [repeated 3x across cluster]
(RolloutWorker pid=101042)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 396, in close [repeated 6x across cluster]
(RolloutWorker pid=101042)     connection.check().close(wait) [repeated 3x across cluster]
(RolloutWorker pid=101042)     self._sendCmd(tc.CMD_CLOSE, None, None) [repeated 3x across cluster]
(RolloutWorker pid=101042)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 231, in _sendCmd [repeated 3x across cluster]
(RolloutWorker pid=101042)     return self._sendExact() [repeated 3x across cluster]
(RolloutWorker pid=101042)   File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 137, in _sendExact [repeated 3x across cluster]
(RolloutWorker pid=101042)     raise FatalTraCIError("Connection closed by SUMO.") [repeated 3x across cluster]
(RolloutWorker pid=101042) traci.exceptions.FatalTraCIError: Connection closed by SUMO. [repeated 3x across cluster]
@lvyx21
Copy link

lvyx21 commented Jul 12, 2024

I have the same error. Did you solve it?

@VeigarJia
Copy link

I have the same error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@VeigarJia @Kunal-Kumar-Sahoo @lvyx21