SDVRP env: Why is the masking for the current loc not needed??

[ShuN6211]

I'm currently working on implementing my own env that is partially related to SDVRP env.
In the masking for SDVRP, it seems the current node is not masked. Because reward is the negative tour length, does staying at a same node result in 0, which is the max reward?? (but can't pass check_solution_validity method)
This case may be a trivial, but it seems to be allowed.
Is there any reason for the masking for the current node not needed?
Best regards

rl4co/rl4co/envs/routing/sdvrp/env.py

Lines 132 to 140 in b516158

	@staticmethod
	def get_action_mask(td: TensorDict) -> torch.Tensor:
	mask_loc = (td["demand_with_depot"][..., 1:] == 0) | (
	td["used_capacity"] >= td["vehicle_capacity"]
	)
	mask_depot = (td["current_node"] == 0).squeeze(-1) & (
	(mask_loc == 0).int().sum(-1) > 0
	)
	return ~torch.cat((mask_depot[..., None], mask_loc), -1)

[fedebotu]

Hi @ShuN6211 !

The current node may not be masked only if it is the depot:

     mask_depot = (td["current_node"] == 0).squeeze(-1) & (  # (1)
         (mask_loc == 0).int().sum(-1) > 0 # (2)
     ) 

this means roughly: "you cannot go to the depot if (1) you are already at the depot (which has index 0) and (2) there are still unvisited node that you can visit outside of it"

Note that if the episode is over, you are always allowed to stay in the depot indefinetely to wait for other episodes in the same batch to finish.

Does this answer your question? :)

[ShuN6211]

Hi @fedebotu! Thanks for answering!! I feel we need more discussion on the topic!

(1) you are already at the depot (which has index 0) and (2) there are still unvisited node that you can visit outside of it

I understand this point. 👌 My concern is about the locs rather than the depot.

mask_loc = (td["demand_with_depot"][..., 1:] == 0) | ( 
         td["used_capacity"] >= td["vehicle_capacity"] 
     ) 

In my understanding, this means "you can go to the loc that still haves unfilled demand, but if remaining vehicle capacity is empty, you can't go to any of locs " ...Oh, I understand the things during editing this message!!
I suspected the constructive policy(Decoding strategy's step) may select the same loc in a raw if mask for the current loc is not provided. (ex. actions = [depot, 1, 1, 1, ...], and reward of this episode is negative of distance between the depot and the loc indexed 1, rather big than that of feasible solutions.)
However. this case could never happen because "(1) if demand of the loc is filled, that loc will be never visited in the episode.
(2) if the demand is not filled but remain, the remaining vehicle's capacity is empty and should go back to the depot".

Conclusion: masking for current locs is not needed!

Well, this discussion was a great help for me!! thanks a lot!! (very minor point...😢)

[fedebotu]

Awesome! Hope this helps others too to clarify doubts~