Feature Request : Segmentation model

[bluesky314]

The paper lists an easy way to use the model for segmentation. I really hope there is enough flexibility in your code to allow for that alteration.

Following [16], we modify our EfficientDet model to keep feature level {P2,P3,...,P7} in BiFPN, but only use P2 for the fi- nal per-pixel classification. For simplicity, here we only evaluate a EfficientDet-D4 based model, which uses a Ima- geNet pretrained EfficientNet-B4 backbone (similar size to ResNet-50). We set the channel size to 128 for BiFPN and 256 for classification head. Both BiFPN and classification head are repeated by 3 times.

[rwightman]

@bluesky314 I would like to try this, but need to get obj detection training running first, a bit busy for a while so not sure when I'll get to it quite yet

[bluesky314]

Ok, I think if you get OD to run then most of the coding work would be done. We only need to add another path by changing bifpn_sum_config. Then we just take the last P2 and attach a segmentation head.

[rwightman]

@bluesky314 yeah, it should be fairly straight forward, but still making big improvements in the core model/post processing.

One concern I have with the segmentation with the Tensowflow SAME equivalent padding enabled is that the feature maps are likely not lined up well (the reason for aligned Xception for DeepLab). Symmetric PyTorch style padding will likely be better there as that is essnetially what aligned Xception was, manually implement PyTorch padding in TF. That should be easy to flip for this code base, but may require some tweaks to make the map sizing sensible.

[lucasjinreal]

@rwightman Do u have plan to switch Yolact backbone to efficientnet to do instance segmentation? It's also most same in terms of objecet detection part (only difference is backbone and FPN) both retina architecture and design. it would be very awsome if instance segmentation implemented upon efficientdet. (I believe Yolact is at the bottle neck of it's detector, but it's simplicity and speed quite amazing)

[faheemullah]

@rwightman, @bluesky314, How will be its diagram if we add segmentation also to EfficientDet in order to get results like Mask R-CNN? The paper has explained briefly but I couldn't get that.