ID	139514519
Max Iter	270000
lr sched	3x
Backbone Layers	101
train time (s/iter)	0.392
Training Memory (GB)	6.0
inference time (s/im)	0.066

ID	139514544
Max Iter	90000
lr sched	1x
Backbone Layers	50
train time (s/iter)	0.304
Training Memory (GB)	4.8
inference time (s/im)	0.053

ID	139514569
Max Iter	270000
lr sched	3x
Backbone Layers	50
train time (s/iter)	0.302
Training Memory (GB)	4.8
inference time (s/im)	0.053

Panoptic FPN

facebookresearch / detectron2

Last updated on Feb 19, 2021

Parameters 65 Million

inference time (s/im) 0.066

File Size 248.51 MB

Training Data MS COCO

Training Resources 8 NVIDIA V100 GPUs

Training Time 1.22 days

Architecture	Panoptic FPN, Mask R-CNN, RoiAlign, ResNet
ID	139514519
Max Iter	270000
lr sched	3x
Backbone Layers	101
train time (s/iter)	0.392
Training Memory (GB)	6.0
inference time (s/im)	0.066
SHOW MORE
SHOW LESS

Parameters 46 Million

inference time (s/im) 0.053

File Size 175.83 MB

Training Data MS COCO

Training Resources 8 NVIDIA V100 GPUs

Training Time 8 hours

Architecture	Panoptic FPN, Mask R-CNN, RoiAlign, ResNet
ID	139514544
Max Iter	90000
lr sched	1x
Backbone Layers	50
train time (s/iter)	0.304
Training Memory (GB)	4.8
inference time (s/im)	0.053
SHOW MORE
SHOW LESS

Parameters 46 Million

inference time (s/im) 0.053

File Size 175.83 MB

Training Data MS COCO

Training Resources 8 NVIDIA V100 GPUs

Training Time 23 hours

Architecture	Panoptic FPN, Mask R-CNN, RoiAlign, ResNet
ID	139514569
Max Iter	270000
lr sched	3x
Backbone Layers	50
train time (s/iter)	0.302
Training Memory (GB)	4.8
inference time (s/im)	0.053
SHOW MORE
SHOW LESS

README.md

Summary

Panoptic FPN endows Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone. This allows for the model to be applied to the panoptic segmentation task.

How do I load this model?

There are several Panoptic FPN models available in Detectron2, with different backbones and learning schedules.

To load from the Detectron2 model zoo:

from detectron2 import model_zoo
model = model_zoo.get("COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml", trained=True)

Replace the configuration path with the variant you want to use. You can find the paths in the model summaries at the top of this page.

How do I train this model?

You can follow the Getting Started guide on Colab to see how to train a model.

You can also read the official Detectron2 documentation.

Citation

@misc{wu2019detectron2,
  author =       {Yuxin Wu and Alexander Kirillov and Francisco Massa and
                  Wan-Yen Lo and Ross Girshick},
  title =        {Detectron2},
  howpublished = {\url{https://github.com/facebookresearch/detectron2}},
  year =         {2019}
}

Results

Panoptic Segmentation on COCO minival

Panoptic Segmentation

BENCHMARK	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
COCO minival	Panoptic FPN (R101-FPN, 3x)	PQ	43.0	# 1
		boxAP	42.4	# 1
		maskAP	38.5	# 1
COCO minival	Panoptic FPN (R50-FPN, 3x)	PQ	41.5	# 2
		boxAP	40.0	# 2
		maskAP	36.5	# 2
COCO minival	Panoptic FPN (R50-FPN, 1x)	PQ	39.4	# 3
		boxAP	37.6	# 3
		maskAP	34.7	# 3