What is Semantic Segmentation?
Semantic segmentation is a computer vision task that labels every pixel of an image with a class.
Long, Shelhamer and Darrell describe the task as labeling each pixel with the class of its enclosing object or region. Their fully convolutional networks take an image of any size. They return a label map of the same size.
Related tasks
Instance segmentation detects each object and segments it separately. Semantic segmentation does not. It gives the same label to two cracks. Panoptic segmentation combines both tasks and gives each pixel in the scene one consistent label.
How a model is built and scored
Many models follow the U-Net design. A contracting path captures context. A symmetric expanding path restores the position of each label. Its authors trained it on very few images.
Two common scores are pixel accuracy and mean intersection over union (mean IU). Pixel accuracy is the number of correct pixels divided by all pixels. Mean IU compares predicted and true pixels for each class and averages the result.
Use in airfield inspection
The PCI of an airfield pavement is determined under ASTM D5340 (FAA AC 150/5380-6C, section 2.4). The FAA circular does not mention image segmentation. This page makes no claim about how well a segmentation model finds pavement distress.
Frequently Asked Questions
- What is the difference between semantic, instance and panoptic segmentation?
- Semantic segmentation gives each pixel a class but does not separate objects of the same class. Instance segmentation detects and segments each object on its own. Panoptic segmentation combines both, so that every pixel in the scene gets one consistent label.
- How is a semantic segmentation model scored?
- Two common scores are pixel accuracy and mean intersection over union (mean IU). Pixel accuracy is the share of pixels with the correct class. Mean IU averages, over all classes, the overlap between predicted and true pixels divided by their union.
- What is a U-Net?
- A U-Net is a convolutional network for segmentation with a contracting path that captures context and a symmetric expanding path that gives precise localization. Ronneberger, Fischer and Brox introduced it in 2015 for biomedical images and trained it on very few images.
References
- fcn-long-2015
- Fully Convolutional Networks for Semantic Segmentation Long, Shelhamer and Darrell, CVPR 2015 (arXiv:1411.4038)
- unet-2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation Ronneberger, Fischer and Brox (arXiv:1505.04597)
- panoptic-2018
- Panoptic Segmentation Kirillov, He, Girshick, Rother and Dollar (arXiv:1801.00868, CVPR 2019)
- faa-ac-150-5380-6c
- AC 150/5380-6C, Guidelines and Procedures for Maintenance of Airport Pavements FAA
This is a reference entry. TarmacView provides drone inspection services and doesn't sell the equipment or materials described in this glossary.
