On the use of the overlapping area matrix for image segmentation evaluation: A survey and new performance measures

Department of Mathematic and Computer Sciences , University of the Balearic Islands, Palma, Balearic Islands, Spain
Pattern Recognition Letters (Impact Factor: 1.55). 01/2006; DOI: 10.1016/j.patrec.2006.05.002
Source: DBLP

ABSTRACT The development of common and reasonable criteria for evaluating and comparing the performance of segmentation algorithms has always been a concern for researchers in the area. As it is discussed in the paper, some of the measures proposed are not adequate for general images (i.e. images of any sort of scene, without any assumption about the features of the scene objects or the illumination distribution) because they assume a certain distribution of pixel gray-level or colour values for the interior of the regions. This paper reviews performance measures not performing such an assumption and proposes a set of new performance measures in the same line, called the percentage of correctly grouped pixels (CG), the percentage of over-segmentation (OS) and the percentage of under-segmentation (US). Apart from accounting for misclassified pixels, the proposed set of new measures are intended to compute the level of fragmentation of reference regions into output regions and vice versa. A comparison involving similar measures is provided at the end of the paper.

  • Source
    • "Given all performance measures that can be based on pixel counts or object-based detection rates, a final task of interest is to rank the detection algorithms according to their overall performance . Most of the studies (Huang and Dom, 1995; Hoover et al., 1996; Mariano et al., 2002; Ortiz and Oliver, 2006; Jiang "
    [Show abstract] [Hide abstract]
    ABSTRACT: We propose a new procedure for quantitative evaluation of object detection algorithms. The procedure consists of a matching stage for finding correspondences between reference and output objects, an accuracy score that is sensitive to object shapes as well as boundary and fragmentation errors, and a ranking step for final ordering of the algorithms using multiple performance indicators. The procedure is illustrated on a building detection task where the resulting rankings are consistent with the visual inspection of the detection maps.
    Pattern Recognition Letters 07/2010; 31:1128-1137. DOI:10.1016/j.patrec.2009.10.016 · 1.55 Impact Factor
  • Source
    • "(black or white) and two phases are defined (Ortiz and Oliver, 2006; 123 Seul et al., 2006; Gonzalez and Woods, 1992; Russ, 1998; Bernd, 124 1997). In this case, binary segmentation was fixed to the threshold "
    [Show abstract] [Hide abstract]
    ABSTRACT: In this research the spatial distribution characterization of niobium phosphate into bleached cellulose was carried out combining processing and images analysis obtained by SEM and statistical methodologies. The objective is to investigate the deposit composition and phosphate morphology by using complementary analytical techniques. Based on the proposed methodology, parameters of niobium phosphate agglomerates (size and shape) and fiber morphology were evaluated depending on gray-levels (average luminance and fiber type): fiber characteristics (morphology) were measured. For the test method proposed, a specific region of cellulose/NbOPO(4) x nH(2)O composite was analyzed. This method involves area fraction measuring with a conditional probabilistic analysis. The analyzed fields were divided in different ways, called 'Scanning' and as a result, in quantitative terms, the phosphate deposition was described as spatial distribution homogeneous or inhomogeneous. The quantitative microscopy as a non-destructive testing provides relevant information when it is combined with statistic analysis.
    Micron 03/2010; 41(5):402-11. DOI:10.1016/j.micron.2010.02.012 · 2.06 Impact Factor
  • Source
    • "Let C ij be the number of pixels in the i'th object in a reference map that overlap with the j'th object in an output map produced by an algorithm. Ortiz and Oliver [20] "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper presents the initial results of the algorithm performance contest that was organized as part of the 5th IAPR Workshop on Pattern Recognition in Remote Sensing (PRRS 2008). The focus of the 2008 contest was automatic building detection and digital surface model (DSM) extraction. A QuickBird data set with manual ground truth was used for building detection evaluation, and a stereo Ikonos data set with a highly accurate reference DSM was used for DSM extraction evaluation. Nine submissions were received for the building detection task, and three submissions were received for the DSM extraction task. We provide an overview of the data sets, the summaries of the methods used for the submissions, the details of the evaluation criteria, and the results of the initial evaluation.
    Pattern Recognition in Remote Sensing (PRRS 2008), 2008 IAPR Workshop on; 01/2009
Show more