Figure 5 - available via license: Creative Commons Zero 1.0
Content may be subject to copyright.
Source publication
We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for zero-shot anomaly segmentation with hybrid prompt regularization to improve the adaptability of modern foundation models. Existing anomaly segmentation models typically rely on domain-specific fine-tuning, limiting their generalization across countless anomaly patterns. In this w...
Context in source publication
Context 1
... prompt (P C ). With the incorporation of anomaly confidence prompts, we limit the number of anomaly regions, which effectively reduces false positives, leading to 0.72% F p average improvements across all categories, as shown in Table 2. The influence of the hyperparameter K in P C is illustrated in Fig. 5. The figure shows that performance initially increases as K improves, as more anomaly regions are accurately detected. However, when K exceeds a certain threshold (around K = 5), the performance drops slightly as more regions are wrongly identified as ...
Similar publications
The quality of videos varies due to the different capabilities of sensors. Video super-resolution (VSR) is a technology that improves the quality of captured video. However, the development of a VSR model is very costly. In this paper, we present a novel approach for adapting single-image super-resolution (SISR) models to the VSR task. To achieve t...
Citations
... The prompt-based model serves as the core of this innovative method, designed to segment anomalies without the necessity of extensive training. This model is structured around a three-part architecture, consisting of an anomaly segment model, a foundation model, and the prompt-based model itself [13]. ...
... The anomaly segmentation model delves into three key anomaly segmentation approaches: encoder-decoder models, feature embedding-based models, and zero-shot tech-niques [13][14][15]. In domains like computer vision, industrial quality control, and cybersecurity, anomaly segmentation is vital. ...
... An- (b) Predicted ground truth. Fig. 8. Visualization comparisons of zero-shot anomaly segmentation for CLIPSeg [38], FL-2 (Florence-2) [39], SAA (Segment Any Anomaly) [13], and Our model. and recall, albeit trailing behind our model and SVM in overall efficacy. ...
This paper proposes an end-to-end pipeline to detect broken eggs in a holder without extensive training, employing a two-step image segmentation and processing approach using saliency scores, all without relying on a large amount of labeled data. The process begins by inputting an egg image with text prompts into Grounding DINO, which returns an egg bounding box. This is followed by the segment anything model (SAM), which extracts the egg’s segmented region. The segmented region is then divided into two crucial components for detection: a binary mask image and a background-removed egg image. The innovation in our method lies in using the saliency score of the estimated anomaly region by employing image processing techniques to effectively distinguish between intact and broken eggs. To validate our approach, we compare it to well-known models such as SVM, XGBoost, and YOLOv8, and we also conduct zero-shot experiments with CLIPSeg, Florence-2, and SAA. In our experimental setup, we utilize 50 egg holder images, each containing both intact and broken eggs. We carefully cropped and processed 30 eggs (arranged in a 6×5 grid) from each holder, resulting in a comprehensive testing dataset totaling 1,500 images. Our results demonstrate the robustness of our method, achieving an impressive 99.56% accuracy in detecting both intact and broken eggs. This breakthrough promises significant advancements in the field of broken egg detection, with broad applications across diverse industries, including food safety, quality control, and automated packaging systems.
... The other way around, the SAM can generate masks for objects in bounding boxes generated by a zero-shot object detector, e.g., Grounding DINO [104], as presented in [91]. [105] uses the same approach to segment anomalies by incorporating domain expert knowledge to prompt the models. [106] assembles CLIP and the SAM to count objects. ...
... Automated, early inspection for trash and damage offers the potential to streamline the process. Since the type and appearance of trash or damage is unknown in advance -a typical use case for anomaly detection, in conjunction with the SAM as discussed in [105]. Experimentally, we used GroundedSAM [91] for early visual identification of trash and unassembled parts using textual prompts, e.g., "plastic wrapping", "fastener" or "clamp". ...
Deploying deep learning-based applications in specialized domains like the aircraft production industry typically suffers from the training data availability problem. Only a few datasets represent non-everyday objects, situations, and tasks. Recent advantages in research around Vision Foundation Models (VFM) opened a new area of tasks and models with high generalization capabilities in non-semantic and semantic predictions. As recently demonstrated by the Segment Anything Project, exploiting VFM's zero-shot capabilities is a promising direction in tackling the boundaries spanned by data, context, and sensor variety. Although, investigating its application within specific domains is subject to ongoing research. This paper contributes here by surveying applications of the SAM in aircraft production-specific use cases. We include manufacturing, intralogistics, as well as maintenance, repair, and overhaul processes, also representing a variety of other neighboring industrial domains. Besides presenting the various use cases, we further discuss the injection of domain knowledge.
