Nan Guo’s research while affiliated with Chinese Association for Artificial Intelligence and other places

Smoke and fire recognition are of great importance on foreseeing fire disasters and preventing environmental pollution by monitoring the burning process of objects (e.g., straw, fuels). However, since fire images suffer from problems like the variability of the features, complexity of scenarios, interference from background, changeable weather conditions as well as image quality problems, identifying smoke and fire accurately and promptly from a given image still remains a substantial challenge. Automatically learning the features of smoke images by CNNs has improved the target recognition ability compared to traditional approaches, nonetheless, convolutions and pooling operations in CNNs may cause severe information loss which may lead to misjudgment. To tackle the above problems, this paper proposed a hybrid attention model based on the characteristics of smoke images. This model adopted multiple optimized attention mechanism in several stages to quickly and precisely capture the important features, achieving state-of-the-art performance on smoke and fire recognition in terms of accuracy and speed. Our proposed module mainly consists of two stages: pooling and attention. In the first stage, we conducted several newly proposed first-order pooling methods. Through traversing the data space in a larger scope, features are better reserved, thus constructing a more intact feature space of smoke and fire in an image. In the second stage, feature maps are aggregated together to perform channel and spatial attention. The channel and spatial dependencies allow us to quickly catch the important features presented in an image. By fully exploring the feature space and prominent salient features, characteristics of smoke and fire are better presented so as to obtain better smoke and fire detection results. Experiments have been conducted on public smoke detection dataset and new proposed fine-grained smoke and fire detection database. Experimental results revealed that the proposed method outperformed popular deep CNNs and existing prevalent attention models for smoke and fire detection problems.

Convolutional Neural Networks (CNNs) can be generally regarded as learning-based visual systems for computer vision tasks. By imitating the operating mechanism of the human visual system (HVS), CNNs can even achieve better results than human beings in some visual tasks. However, they are primary when compared to the HVS for the reason that the HVS has the ability of active vision to promptly analyze and adapt to specific tasks. In this study, a new unified pooling framework was proposed and a series of pooling methods were designed based on the framework to implement active vision to CNNs. In addition, an active selection pooling (ASP) was put forward to reorganize existing and newly proposed pooling methods. The CNN models with ASP tend to have a behavior of focus selection according to tasks during training process, which acts extrememly similar to the HVS.

Meta-learning is one of the latest research directions in machine learning, which is considered to be one of the most probably ways to realize strong artificial intelligence. Meta-learning focuses on seeking solutions for machines to learn to learn like human beings do - to recognize things through only few sample data and quickly adapt to new tasks. Challenges occur in how to train an efficient machine model with limited labeled data, since the model is easily over-fitted. In this paper, we address this obvious but important problem and propose a metric-based meta-learning model, which combines attention mechanisms and ensemble learning method. In our model, we first design a dual path attention module which considers both channel attention and spatial attention module, and the attention modules have been stacked to conduct a meta-learner for few shot meta-learning. Then, we apply an ensemble method called snap-shot ensemble to the attention-based meta-learner in order to generate more models in a single episode. Features abstracted from the models are put into the metric-based architecture to compute a prototype for each class. Our proposed method intensifies the feature extracting ability of backbone network in meta-learner and reduces over-fitting through ensemble learning and metric learning method. Experimental results toward several meta-learning datasets show that our approach is effective.

In recent years, people’s daily lives have become inseparable from a variety of electronic devices, especially mobile phones, which have undoubtedly become necessity in people’s daily lives. In this paper, we are looking for a reliable way to acquire visual quality of the display product so that we can improve the user’s experience with the display product. This paper proposes two major contributions: the first one is the establishment of a new subjective assessment database (DPQAD) of display products’ screen images. Specifically, we invited 57 inexperienced observers to rate 150 screen images showing the display product. At the same time, in order to improve the reliability of screen display quality score, we combined the single stimulation method with the stimulation comparison method to evaluate the newly created display products’ screen images database effectively. The second one is the development of a new no-reference image quality assessment (IQA) metric. For a given image of the display product, first our method extracts 27 features by analyzing the contrast, sharpness, brightness, etc., and then uses the regression module to obtain the visual quality score. Comprehensive experiments show that our method can evaluate natural scene images and screen content images at the same time. Moreover, compared with ten state-of-the-art IQA methods, our method shows obvious superiority on DPQAD.

In recent years, increasing attention has been paid to devising tone mapping operators, which convert specialized high dynamic range (HDR) images to standard low dynamic range (LDR) ones for visualization on daily monitors. However, there lacks a reliable evaluation criterion for comparing distinct tone mapping operators, which is of great significance to the design and optimization of tone mapping methods. In this paper we propose an effective tone mapping evaluation system (TMES) based on a two-stage framework. In the first stage, features are extracted in view of the observations that luminance information, color saturation, statistical naturalness, structural fidelity and visual saliency have different and determinate influences on the perceptual quality of tone-mapped LDR images. In the second stage, the extracted features are integrated with a data-driven optimization strategy, which iteratively learns the parameters by applying thousands of collected tone-mapped LDR and natural images. Our TMES evaluation system can be implemented with or without reference HDR images, serving for the optimization and monitoring of tone mapping methods. Experiments conducted on three databases prove the superiority of our quality evaluation system.

Recent years have witnessed the widespread applications of mobiles and other portable electronic devices, involving a variety of screen content photos. Different from natural scene photos, screen content photos are composed of more lines, fewer colors and unique text messages. Thus, it is difficult to realize the satisfactory enhancement effect of screen content photos using traditional image enhancement technology since they are designed for the natural scene. In this paper, we develop a novel enhancement model for screen content photos by considering text and picture separately. To be specific, we first use a fully convolutional network to divide a screen content photo into three independent parts: picture region, foreground text region, and background. Second, an optimal modification of histogram is used to automatically enhance the picture region’s contrast, and the guided image filter is used to enhance the foreground text region. Third, the enhanced picture region, the enhanced foreground text region, and background are fused to obtain the final enhanced image. Experimental results show that our model has produced less noise and derived outstanding enhancement effect than the popular enhancement techniques.

Recent years have witnessed numerous successful applications of incorporating attention module into feed-forward convolutional neural networks. Along this line of research, we design a novel lightweight general-purpose attention module by simultaneously taking channel attention and spatial attention into consideration. Specifically, inspired by the characteristics of channel attention and spatial attention, a nonlinear hybrid method is proposed to combine such two types of attention feature maps, which is highly beneficial to better network fine-tuning. Further, the parameters of each attention branch can be adjustable for the purpose of making the attention module more flexible and adaptable. From another point of view, we found that the currently popular SE, and CBAM modules are actually two particular cases of our proposed attention module. We also explore the latest attention module ADCM. To validate the module, we conduct experiments on CIFAR10, CIFAR100, Fashion MINIST datasets. Results show that, after integrating with our attention module, existing networks tend to be more efficient in training process and have better performance as compared with state-of-the-art competitors. Also, it is worthy to stress the following two points: (1) our attention module can be used in existing state-of-the-art deep architectures and get better performance at a small computational cost; (2) the module can be added to existing deep architectures in a simple way through stacking the integration of networks block and our module.