Xin Jin

• PhD
• Professor (Associate) at Beijing Electronic Science and Technology Institute

Datasets play a pivotal role in training visual models, facilitating the development of abstract understandings of visual features through diverse image samples and multidimensional attributes. However, in the realm of aesthetic evaluation of artistic images, datasets remain relatively scarce. Existing painting datasets are often characterized by l...

There are many techniques for recoloring images with different effects and improving color discrimination in patients with color vision defects. However, certain issues still persist, such as the unnatural and discordant colors of objects in the converted image. To address these problems, we have explored a comprehensive set of methods to achieve i...

In spite of great success in many image recognition tasks achieved by recent deep models, directly applying them to recognize low-resolution images may suffer from low accuracy due to the missing of informative details during resolution degradation. However, these images are still recognizable for subjects who are familiar with the corresponding hi...

Image aesthetic evaluation is a highly prominent research domain in the field of computer vision. In recent years, there has been a proliferation of datasets and corresponding evaluation methodologies for assessing the aesthetic quality of photographic works, leading to the establishment of a relatively mature research environment. However, in cont...

Despite the existence of numerous methods for recoloring images with diverse effects, challenges such as unnatural and inharmonious colors of the converted objects still persist. To address these issues, we have developed a novel approach to image recoloration. Our method ensures that the resulting images possess three crucial properties: naturalne...

In recent years, image aesthetic quality assessment has become increasingly popular. In addition to numerical assessment, aesthetic captioning has been proposed to capture the overall aesthetic impression of an image. To further advance this field, we address a task of aesthetic attribute assessment, which is the aesthetic multi-attributes captioni...

With the rise of the information age, digitalization and paperless processes have become the norm in managing archive images. However, research on extracting and managing image content from archives is still in its early stages, and is primarily focused on recognizing fixed-format archive images. As a result, there is a lack of technology for extra...

This paper introduces a novel watermarking algorithm based on the U-Net Network to manage and trace the source of fax document leaks. The goal is to enable the embedding and extraction of watermark information in fax text images. To adapt the U-Net structure to the unique characteristics of text images transmitted through fax channels and cross-med...

Archives contain valuable historical information and must be properly preserved. However, traditional archival materials are vulnerable to damage from water, fire, and mold, making long-term storage difficult. To address this issue, digital archives have been established for management. As a result, effective storage, detection, extraction, and uti...

Although computational aesthetics evaluation has made certain achievements in many fields, its research of music performance remains to be explored. At present, subjective evaluation is still a ultimate method of music aesthetics research, but it will consume a lot of human and material resources. In addition, the music performance generated by AI...

With the continuous development of social software and multimedia technology, images have become a kind of important carrier for spreading information and socializing. How to evaluate an image comprehensively has become the focus of recent researches. The traditional image aesthetic assessment methods often adopt single numerical overall assessment...

Computational aesthetics evaluation has made great achievements in the field of visual arts, but the research work on music still needs to be explored. Although the existing work of music generation is very substantial, the quality of music score generated by AI is relatively poor compared with that created by human composers. The music scores crea...

Image forgery localization aims to identify forged regions by capturing subtle traces from high-quality discriminative features. In this paper, we propose a Transformer-style network with two feature extraction branches for image forgery localization, and it is named as Two-Branch Transformer (TBFormer). Firstly, two feature extraction branches are...

With the development of the information age, the awareness of archives informatization of governments, organizations and groups at all levels is constantly increasing. The key content extraction of scanned archive images has been widely concerned by researchers. Based on the deep learning method, we extract text and key elements (ID photo, seal, da...

In recent years, due to the improvement of scientific research methods and the wide-open source and acquisition of related data sets, face stylization has become a hot research field and application direction. There is a need to stylize face images in many applications, such as camera beauty, artistic photo processing, etc. However, most of the cur...

In the tasks of image aesthetic quality assessment, it is difficult to reach both the high score area and low score area due to the normal distribution of aesthetic datasets. To reduce the error in labelling and solve the problem of normal data distribution, we propose a new aesthetic mixed dataset with classification and regression called AMD-CR,...

In film art, the usage of various fields of view in a movie reflects the shooting style of the director or artist. Analyzing which field of view was used by the shot can help us achieve the semi-automatic design and generation of the storyboard. Before achieving the generation of movie shots, the corresponding field of view recognition method and f...

Nowadays, the portrait image aesthetic attribute evaluation is extremely valuable in the research area, but there is no either specialized portrait image aesthetic attribute dataset or specialized portrait image aesthetic dataset. At the same time, the lightweight image aesthetic attribute evaluation model is also valuable in the market. In this pa...

Aesthetic assessment of images can be categorized into two main forms: numerical assessment and language assessment. Aesthetics caption of photographs is the only task of aesthetic language assessment that has been addressed. In this paper, we propose a new task of aesthetic language assessment: aesthetic visual question and answering (AVQA) of ima...

With the vigorous development of mobile photography technology, major mobile phone manufacturers are scrambling to improve the shooting ability of equipments and the photo beautification algorithm of software. However, the improvement of intelligent equipments and algorithms cannot replace human subjective photography technology. In this paper, we...

Image aesthetic quality assessment is popular during the last decade. Besides numerical assessment, nature language assessment (aesthetic captioning) has been proposed to describe the generally aesthetic impression of an image. In this paper, we propose aesthetic attribute assessment, which is the aesthetic attributes captioning, i.e., to assess th...

In recent years, image generation has made great strides in improving the quality of images, producing high-fidelity ones. Also, quite recently, there are architecture designs, which enable GAN to unsupervisedly learn the semantic attributes represented in different layers. However, there is still a lack of research on generating face images more c...

There are many tasks on image aesthetic assessment, such as aesthetic classification, scoring, score distribution prediction, and captions. Due to the distribution of the aesthetic score is unbalanced, the assessment models always output scores near the mean score. In this paper, we propose a fine-grained regression method for aesthetics score regr...

With the continuous development of social software and multimedia technology, images have become a kind of important carrier for spreading information and socializing. How to evaluate an image comprehensively has become the focus of recent researches. The traditional image aesthetic assessment methods often adopt single numerical overall assessment...

Automatic ultrasound image segmentation plays an important role in early diagnosis of human diseases. This paper introduces a novel and efficient encoder–decoder network, called Lightweight Attention Encoder–Decoder Network (LAEDNet), for automatic ultrasound image segmentation. In contrast to previous encoder–decoder networks that involve complica...

When taking photos of a suspicious missing child in the street and posting them to the social network is becoming a feasible way to find missing children, the exposure of photos may cause privacy issues. To address this problem, we propose Double-Blinded Finder, an efficient and double-blinded system for finding missing children via low-dimensional...

In the tasks of image aesthetic quality evaluation, it is difficult to reach both the high score area and low score area due to the normal distribution of aesthetic datasets. To reduce the error in labeling and solve the problem of normal data distribution, we propose a new aesthetic mixed dataset with classification and regression called AMD-CR, a...

In industry, there exist plenty of scenarios where old gray photos need to be automatically colored, such as video sites and archives. In this paper, we present the HistoryNet focusing on historical person's diverse high fidelity clothing colorization based on fine grained semantic understanding and prior. Colorization of historical persons is real...

How to realize data transmission between Intranet and extranet devices in physical isolation is an important problem. QR code can be used to identify text, image information, can be used as a carrier of information exchange. In this paper, QR code technology is used to propose a data transmission system between the Internet and the Internet under t...

For indoor scenes, the fourth-order spherical harmonic function is used to model the illumination, resulting in that 48 spherical harmonic coefficients are used to represent the whole scene. The illumination contained in the low dynamic range image is insufficient, so high dynamic range environment maps are adopted in this part, and the aim is to p...

When inserting a virtual object into an outdoor image, the recovery of sun orientation has a critical effect on the fusion of virtual objects and real scenes. For the problem of sun orientation estimation from a single outdoor image, a new end-to-end learning method is proposed in this paper. A luminance channel of sky region is introduced into the...

With the development of big data technology, privacy protection of computer vision algorithm has become a hot research field. To protect privacy of both terminal and cloud in the object detection algorithm, Jin et al. proposed a method to represent an image with multiple random sub-images. The cloud detects objects from these random sub-images inst...

Face images contain rich and diverse information and have become the focus of research in the field of computer vision. The current research on facial images includes facial recognition, facial image makeup transfer, facial image segmentation, facial image illumination transfer, facial image beautification and rendering. Among them, the illuminatio...

Viola-Jones algorithm (Viola and Jones 2001; Int J Comput Vis 57(2):137–154, 2004) is a milestone in the development of face detection technology. It greatly improves the efficiency of face detection on the premise of ensuring the accuracy, which indicates that the research results in the field of computer vision have the ability to put into practi...

The huge computational overhead limits the inference of convolutional neural networks on mobile devices for object detection, which plays a critical role in many real-world scenes, such as face identification, autonomous driving, and video surveillance. To solve this problem, this paper introduces a lightweight convolutional neural network, called...

With the development of deep learning technology, many deep learning methods have been applied to font recognition and generation. However, few studies focus on font inpainting problems. This paper is dedicated to repairing damaged fonts based on style to repair damaged fonts in a better way. In this paper, we propose a CGAN (Conditional Generative...

With the development of cloud computing, the storage and processing of massive visual media data has gradually transferred to the cloud server. For example, if the intelligent video monitoring system cannot process a large amount of data locally, the data will be uploaded to the cloud. Therefore, how to process data in the cloud without exposing th...

The task of aesthetic quality assessment is complicated due to its subjectivity. In recent years, the target representation of image aesthetic quality has changed from a one-dimensional binary classification label or numerical score to a multi-dimensional score distribution. According to current methods, the ground truth score distributions are str...

Nowadays, more and more people choose to upload their data and information to cloud. The cloud saves users’ data and information and starts providing computing services and artificial intelligence analysis. However, the privacy of users’ information and data will be completely exposed to the cloud without any protection. In this paper, we propose t...

The image-based virtual illumination technology directly changes the illumination effect of objects on the image. It does not require complex light propagation simulation calculations, it is an image-based rendering technology. The image mainly relies on the imaging of the visual sensor, and uses the virtual illumination technology of the image to...

With the increasing popularity of multimedia data on the internet and our daily life, the way people get information is becoming more diverse, video is becoming an important part of our dailylife and work. At the same time, the popularity of video also brings new security risks and challenges, more and more people are paying attention to video secu...

Complex object illumination transfer is a special challenge in computer vision. In our paper, we put forward a method for complex object illumination transfer. Firstly, the input object image was divided into object components by semantic analysis, to find the reference object images consistent with the object component material in the physical wor...

Complex object illumination transfer presents a particular challenge in computer vision. We took up the challenge and proposed a novel complex object illumination transfer method through semantic and material parsing and composition. Firstly, we parsed the subject complex object into several parts through semantic parsing. We could easier match sim...

Posting photos of suspected missing children on the street and posting them to social networks can help find missing children, but posting them to the Internet without protection may create privacy issues. In order to solve this problem, we propose Double-Blinded Finder, which is mainly used for low-dimensional multi-attribute matching of children’...

With the development of cloud computing, the storage and processing of massive visual media data has gradually transferred to the cloud server. For example, if the intelligent video monitoring system cannot process a large amount of data locally, the data will be uploaded to the cloud. Therefore, how to process data in the cloud without exposing th...

When a virtual object is inserted into an outdoor image, the recovery of scene illumination has a critical effect on the mix of virtual objects and actual reality. There are two main parts of the object in the outdoor scene: the sun and the sky. In order to represent the illumination conditions of these two natural illumination, this paper uses the...

The aesthetic quality assessment of image is a challenging work in computer vision field. The recent research work used the deep convolutional neural network to evaluate the aesthetic quality of images. However, the score of image data sets has a strongly normal distribution, which makes the training of neural network easy to be over-fitting. In ad...

Image completion has always been an important research area of image processing. With the continuous development of the deep learning model in recent years, further progress has been made in the repair of images. In this paper, we focused on realistic and painting portrait data, studied on semantic inpainting techniques based on regional completion...

Nowadays, with the maturity and wide application of face recognition technology, the recognition accuracy, recognition efficiency, and data security have attracted people’s attention. However, when face recognition is performed, face information is completely exposed to the cloud server without any protection measures. Therefore, a series of proble...

In spite of great success in many image recognition tasks achieved by recent deep models, directly applying them to recognize low-resolution images may suffer from low accuracy due to the missing of informative details during resolution degradation. However, these images are still recognizable for subjects who are familiar with the corresponding hi...

The illumination effect is essential for the realistic results in images which are created by inserting virtual objects into real scene. For outdoor scenes, automatic estimation of sun orientation condition from a single outdoor image is fundamental for inserting 3D models to a single image. Traditional methods for outdoor sun orientation estimatio...

The aesthetic quality assessment of image is a challenging work in computer vision field. The recent research work used the deep convolutional neural network to evaluate the aesthetic quality of images. However, the score of image data sets has a strongly normal distribution, which makes the training of neural network easy to be over-fitting. In ad...

Occluded face recognition, which has an attractive application in the visual analysis field, is challenged by the missing cues due to heavy occlusions. Recently, several face inpainting methods based on generative adversarial networks (GANs) fill in the occluded parts by generating images fitting the real image distributions. They can lead to a vis...

With the rapid progress of virtual reality and augmented reality technologies, 3D contents are the next widespread media in many applications. Thus, the protection of 3D models is primarily important. Encryption of 3D models is essential to maintain confidentiality. Previous work on encryption of 3D surface model often considers the point clouds, t...

It is difficult and challenging to evaluate the aesthetics quality of images from multiple angles. Since humans’ perception of images comes from many factors, the integrated image aesthetic quality assessment cannot be easily summarized by few attributes. A comprehensive evaluation is supposed to predict many aesthetic attributes across not only on...

Recently, cloud storage and processing have been widely adopted. Mobile users in one family or one team may automatically backup their photos to the same shared cloud storage space. The powerful face detector trained and provided by a 3rd party may be used to retrieve the photo collection which contains a specific group of persons from the cloud st...

We propose a novel AR CAPTCHA that first uses Augmented Reality to design CAPTCHA. Users should use their cameras on mobile devices to capture a marker in the 3D physical world or PC screens to find the appropriate angle to recognize each 3D character rendered on the marker in a 3D registration manner, which is very hard for machines or robots to d...

The recent years have witnessed great advances for semantic segmentation using deep convolutional neural networks (DCNNs). However, a large number of convolutional layers and feature channels lead to semantic segmentation as a computationally heavy task, which is disadvantage to the scenario with limited resources. In this paper, we design an effic...

Convolutional neural networks (CNNs) have shown vulnerable to adversarial examples which contain imperceptible perturbations. In this paper, we propose an approach to defend against adversarial examples with soft decision trees embedding. Firstly, we extract the semantic features of adversarial examples with a feature extraction network. Then, a sp...

Image aesthetic quality assessment has been a relatively hot topic during the last decade. Most recently, comments type assessment (aesthetic captions) has been proposed to describe the general aesthetic impression of an image using text. In this paper, we propose Aesthetic Attributes Assessment of Images, which means the aesthetic attributes capti...

Image aesthetic quality assessment has been a relatively hot topic during the last decade. Most recently, comments type assessment (aesthetic captions) has been proposed to describe the general aesthetic impression of an image using text. In this paper, we propose Aesthetic Attributes Assessment of Images, which means the aesthetic attributes capti...

To meet the women appearance needs, we present a novel virtual experience approach of facial makeup transfer, developed into windows platform application software. The makeup effects could present on the user's input image in real time, with an only single reference image. The input image and reference image are divided into three layers by facial...

To meet the women appearance needs, we present a novel virtual experience approach of facial makeup transfer, developed into windows platform application software. The makeup effects could present on the user’s input image in real time, with an only single reference image. The input image and reference image are divided into three layers by facial...

The aesthetic quality assessment of images is a challenging work in the field of computer vision because of its complex subjective semantic information. The recent research work can utilize the deep convolutional neural network to evaluate the overall score of the image. However, the focus in the field of aesthetic is often not limited to the total...

Recent deep trackers have achieved impressive performance in visual tracking. Typically, these trackers apply complex deep networks with massive parameters for representing objects, which makes their deployment on resource-limited devices very challenging due to two reasons: (1) high computation complexity, and (2) high storage footprint. To addres...

Image relighting is to change the illumination of an image to a target illumination effect without known the original scene geometry, material information and illumination condition. We propose a novel outdoor scene relighting method, which needs only a single reference image and is based on material constrained layer decomposition. Firstly, the ma...

The illumination effect is essential for the realistic results in images which are created by inserting virtual objects into real scene. For outdoor scenes, automatic estimation of sun orientation condition from a single outdoor image is fundamental for inserting 3D models to a single image. Traditional methods for outdoor sun orientation estimatio...

In this study, the authors address a challenging problem of aesthetic image classification, which is to label an input image as high- or low-aesthetic quality. We take both the local and global features of images into consideration. A novel deep convolutional neural network named ILGNet is proposed, which combines both the inception modules and a c...

To protect the contents of images in the mobile internet era during image storage and transmission, image encryption has achieved a tremendous success during the last decades. Traditional color image encryption method often use the RGB color space. We have the observation that in non-RGB color spaces, the luminance channels often contain more infor...

When combining very different images which often contain complex objects and backgrounds, producing consistent compositions is a challenging problem requiring seamless image editing. In this paper, we propose a general approach, called object-aware image editing, to obtain consistency in structure, color, and texture in a unified way. Our approach...