Ralph R. Martin’s research while affiliated with Cardiff University and other places

Understanding how people perceive the visual complexity of shapes has important theoretical as well as practical implications. One school of thought, driven by information theory, focuses on studying the local features that contribute to the perception of visual complexity. Another school, in contrast, emphasizes the impact of global characteristics of shapes on perceived complexity. Inspired by recent discoveries in neuroscience, our model considers both local features of shapes: edge lengths and vertex angles, and global features: concaveness, and is in 92% agreement with human subjective ratings of shape complexity. The model is also consistent with the hierarchical perceptual learning theory, which explains how different layers of neurons in the visual system act together to yield a perception of visual shape complexity.

Accurate camera pose estimation is essential and challenging for real world dynamic 3D reconstruction and augmented reality applications. In this paper, we present a novel RGB-D SLAM approach for accurate camera pose tracking in dynamic environments. Previous methods detect dynamic components only across a short time-span of consecutive frames. Instead, we provide a more accurate dynamic 3D landmark detection method, followed by the use of long-term consistency via conditional random fields, which leverages long-term observations from multiple frames. Specifically, we first introduce an efficient initial camera pose estimation method based on distinguishing dynamic from static points using graph-cut RANSAC. These static/dynamic labels are used as priors for the unary potential in the conditional random fields, which further improves the accuracy of dynamic 3D landmark detection. Evaluation using the TUM \zjc{and Bonn RGB-D dynamic datasets shows that our approach significantly outperforms state-of-the-art methods, providing much more accurate camera trajectory estimation in a variety of highly dynamic environments. We also show that dynamic 3D reconstruction can benefit from the camera poses estimated by our RGB-D SLAM approach.

In order to accurately count the number of animals grazing on grassland, we present a livestock detection algorithm using modified versions of U-net and Google Inception-v4 net. This method works well to detect dense and touching instances. We also introduce a dataset for livestock detection in aerial images, consisting of 89 aerial images collected by quadcopter. Each image has resolution of about 3000×4000 pixels, and contains livestock with varying shapes, scales, and orientations. We evaluate our method by comparison against Faster RCNN and Yolo-v3 algorithms using our aerial livestock dataset. The average precision of our method is better than Yolo-v3 and is comparable to Faster RCNN.

In this work, we address a novel and practical problem of automatically generating a room design from given room function and basic geometry, which can be described as picking appropriate objects from a given database, and placing the objects with a group of pre-defined criteria. We formulate both object selection and placement problems as probabilistic models. The object selection is first formulated as a supervised generative model, to take room function into consideration. Object placement problem is then formulated as a Bayesian model, where parameters are inferred with Maximizing a Posteriori (MAP) objective. We solve the placement problem efficiently by introducing a solver based on Markov Chain Monte Carlo with a specific proposal function designed for the problem.

We propose a novel framework for semantically enhancing a 3D indoor scene in agreement with a user-provided guide word. To do so, we make changes to furniture colors and place small objects in the scene. The relevance of specific furniture colors and small objects to each guide word is learned from a database of annotated images, taking into account both their frequency and specificity to that guide word. Enhancement suggestions are generated by optimizing a scoring function, which combines the relevance of both enhancement factors, i.e., furniture colors and small objects. During optimization, a submodular set function is adopted to ensure that a diverse set of enhancement suggestions is produced. Our experiments show that this framework can generate enhancement suggestions that are both compatible with the input guide word, and comparable to ones designed by humans.

In this work, we address a novel and practical problem of automatically generating a room design from given room function and basic geometry, which can be described as picking appropriate objects from a given database, and placing the objects with a group of pre-defined criteria. We formulate both object selection and placement problems as probabilistic models. The object selection is first formulated as a supervised generative model, to take room function into consideration. Object placement problem is then formulated as a Bayesian model, where parameters are inferred with Maximizing a Posteriori (MAP) objective. By introducing a solver based on Markov Chain Monte Carlo (MCMC), the placement problem is solved efficiently.

We present a system that automatically recovers scene geometry and illumination from a video, providing a basis for various applications. Previous image based illumination estimation methods require either user interaction or external information in the form of a database. We adopt structure-from-motion and multi-view stereo for initial scene reconstruction, and then estimate an environment map represented by spherical harmonics (as these perform better than other bases). We also demonstrate several video editing applications that exploit the recovered geometry and illumination, including object insertion (e.g., for augmented reality), shadow detection, and video relighting.

Objective image quality metrics (IQMs) potentially benefit from the addition of visual saliency. However, challenges to optimising the performance of saliency-based IQMs remain. A previous eye-tracking study has shown that gaze is concentrated in fewer places in images with highly salient features than in images lacking salient features. From this, it can be inferred that the former are more likely to benefit from adding a saliency term to an IQM. To understand whether these ideas still hold when using computational saliency instead of eyetracking data, we first conducted a statistical evaluation using 15 state of the art saliency models and 10 well-known IQMs. We then used the results to devise an algorithm which adaptively incorporates saliency in IQMs for natural scenes, based on saliency dispersion. Experimental results demonstrate this can give significant improvements.

Panoramic video stitching consists of two major steps: remapping each candidate video stream to its final position and compositing them to generate seamless results. Given videos captured with cameras in fixed relative positions, the remapping step can be done directly using a precomputed look-up table. Greater challenges lie in the more time-consuming composition step. Real world applications typically use blending to perform composition; the performance of the whole system largely depends on the efficiency of the blending algorithm. In this paper, we provide in-depth analysis of the application of several state-of-the-art image blending techniques to realtime panoramic video stitching, as realtime panoramic video stitching enables near-immediate broadcast. Test videos were captured under various conditions, and stitched using various blending methods. Both computational efficiency and quality of composition results were evaluated. Source code and test videos are all publicly available.