Neil D. B. Bruce

Neil D. B. Bruce
University of Manitoba | UMN · Department of Computer Science

Assistant Professor

About

93
Publications
13,300
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
3,718
Citations
Additional affiliations
July 2012 - present
University of Manitoba
Position
  • Professor (Assistant)
January 2009 - present
The University of York
Position
  • York University
January 2009 - February 2011
York University
Position
  • PostDoc Position

Publications

Publications (93)
Preprint
Full-text available
Deep spatiotemporal models are used in a variety of computer vision tasks, such as action recognition and video object segmentation. Currently, there is a limited understanding of what information is captured by these models in their intermediate representations. For example, while it has been observed that action recognition algorithms are heavily...
Preprint
Full-text available
Existing weakly or semi-supervised semantic segmentation methods utilize image or box-level supervision to generate pseudo-labels for weakly labeled images. However, due to the lack of strong supervision, the generated pseudo-labels are often noisy near the object boundaries, which severely impacts the network's ability to learn strong representati...
Preprint
Full-text available
In this paper, we present a strategy for training convolutional neural networks to effectively resolve interference arising from competing hypotheses relating to inter-categorical information throughout the network. The premise is based on the notion of feature binding, which is defined as the process by which activations spread across space and la...
Preprint
Full-text available
In this paper, we challenge the common assumption that collapsing the spatial dimensions of a 3D (spatial-channel) tensor in a convolutional neural network (CNN) into a vector via global pooling removes all spatial information. Specifically, we demonstrate that positional information is encoded based on the ordering of the channel dimensions, while...
Preprint
Full-text available
In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An implication of this is that a filter may know what it is looking at, but not where it is positioned in the image. In this paper, we first test this hypothesis and reveal...
Preprint
Full-text available
Contrasting the previous evidence that neurons in the later layers of a Convolutional Neural Network (CNN) respond to complex object shapes, recent studies have shown that CNNs actually exhibit a `texture bias': given an image with both texture and shape cues (e.g., a stylized image), a CNN is biased towards predicting the category corresponding to...
Preprint
In this paper, we present a strategy for training convolutional neural networks to effectively resolve interference arising from competing hypotheses relating to inter-categorical information throughout the network. The premise is based on the notion of feature binding, which is defined as the process by which activation's spread across space and l...
Article
Saliency and visual attention have been studied in a computational context for decades, mostly in the capacity of predicting spatial topographical saliency maps or simulated heatmaps. Spatial selection by an attentive mechanism is, however, inherently a sequential sampling process in humans. There have been recent efforts in analyzing and modeling...
Preprint
Saliency detection has been widely studied because it plays an important role in various vision applications, but it is difficult to evaluate saliency systems because each measure has its own bias. In this paper, we first revisit the problem of applying the widely used saliency metrics on modern Convolutional Neural Networks(CNNs). Our investigatio...
Article
Existing advanced saliency systems have been proposed using Convolutional Neural Networks (CNNs), the prior knowledge of objectness is crucial for saliency detection. However, we show that the use of objectness may also limit the power of CNNs on the images which do not contain a salient object. Besides, one previous study has shown applying a deep...
Preprint
In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An implication of this is that a filter may know what it is looking at, but not where it is positioned in the image. Information concerning absolute position is inherently u...
Article
In this article, the authors propose a deep learning framework for malware classification. There has been a huge increase in the volume of malware in recent years which poses serious security threats to financial institutions, businesses, and individuals. In order to combat the proliferation of malware, new strategies are essential to quickly ident...
Preprint
In this paper, we present a canonical structure for controlling information flow in neural networks with an efficient feedback routing mechanism based on a strategy of Distributed Iterative Gating (DIGNet). The structure of this mechanism derives from a strong conceptual foundation and presents a light-weight mechanism for adaptive control of compu...
Article
Full-text available
Salient object detection is a problem that has been considered in detail and many solutions have been proposed. In this paper, we argue that work to date has addressed a problem that is relatively ill-posed. Specifically, there is not universal agreement about what constitutes a salient object when multiple observers are queried. This implies that...
Preprint
Full-text available
The Saliency Model Implementation Library for Experimental Research (SMILER) is a new software package which provides an open, standardized, and extensible framework for maintaining and executing computational saliency models. This work drastically reduces the human effort required to apply saliency algorithms to new tasks and datasets, while also...
Preprint
Full-text available
In this paper, we present an approach for Recurrent Iterative Gating called RIGNet. The core elements of RIGNet involve recurrent connections that control the flow of information in neural networks in a top-down manner, and different variants on the core structure are considered. The iterative nature of this mechanism allows for gating to spread in...
Preprint
In this paper, we present an approach for Recurrent Iterative Gating called RIGNet. The core elements of RIGNet involve recurrent connections that control the flow of information in neural networks in a top-down manner, and different variants on the core structure are considered. The iterative nature of this mechanism allows for gating to spread in...
Preprint
Salient object detection is a problem that has been considered in detail and many solutions proposed. In this paper, we argue that work to date has addressed a problem that is relatively ill-posed. Specifically, there is not universal agreement about what constitutes a salient object when multiple observers are queried. This implies that some objec...
Preprint
Full-text available
Much research has examined models for prediction of semantic labels or instances including dense pixel-wise prediction. The problem of predicting salient objects or regions of an image has also been examined in a similar light. With that said, there is an apparent relationship between these two problem domains in that the composition of a scene and...
Preprint
Full-text available
Effective integration of local and global contextual information is crucial for semantic segmentation and dense image labeling. We develop two encoder-decoder based deep learning architectures to address this problem. We first propose a network architecture called Label Refinement Network (LRN) that predicts segmentation labels in a coarse-to-fine...
Conference Paper
Full-text available
Understanding human gaze behaviour has benefits from scientific understanding to many application domains. Current practices constrain possible use cases, requiring experimentation restricted to a lab setting or controlled environment. In this paper, we demonstrate a flexible unconstrained end-to-end solution that allows for collection and analysis...
Conference Paper
Full-text available
In recent years, there have been significant advances in deep learning applied to problems in high-level vision tasks (e.g. image classification, object detection, semantic segmentation etc.) which has been met with a great deal of success. State-of-the-art methods that have shown impressive results on recognition tasks typically share a common str...
Article
Full-text available
Salient object detection is a problem that has been considered in detail and many solutions proposed. In this paper, we argue that work to date has addressed a problem that is relatively ill-posed. Specifically, there is not universal agreement about what constitutes a salient object when multiple observers are queried. This implies that some objec...
Article
Purpose: The accurate prediction of intra-fraction lung tumor motion is required to compensate for system latency in image-guided adaptive radiotherapy systems. The goal of this study was to identify an optimal prediction model that has a short learning period so that prediction and adaptation can commence soon after treatment begins, and requires...
Conference Paper
We designed and evaluated a series of teleoperation interface techniques that aim to draw operator attention while mitigating negative effects of interruption. Monitoring live teleoperation video feeds, for example to search for survivors in search and rescue, can be cognitively taxing, particularly for operators driving multiple robots or monitori...
Article
Full-text available
We consider the problem of semantic image segmentation using deep convolutional neural networks. We propose a novel network architecture called the label refinement network that predicts segmentation labels in a coarse-to-fine fashion at several resolutions. The segmentation labels at a coarse resolution are used together with convolutional feature...
Article
We consider the problem of localizing and segmenting objects in weakly labeled video. A video is weakly labeled if it is associated with a tag (e.g. YouTube videos with tags) describing the main object present in the video. It is weakly labeled because the tag only indicates the presence/absence of the object, but does not give the detailed spatial...
Article
Purpose:In image-guided adaptive radiotherapy systems, prediction of tumor motion is required to compensate for system latencies. However, due to the non-stationary nature of respiration, it is a challenge to predict the associated tumor motions. In this work, a systematic design of the neural network (NN) using a mixture of online data acquired du...
Article
Yarbus' pioneering work in eye tracking has been influential to methodology and in demonstrating the apparent importance of task in eliciting different fixation patterns. There has been renewed interest in Yarbus' assertions on the importance of task in recent years, driven in part by a greater capability to apply quantitative methods to fixation d...
Conference Paper
In viewing an image or real-world scene, different observers may exhibit different viewing patterns. This is evidently due to a variety of different factors, involving both bottom-up and top-down processing. In the literature addressing prediction of visual saliency, agreement in gaze patterns across observers is often quantified according to a mea...
Chapter
Collaboration is a necessary, everyday human activity, yet computing environments specifically designed to support collaborative tasks have typically been aimed toward groups of experts in extensive, purpose-built environments. The cost constraints and design complexities of fully-networked, multi-display environments have left everyday computer us...
Conference Paper
Full-text available
In this paper we present a definition for visual saliency grounded in information theory. This proposal is shown to relate to a variety of classic research contributions in scale-space theory, interest point detection, bilateral filtering, and to existing models of visual saliency. Based on the proposed definition of visual saliency, we demonstrate...
Article
Full-text available
Saliency maps produced by different algorithms are often evaluated by comparing output to fixated image locations appearing in human eye tracking data. There are challenges in evaluation based on fixation data due to bias in the data. Properties of eye movement patterns that are independent of image content may limit the validity of evaluation resu...
Conference Paper
Full-text available
We introduce a layout manager that exploits the robust sensing capabilities of next-generation head-worn displays by embedding virtual application windows in the user's surroundings. With the aim of allowing users to find applications quickly, our approach leverages spatial memory of a known body-centric configuration. The layout manager balances m...
Article
Full-text available
In this paper, we examine the problem of learning sparse representations of visual patterns in the context of artificial and biological vision systems. There are a myriad of strategies for sparse coding that often result in similar feature properties for the learned feature set. Typically this results in a bank of Gabor-like or edge filters that ar...
Article
In the past decade, a large number of computational models of visual saliency have been proposed. Recently a number of comprehensive benchmark studies have been presented, with the goal of assessing the performance landscape of saliency models under varying conditions. This has been accomplished by considering fixation data, annotated image regions...
Article
Full-text available
Image processing has been applied for aesthetic or artistic purposes to produce a range of visual effects including abstracted images, painterly renderings, and comic-book, or cartoon effects. In this paper we examine the problem of transforming standard RGB images to having an appearance reminiscent of older console games. This is achieved by way...
Conference Paper
Full-text available
Given the significant number of potential applications, visual saliency has increasingly become an area of interest in image and vision research. Many different strategies for predicting visual saliency have been proposed, that differ in their composition or rationale, and with a significant focus on improving performance across standard benchmarks...
Article
Full-text available
Most saliency algorithms rely on a filter processing stage in which an image is analyzed using a bank of convolution kernels. When applying a convolution to an image, however , a region of pixels with thickness equal to one-half the kernel width at the image border is left undefined due to insufficient input (this undefined region is hereafter refe...
Article
In this article, we examine how emotional and perceptual stimulus factors influence visual search efficiency. In an initial task, we run a visual search task, using a large number of target/distractor emotion combinations. In two subsequent tasks, we then assess measures of perceptual (rated and computational distances) and emotional (rated valence...
Conference Paper
Full-text available
We consider the problem of segmenting objects in weakly labeled video. A video is weakly labeled if it is associated with a tag (e.g. Youtube videos with tags) describing the main object present in the video. It is weakly labeled because the tag only indicates the presence/absence of the object, but does not give the detailed spatial/temporal locat...
Conference Paper
Full-text available
In this paper, we explore the problem of analyzing gaze patterns towards attributing greater meaning to observed fixations. In recent years, there have been a number of efforts that attempt to categorize fixations according to their properties. Given that there are a multitude of factors that may contribute to fixational behavior, including both bo...
Article
Mobile devices are endowed with significant sensing capabilities. However, their ability to 'see' their surroundings, during active use, is limited. We present Surround-See, a self-contained smartphone equipped with an omni-directional camera that enables peripheral vision around the device to augment daily mobile tasks. Surround-See provides mobil...
Conference Paper
Full-text available
In this paper we consider the problem of dynamic range compression from multiple exposures in the absence of raw images, radiometric response functions, or irradiance information. This is achieved in a rapid and relatively simplistic fashion by merging image content across provided exposures. The premise of the proposal lies in assuming as one impo...
Conference Paper
Full-text available
This paper describes a biologically motivated local context operator to improve low-level visual feature representations. The computation borrows the idea from the primate visual system that different visual features are computed with different speeds in the visual system and thus they can positively affect each other via early recurrent modulation...
Article
Full-text available
The stereo correspondence problem is a topic that has been the subject of considerable research effort. What has not yet been considered is an analogue of stereo correspondence in the domain of attention. In this chapter, the authors bring this problem to light, revealing important implications for computational models of attention, and in particul...
Conference Paper
Full-text available
The human brain uses visual attention to facilitate object recognition. Traditional theories and models envision this attentional mechanism either in a pure feedforward fashion for selection of regions of interest or in a top-down task-priming fashion. To these well-known attentional mechanisms, we add here an additional novel one. The approach is...
Article
Full-text available
In visual search experiments there exist a variety of experimental paradigms in which a symmetric set of experimental conditions yields asymmetric corresponding task performance. There are a variety of examples of this that currently lack a satisfactory explanation. In this paper, we demonstrate that distinct classes of asymmetries may be explained...
Article
Formal arguments exist establishing that the complexity of visual search prohibits extensive analysis of all visual content in parallel. It follows that the task of selecting important content out of the enormous pool of incoming sensory input may be regarded as a critical component of animal vision; theoretically as well as practically this remain...
Conference Paper
In recent years, many principled probabilistic definitions for the determination of visual saliency have been proposed. Moreover, there has been increased focus on the role of context in the determination of visual salience. Prior efforts have shed some light on how context may help in predicting the location of, or presence of features associated...
Conference Paper
Full-text available
In this paper, we consider whether statistical regularities in natural images might be exploited to provide an improved selection criterion for interest points. One approach that has been particularly influential in this domain, is the Harris corner detector. The impetus for the selection criterion for Harris corners, proposed in early work and whi...