Jonathan Connell

Jonathan Connell
Massachusetts Institute of Technology | MIT · Department of Computational Science and Engineering

Ph.D.

About

176
Publications
74,105
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
9,223
Citations
Additional affiliations
January 2014 - present
IBM
Position
  • Researcher
September 1993 - December 2013
IBM
Position
  • Researcher
July 1989 - August 1993
IBM
Position
  • Researcher

Publications

Publications (176)
Preprint
Home robots may come with many sophisticated built-in abilities, however there will always be a degree of customization needed for each user and environment. Ideally this should be accomplished through one-shot learning, as collecting the large number of examples needed for statistical inference is tedious. A particularly appealing approach is to s...
Preprint
The visual world is very rich and generally too complex to perceive in its entirety. Yet only certain features are typically required to adequately perform some task in a given situation. Rather than hardwire-in decisions about when and what to sense, this paper describes a robotic system whose behavioral policy can be set by verbal instructions it...
Preprint
In this paper we present a system that detects and tracks objects and agents, computes spatial relations, and communicates those relations to the user using speech. Our system is able to detect multiple objects and agents at 30 frames per second using a RGBD camera. It is able to extract the spatial relations in, on, next to, near, and belongs to,...
Preprint
We introduce a technique that can automatically tune the parameters of a rule-based computer vision system comprised of thresholds, combinational logic, and time constants. This lets us retain the flexibility and perspicacity of a conventionally structured system while allowing us to perform approximate gradient descent using labeled data. While th...
Preprint
Spoken language is a convenient interface for commanding a mobile robot. Yet for this to work a number of base terms must be grounded in perceptual and motor skills. We detail the language processing used on our robot ELI and explain how this grounding is performed, how it interacts with user gestures, and how it handles phenomena such as anaphora....
Data
We describe the effective use of online learning to enhance the conversational capabilities of a concierge robot that we have been developing over the last two years. The robot was designed to interact naturally with visitors and uses a speech recognition system in conjunction with a natural language classifier. The online learning component monito...
Article
In this paper we present our work with the Pepper robot, a service robot from SoftBank Robotics. We had two main goals in this work: improving the autonomy of this robot by increasing its awareness of the environment; and enhance the robot ability to interact with its users. To achieve this goals, we used ROS, a modern open-source framework for dev...
Conference Paper
Full-text available
We describe the effective use of online learning to enhance the conversational capabilities of a concierge robot that we have been developing over the last two years. The robot was designed to interact naturally with visitors and uses a speech recognition system in conjunction with a natural language classifier. The online learning component monito...
Article
Full-text available
IBM Research is engaged in a research program in symbiotic cognitive computing to investigate how to embed cognitive computing in physical spaces. This article proposes five key principles of symbiotic cognitive computing: context, connection, representation, modularity, and adaptation, along with the requirements that flow from these principles. W...
Patent
Full-text available
A method, an apparatus and an article of manufacture for generating a cardiovascular measurement for individual identification. The method includes acquiring at least one depiction of cardiac anatomy from an individual, extracting at least one quantified representation of cardiac anatomy from the at least one depiction, defining at least one compar...
Article
Very large-scale biometric systems are becoming mainstream in nationwide identity cards and mobile secure payment methods. As with other Big Data systems, biometric systems contend with the “four V” challenges that involve the effective managing of the complex life cycle and operations of identity information—despite the immense enrollment database...
Article
Full-text available
In this work, we study the performance of a two-stage ensemble visual machine learning framework for classification of medical images. In the first stage, models are built for subsets of features and data, and in the second stage, models are combined. We demonstrate the performance of this framework in four contexts: 1) The public ImageCLEF (Cross...
Patent
Full-text available
Mechanisms are provided for determining the physical location of a physical asset in a physical area. A plurality of physical assets are controlled to cause each physical asset to output a visual output pattern on visual output elements of the physical asset. An image of a target physical asset is captured that has the current state of the visual o...
Patent
Full-text available
Method, apparatus and computer program product compare biometrics in an anonymous manner. A first collection of biometrics is transformed using a first cancelable non-invertible biometric transform to create a first collection of transformed biometrics. A second collection of biometrics is transformed using the first cancelable non-invertible biome...
Patent
Full-text available
The invention provides a method, system, and program product for detecting an object in a digital image. In one embodiment, the invention includes: deriving an initial object indication mask based on pixel-wise differences between a first digital image and a second digital image, at least one of which includes the object; performing an edge finding...
Conference Paper
Full-text available
In this work, we present a framework for medical image modality recognition based on a fusion of both visual and text classification methods. Experiments are performed on the public ImageCLEF 2013 medical image modality dataset, which provides figure images and associated fulltext articles from PubMed as components of the benchmark. The presented v...
Patent
The system and method obscures descriptive image information about one or more images. The system comprises a selector for selecting the descriptive image information from one or more of the images, a transformer that transforms the descriptive information into a transformed state, and an authorizer that provides authorization criteria with the ima...
Patent
Full-text available
Methods and apparatus are provided for maintaining a background image model in a background subtraction system using accumulated motion. A background image model is maintained by obtaining a map of accumulated motion; and adjusting the background image model based on the map of accumulated motion. The map of accumulated motion may be obtained, for...
Patent
Full-text available
Systems and methods for detecting visual objects by employing multiple cues include statistically combining information from multiple sources into a saliency map, wherein the information may include color, texture and/or motion in an image where an object is to be detected or background determined. The statistically combined information is threshol...
Conference Paper
Iris recognition has gained popularity due to factors such as its perceived high accuracy, significant usability advantages attributed to its non-contact acquisition method, and the availability of low cost sensors due to improvements in technology. However, non-contact biometrics authentication systems are vulnerable to different types of attacks...
Patent
Techniques for generating a distorted fingerprint representation for a given fingerprint image are provided. First, at least one fingerprint feature point from a given fingerprint image is selected. At least one representation of a region proximate to the selected fingerprint feature point is then generated. Next, the representation of the region p...
Patent
Techniques for generating a distorted template for a given facial image are provided. First, at least one facial feature point from a given facial image is selected. The selected facial feature has location information associated therewith. A representation of a region proximate to the selected facial feature point is then generated. Next, the loca...
Patent
Full-text available
A method, information processing system, and computer program storage product annotate video images associated with an environmental situation based on detected actions of a human interacting with the environmental situation. A set of real-time video images are received that are captured by at least one video camera associated with an environment p...
Conference Paper
Full-text available
Management and monitoring of data centers is a growing field of interest, with much current research, and the emergence of a variety of commercial products aiming to improve performance, resource utilization and energy efficiency of the computing infrastructure. Despite the large body of work on optimizing data center operations, few studies actual...
Article
Full-text available
Management and monitoring of data centers is a growing field of interest, with much current research, and the emergence of a variety of commercial products aiming to improve performance, resource utilization and energy efficiency of the computing infrastructure. Despite the large body of work on optimizing data center operations, few studies actual...
Patent
A system and method for performing an identity search in a database of iris images for a probe iris image includes generating a full-length iris code and a compact iris code for iris images in a database, and generating a full-length iris code and a compact iris code for a probe iris image. The compact iris code for the probe image is scored agains...
Patent
In one embodiment, the invention is a method and apparatus for repeatable facial distortion. One embodiment of a method for generating a secure facial image from an original facial image includes receiving the original facial image and a key, the key being associated with a subject depicted in the original facial image, and distorting the original...
Article
Full-text available
Today retail video analytics has gone beyond the traditional domain of security and loss prevention by providing retailers insightful business intelligence such as store traffic statistics and queue data. Such information allows for enhanced customer experience, optimized store performance, reduced operational costs, and ultimately higher profitabi...
Conference Paper
This paper describes our Extensible Language Interface (ELI) for robots. The system is intended to interpret far-field speech commands in order to perform fetch-and-carry tasks, potentially for use in an eldercare context. By "extensible" we mean that the robot is able to learn new nouns and verbs by simple interaction with its user. An associated...
Conference Paper
Full-text available
In this study, we propose a novel biometric signature for human identification based on anatomically unique structures of the left ventricle of the heart. An algorithm is developed that analyzes the 3 primary anatomical structures of the left ventricle: the endocardium, myocardium, and papillary muscles. Comparisons of these analyses between probe...
Conference Paper
It is becoming increasingly clear that it is humanly impossible to analyze a deluge of data from cameras and other sensors in a variety of applications including surveillance, railroad inspection, driver assistance. The practical systems that we built, although in pursuit of different business objectives, share a common goal, which is to intelligen...
Conference Paper
Full-text available
We will demonstrate a robot for data center energy management, in action, on a simulated data center floor. We shall highlight the robot's navigation, tile and obstacle classification, event scheduling and preemption capabilities, along with its ability to discover charging docks, and successfully dock with extreme precision. We shall also show sim...
Article
Full-text available
Humans, as well as many living organisms, are gifted with the power of “seeing” and “understanding” the environment around them using their eyes. The ease with which humans process and understand the visual world is very deceiving and often prompts us to underestimate the effort and methods needed to build practical, effective, and inexpensive comp...
Conference Paper
We describe a system that controls whether the headlights of a vehicle are in the highbeam or lowbeam state based on input from a forward looking video camera. The core of the system relies on conventional computer vision techniques, albeit with a sophisticated spot finder front-end. Despite this architecture we are able to use an automated supervi...
Chapter
Biometrics-based authentication systems offer advantages over the present practices of knowledge and/or possession-based authentication systems. However, when using biometrics, the overall authentication architecture needs to be reexamined to ensure that no new weak security points are introduced. After analyzing a pattern recognition-based threat...
Chapter
For learning to be useful on real robots, whatever algorithm is used must converge in some “reasonable” amount of time. If each trial step takes on the order of seconds, a million steps would take several months of continuous run time. In many cases such extended runs are neither desirable nor practical. In this chapter we discuss how learning can...
Chapter
In this chapter we provide an overview of the field of robot learning. We first discuss why robot learning is interesting and explain what is hard about it. We then characterize the robot learning problem and point out some major issues that need to be addressed. Next we survey some established techniques which are relevant to robot learning and th...
Conference Paper
Full-text available
This talk provides a retrospective on the first decade of autonomic computing, an assessment of the extent to which the original vision has been realized, and some discussion and speculation about the the remaining research challenges. Nearly a decade ...
Conference Paper
Full-text available
We describe an inexpensive robot that serves as a physical autonomic element, capable of navigating, mapping and monitoring data centers with little or no human involvement, even ones that it has never seen before. Through a series of real experiments and simulations, we establish that the robot is sufficiently accurate, efficient and robust to be...
Conference Paper
Full-text available
We describe an inexpensive autonomous robot capable of navigating previously unseen data centers and monitoring key metrics such as air temperature 1 . The robot provides real-time navigation and sensor data to commercial IBM software, thereby enabling real-time generation of the data center layout, a thermal map and other visualizations of energy...
Conference Paper
Many modern retail stores have self-checkout stations where customers can ring up their own orders without the assistance of any store personnel. To promote customer honesty these systems often weigh each item as it is placed in the bag to confirm that it has the expected mass for the product scanned. In our system we augment this basic check with...
Conference Paper
Full-text available
The texture in a human iris has been shown to have good individual distinctiveness and thus is suitable for use in reliable identification. A conventional iris recognition system unwraps the iris image and generates a binary feature vector by quantizing the response of selected filters applied to the rows of this image. Typically there are 360 angu...
Conference Paper
Full-text available
There have been claims of very high information content in iris texture, higher even than in fingerprints. This makes iris attractive for large scale identification systems with possibly millions of people. However, some systems operate by performing N 1:1 matches of the probe against the database. This can get prohibitively expensive in terms of c...
Article
Techniques for performing audio-visual speech recognition, with improved recognition performance, in a degraded visual environment. For example, in one aspect of the invention, a technique for use in accordance with an audio-visual speech recognition system for improving a recognition performance thereof includes the steps/operations of: (i) select...
Conference Paper
Full-text available
Cancelable biometric systems are gaining in popularity for use in person authentication for applications where the privacy and security of biometric templates are important considerations. A variety of approaches have been proposed in the literature. In this work, we have chosen two (a registration based and a registration free) techniques and perf...
Conference Paper
Full-text available
A person only has two irises - if his pattern is stolen he quickly runs out of alternatives. Thus methods that protect the true iris pattern need to be adopted in practical biometric applications. In particular, it is desirable to have a system that can generate a new unique pattern if the one being used is lost, or generate different unique patter...
Conference Paper
Full-text available
Iris-based human recognition is very attractive because of the high accuracy achievable. However, existing encoding methods are unable to handle iris images acquired when the ambient lighting is non-uniform. In this paper we propose a novel encoding technique which can handle images acquired under such conditions. The method is based on 2D median f...
Conference Paper
Full-text available
Cancelable biometrics bridges the gap between the convenience of biometric authentication and security offered by cryptographic systems. There have been several prior attempts to construct cancelable templates out of fingerprint biometrics. However, existing approaches require pre-alignment of fingerprint images and therefore, are not robust to ima...
Conference Paper
Full-text available
One of the main challenges in building an efficient and scalable automatic fingerprint identification system is to identify features which are highly discriminative and are reproducible across different prints of the same finger. Most existing fingerprint matching approaches rely on minutiae geometry. Relatively, little effort has gone into analyzi...
Conference Paper
Full-text available
Iris segmentation is an important first step for high accuracy iris recognition. A robust iris segmentation procedure should be able to handle noise, occlusion and non-uniform lighting. It also impacts system accuracy - high FAR or FRR values may come directly from bad or wrong segmentations. In this paper a simple new approach for iris segmentatio...
Conference Paper
Full-text available
We present a face reconstruction approach for revocable face matching. The proposed approach generates photometrically valid cancelable face images by following the image formation process. Given a face image, the approach estimates facial albedo followed by a subject-specific key based photometric deformation to generate a cancelable face image. T...
Conference Paper
Full-text available
There are a wide variety of approaches to Artificia l Intelligence. Yet interestingly we find that these can all be grouped into four broad categories: Silver Bullets, Core Values, Emergence, and Emulation. We will explain the methodological underpinnings of these categories and give examples of the type of work being pursued in each. Understanding...
Conference Paper
Full-text available
AI has many techniques and tools at its disposal, y et seems to be lacking some special "juice" needed to create a true being. We propose that the missing ingredients are a general theory of motivation and an operational understanding of natural language. The motivation part comes largely f rom our animal heritage: a real-world agent must continual...
Conference Paper
Full-text available
We present a system for automatically detecting driver's speech in the automobile domain using visual-only information extracted from the driver's mouth region. The work is motivated by the desire to eliminate manual push-to-talk activation of the speech recognition engine in newly designed voice interfaces in the typically noisy car environment, a...
Conference Paper
Full-text available
Pervasive sensor based systems are transforming Information Technology systems from being transactional in nature to being observational in nature. Observational systems are inherently distributed and capture information at a much finer grain of space and time. Enabling and building such systems also poses many technology challenges, extracting inf...
Chapter
Traditional authentication schemes utilize tokens or depend on some secret knowledge possessed by the user for verifying his or her identity. Although these techniques are widely used, they have several limitations. Both tokenand knowledge-based approaches cannot differentiate between an authorized user and an impersonator having access to the toke...
Conference Paper
With increased concern for physical security, video surveillance is becoming an important business area. Similar camera-based system can also be used in such diverse applications as retail-store shopper motion analysis and casino behavioral policy monitoring. There are two aspects of video surveillance that require significant computing power: imag...
Conference Paper
Full-text available
Biometrics offers usability advantages over traditional token and password based authentication schemes, but raises privacy and security concerns. When compromised, credit cards and passwords can be revoked or replaced while biometrics are permanently associated with a user and cannot be replaced. Cancelable biometrics attempts to solve this by con...
Conference Paper
The cumulative match curve (CMC) is used as a measure of 1: m identification system performance. It judges the ranking capabilities of an identification system. The receiver operating characteristic curve (ROC curve) of a verification system, on the other hand, expresses the quality of a 1:1 matcher. The ROC plots the false accept rate (FAR) of a 1...
Conference Paper
As smart surveillance technology becomes a critical component in security infrastructures, the system architecture assumes a critical importance. This paper considers the example of smart surveillance in an airport environment. We start with a threat model for airports and use this to derive the security requirements. These requirements are used to...
Article
In recent years, closed-circuit television (CCTV) cameras have gained widespread use worldwide. Human operators monitor CCTV systems, unobtrusive or deliberately hidden cameras allow spying and voyeurism, and video surveillance, which make CCTVs a tool for state control and oppression. The use of surveillance is spreading as the hardware becomes mo...
Article
Full-text available
Situation awareness is the key to security. Awareness requires information that spans multiple scales of space and time. Smart video surveillance systems are capable of enhancing situational awareness across multiple scales of space and time. However, at the present time, the component technologies are evolving in isolation. To provide comprehensiv...
Conference Paper
The need for a large sample size grows exponentially with the dimensionality of the feature space ("curse of dimensionality"), which increases the labor cost during the training procedure and severely restricts the number of the practical applications. While feature selection methods can often alleviate the problems associated with the curse of dim...
Article
This report describes a working autonomous mobile robot whose only goal is to collect and return empty soda cans. It operates in an unmodified office environment occupied by moving people. The robot is controlled by a collection of over 40 independent "behaviors'' distributed over a loosely coupled network of 24 processors. Together this ensemble h...
Article
Full-text available
We present the results of an implemented system for learning structural prototypes from grey-scale images. We show how to divide an object into subparts and how to encode the properties of these subparts and the relations between them. We discuss the importance of hierarchy and grouping in representing objects and show how a notion of visual simila...
Conference Paper
One of the key components of tele-presence systems is automatic awareness of the remote environment. This very same capability of automatic situation awareness is currently being developed and deployed in the context of the next generation smart surveillance systems. Smart surveillance systems use a number of automatic video analysis techniques lik...
Article
In this paper we describe a working, implemented controller for a real, physical mobile robot arm. The controller is composed of a collection of 15 independent behaviors which run, in real time, on a set of 8 loosely coupled on-board 8-bit microprocessors. We describe how these behaviors cooperate to actually seek out and retrieve objects using loc...
Article
It is well known that frontal video of the speaker’s mouth region contains significant speech information that, when combined with the acoustic signal, can improve accuracy and noise robustness of automatic speech recognition (ASR) systems. However, extraction of such visual speech information from full-face videos is computationally expensive, as...
Conference Paper
Full-text available
The detection and tracking of people lie at the heart of many current and near-future applications of computer vision. We describe a background subtraction system designed to detect moving objects in a wide variety of conditions, and a second system to detect objects moving in front of moving backgrounds. Detected foreground regions are tracked wit...
Conference Paper
Full-text available
Much progress has been achieved during the past two decades in audio-visual automatic speech recognition (AVASR). However, challenges persist that hinder AVASR deployment in practical situations, most notably, robust and fast extraction of visual speech features. We review our efforts in overcoming this problem, based on an appearance-based visual...
Article
There are a number of sensor possibilities for mobile robots. Unfortunately many of these are relatively expensive (e.g., laser scanners) or only provide sparse information (e.g., sonar rings). As an alternative, vision-based navigation is very attractive because cameras are cheap these days and computer power is plentiful. The trick is to figure o...
Conference Paper
Data hiding techniques can be employed to enhance the security of biometrics-based authentication systems. In our previous work, we proposed a method to hide challenge responses in a WSQ-compressed fingerprint image. In this paper, we extend the work to analyze the security holes in the data hiding method and enhance the hiding technique to thwart...
Conference Paper
Full-text available
Smart surveillance, is the use of automatic video analysis technologies in video surveillance applications. This paper attempts to answer a number of questions about smart surveillance: What are the applications of smart surveillance? What are the system architectures for smart surveillance? What are the key technologies? What are the some of the k...
Conference Paper
Full-text available
Biometrics-based personal authentication systems are becoming popular with increased demand on security. A biometrics is expected to have significant amount of discriminatory information in representing uniqueness of a person. This discriminatory information about the biometric is loosely defined as the individuality. Even though some individuality...
Article
Full-text available
The increasing need for sophisticated surveillance systems and the move to digital surveillance infrastructure has transformed surveillance into a large scale data analysis and management challenge. Smart surveillance systems use automatic image understanding techniques to extract information from the surveillance data. While the majority of the re...
Chapter
Earlier in this book (Chapter 5) we described the evaluation of biometric systems in terms of their error rates. But two important questions to consider when comparing these measures, principally the False Accept and False Reject Rates, is how accurately we know the error rates, and how much can they be trusted for making decisions.
Chapter
Automated biometric authentication systems help to alleviate the problems associated with existing methods of user authentication. Biometrics can improve convenience or security, or ideally both. However, security weak points will exist or will be introduced in any biometric installation, unintended by the designers of the system. These weak points...