Michèle Gouiffès

• Professor (Associate) at University of Paris-Saclay

Abnormal event detection in videos is a challenging problem, partly due to the multiplicity of abnormal patterns and the lack of their corresponding annotations. In this paper, we propose new constrained pretext tasks to learn object level normality patterns. Our approach consists in learning a mapping between down-scaled visual queries and their c...

The Vera Icona installation-performance considers the relationship between face and image in contemporary society. It is inspired by the legend of the Vera Icona, an “icon made without hands” (said to have come into existence miraculously rather than by human creation), which is part of the political and philosophical history of images, from the re...

This paper describes a mobile application that builds and updates a 3D model of an indoor environment, including walls, floor and openings, by a simple scan performed using a tablet equipped with a depth sensor. This algorithm is fully implemented on the device, does not require internet connection and runs in real-time, i.e., at five frames per se...

This paper addresses video anomaly detection problem for videosurveillance. Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy, in which our model learns object-centric normal patterns without seeing anomalous samples during training. The main contributions consist in coupling pre...

A very low dimension frame-level motion descriptor is herein proposed with the capability to represent incomplete dynamics, thus allowing online action prediction. At each frame, a set of local trajectory kinematic cues are spatially pooled using a covariance matrix. The set of frame-level covariance matrices forms a Riemannian manifold that descri...

Sign Languages (SLs) are visual–gestural languages that have developed naturally in deaf communities. They are based on the use of lexical signs, that is, conventionalized units, as well as highly iconic structures, i.e., when the form of an utterance and the meaning it carries are not independent. Although most research in automatic Sign Language...

This study proposes a strategy to refine optical flow based on the estimated reliability maps. These maps are firstly estimated a posteriori after the motion estimation by the well‐known Kanade–Lucas–Tomasi (KLT). With two new defined criteria based, respectively, on the optical flow local variance and the temporal evolution of the KLT residuals, a...

We present baseline results for a new task of automatic segmentation of Sign Language video into sentence-like units. We use a corpus of natural Sign Language video with accurately aligned subtitles to train a spatio-temporal graph convolutional network with a BiLSTM on 2D skeleton data to automatically detect the temporal boundaries of subtitles....

This paper presents an approach towards an automatic annotation system for French Sign Language (LSF). Such automation aims to reduce the processing time and the subjectivity of manual annotations done by linguists in order to study the sign language and simplify indexing for automatic signs recognition. The described system uses face and body keyp...

This paper presents an on the fly planar segmentation algorithm that runs on a tablet equipped with a depth sensor and which uses a motion tracking algorithm. Our algorithm segments each incoming point cloud from the depth sensor and it then updates a global model containing all of the previously identified planes. Consequently, identical planes ar...

Gardien du Temple is an interactive installation where the concealment or revelation of poems and images is achieved through diminished reality, a new approach to augmented reality using a projector-camera system. By capturing the environment with a camera and canceling parts of the scene by reprojecting inverted images, Gardien du Temple questions...

This paper presents a mobile platform that uses a new generation of tablets equipped with a depth sensor to perform a real time 3D reconstruction of an indoor environment. The platform generates a 3D model where the strucutral elements are identified: ground, ceiling, walls and openings. The 3D model is used for the evaluation of both the geometric...

This paper presents a preliminary study which aims to investigate the tangible navigation in 3D Virtual Environments with new tablets (ex. Google Tango) that provide a self-contained 3D full tracking (ex. Google Tango, Intel RealSense). The tangible navigation was compared with tactile navigation techniques used in standard 3D applications (ex. vid...

The creation of interactive installations is a process which implies to bring back together phenomenological evaluations of visitors' experiences and motion analyses which inherently highlight the core material of interaction design. In this paper, we present an interactive and immersive work, See me through you, which ambition to provide a rich ex...

This paper presents a modular system for both abnormal event detection and categorization in videos. Complementary normalcy models are built both globally at the image level and locally within pixels blocks. Three features are analyzed: (1) spatio-temporal evolution of binary motion where foreground pixels are detected using an enhanced background...

Spatial Augmented Reality applications generally use projector-camera systems to control the visual projection appearance by comparing the initial projected and the acquired images. To obtain an accurate geometric compensation, a non-intrusive feature-point matching approach can be exploited which must handle complex photometric distortions due to...

This work introduces a novel action descriptor that represents activities instantaneously in each frame of a video sequence for action recognition. The proposed approach first characterizes the video by computing kinematic primitives along trajectories obtained by semi-dense point tracking in the video. Then, a frame level characterization is achie...

Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed app...

Projector-camera systems are designed to improve the projection quality by comparing original images with their captured projections, which is usually complicated due to high photometric and geometric variations. Many research works address this problem using their own test data which makes it extremely difficult to compare different proposals. Thi...

Poster présenté à IHM 2015.

L'imagerie médicale révèle l'intérieur du corps humain sans physiquement ouvrir ce dernier. Devenue nécessaire dans l'établissement d'un grand nombre de diagnostics cliniques, l'imagerie médicale donne à voir de différentes façons l'anatomie et la fonction des organes internes. La présence de plus en plus prégnante dans notre quotidien de ces image...

In this paper, a novel approach that allows color compensated projection on an arbitrary surface is presented. Assuming that the geometry of the surface is known, this method can be used in dynamic environments, where the surface color is not static. A simple calibration process is performed offline and only a single input image under reference ill...

Covariance matching techniques have recently grown in interest due to their good performances for object retrieval, detection, and tracking. By mixing color and texture information in a compact representation, it can be applied to various kinds of objects (textured or not, rigid or not). Unfortunately, the original version requires heavy computatio...

The paper proposes contributions for mean-shift (MS) and covariance tracking (CT), and makes these two complementary methods cooperate. While MS runs fast and can handle non-rigid objects represented by their color distribution, CT is more time-consuming but achieves a generic tracking by mixing color and texture information. Each method is modifie...

This paper presents a novel way to embed local binary texture information in the form of local binary patterns (LBP) into the covariance descriptor. Contrary to previous publications, our method is not based on the LBP decimal values where arithmetic operations have no texture meaning. Our method uses the angles described by the uniform LBP pattern...

The success of matching algorithms relies on the definition of features which are both invariant against the geometric distortions to be considered, and distinctive enough to avoid ambiguities. This paper addresses the problem of color feature points matching under photometric and geometric changes. Considering the popular SURF descriptor, it analy...

HDRI (High Dynamic Range Imaging) techniques are used to produce dynamic and well-contrasted images of real-world luminance, by capturing several images of the same scene through exposure bracketing. In the same vein, we have developed a new approach to image fusion from a series of photographs of the same scene taken at different timestamps. When...

Covariance matching techniques have recently grown in interest due to their good performances for object retrieval, detection and tracking. By mixing color and texture information in a compact representation, it can be applied to various kinds of objects (textured or not, rigid or not). Unfortunately, the original version requires heavy computatio...

This paper addresses the multi-target tracking problem with the help of a matching method where moving objects are detected in each frame, tracked when it is possible and matched by similarity of covariance matrices when difficulties arrive. Three contributions are proposed. First, a compact vector based on color invariants and Local Binary Pattern...

Depth perception is a crucial but greedy task for most mobile robots to navigate in their environment and avoid obstacles. Generally, it does not need to be completed at permanent full precision but can be done in a coarse-to-fine strategy. In the present paper, a novel method is proposed to enhance a very sparse disparity map provided by a block m...

Since modeling reflections in image processing is a difficult task, most computer vision algorithms assume that objects are Lambertian and that no lighting change occurs. Some photometric models can partly answer this issue by assuming that the lighting changes are the same at each point of a small window of interest. Through a study based on specu...

Résumé Dans cet article, nous proposons une approche pour la reconnaissance des activités humaines à partir des vidéos capturées à l'aide des caméras monoculaires. Nous présentons une méthode basée sur la quantifi-cation vectorielle de descripteurs de mouvement. Ces descripteurs sont calculés à partir de l'orientation et la magnitude des vecteurs d...

In this paper, we propose an approach for human activity categorizing based on the use of optical flow direction and magnitude features. The main contribution of this paper is the feature representation that mirrors the geometry of the human body and relationships between its moving regions when performing activities. The features are quantified us...

This paper proposes a color tracking strategy designed to improve the robustness against luminance and saturation changes due to illumination variations. On the one hand, color is helpful in terms of photometric invariance and separability power. On the other hand, it is more costly in time and resources and most color invariants are ill-defined at...

The paper presents a spatio-color clustering method. First, the RGB space is converted into a spherical representation in order to better reveal the body color vectors of the image. The classification is performed after the color connectedness degrees (CCD) of a color interval which embeds jointly 1) colorimetric information; 2) the probability tha...

This paper introduces an algorithm to automatically and continuously select the most appropriate color space to use in order to improve the performances of visual tracking. Eight color spaces are tested, and the Mean-Shift (MS) tracker is considered. The selection of the colorspace is made using an evaluation criterion based on the quality of the w...

Level sets appear a suitable basis to decomposing an image into a meaningful organization of basic entities. Handling images by a basis of associated sets proves more effective than considering images as the collection of acquired intensity values, even in several frequency bands, due to the various noise sources. Then, level lines and sets are mor...

This paper proposes a novel definition of color sets, called body color sets and lines. The proposed technique refers to the dichromatic reflectance model, which states that the colors of a uniform lambertian object are roughly located around straight vectors going from the black to the body color, in relation to physical properties of the object....

The paper presents a spatio-color classification in a chrominanceluminance space related to the dichromatic model. The unsupervised adaptive clustering is performed after the color connectedness degrees (CCD) of a color interval which embeds jointly 1) colorimetric information; 2) the probability that a given color is connected (in the image) to a...

This paper proposes an extension to the mean shift tracking by using XY projection-histograms to model the object. More than providing statistical information about the target to track, they embed information about the spatial arrangement of pixels. This approach, without any complexity increase, provides a better robustness and quality of the trac...

This paper proposes an extension to the mean shift tracking. We introduce the color connectedness degrees (CCD) which, more than providing statistical information about the target to track, embeds information about the amount of connectedness of the color intervals which compose the target. With a low increase of complexity, this approach provides...

This paper proposes a novel definition of color lines and sets, based on the dichromatic model for lambertian objects. The ends of the body vectors are robustly detected, from the clearest to the darkest through to a multi-level 2D histogram analysis. Finally, instead of classically defining the topographic map along one sole luminance direction, o...

This paper proposes a contribution for obstacles detection and tracking in the context of car-driving assistance. On the basis of the uv-disparity9 designed for in-vehicle stereovision, we propose a robust procedure to detect the objects located on the road plane. This detection is used as an initialization stage for a real-time template matching p...

Level sets appear a suitable basis to decomposing an image into a meaningful organization of basic entities. Handling images by a basis of associated sets proves more effective than considering images as the collection of acquired intensity values, even in several frequency bands, due to the various noise sources. Then, level lines and sets are mor...

This paper addresses the problem of visual target tracking by use of robust primitives. More precisely, we evaluate the use of color segments features in a matching procedure and compare the dichromatic color lines (Gouiffès and Zavidovique, 2008) with the existing ones, defined in the HSV color space. The motion parameters of the target to track a...

Unlike edges, level lines are closed and less sensitive to external parameters. They provide a compact geometrical representation of images and they are, to some extent, robust to contrast changes. This paper proposes a novel and vectorial representation of color lines which does not require any color conversion. The topographic map is defined alon...

Topographic maps are an interesting alternative to edge-based techniques common in computer vision applications. Indeed, unlike edges, level lines are closed and less sensitive to external parameters. They provide a compact geometrical representation of images and they are, to some extent, robust to contrast changes. The aim of this paper is to pro...

The objective of this work is to develop a reliable image processing technique to measure the vibration parameters on every part of MEMS resonators using microscopic images of the vibrating devices. Images of resonators vibrating in high frequencies are characterized by the blurs whose point spread functions (PSFs) are expressed in a parametric for...

This paper addresses the problem of robust feature points tracking by using specific color invariants -robust to specular reflections, lighting changes and to some extent to color lighting changes-when they are relevant and photometric normalization in the opposite case. Indeed, most color invariants become noisy or irrelevant for low saturation an...

This article proposes a local photometric model that compensates for specular highlights and lighting variations due to position and intensity changes. We define clearly on which assumptions it is based, according to widely used reflection models. Moreover, its theoretical validity is studied according to few configurations of the scene geometry (l...

This article proposes several colour points tracking methods which are robust to illumination changes. Firstly, the illumination correction is achieved by computing simultaneously a local photometric model and a motion model during the image sequence. Secondly, some colour invariants are used to compensate in each point for the illumination changes...

Since the precise modeling of reflection is a dicult task, most feature points trackers assume that objects are lambertian and that no lighting change occurs. To some extent, a few approaches answer these issues by computing an ane photometric model or by achieving a photometric normalization. Through a study based on specular reflection models, we...

Since the precise modeling of reflection is a difficult task, most feature points trackers assume that objects are lambertian and that no lighting change occurs. To some extent, a few approaches answer these issues by computing an affine photometric model or by achieving a photometric normalization. Through a study based on specular reflection mode...

The ink reflectance strongly depends on its opacity and on the color of the body on which it is printed. In this paper, we focus on this problem, which is linked to ink transparency. We propose some attributes, which are invariant to the background color and to the quantity of ink printed. Some experimental results prove the theorical results. Then...

- La plupart des méthodes de suivi de points suppose que les objets sont lambertiens et qu'aucun changement d'éclairage ne se produit dans la scène. Dans une certaine mesure, le suivi avec normalisation photométrique ainsi que l'approche proposée par Soatto et al. [3] répondent à ce problème. Par une étude de la modélisation des variations d'illumi...

In this article, we study the color appearance of the ink printed on a background, according to both its concentration and the background color. We find some attributes, the concentration quotients ratios, that are more invariant to the ink concentration than simple color attributes. Our work deals with traceability of porcine products. We have to...

This article describes our method to measure and model the reflectance of an anisotropic and colorimetricaly inhomogeneous material : the ham. The main difficulty is to find a model which is suitable to all hams. Moreover, we have to face difficulties inherent to the measurement system itself. It doesn't enable to make measurements for all the posi...

Cet article propose une technique de suivi de points différentielle robuste aux changements d'illumination par la coopération d'attributs couleur invariants et d'une normalisation photométrique en fonction du contexte. En effet, la plupart des invariants couleur s'avèrent bruités ou peu pertinents dans le cas d'une faible saturation et/ou d'une fai...