Remi Ronfard

Remi Ronfard
National Institute for Research in Computer Science and Control | INRIA · IMAGINE - Intuitive Modeling and Animation for Interactive Graphics and Narrative Environments Research Team

PhD

About

119
Publications
33,912
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
4,666
Citations
Introduction
I am a Research Director at Inria with a rich experience in domains ranging from geometric design to 3D animation and computer vision. I have been R&D director for virtual cinematography at Xtranormal Technologies, where I created the patented "magicam" system. I am the scientific leader of the ANIMA team at INRIA, where my research agenda focuses on creating new software tools for "directing virtual worlds". I am also an Associate Researcher in the Spatial Media team at ENSADLAB in Paris.
Additional affiliations
July 2020 - present
National Institute for Research in Computer Science and Control
Position
  • Managing Director
Description
  • The ANIMA team conducts research in geometric modeling, physical modeling, semantic modeling and aesthetic modeling of virtual story worlds.
September 2018 - present
École Nationale Supérieure des Arts Décoratifs
Position
  • Research Associate
Description
  • The spatial media team conducts research in immersive virtual reality with the goal of creating new artistic forms in the four dimensions of space and time.
May 2016 - June 2020
National Institute for Research in Computer Science and Control
Position
  • Research Director
Education
February 1988 - February 1991
MINES ParisTech
Field of study
  • Computer Science
September 1983 - June 1986
MINES ParisTech
Field of study
  • Engineering

Publications

Publications (119)
Conference Paper
Full-text available
Preserving the cultural heritage of the performing arts raises difficult and sensitive issues, as each performance is unique by nature and the juxtaposition between the performers and the audience cannot be easily recorded. In this paper, we report on an experimental research project to preserve another aspect of the performing arts—the history of...
Article
Full-text available
We present a space-time abstraction for the sketch-based design of character animation. It allows animators to draft a full coordinated motion using a single stroke called the space-time curve (STC). From the STC we compute a dynamic line of action (DLOA) that drives the motion of a 3D character through projective constraints. Our dynamic models fo...
Article
Full-text available
We introduce the Vector Animation Complex (VAC), a novel data structure for vector graphics animation, designed to support the modeling of time-continuous topological events. This allows features of a connected drawing to merge, split, appear, or disappear at desired times via keyframes that introduce the desired topological change. Because the res...
Conference Paper
Full-text available
We describe an optimization-based approach for automatically creating well-edited movies from a 3D animation. While previous work has mostly focused on the problem of placing cameras to produce nice-looking views of the action, the problem of cutting and pasting shots from all available cameras has never been addressed extensively. In this paper, w...
Conference Paper
Full-text available
We introduce a generative model for learning person and costume specific detectors from labeled examples. We demonstrate the model on the task of localizing and naming actors in long video sequences. More specifically, the actor's head and shoulders are each represented as a constellation of optional color regions. Detection can proceed despite cha...
Article
OpenKinoAI is an open source framework for postproduction of ultrahigh definition video, which makes it possible to emulate professional multiclip editing techniques for the case of single-camera recordings. OpenKinoAI includes tools for uploading raw video footage of live performances on a remote web server; detecting, tracking, and recognizing th...
Article
The goal of our paper is to demonstrate through artistic experimentation that the concept of montage can be an effective vector for conveying complex narratives in immersive, room-scale virtual reality experiences. We review the particularities of virtual reality mediations regarding narrative, and we propose the new conceptual framework of « spati...
Article
Over the last forty years, researchers in computer graphics have proposed a large variety of theoretical models and computer implementations of a virtual film director, capable of creating movies from minimal input such as a screenplay or storyboard. The underlying film directing techniques are also in high demand to assist and automate the generat...
Preprint
OpenKinoAI is an open source framework for post-production of ultra high definition video which makes it possible to emulate professional multiclip editing techniques for the case of single camera recordings. OpenKinoAI includes tools for uploading raw video footage of live performances on a remote web server, detecting, tracking and recognizing th...
Conference Paper
Full-text available
We present a method for easily drafting expressive character animation by playing with instrumented rigid objects. We parse the input 6D trajectories (position and orientation over time) – called spatial motion doodles – into sequences of actions and convert them into detailed character animations using a dataset of parameterized motion clips which...
Preprint
Full-text available
Manipulation actions transform objects from an initial state into a final state. In this paper, we report on the use of object state transitions as a mean for recognizing manipulation actions. Our method is inspired by the intuition that object states are visually more apparent than actions from a still frame and thus provide information that is co...
Conference Paper
Full-text available
Describing the history of a terrain from a vertical geological cross-section is an important problem in geology, called geological restoration. Designing the sequential evolution of the geometry is usually done manually, involving many trials and errors. In this work, we recast this problem as a storyboarding problem, where the different stages in...
Conference Paper
Full-text available
In this paper, we describe experiments with techniques for locating foods and recognizing food states in cooking videos. We describe production of a new data set that provides annotated images for food types and food states. We compare results with two techniques for detecting food types and food states, and then show that recognizing type and stat...
Conference Paper
Full-text available
This paper presents Figurines, an offline framework for narrative creation with tangible objects, designed to record storytelling sessions with children, teenagers or adults. This framework uses tangible diegetic objects to record a free narrative from up to two storytellers and construct a fully annotated representation of the story. This represen...
Article
Full-text available
An important problem in computer animation of virtual characters is the expression of complex mental states during conversation using the coordinated prosody of voice, rhythm, facial expressions, and head and gaze motion. In this work, the authors propose an expressive conversion method for generating natural speech and facial animation in a variet...
Article
In this work we explore the capability of audiovisual prosodic features (such as fundamental frequency, head motion or facial expressions) to discriminate among different dramatic attitudes. We extract the audiovisual parameters from an acted corpus of attitudes and structure them as frame, syllable and sentence-level features. Using Linear Discrim...
Article
Full-text available
Recordings of stage performances are easy to capture with a high-resolution camera, but are difficult to watch because the actors' faces are too small. We present an approach to automatically create a split screen video that transforms these recordings to show both the context of the scene as well as close-up details of the actors. Given a static r...
Conference Paper
Full-text available
Pretend play is a storytelling technique, naturally used from very young ages, which relies on objectsubstitution to represent the characters of the imagined story. We propose a system which assists thestoryteller by generating a virtualized story from a recorded dialogue performed with 3D printed figurines.We capture the gestures and facial expres...
Chapter
Full-text available
: Nous proposons d'aborder les conditions d'une simulation numérique du travail théâtral à travers la présentation des enjeux d'un projet de recherche en cours de construction, intitulé THéâtre sur mAquette virtueLle Interactive Augmentée (THALIA). Cette démarche de simulation implique un contrôle informatique, dont nous évaluerons la nature à la l...
Conference Paper
Full-text available
When creating real or computer graphics movies, the questions of how to layout elements on the screen, together with how to move the cameras in the scene are crucial to properly conveying the events composing a narrative. Though there is a range of techniques to automatically compute camera paths in virtual environments, none have seriously conside...
Conference Paper
Full-text available
The focus of this study is the generation of expressive audiovisual speech from neutral utterances for 3D virtual actors. Taking into account the segmental and suprasegmental aspects of audiovisual speech, we propose and compare several computational frameworks for the generation of expressive speech and face animation. We notably evaluate a standa...
Conference Paper
Full-text available
For this paper, in the context of the French research project Spectacle en Ligne(s), we have recorded the entire set of rehearsals of one theater and one opera production using state-of-the-art video equipment. The resulting raw video and audio tracks as well as manually generated annotation data were then preprocessed in order to localize actors a...
Conference Paper
Full-text available
Cartoonists and animators often use lines of action to emphasize dynamics in character poses. In this paper, we propose a physically-based model to simulate the line of action's motion, leading to rich motion from simple drawings. Our proposed method is decomposed into three steps. Based on user-provided strokes, we forward simulate 2D elastic moti...
Conference Paper
Full-text available
In this position paper, we argue that advances in intelligent cinematography require better models of the multimodal structure of filmic discourse, and of the inferences made by an audience while films are being watched. Such questions have been addressed by film scholars and cognitive scientists in the past, but their models have not so far had su...
Conference Paper
Full-text available
Vertical video editing is the process of digitally editing the image within the frame as opposed to horizontal video editing, which arranges the shots along a timeline. Vertical editing can be a time-consuming and error-prone process when using manual key-framing and simple interpolation. In this paper, we present a general framework for automatica...
Conference Paper
Full-text available
Through a precise 3D animated reconstruction of a key scene in the movie "Back to the Future" directed by Robert Zemekis, we are able to make a detailed comparison of two very different versions of editing. The first version closely follows film editor Arthur Schmidt original sequence of shots cut in the movie. The second version is automatically g...
Article
Full-text available
The AAAI-14 Workshop program was held Sunday and Monday, July 27–28, 2012, at the Québec City Convention Centre in Québec, Canada. Canada. The AAAI-14 workshop program included fifteen workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Robotics; Artificial Intelligence Applied to Assistive...
Article
We describe an optimization-based approach for automatically creating well-edited movies from a 3D animation. While previous work has mostly focused on the problem of placing cameras to produce nice-looking views of the action, the problem of cutting and pasting shots from all available cameras has never been addressed extensively. In this paper, w...
Article
Full-text available
The pre-production stage in a film workflow is important to save time during production. To be useful in stereoscopic 3-D movie-making, storyboards and previz tools need to be adapted in at least two ways. First, it should be possible to specify the desired depth values with suitable and intuitive user interfaces. Second, it should be possible to p...
Article
Full-text available
This paper presents a system that generates cinematic replays for dialogue-based 3D video games. The system exploits the narrative and geometric information present in these games and automatically computes camera framings and edits to build a coherent cinematic replay of the gaming session. We propose a novel importance-driven approach to cinemati...
Conference Paper
Full-text available
The purpose of this work is to evaluate the contribution of audio-visual prosody to the perception of complex mental states of virtual actors. We propose that global audio-visual prosodic contours - i.e. melody, rhythm and head movements over the utterance - constitute discriminant features for both the generation and recognition of social attitude...
Conference Paper
Full-text available
Figure 1: Camera behaviors for a cinematic replay: (a) CU on F rank 3/4backright screenleft and Lili center (b) CU on Lili 3/4backright screenleft and F rank center , (c) POV F rank MCU on Lili screencenter and (d) POV Lili CU on F rank screencenter Abstract This paper presents a system that generates cinematic replays for dialogue-based 3D video g...
Conference Paper
Full-text available
We propose a framework for automatically generating multiple clips suitable for video editing by simulating pan-tilt-zoom camera movements within the frame of a single static camera. Assuming important actors and objects can be localized using computer vision techniques, our method requires only minimal user input to define the subject matter of ea...
Conference Paper
Full-text available
The purpose of this work is to evaluate the contribution of audio-visual prosody to the perception of complex mental states of virtual actors. We propose that global audio-visual prosodic contours -i.e. melody, rhythm and head movements over the utterance -consti-tute discriminant features for both the generation and recognition of social attitudes...
Article
Full-text available
Basic topological modeling, such as the ability to have several faces share a common edge, has been largely absent from vector graphics. We introduce the vector graphics complex (VGC) as a simple data structure to support fundamental topological modeling operations for vector graphics illustrations. The VGC can represent any arbitrary non-manifold...
Conference Paper
Full-text available
Story generation (including interactive narrative) consists of creating a narrative experience on computer by generating narrative events. It requires building an abstract computational model that can generate a variety of narrative events from a limited set of authored content. These models implement a story logic, as they formalize the occurrence...
Conference Paper
Full-text available
One problem common to all researchers in the field of virtual cinematography and editing is to be able to assess the quality of the output of their systems. There is a pressing requirement for appropriate evaluations of proposed models and techniques. Indeed, though papers are often accompanied with example videos, showing subjective results and oc...
Article
Full-text available
Brief statement: Nicolas Szilas is a doctor in Cognitive Science who has been researching in interactive narrative for the last 15 years. He is the creator of IDtension, a system for Interactive Drama, based on computational models of narrative. Since 2006, he is working on games and interactive narrative for learning. He is particularly interested...
Article
Full-text available
he line of action is a conceptual tool often used by cartoonists and illustrators to help make their figures more consistent and more dramatic. We often see the expression of characters--may it be the dynamism of a super hero, or the elegance of a fashion model--well captured and amplified by a single aesthetic line. Usually this line is laid down...
Conference Paper
Full-text available
The automated computation of appropriate viewpoints in complex 3D scenes is a key problem in a number of computer graphics applications. In particular, crowd simulations create visually complex environments with many simultaneous events for which the computation of relevant viewpoints remains an open issue. In this paper, we propose a system which...
Conference Paper
Full-text available
The article presents a joint audio-video approach towards speaker identity conversion, based on statistical methods originally introduced for voice conversion. Using the experimental data from the 3D BIWI Audiovisual corpus of Affective Communication, mapping functions are built between each two speakers in order to convert speaker-specific feature...
Article
Full-text available
The prose storyboard language is a formal language for describing movies shot by shot, where each shot is described with a unique sentence. The language uses a simple syntax and limited vocabulary borrowed from working practices in traditional movie-making, and is intended to be readable both by machines and humans. The language is designed to serv...
Chapter
Full-text available
Dans ce chapitre, nous abordons la question de la notation de la mise en scène sous l'angle de la théorie de la notation de Goodman, qui fixe des contraintes syntaxiques et sémantiques beaucoup plus strictes qu'il n'est de coutume dans le domaine des études théâtrales. Ceci nous amène à proposer un schéma de notation restreint à un petit nombre d'a...
Article
Full-text available
http://www.generativeart.com
Conference Paper
Full-text available
Generating interactive narratives as movies requires knowledge in cinematography (camera placement, framing, lighting) and film editing (cutting between cameras). We present a framework for generating a well-edited movie from interactively generated scene contents and cameras. Our system computes a sequence of shots by simultaneously choosing which...