Frederic Fol Leymarie

• PhD
• Professor (Full) at Goldsmiths University of London

Using a single RGB camera to obtain accurate body dimensions, rather than measuring these manually or via more complex multicamera systems or more expensive 3D scanners, has a high application potential for the apparel industry. We present a system that estimates upper human body measurements using a hybrid set of techniques from both classic compu...

This invited special session of IGS 2023 presents the works carried out at Laboratoire Scribens and some of its collaborating laboratories. It summarises the 17 talks presented in the colloquium #611 entitled « La lognormalité: une fenêtre ouverte sur le contrôle neuromoteur» (Lognormality: a window opened on neuromotor control), at the 2023 confer...

We present a system that estimates upper human body measurements using a set of computer vision and machine learning techniques. In a nutshell, the main steps involve: (1) using a portable camera (such as from a smartphone); (2) improving the image quality; (3) isolating the human body from the surrounding environment; (4) performing a calibration...

We present a system that estimates upper human body measurements using a set of computer vision and machine learning techniques. In a nutshell, the main steps involve: (1) using a portable camera (such as from a smartphone); (2) improving the image quality; (3) isolating the human body from the surrounding environment; (4) performing a calibration...

We present a survey of mixed-initiative methods for the creation of content for video games. We also propose a definition of what mixed-initiative implies, as the term lacks a clear specification. The survey includes works not directly aimed at video games but which create content that can potentially be used in games, such as art programs utilizin...

The interrelationship among art, intelligence, and machine has important implications for the visual arts as part of a general education. Here, Frederic Fol Leymarie (FFL), a computer scientist and engineer at Goldsmiths College, and Seymour Simmons III (SS3), an artist and art educator from Winthrop University, South Carolina, discuss these issues...

Until recently, Virtual Reality (VR) applications relied on controllers to enable user interaction in virtual environments. With advances in tracking technology, HMDs are now able to track the movements of users’ hands in real-time with significantly greater accuracy, allowing us to interact with the digital world directly with our hands. However,...

We develop a method to automatically segment a font’s glyphs into a set of overlapping and intersecting strokes with the aim of generating artistic stylizations. The segmentation method relies on a geometric analysis of the glyph’s outline, its interior, and the surrounding areas and is grounded in perceptually informed principles and measures. Our...

Commercial camera traps are usually triggered by a Passive Infra-Red (PIR) motion sensor necessitating a delay between triggering and the image being captured. This often seriously limits the ability to record images of small and fast moving animals. It also results in many “empty” images, e.g., owing to moving foliage against a background of diffe...

We present the development process of Bioblox2-5D, an educational biology game aimed at teenagers. The game content refers to protein docking and aims to improve learning about molecular shape complexity, the roles of charges in molecular docking and the scoring function to calculate binding affinity. We developed the game as part of a collaboratio...

This paper presents the network bending framework, a new approach for manipulating and interacting with deep generative models. We present a comprehensive set of deterministic transformations that can be inserted as distinct layers into the computational graph of a trained generative neural network and applied during inference. In addition, we pres...

In less than a year's time, March 2022 will mark the twentieth anniversary of the first documented game jam, the Indie Game Jam, which took place in Oakland, California in 2002. Initially, game jams were widely seen as frivolous activities. Since then, they have taken the world by storm. Game jams have not only become part of the day-to-day process...

A dominant theory of embodied aesthetic experience (Freedberg & Gallese, 2007, Trends in Cognitive Sciences, 11, 197) posits that the appreciation of visual art is linked to the artist’s movements when creating the artwork, yet a direct link between the kinematics of drawing actions and the aesthetics of drawing outcomes has not been experimentally...

In less than a year’s time, March 2022 will mark the twentieth anniversary of the first documented game jam, the Indie Game Jam, which took place in Oakland, California in 2002. Initially, game jams were widely seen as frivolous activities. Since then, they have taken the world by storm. Game jams have not only become part of the day-to-day process...

We present a review of methods for procedurally generating the morphology of virtual creatures. We include a range of methods, with the main groups being from ALife over art to video games. Even though at times these groups overlap, for clarity we have kept this distinction. By including the word virtual, we mean that we focus on methods for simula...

Based on the Method of Loci, the following experiment compares the effect of two different virtual environments on participants’ memory performance. The primary task consists of remembering a sequence of random playing cards. Each virtual environment is based on a different architectural style with a different layout. One is inspired by a Palladian...

We present a system built to generate arrangements of three-dimensional models for aesthetic evaluation, with the aim to support an artist in their creative process. We explore how this system can automatically generate aesthetically pleasing content for use in the media and design industry, based on standards originally developed in master artwork...

We introduce a new framework for manipulating and interacting with deep generative models that we call network bending. We present a comprehensive set of deterministic transformations that can be inserted as distinct layers into the computational graph of a trained generative neural network and applied during inference. In addition, we present a no...

Je présenterai quelques réflexions sur les potentiels qui se trouvent à l'intersection de l'art, des machines (y compris l'IA) et de l'intelligence. Je ne souhaite pas simplement donner un aperçu de la pratique actuelle des artistes utilisant, ou parfois développant, certains aspects de l’utilisation des machines dans leur pratique, mais plutôt dis...

Our goal is to be able to reproduce computationally calligraphic traces, such as found in the art practices of graffiti and various forms of more traditional calligraphy, while mimicking their production process. To this end, we propose a method that allows to reconstruct kinematics solely from the geometric samples of handwritten traces in the for...

Motivation: The 3D structure of chromatin in the nucleus is important for gene expression and regulation. Chromosome conformation capture techniques, such as Hi-C, generate large amounts of data showing interaction points on the genome but these are hard to interpret using standard tools. Results: We have developed CSynth, an interactive 3D geno...

Deep neural networks have become remarkably good at producing realistic deepfakes, images of people that (to the untrained eye) are indistinguishable from real images. Deepfakes are produced by algorithms that learn to distinguish between real and fake images and are optimised to generate samples that the system deems realistic. This paper, and the...

Graffiti art is a controversial art form, and as such there has been little empirical work assessing its aesthetic value. A recent study examined image statistical properties of text-based artwork and revealed that images of text contain less global structure relative to fine detail compared to artworks. However, previous research did not include g...

We introduce a new framework for interacting with and manipulating deep generative models that we call network bending. We present a comprehensive set of deterministic transformations that can be inserted as distinct layers into the computational graph of a trained generative neural network and applied during inference. In addition, we present a no...

While the games industry is moving towards procedural content generation (PCG) with tools available under popular platforms such as Unreal, Unity or Houdini, and video game titles like No Man's Sky and Horizon Zero Dawn taking advantage of PCG, the gap between academia and industry is as wide as it has ever been, in terms of communication and shari...

Deep neural networks have become remarkably good at producing realistic deepfakes, images of people that are (to the untrained eye) indistinguishable from real images. These are produced by algorithms that learn to distinguish between real and fake images and are optimised to generate samples that the system deems realistic. This paper, and the res...

The chapter presents an approach for the interactive definition of curves and motion paths based on Gaussian mixture model (GMM) and optimal control. The input of our method is a mixture of multivariate Gaussians defined by the user, whose centers define a sparse sequence of key-points, and whose covariances define the precision required to pass th...

With the goal of casting a spotlight on the posture of the creative community at this crucial moment in human technological history, we present herein a thematic overview of the 23 articles published in the recent Arts Special Issues “The Machine as Art (in the 20th Century)” and “The Machine as Artist (for the 21st Century)”. Surprisingly, several...

In this article we present the use of dispersive flies optimisation (DFO) for swarms of particles active on a medialness map – a 2D field representation of shape informed by perception studies. Optimising swarms activity permits to efficiently identify shape-based keypoints to automatically annotate movement and is capable of producing meaningful q...

The 3D structure of chromatin in the nucleus is important for gene expression and regulation. Chromosomal conformation capture techniques, such as Hi-C, generate large amounts of data showing interaction points on the genome but these are hard to interpret using standard tools. We have developed CSynth, a high performance 3D genome browser and real...

We propose a computational framework to learn stylisation patterns from example drawings or writings, and then generate new trajectories that possess similar stylistic qualities. We particularly focus on the generation and stylisation of trajectories that are similar to the ones that can be seen in calligraphy and graffiti art. Our system is able t...

We present a method for the interactive generation of stylised letters, curves and motion paths that are similar to the ones that can be observed in art forms such as graffiti and calligraphy. We define various stylisations of a letter form over a common geometrical structure, which is given by the spatial layout of a sparse sequence of targets. Di...

We propose a computational framework to learn stylisation patterns from example drawings or writings, and then generate new trajectories that possess similar stylistic qualities. We particularly focus on the generation and stylisation of trajectories that are similar to the ones that can be seen in calligraphy and graffiti art. Our system is able t...

We present the Sigma Lognormal model as a potential tool for curve generation in computer graphics related applications. We discuss its extension and parameterisations for the interactive definition of handwriting, drawing and calligraphic trajectories. This results in an efficient trajectory synthesis method, that has a user interface similar to t...

We describe a methodology for the interactive definition of curves and motion paths using a stochastic formulation of optimal control. We demonstrate how the same optimization framework can be used in different ways to generate curves and traces that are geometrically and dynamically similar to the ones that can be seen in art forms such as calligr...

In this article we explore the practical use of medialness informed by perception studies as a representation and processing layer for describing a class of works of visual art. Our focus is towards the description of 2D objects in visual art, such as found in drawings, paintings, calligraphy, graffiti writing, where approximate boundaries or lines...

With the understanding that art and technology are continuing to experience an historic and rapidly intensifying rapprochement—but with the understanding as well that accounts thereof have tended to be constrained by scientific/engineering rigor on the one hand, or have tended to swing to the opposite extreme—it is the goal of this special issue of...

In this paper two swarm intelligence algorithms are used, the first leading the “attention” of the swarm and the latter responsible for the tracing mechanism. The attention mechanism is coordinated by agents of Stochastic Diffusion Search where they selectively attend to areas of a digital canvas (with line drawings) which contains (sharper) corner...

We present an approach to generate rapid and fluid drawing movements on a compliant Baxter robot, by taking advantage of the kinematic redundancy and torque control capabilities of the robot. We concentrate on the task of reproducing graffiti-stylised letter-forms with a marker. For this purpose, we exploit a compact lognormal-stroke based represen...

We propose a perception-based medial point description of a natural form (2D: static or in articulated movement) as a framework for a shape representation which can then be efficiently used in biological species identification and matching tasks. Medialness is defined by adapting and refining a definition first proposed in the cognitive science lit...

The research aims to develop novel techniques able to recognise different sequential gestures, to the level where they will describe and compute articulated movements in real time. In the context of live media arts, the research outcomes would change the paradigm of creating, learning, performing, designing for live media arts, by giving feedback o...

FoldSynth is an interactive platform designed to help understand the characteristics and commonly used visual abstractions of molecular strands with an emphasis on proteins and DNA. It uses a simple model of molecular forces to give real time interactive animations of the folding and docking processes. The shape of a molecular strand is shown as a...

We study the use of the generative systems known as computational ecosystems to convey artistic and narrative aims. These are virtual worlds running on computers, composed of agents that trade units of energy and emulate cycles of life and behaviors adapted from biological life forms. In this article we propose a conceptual framework in order to un...

In this paper we describe a system aimed at the generation and analysis of graffiti tags.We argue that the dynamics of the movement involved in generating tags is in large part - and at a higher degree with respect to many other visual art forms - determinant of their stylistic quality. To capture this notion computationally, we rely on a biophysic...

Title: Decoding De Kooning Swarms of ants and birds set off to decode a complex painting by Willem De Kooning in their own swarmic way. The step-by-step behaviour of the swarms is detailed in [1]. This video displays consecutive cycles of the behaviour of the swarms. A cycle begins when swarms choose a line from the canvas on the right and produc...

This paper presents two complementary approaches to automatically classify pottery sherds: one that focuses on the sherd’s profile and the other that examines visual features of the sherd’s surface. The methods are validated using a set of pottery sherds that were collected during surveys at the ancient site of Koroneia (Greece), which were carried...

We analyse works of digital art that use a technique from artificial life (ALife) called computational ecosystems (CEs). These are systems running on computers where agents are organized in a hierarchical structure (of a food-chain) and trade token units (of energy and biomass) as a way of promoting community dynamics. We analyse a collection of fo...

We have recently introduced the idea of using a perception-based medial point description of a deforming or moving 2D object as a framework for a part-based shape representation which can then be efficiently used in dynamic pattern recognition tasks. The first step is one of fuzzy medialness measurements of 2D segmented objects from intensity image...

We introduce the idea of using a perception-based medial point description of a biological form (such as a 2D profile of a moving animal) as a basis for movement computing which delivers computational schemes to automatically annotate movement and be capable of producing meaningful qualitative descriptions. We distinguish interior from exterior sha...

We introduce the idea of using a perception-based medial point description of a natural form (2D static or in movement) as a framework for a part-based shape representation which can then be efficiently used in biological species identification and matching tasks. The first step is one of fuzzy medialness measurements of 2D segmented objects from i...

We describe Paul, a robotic installation that produces observational face drawings of people. Paul is a naive drawer: it does not have highlevel knowledge of the structures constitutive of the human face (such as the mouth, nose, eyes) nor the capability of learning expertise based on experience as a human would. However, Paul is able to draw using...

One of the most interesting—if frustrating—aspects of charting the history of computer art is trying to understand the intersections of specific technologies and artistic experimentation. It is rarely as clear-cut as a simple linear influence of one to the other, partly because artists are able to envision all kinds of possibilities that technology...

This paper presents two complementary approaches to automatically classify pottery sherds: one that focuses on the sherd’s profile and the other that examines visual features of the sherd’s surface. The methods are validated using a set of pottery sherds that were collected during surveys at the ancient site of Koroneia (Greece), which were carried...

Most of what is known about visual perception in humans is due to empirical work relying on computerised display. In this position paper, we argue that this technology does not permit the investigation of the enactive processes that support visual experience. We discuss the requirements for investigation of situated, embodied interaction of an obse...

In this paper we describe Paul, a robotic installation that produces observational sketches of people. The sketches produced have been considered of interest by fine art professionals in recent art fairs and exhibitions, as well as by the public at large. We identify factors that may account for the perceived qualities of the produced sketches. A t...

Experiments that test perceptual illusions and movement perception have relied predominantly on observing participant response to screen-­based phenomena. There are a number of inherent problems to this experimental method as it involves flicker, ignores depth perception and bypasses the proprioceptive system, in short it is psychophysically distin...

In this paper we introduce a novel approach to dance choreographies in virtual worlds. We present a dance performed by avatars in a virtual world, where a computational ecosystem provides a mechanism driving the actions and movements of the avatars. First, we discuss the background and motivations, and describe the performance. Then, we describe th...

"Vision is continuous creation of meaning from images." - R. L. Gregory [15] Abstract. Shape is the structure of a localised field constructed in relation to an object, which e.g. consists of geometric entities in association with each sample of the object being scrutinized. To identify or characterise shape, we need to "probe" the field [27,35]. I...

Synthetic biology as an artistic technique draws on the exploration of the mechanisms used by organisms to process information and develop their morphologies to address a diverse range of aesthetic agendas. The instrumentalisation of biological processes and methodologies by artists is called Evolutionary Art (EvoArt) and this practice has been att...

We present the latest developments in modeling 3D biomedical data via the Medial Scaffold (MS), a 3D acyclic oriented graph representation of the Medial Axis (MA) [LK07, SP08]. TheMS (and associated 3DMA) can be computed as the result of the singularities of a geometric wave propagation simulation. We consider here some of the potential application...

We propose an algorithm for surface reconstruction from unorganized points based on a view of the sampling process as a deformation from the original surface. In the course of this deformation the Medial Scaffold(MS)(MS) — a graph representation of the 3D Medial Axis(MA)(MA) — of the original surface undergoes abrupt topological changes (transition...

We consider the problem of generating art with a computer system, as it relates to the un- derstanding of a drawing style. Our system in its present development is able to sketch faces automatically, starting from a picture, typically a photographic snapshot of a scene with humans. Once a digital image is considered, the system automatically finds...

A novel biological software approach to define and evolve 3D computer art forms is described based on a re-implementation of the FormGrow system produced by Latham and Todd at IBM in the early 1990’s. This original work is extended by using DNA sequences as the input to generate complex organic-like forms. The translation of the DNA data to 3D grap...

We conclude the book by covering a wide spectrum of applications of medial symmetries of shape from the infinitely large toward the infinitely small. Our journey starts with a dynamic model of the formation and evolution of galaxies. We move on to the description of geographical information at the scale of regions of planet Earth. Next is the repre...

xTNZ, is focused on the exploration of the possibilities of using artificial life in the context of art. My aim isto develop an ecosystem based on a real-time three-dimensional PC based system sustaining a “living” virtual environment. The entities populating this virtual world have been designed to be active and responsive. They behave and interact...

We propose an algorithm for surface reconstruction from unorganized points based on a view of the sampling process as a deformation from the original surface. In the course of this deformation the Medial Scaffold (MS) - a graph representation of the 3D Medial Axis (MA) - of the original surface undergoes abrupt changes (transitions) such that the M...

We introduce the use of weighted medial graph structures to represent sounds and musical events. Such structures permit the building of hierarchical neighborhood relationships (symmetries) while mapping time to a weight (of a note, or event). This is useful to relate perceptual organisations (groupings in the sense of Gestalt theory) both visually...

We present a novel "biological" approach to define and evolve 3D art forms. The work combines a re-implementation of the Form-Grow system of Todd and Latham [Todd and Latham 1992] with an external source to define the shapes: DNA sequences.

We introduce the notion of the medial scaffold, a hierarchical organization of the medial axis of a 3D shape in the form of a graph constructed from special medial curves connecting special medial points. A key advantage of the scaffold is that it captures the qualitative aspects of shape in a hierarchical and tightly condensed representation. We p...

We present a new method for controlling kinematic joint chains in animation rigs which addresses significant shortcomings of traditional forward and inverse kinematic systems, combining the best aspects of each while eliminating their respective trade-offs.

In Aesthetic Computing, key scholars and practitioners from art, design, computer science, and mathematics lay the foundations for a discipline that applies the theory and practice of art to computing. Aesthetic computing explores the way art and aesthetics can play a role in different areas of computer science. One of its goals is to modify comput...

This volume starts from an interdisciplinary expertise of the contributors, and chooses to work on the very origins of conscious qualitative states in perception. The leading research paradigm can be synthesized in ‘ phenomenology to neurons to stimuli, and backwards’, since as a starting point it has taken the phenomenal appearances in the visual...

The Mid-Ocean Studio, Brown University's SHAPE lab, and Goldsmiths College are collaborating on a prototype system for the automatic evolution of biomimetic sculpture using structural automata. This collaboration is resulting in effective, computerized means to autogenerate large, increasingly complex works of art, and allowing for a long-anticipat...

A prototype system for the automatic evolution of biomimetic structures using structural automata is described and its utility for generating digital sculpture is demonstrated. Sculptures are generated from a primordial shape which is represented in terms of a triangular mesh and sculpture is created by extending the original surface using tetrahed...

The reconstruction of objects from data in practical applications often leads to surfaces with small perturbations and other artifacts, which make the detection of their ridges and generalized axes difficult. We propose an approach to smoothing small structures while preserving ridges, which is based on the medial axis structure of the surface. The...

This work proposes a method for global registration based on matching 3D medial structures of unorganized point clouds or triangulated meshes. Most practical known methods are based on the iterative closest point (ICP) algorithm, which requires an initial alignment close to the globally optimal solution to ensure convergence to a valid solution. Fu...

The shock scaffold is a hierarchical organization of the medial axis in 3D consisting of special medial points and curves connecting these points, thereby forming a geometric directed graph, which is key in applications such as object recognition. In this paper we describe a method for computing the shock scaffold of realistic datasets, which invol...

Available in film copy from University Microfilms International. Thesis (Ph. D.)--Brown University, 2003. Vita. Thesis advisor: Benjamin B. Kimia. Includes bibliographical references (leaves 234-262).

The usefulness of the 3D Medial Axis (MA) is dependent on both the availability of accurate and stable methods for computing individual MA points and on schemes for deriving the local structure and connectivity among these points. We propose a framework which achieves both by combining the advantages of exact bisector computations used in computati...

We present a discrete framework for 3D wave propagation to support morphological computations with an emphasis on the recovery of the medial axis of a 3D solid, a collection of surface patches, or a data set of unorganized points. The wave propagation is implemented on a discrete lattice, where initial surfaces are considered as sources of propagat...

A heretofore unsolved problem of great archaeological importance is the automatic assembly of pots made on a wheel from the hundreds (or thousands) of sherds found at an excavation site. An approach is presented to the automatic estimation of mathematical models of such pots from 3D measurements of sherds. A Bayesian approach is formulated beginnin...

A heretofore unsolved problem of great archaeological importance is the automatic assembly of pots made on a wheel from the hundreds (or thousands) of sherds found at an excavation site. An approach is presented to the automatic estimation of mathematical models of such pots from 3D measurements of sherds. The overall approach is formulated and des...

The usefulness of the 3D medial axis (MA) is dependent on both the availability of accurate and stable methods for computing individual MA points and on schemes for deriving the local structure and connectivity among these points. We review the leading approaches for computing the 3D MA and the different fields of application for this representatio...

The notion of curvature of planar curves has emerged as one of the most powerful for the representation and interpretation of objects in an image. Although curvature extraction from a digitized object contour would seem to be a rather simple task, few methods exist that are at the same time easy to implement, fast, and reliable in the presence of n...