Norman Badler

Norman Badler
University of Pennsylvania | UP · Department of Computer and Information Science

PhD, Univ. of Toronto CS

About

540
Publications
97,054
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
17,370
Citations

Publications

Publications (540)
Chapter
With academic and public exposure, our work comes to the attention of the Crew Station Design Section at NASA Johnson Space Center. We obtain funding from them to program an interactive human modeling and ergonomic analysis system called TEMPUS. This initiates a multi-year association with NASA, and later leads to additional project work with NASA...
Chapter
Along the way to building digital bodies, we needed to add faces. We started with a physics-based animation model. I met with Paul Ekman and we adopted and programmed his psychologically-motivated Facial Action Coding System. Other students added speech coordination and eye movements. We held the first NSF Workshop on face animation. The invited at...
Chapter
We combine speech synthesis, pointing behaviors, and locomotion to build a virtual weatherperson. He was an early exploration into digital people for real-time announcer duties. Speech and gestures were indicated with mark-up annotations added to the text prior to animation.
Chapter
Because Jack was embeddable into other software systems, it was easy to control its movements through real-time commands. The emergence of large-scale military simulations led to incorporating multiple Jack instances as dismounted soldiers into distributed interactive simulations and virtual environments. When the Navy started using Jack in trainin...
Chapter
Now I had to go to graduate school. Ginny and I apply to the University of Toronto. We are accepted and I receive a four year Fellowship. During my first year as a Mathematics graduate student, I take an AI class to further my knowledge in computer vision. I take a course from famous polyhedra mathematician Coxeter. While doing a programming projec...
Chapter
We explored expansion of the virtual agent’s sensory capabilities. A mine simulation required hazard detection and exposure accumulation. A sense of hearing required sound simulation in a cluttered environment. Our agents “hear” sound signals distorted and attenuated by obstacles, including other virtual humans. When they received a sound they trie...
Chapter
Some of my Ph.D. students join a local financial firm, Susquehanna International Group, and convince the company to fund a full-scale renovation of the former ENIAC space we were occupying as the Computer Graphics Lab. The SIG Center is on the Penn Engineering tour, and we hear funny, fictional comments about what we do.
Chapter
The culmination of nearly four decades of our movement notation studies yielded an empirically-validated mapping from the OCEAN five-factor personality dimensions into animation parameters. We could now take any existing motion and bias the perceptual result to portray any desired personality parameter setting.
Chapter
Returning to my desire to link text descriptions with human activities, colleagues Bonnie Webber and Mark Steedman and I form the animation from natural language seminar, ANIML The goal was to build more “intelligence” into the digital humans we already possessed, moving them beyond computer graphics representations of body shape and motion.
Chapter
With an anthropologist colleague, Clark Erisckson, we taught an undergraduate course focusing on 3D modeling of anthropologically-significant sites and artifacts. The relationship between the environment and its inhabitants, mostly in South America, is a central theme.
Chapter
University faculty are expected to teach, but learning how to teach is often left to chance. My experiences with other teaching professionals and my gravitation to individualized instruction are part of my own evolution.
Chapter
While trying to animate digital people, I encountered the world of movement notations through early Penn colleague Steve Smoliar and ex-ballet dancer Lynne Weber. We study how such descriptive systems might be used as graphical digital inputs for a human animation system. We explore Labanotation, but learn that it describes what to move but not how...
Chapter
Jan Allbeck realizes our goal to animate meaningful and functional groups of people. Her CAROSA system uses common user interface tools to create scheduled, reactive, opportunistic, and appropriately random actions. We further expand this approach with smart events which co-opt appropriate agents into performing required tasks.
Chapter
With colleague Ruzena Bajcsy, we purchased an early full color raster graphics display system. Freeing ourselves from the line drawings necessitated by prior technology, I invented a sphere-based representation for digital humans called Bubblepeople. They portray a softer, more three-dimensional human form and capture some public attention. Bubblew...
Chapter
As our research enterprise grew, we were fortunate to find funding sources to supply new technology and support Ph.D. students. We become part of the U.S. Army Artificial Intelligence Center at Penn, and that helps expose our work to wider sponsorship through government and military sources. Through mutual video interests with a retiring Electrical...
Chapter
Unlike early seated operator ergonomic systems, Jack could freely walk in any environment. We added path planning for obstacle avoidance, and an adjustment for external influences that could change posture or gait for additional realistic behavior. We investigated the possibility of animating more accurate leg, knee, ankle, and foot motions during...
Chapter
Human eye movements are not random, nor is a fixed gaze realistic. Together with my son Jeremy Badler and Ph.D. students, we explore eye movement and saccade models that reflect the conversational state of the digital human. Thinking, speaking, listening, and attending to objects in the environment all influence gaze behaviors.
Chapter
Our desire for motion capture and real-time interaction led to the construction of the LiveActor virtual reality room. One particular project included constructing a haptic tactor sleeve to give a VR user direct feedback about collisions with virtual objects during maintenance activities. I speculate on why it wasn’t as well utilized as I had expec...
Chapter
An unexpected inspiration and a collaboration with my son David Badler, led to a second spin-off from the computer graphics lab: a software system to display webpages of graduates during [long] commencement exercises. The company, MarchingOrder, is the major player in the graduation event management niche.
Chapter
Now a responsible adult, I need a decent summer job. In desperation I look in the phone book Yellow Pages and accidently hit upon the chance of a lifetime. I join Kramer Research working on an optical character recognition project, become lead programmer, and learn some computer vision techniques on my own. I nearly drop out of UCSB in my Senior ye...
Chapter
I start the Center for Digital Visualization (ViDi) to broaden the scope of our collaborations with artists, anthropologists, archaeologists, and art historians. We engage numerous undergraduates in these projects.
Chapter
We realized that human movements are often precipitated by environmental comfort factors. We gave our agents the perception of temperature and proximity of other heat sources (including other people). They responded spontaneously to improve their comfort index without explicitly being told to move.
Chapter
The thesis work now tied together a number of personal threads. I programmed a 3D animation system to create synthetic computer graphics movies. The primary methodology for the analysis and movement description components surprisingly hark back to the Russian language classes I took at UCSB.
Chapter
Our work has been featured in a number of technical, professional, and mass media venues. An unexpected appearance in a novel is only discovered through a coincidence.
Chapter
My first Ph.D. student was from the Architecture Department. He wanted to work with computer graphics but no one in architecture had any relevant experience at that time. This relationship would turn out to be fortuitous for me, but also valuable for Ginny’s archaeology research.
Chapter
The ANIML seminar eventually settled on the problem of animating textual instructions. This proved to be a rich domain of interest to multiple sponsors, including NASA and the Air Force. The latter had major problems and incentives to create human maintainer simulations. We exploit our own programming structure built on Jack, called Parallel Transi...
Chapter
The arrival of Fulbright Scholar Nuria Pelechano into our research group led to an in depth study of crowd simulation. We explored inter-agent communication, enhancements to the social force movement model, and the live, virtual reality experience of presence in a crowd. Ben Sunshine-Hill invents alibi generation, in order to justify agent pedestri...
Chapter
By 1996, I was running a small business with the University. Jack was an established product with full-time staff support, documentation, and even international distributors. It was time to launch a start-up and take Jack commercial. This was Penn’s first successful software spin-off company.
Chapter
I reflect on the need for lifelong goals, recognize collaborations, and honor additional Ph.D. students outside the digital human domain. I also address some personal academic failures. A retirement party was my chance to applaud the post-degree achievements of selected individuals who greatly influenced me over my academic career. Meaningful resea...
Chapter
Circumstances landed me in the University of California at Santa Barbara. During Freshman year, I met my future wife Ginny, and we applied to and were accepted into a brand new College of Creative Studies at UCSB. This experience was seminal in building my math and teaching skills, but humbling next to the achievements of my peers in the Math progr...
Chapter
By adding Parallel Transition Networks, we can have multiple digital people interacting with each other and their environment. We add planning capabilities and speech generation. Led by colleague Justine Cassell, we create a synthesized full body animated dialogue demonstration. We extend some of these concepts in an interactive, language-driven, m...
Chapter
After retiring from my tenured faculty position at Penn I took a part-time position as Head of Research at a geospatial visualization company, Cesium, run by a former Masters’ student, Patrick Cozzi. I started to investigate using machine learning to generate 2D video crowds for audience venues.
Chapter
Artists and computer graphics mesh naturally. We always welcomed artists into the research lab. After an attempt to launch a Masters in Virtual Environments degree at Penn, our Dean instead created an undergraduate engineering degree called Digital Media Design. We soon figured out what and how to teach, eventually focussing on programming skills....
Chapter
My four year Canadian fellowship had ended and I needed to teach courses at Toronto for continued income. I thought I would stay there on the faculty, but the desired offer didn’t materialize. I needed to find a real job. An opportunity to apply to the University of Pennsylvania arrived in the mail one day. I interviewed there and during my visit l...
Chapter
Building off my professional experience in optical character recognition from Kramer Research, I settle into Toronto’s computer science PhD program and keep a detailed notebook of everything I read or think about. I’m certain that computer vision will be a component in my thesis, somehow. I maintain a strong interest in computer graphics and animat...
Chapter
The culmination of our efforts to combine language and animation produce an extensive definition of a Parametrized Action Representation (PAR). PARs interface with lower-level human motion controllers to achieve a language-driven level of control. We demonstrate this approach with a virtual checkpoint scenario where animated agent behaviors can be...
Chapter
I decide that having computers understand moving image sequences could be a PhD thesis topic. I make a pilgrimage to MIT to hear firsthand from AI Lab head Patrick Winston if he knows of anyone else working in the area. Armed with some indirect assurances that there were not, I commence working on moving image analysis as a thesis topic. I spent co...
Chapter
The ViDi Center organizes a study on whether certain small objects might have been hard to see in an ancient Near Eastern site. A combination of archaeological, architectural, illumination, material appearance, and perceptual factors help address a visibility hypothesis.
Chapter
My origin story begins in grade school, with exposure to digital computers and polyhedra. My family was not academic. By high school I had learned to program in Fortran in an NSF-sponsored summer course. I sought to combine math and computers somehow, but not always with any encouragement.
Chapter
My interest in motion capture arose from a different direction: a need to digitize 3D models from real objects. A series of digitizing systems led us to use these technologies attached to a body to capture 3D motions in a small space in real time. Using inverse kinematics, we could capture whole body motions with just six attached sensors.
Chapter
With some much-needed external funding, I could start my own research program. We revisited modeling human figures for computer vision movement analysis. We wrote new graphics programs to replace my Ph.D. systems. I become the computer graphics “guy,” but retain a strong interest in computer vision techniques. Early research connected vision and gr...
Chapter
Because combat produces injuries, the role of the medic is crucial in warfare. We create a physiological model for Jack so that a virtual medic, interactively controlled by a user, can practice trauma diagnosis and response to injury. The physiological model mirrors major human respiratory and circulatory system behaviors.
Chapter
An important step up from Parametrized Actions is the organization of these activities into stories. Animating human-robot interactions is the theme of an Army-sponsored project. My postdoc, Mubbasir Kapadia, leads a two year concerted effort in crowd motions, navigation planning, and story systems. Behavior trees are a programming structure organi...
Chapter
The professional community centered on the ACM Special Interest Group in Computer Graphics has been an inspiration. I was involved in its early evolution and relish the human connections it has fostered over five decades. I tried to forge better connections between the computer vision and graphics communities, but was less than successful. I also s...
Chapter
The Computer and Information Science faculty at Penn had just been established as a Department in 1972. The department Chair, Aravind Joshi, began to build up a cadre of young faculty. Some of them were highly relevant to my interests in computer vision and movement descriptions in natural language. The collaborations were natural and meaningful. I...
Chapter
I muse on what it means to obtain a “Doctor of Philosophy” degree and what my students have meant to me.
Article
I have been fortunate to have experienced and contributed to the software development of virtual humans. Whether they are Avatars, Conversational Agents, game NPCs, or Virtual Influencers, they all have origins in computer graphics (CG). Indeed, the “DNA” of these human models encompasses a large set of CG component technologies including 3-D model...
Conference Paper
Full-text available
There has recently been an explosion of interest in creating largescale shared virtual spaces for multiplayer content. However, rendering player-controllable avatars in real-time creates latency issues when scaling to thousands of players. We introduce a human audience video dataset to support applications in deep learning-based 2D video audience s...
Article
This paper explores new narrative generation paradigms for open world problems. We propose a speed-up variant of partial planner–accelerated partial order planner, that can automatically generate narratives for large plan spaces. To incorporate real-time free-form user interaction, a dynamic partial planning technique has been introduced to self-re...
Article
A vast body of literature has dealt with the challenges of creating the impression of human appearance and human-like motion in the animation of game characters. In this paper, we further refine these efforts by creating a flexible environment for animating game characters endowed with personality, which is a core descriptor of stable characteristi...
Conference Paper
Full-text available
Developing computational methods for bodily expressed emotion understanding can benefit from knowledge and approaches of multiple fields, including computer vision, robotics, psychology/psychiatry, graphics, data mining, machine learning, and movement analysis. The panel, consisting of active researchers in some closely-related fields, attempts to...
Chapter
Developing computational methods for bodily expressed emotion understanding can benefit from knowledge and approaches of multiple fields, including computer vision, robotics, psychology/psychiatry, graphics, data mining, machine learning, and movement analysis. The panel, consisting of active researchers in some closely-related fields, attempts to...
Article
Full-text available
Statistical analysis of accidents in recent years shows that crowd crushes have become significant non-combat, non-environmental public disasters. Unlike common accidents such as fires, crowd crushes may occur without obvious external causes, and may arise quickly and unexpectedly in otherwise normal surroundings. We use physics-based simulations t...
Article
Full-text available
Crowd simulation addresses algorithmic approaches to steering, navigation, perception, and behavioral models. Significant progress has been achieved in modeling interactions between agents and the environment to avoid collisions, exploit empirical local decision data, and plan efficient paths to goals. We address a relatively unexplored dimension o...
Article
Real-time performance and accuracy are two most challenging requirements in virtual surgery training. These difficulties limit the promotion of advanced models in virtual surgery, including many geometric and physical models. This paper proposes a physical model of virtual soft tissue, which is a twist model based on the Kriging interpolation and m...
Chapter
People, when part of a crowd, are able to perform unusual behavior, which would not be performed by a single person (LeBon 1895). A crowd is a powerful entity and its understanding is very important, specially regarding safety issues. The understanding of crowd motion can provide enough information in order to map people features and behaviors that...
Chapter
This chapter introduces some important aspects of crowd dynamics and crowd evacuation, and discusses regulations concerning evacuation processes as discussed and presented in some countries.
Chapter
When evaluating or simulating crowd egress situations, there are several parameters that should be taken into account. For instance, the initial distribution of the people and/or local densities are important to assess possible hazardous events; tracking people or detecting main flows can be very useful to identify main escape routes or bottlenecks...
Chapter
This chapter presents some current technologies in crowd simulation and it is organized into two parts. First, we present and discuss the main existing state-of-the-art technologies developed with an explicit goal to simulate crowds. Then we present in detail CrowdSim, a new crowd simulation software developed by the authors. This is not a commerci...
Chapter
When analyzing an egress process simulation, it is possible to extract data that can be used to provide a deep analysis of scenarios. Before something goes wrong, a simulation project can identify attention points related to people’s comfort and safety when in egress.
Book
This book describes from a computer science viewpoint the software, methods of simulating and analysing crowds with a particular focus on the effects of panic in emergency situations. The power of modern technology impacts on modern life in multiple ways every day. A variety of scientific models and computational tools have been developed to impro...
Conference Paper
Crowd simulation has become an important area, mainly in entertainment and security applications. In particular, this area has been explored in safety systems to evaluate environments in terms of people comfort and security. In general, the evaluation involves the execution of one or more simulations in order to provide statistical information abou...
Article
Evacuation planning is an important and difficult task in building design. The proposed framework can identify optimal evacuation plans using decision points, which control the ratio of agents that select a particular route at a specific spatial location. The authors optimize these ratios to achieve the best evacuation based on a quantitatively val...
Article
A major goal of research on virtual humans is the animation of expressive characters that display distinct psychological attributes. Body motion is an effective way of portraying different personalities and differentiating characters. The purpose and contribution of this work is to describe a formal, broadly applicable, procedural, and empirically...
Conference Paper
Recognizing and captioning the occurrence of virtual episodes can add descriptive capabilities to games and simulations featuring human-like agents. We introduce a captioning heuristic for multi-character animations. The input of our algorithm consists of the traces (lowest-level procedure names) of each character's animation, such as walking, runn...
Article
A major goal of research on virtual humans is the animation of expressive characters that display distinct psychological attributes. Body motion is an effective way of portraying different personalities and differentiating characters. The purpose and contribution of this work is to describe a formal, broadly applicable, procedural, and empirically...
Book
This book provides a deep understanding of state-of-art methods for simulation of heterogeneous crowds in computer graphics. It will cover different aspects that are necessary to achieve plausible crowd behaviors. The book will be a review of the most recent literature in this field that can help professionals and graduate students interested in th...
Conference Paper
The processing time to simulate crowds for games or simulations is a real challenge. While the increasing power of processing capacity is a reality in the hardware industry, it also means that more agents, better rendering and most sophisticated Artificial Intelligence (AI) methods can be used, so again the computational time is an issue. Despite t...
Conference Paper
A variety of scientific models and computational tools have been developed to improve human safety and comfort in built environments. In this work we discuss the use of crowd simulation to reproduce and to evaluate egress performance in specific scenarios. We present CrowdSim, a crowd simulation tool designed to automatically reproduce crowd behavi...
Article
Crowds arise in a variety of situations, such as public concerts and sporting matches. In typical conditions, the crowd moves in an orderly manner, but panic situations may lead to catastrophic results. We propose a computer vision method to identify motion pattern changes in human crowds that can be related to an unusual event. The proposed approa...
Chapter
Following seminal work [225] on flocking behaviors using particle systems, the field of crowd simulation has grown into a well-developed, multi-faceted area of study [50, 271]. Steering, or goal-directed collision avoidance, is the layer of intelligence that interfaces with navigation to move an agent along its planned path by performing a series o...
Chapter
Crowd simulations can better depict real populations if they include agent traits that introduce character heterogeneity. The memory model presented is one avenue for differentiating indi– viduals. Agents are capable of remembering objects and their locations they encounter in the environment. These memories can be strengthened through repeated enc...
Chapter
There exists a wealth of research [118, 208, 272] that separately addresses the problems of charac– ter animation, steering, navigation, and behavior authoring, with many open challenges [125] that need to be addressed in an effort to arrive at a common framework for simulating autonomous virtual humans for the next generation of narrative-driven i...
Chapter
ADAPT is a modular, flexible platform which provides a comprehensive feature set for anima– tion, navigation, and behavior tools needed for end-to-end simulation development. By allowing a user to independently incorporate a new animation choreographer or steering system, and make those components immediately accessible to the behavior level withou...
Chapter
Agent-based sound perception using packet representation and propagation model. The green arrow in the scene is the sound source position, and the agents’ captions on top show what they just heard. Green indicates correct perception, blue and cyan indicate approximate perception, and red is an incorrectly perceived signal. These sound candidates or...
Chapter
We have integrated sound propagation and human-like perception into virtual human simulations. While sound propagation and synthesis have been explored in computer graphics, and there exist extensive studies on auditory perception in psychology, ours is the first work to enable virtual humans to plausibly hear, listen, and react to auditory trigger...
Chapter
The agent control methods discussed so far have a common feature: movement directives come from essentially omniscient navigation algorithms. Footstep steering uses a local context model based on explicit knowledge of surrounding obstacles and agents, while navigation proceeds with search algorithms exploiting multiple-level spatial abstractions. W...
Chapter
We have covered two key aspects of navigation planning for autonomous agents: the importance of having well-defined abstract representations of the environment and planning in multiple do– mains to achieve paths of high quality with good performance.
Chapter
Real human crowds and populations are not uniform. Not everyone performs the same behaviors and even similar behaviors are not performed in the same ways [126], and subtle movement differences denote widely different personalities [196]. Physical human populations are heterogeneous, so virtual populations should reflect that. Individual differences...
Chapter
Complex virtual worlds with sophisticated, autonomous virtual populations are an essential tool in developing simulations for security, disaster prevention, interactive training experiences and entertainment. Directed interactions between virtual characters help tell rich narratives within the virtual world and personify the characters in the mind...
Chapter
Autonomous agents navigating virtual environments need a robust abstract representation of the walkable or “free” space. Deciding the most adequate representation—the navigation mesh (NavMesh) [258]—can be a challenge with strong dependencies on the application and the size of the crowd. Ideally the navigation mesh should be formed by cells with a...
Article
In the social psychology literature, crowds are classified as audiences and mobs. Audiences are passive crowds, whereas mobs are active crowds with emotional, irrational and seemingly homogeneous behavior. In this study, we aim to create a system that enables the specification of different crowd types ranging from audiences to mobs. In order to ach...
Conference Paper
Full-text available
Many interactive 3D games utilize motion capture for both character animation and user input. These applications require short, meaningful sequences of data. Manually producing these segments of motion capture data is a laborious, time-consuming process that is impractical for real-time applications. We present a method to automatically produce sem...
Book
This volume presents novel computational models for representing digital humans and their interactions with other virtual characters and meaningful environments. In this context, we describe efficient algorithms to animate, control, and author human-like agents having their own set of unique capabilities, personalities, and desires. We begin with t...
Article
To improve the accuracy and interactivity of soft tissue deformation simulation, a new plate spring model based on physics is proposed. The model is parameterized and thus can be adapted to simulate different organs. Different soft tissues are modeled by changing the width, number of pieces, thickness, and length of a single plate spring. In this p...
Article
Full-text available
Virtual heritage architectural and cultural reconstructions may be enhanced by populating the environment with simulated people. There are a number of important human modeling issues to address, such as situationally appropriate clothing, occupations, and behaviors. Our interest here is focused on how people interact with portable items in their en...

Network

Cited By