Brian Wandell

• PhD
• Professor (Full) at Stanford University

This paper presents a physics-based simulation that models the complete imaging pipeline from scene radiance to final rendered image. We use this simulation to evaluate sensor designs optimized for high dynamic range (HDR) environments, such as driving through daytime tunnels or in nighttime conditions. The work makes three main contributions: (1)...

Two ideas, proposed by Thomas Young and James Clerk Maxwell, form the foundations of colour science: (i) three types of retinal receptors encode light under daytime conditions, and (ii) colour matching experiments establish the critical spectral properties of this encoding. Experimental quantification of these ideas is used in international colour...

A 2D-line-scan MRI sequence has been reported to directly measure neural responses to stimuli (the “DIANA response”). Attempts to replicate the DIANA response have failed, even with higher field strength and more repetitions. Part of this discrepancy is likely due to a limited understanding of how physiological noise manifests in 2D-line-scan acqui...

Several groups have reported using 2D line-scan MRI sequences in humans and mice to directly measure neural responses to stimuli (the DIANA response). Other groups have been unable to replicate the DIANA response, even with higher field strength and more repetitions. Part of this discrepancy is likely due to a limited understanding of the noise pro...

Diffusion MRI is a complex technique, where new discoveries and implementations occur at a fast pace. The expertise needed for data analyses and accurate and reproducible results is increasingly demanding and requires multidisciplinary collaborations. In the present work we introduce Reproducible Tract Profiles 2 (RTP2), a set of flexible and autom...

The design and evaluation of complex systems can benefit from a software simulation - sometimes called a digital twin. The simulation can be used to characterize system performance or to test its performance under conditions that are difficult to measure (e.g., nighttime for automotive perception systems). We describe the image system simulation so...

Blood and cerebrospinal fluid (CSF) pulse and flow throughout the brain, driven by the cardiac cycle. These fluid dynamics, which are essential to healthy brain function, are characterized by several noninvasive magnetic resonance imaging (MRI) methods. Recent developments in fast MRI, specifically simultaneous multislice acquisition methods, provi...

Image systems simulation software can accelerate innovation by reducing many of the time-consuming and expensive steps in designing, building and evaluating image systems. To realize this potential, it is necessary to build trust in physics-based end-to-end image systems simulations. Towards this goal, we describe and experimentally validate an end...

For more than two centuries scientists and engineers have worked to understand and model how the eye encodes electromagnetic radiation (light). We now understand the principles of how light is transmitted through the optics of the eye and encoded by retinal photoreceptors and light-sensitive neurons. In recent years, new instrumentation has enabled...

Combining image sensor simulation tools with physically based ray tracing enables the design and evaluation (soft prototyping) of novel imaging systems. These methods can also synthesize physically accurate, labeled images for machine learning applications. One practical limitation of soft prototyping has been simulating the optics precisely: lens...

Diffusion MRI is a complex technique, where new discoveries and implementations occur at a fast pace. The expertise needed for data analyses and accurate and reproducible results is increasingly demanding and requires multidisciplinary collaborations. In the present work we introduce Reproducible Tract Profiles (RTP2): a set of flexible and automat...

Combining image sensor simulation tools (e.g., ISETCam) with physically based ray tracing (e.g., PBRT) offers possibilities for designing and evaluating novel imaging systems as well as for synthesizing physically accurate, labeled images for machine learning. One practical limitation has been simulating the optics precisely: Lens manufacturers gen...

We assess the accuracy of a smartphone camera simulation. The simulation is an end-to-end analysis that begins with a physical description of a high dynamic range 3D scene and includes a specification of the optics and the image sensor. The simulation is compared to measurements of a physical version of the scene. The image system simulation accura...

Receptive field properties measured in the reading portion of the ventral occipital-temporal (VOT) cortex are task- and stimulus-dependent. To understand these effects, we analyzed responses in visual field-maps (V1-3, hV4, VO1) whose signals are likely inputs to the VOT. Within these maps, each voxel contains neurons that are responsive to specifi...

We describe an end-to-end image systems simulation that models a device capable of measuring fluorescence in the oral cavity. Our software includes a 3D model of the oral cavity and excitation-emission matrices of endogenous fluorophores that predict the spectral radiance of oral mucosal tissue. The predicted radiance is transformed by a model of t...

We describe and experimentally validate an end-to-end simulation of a digital camera. The simulation models the spectral radiance of 3D-scenes, formation of the spectral irradiance by multi-element optics, and conversion of the irradiance to digital values by the image sensor. We quantify the accuracy of the simulation by comparing real and simulat...

We describe an end-to-end image systems simulation that models a device capable of measuring fluorescence in the oral cavity. Our software includes a 3D model of the oral cavity and excitation-emission matrices of endogenous fluorophores that predict the spectral radiance of oral mucosal tissue. The predicted radiance is transformed by a model of t...

Autonomous driving applications use two types of sensor systems to detect vehicles - depth sensing LiDAR and radiance sensing cameras.We compare the performance (average precision) of a ResNet for vehicle detection in complex, daytime, driving scenes when the input is a depth map [D = d(x,y)], a radiance image [L = r(x,y)], or both [D,L]. (1) When...

The visual field region where a stimulus evokes a neural response is called the receptive field (RF). Analytical tools combined with functional MRI (fMRI) can estimate the RF of the population of neurons within a voxel. Circular population RF (pRF) methods accurately specify the central position of the pRF and provide some information about the spa...

We describe and experimentally validate an end-to-end simulation of a digital camera. The simulation models the spectral radiance of 3D-scenes, formation of the spectral irradiance by multi-element optics, and conversion of the irradiance to digital values by the image sensor. We quantify the accuracy of the simulation by comparing real and simulat...

Autonomous driving applications use two types of sensor systems to identify vehicles - depth sensing LiDAR and radiance sensing cameras. We compare the performance (average precision) of a ResNet for vehicle detection in complex, daytime, driving scenes when the input is a depth map (D = d(x,y)), a radiance image (L = r(x,y)), or both [D,L]. (1) Wh...

The visual field region where a stimulus evokes a neural response is called the receptive field (RF). Analytical tools combined with functional MRI can estimate the receptive field of the population of neurons within a voxel. Circular population RF (pRF) methods accurately specify the central position of the pRF and provide some information about t...

The white matter tracts in the living human brain are critical for healthy function, and the diffusion MRI measured in these tracts is correlated with diverse behavioral measures. The technical skills required to analyze diffusion MRI data are complex: data acquisition requires MRI sequence development and acquisition expertise, analyzing raw-data...

Identifying the plastic and stable components of the visual cortex after retinal loss is an important topic in visual neuroscience and neuro-ophthalmology. Humans with juvenile macular degeneration (JMD) show significant blood-oxygen-level-dependent (BOLD) responses in the primary visual area (V1) lesion projection zone (LPZ), despite the absence o...

We have recently shown that the relative spatial contrast sensitivity function (CSF) of a computational observer operating on the cone mosaic photopigment excitations of a stationary retina has the same shape as human subjects. Absolute human sensitivity, however, is 5- to 10-fold lower than the computational observer. Here we model how additional...

Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (i.e., public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validit...

Through the Human Connectome Project (HCP) our understanding of the functional 37 connectome of the healthy brain has been dramatically accelerated. Given the pressing public 38 health need, we must increase our understanding of how connectome dysfunctions give rise to 39 disordered mental states. Mental disorders arising from high levels of negati...

We quantify the generalization of a convolutional neural network (CNN) trained to identify cars. First, we perform a series of experiments to train the network using one image dataset - either synthetic or from a camera - and then test on a different image dataset. We show that generalization between images obtained with different cameras is roughl...

Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (i.e., public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validit...

We investigate the spatial contrast-sensitivity of modern convolutional neural networks (CNNs) and a linear support vector machine (SVM). To measure performance, we compare the CNN contrast sensitivity across a range of patterns with the contrast sensitivity of a Bayesian ideal observer (IO) with the signal-known-exactly and noise-known-statistical...

We quantify the generalization of a convolutional neural network (CNN) trained to identify cars. First, we perform a series of experiments to train the network using one image dataset - either synthetic or from a camera - and then test on a different image dataset. We show that generalization between images obtained with different cameras is roughl...

The Brain Imaging Data Structure (BIDS) is a community-driven specification for organizing neuroscience data and metadata with the aim to make datasets more transparent, reusable, and reproducible. Intracranial electroencephalography (iEEG) data offer a unique combination of high spatial and temporal resolution measurements of the living human brai...

We investigate the performance of a convolutional neural network (CNN) at detecting a signal-known-exactly in Poisson noise. We compare the network performance with that of a Bayesian ideal observer (IO) that reflects the theoretical optimum in detection performance and a linear support vector machine (SVM). For several types of stimuli, including...

Scientists and engineers have created computations and made measurements that characterize the first steps of seeing. ISETBio software integrates such computations and data into an open-source software package. The initial ISETBio implementations modeled image formation (physiological optics) for planar or distant scenes. The ISET3d software descri...

Imaging systems are increasingly used as input to convolutional neural networks (CNN) for object detection; we would like to design cameras that are optimized for this purpose. It is impractical to build different cameras and then acquire and label the necessary data for every potential camera design; creating software simulations of the camera in...

We are sad to report that Professor Jacob (Jack) Nachmias passed away on March 2, 2019. Nachmias was born in Athens, Greece on June 9, 1928. To escape the Nazis, he and his family came to the United States in 1939. He received his undergraduate degree from Cornell University and then an MA from Swarthmore College, where he worked with Hans Wallach...

We have recently shown that using the information carried by the mosaic of cone excitations of a stationary retina, the relative spatial contrast sensitivity function (CSF) of a computational observer has the same shape as a typical human subject. Absolute human sensitivity, however, is lower than the computational observer by a factor of 5 to 10....

The spectral properties of the ambient illumination provide useful information about time of day and weather. We study the perceptual representation of illumination by analyzing measurements of how well people discriminate between illuminations across scene configurations. More specifically, we compare human performance to a computational-observer...

Reproducible Tract Profiles (RTP) comprises a set of methods to manage and analyze diffusion weighted imaging (DWI) data for reproducible tractography. The tools take MRI data from the scanner and process them through a series of analysis implemented as Docker containers that are integrated into a modern neuroinformatics platform (Flywheel). The pl...

We present a computational-observer model of the human spatial contrast-sensitivity function based on the Image Systems Engineering Toolbox for Biology (ISETBio) simulation framework. We demonstrate that ISETBio-derived contrast-sensitivity functions agree well with ones derived using traditional ideal-observer approaches, when the mosaic, optics,...

Scientists and engineers have created computations and made measurements that characterize the first steps of seeing. ISETBio software integrates such computations and data into an open-source software package. The initial ISETBio implementations modeled image formation (physiological optics) for planar or distant scenes. The ISET3d software descri...

Objective. The nature of artificial vision with a retinal prosthesis, and the degree to which the brain can adapt to the unnatural input from such a device, are poorly understood. Therefore, the development of current and future devices may be aided by theory and simulations that help to infer and understand what prosthesis patients see. Approach....

We describe an open-source simulator that creates sensor irradiance and sensor images of typical automotive scenes in urban settings. The purpose of the system is to support camera design and testing for automotive applications. The user can specify scene parameters (e.g., scene type, road type, traffic density, time of day) to assemble a large num...

Intracranial electroencephalography (iEEG) data offer a unique combination of high spatial and temporal resolution measures of the living human brain. However, data collection is limited to highly specialized clinical environments. To improve internal (re)use and external sharing of these unique data, we present a structure for storing and sharing...

The nature of artificial vision with a retinal prosthesis, and the degree to which the brain can adapt to the unnatural input from such a device, are poorly understood. Therefore, the development of current and future devices may be aided by theory and simulations that help to infer and understand what patients see. A novel computational framework...

We present a computational observer model of the human spatial contrast sensitivity (CSF) function based on the Image Systems EngineeringTools for Biology (ISETBio) simulation framework. We demonstrate that ISETBio-derived CSFs agree well with CSFs derived using traditional ideal observer approaches, when the mosaic, optics, and inference engine ar...

The spectral properties of the ambient illumination provide useful information about time of day and weather. We study the perceptual representation of illumination by analyzing measurements of how well people discriminate between illuminations across scene configurations. More specifically, we compare human performance to a computational-observer...

The ENGAGE study: Integrating neuroimaging, virtual reality and smartphone sensing to understand self-regulation for managing depression and obesity in a precision medicine model, Behaviour Research and Therapy (2017), Precision medicine models for personalizing achieving sustained behavior change are largely outside of current clinical practice. Y...

We summarize the current state of knowledge of the brain's reading circuits, and then we describe opportunities to use quantitative and reproducible methods for diagnosing these circuits. Neural circuit diagnostics-by which we mean identifying the locations and responses in an individual that differ significantly from measurements in good readers-c...

Skilled reading requires rapidly recognizing letters and word forms; people learn this skill best for words presented in the central visual field. Measurements over the last decade have shown that when children learn to read, responses within ventral occipito-temporal cortex (VOT) become increasingly selective to word forms. We call these regions t...

Visual cortex contains a hierarchy of visual areas. The earliest cortical area (V1) contains neurons responding to colour, form and motion. Later areas specialize on processing of specific features. The second visual area (V2) in non-human primates contains a stripe-based anatomical organization, initially defined using cytochrome-oxidase staining...

We compare several major white-matter tracts in human and macaque occipital lobe using diffusion magnetic resonance imaging. The comparison suggests similarities but also significant differences in the tracts. There are several apparently homologous tracts in the 2 species, including the vertical occipital fasciculus (VOF), optic radiation, forceps...

Humans resolve the spatial alignment between two visual stimuli at a resolution that is substantially finer than the spacing between the foveal cones. In this paper, we analyze the factors that limit the information at the cone photoreceptors that is available to make these acuity judgments (Vernier acuity). We use open-source software, ISETBIO ¹ t...

Visual neuroscience has traditionally focused much of its attention on understanding the response properties of single neurons or neuronal ensembles. The visual white matter and the long-range neuronal connections it supports are fundamental in establishing such neuronal response properties and visual function. This review article provides an intro...

Modern technologies are enabling scientists to collect extraordinary amounts of complex and sophisticated data across a huge range of scales like never before. With this onslaught of data, we can allow the focal point to shift towards answering the question of how we can analyze and understand the massive amounts of data in front of us. Unfortunate...

Modern technologies are enabling scientists to collect extraordinary amounts of complex and sophisticated data across a huge range of scales like never before. With this onslaught of data, we can allow the focal point to shift towards answering the question of how we can analyze and understand the massive amounts of data in front of us. Unfortunate...

Visual neuroscience has traditionally focused much of its attention on understanding the response properties of neurons along the visual pathways. This review focuses instead on the properties of the white matter connections between these neurons. Specifically, we provide an introduction to methods to study the human visual white matter using diffu...

Introduction Skilled reading involves rapid recognition of foveally-presented word forms. fMRI measurements have shown that regions in ventral occipito-temporal cortex are trained to rapidly recognize word forms, but the spatial sensitivity of these regions is not well understood. Here we measure its field of view – the portion of the visual world...

We compare the major white matter tracts in human and macaque occipital lobe using diffusion MRI. The comparison suggests similarities but also significant differences in spatial arrangement and relative sizes of the tracts. There are several apparently homologous tracts in the two species, including the vertical occipital fasciculus (VOF), optic r...

Skilled reading requires rapidly recognizing letters and word forms; people learn this skill best for words presented in the central visual field. Measurements over the last decade have shown that when children learn to read, responses within ventral occipito-temporal cortex (VOT) become increasingly selective to word forms. We call these regions t...

Quantitative magnetic resonance imaging (qMRI) aims to quantify tissue parameters by eliminating instrumental bias. We describe qMRI theory, simulations, and software designed to estimate proton density (PD), the apparent local concentration of water protons in the living human brain. First, we show that, in the absence of noise, multichannel coil...

Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form that is appropriate for the application. The n...

Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form that is appropriate for the application. The n...