Fig 3 - uploaded by Sreedhar S. Kumar
Content may be subject to copyright.
Identification of network specific objective functions. (A) Networks of dissociated neurons in vitro exhibit activity characterized by intermittent network-wide spontaneous bursts (SB) separated by periods of reduced activity (raster plot for 60 channels in a DIV 27 network). The shading marks the limits of individual SBs as detected by the burst-detection algorithm. (B) The distribution of Inter-Burst Intervals (IBIs) is approximately lognormal. The histogram shows the IBI distribution for the network in (A). The cumulative of this distribution (red) is predictive of the probability of being interrupted by ongoing activity given the elapsed period of inactivity, i.e. the current state s t . (C) Such a distribution was used to weight response strengths so that each dot represents the mean response strengths that can be evoked over a set of trials, including those that did not lead to stimulation, for a given stimulation latency. The fit predicts the objective function of the optimization problem. The example shows the data for the network shown in Fig 1C. The curve reveals a quasiconcave dependency, a unique global maximum and an optimal latency of % 2.5 s in this network. (D) Fits to the probability of avoiding an interruption (blue), response strengths prediction (orange), and the resulting weighted response curve (orange, dotted) shown for another network. An optimal latency of % 1.5 s emerges in this case. (E) All predicted objective functions for each of the 20 networks studied were quasiconcave and unique choices of optimal stimulus latencies were available. The objective functions were normalized to peak magnitude. doi:10.1371/journal.pcbi.1005054.g003
Source publication
Electrical stimulation of the brain is increasingly used to alleviate the symptoms of a range of neurological disorders and as a means to artificially inject information into neural circuits in neuroprosthetic applications. Machine learning has been proposed to find optimal stimulation settings autonomously. However, this approach is impeded by the...
Contexts in source publication
Context 1
... ranged between hundreds of milliseconds to few seconds. SBs were detected using an algorithm that combined an inter-spike-interval threshold and the number of simultaneously active sites (Fig 3A). Inter-burst-intervals (IBIs) were approximately lognormal distributed (Fig 3B). ...
Context 2
... were detected using an algorithm that combined an inter-spike-interval threshold and the number of simultaneously active sites (Fig 3A). Inter-burst-intervals (IBIs) were approximately lognormal distributed (Fig 3B). Fit- ting algorithms yielded the location and scale parameters (μ and σ) of the corresponding log- normal distribution. ...
Context 3
... ting algorithms yielded the location and scale parameters (μ and σ) of the corresponding log- normal distribution. The cumulative of this distribution was used to estimate the probability of another SB occurring given the period of inactivity that elapsed-or what we term the 'proba- bility of interruption' following an SB (Fig 3B, red line). . The cumulative of this distribution (red) is predictive of the probability of being interrupted by ongoing activity given the elapsed period of inactivity, i.e. the current state s t . ...
Context 4
... then weighted all response strengths with the probability of being able to deliver a stimulus at the corresponding latencies, without being interrupted by ongoing activity. The weighted response strength curve (objective function) thus provides an estimate of the average number of response spikes that can be evoked for each SB (Fig 3C and 3D). A solution that maximizes this estimate is therefore the optimal solution to the proposed trade-off problem, namely, to find the stimulus latency that maximizes the number of response spikes per SB. ...
Citations
... Kumar et al [173] used an in-vitro neural network to find, with RL, the best stimulus latency that would provide the highest response in terms of bursts of action potential. ...
The brain is a highly complex physical system made of assemblies of neurons that work together to accomplish elaborate tasks such as motor control, memory and perception. How these parts work together has been studied for decades by neuroscientists using neuroimaging, psychological manipulations, and neurostimulation. Neurostimulation has gained particular interest, given the possibility to perturb the brain and elicit a specific response. This response depends on different parameters such as the intensity, the location and the timing of the stimulation. However, most of the studies performed so far used previously established protocols without considering the ongoing brain activity and, thus, without adaptively targeting the stimulation. In control theory, this approach is called open-loop control, and it is always paired with a different form of control called closed-loop, in which the current activity of the brain is used to establish the next stimulation. Recently, neuroscientists are beginning to shift from classical fixed neuromodulation studies to closed-loop experiments. This new approach allows the control of brain activity based on responses to stimulation and thus to personalize individual treatment in clinical conditions. Here, we review this new approach by introducing control theory and focusing on how these aspects are applied in brain studies. We also present the different stimulation techniques and the control approaches used to steer the brain. Finally, we explore how the closed-loop framework will revolutionize the way the human brain can be studied, including a discussion on open questions and an outlook on future advances.
... MEAs enable full control of the neuronal activity, providing concurrent recording and stimulation at multiple points of the population with high spatial and temporal resolution. This versatility fostered several studies aiming to control specific features of neuronal activity with closed-loop electrical stimulation in MEAs, such as the mean population firing rate using a proportional controller 23 , the network response probability and latency to stimulation using a proportional-integral controller 24 and response strength to stimulation using reinforcement learning 25,26 . ...
Adaptive neuronal stimulation has a strong therapeutic potential for neurological disorders such as Parkinson's disease and epilepsy. However, standard stimulation protocols mostly rely on continuous open-loop stimulation. We implement here, for the first time in neuronal populations, two different Delayed Feedback Control (DFC) algorithms and assess their efficacy in disrupting unwanted neuronal oscillations. DFC is a well-established closed-loop control technique but its use in neuromodulation has been limited so far to models and computational studies. Leveraging on the high spatiotemporal monitoring capabilities of specialized in vitro platforms, we show that standard DFC in fact worsens the neuronal population oscillatory behaviour and promotes faster bursting, which was never reported in silico. Alternatively, we present adaptive DFC (aDFC) that monitors ongoing oscillation periodicity and self-tunes accordingly. aDFC disrupts collective neuronal oscillations and decreases network synchrony. Furthermore, we show that the intrinsic population dynamics have a strong impact in the susceptibility of networks to neuromodulation. Experimental data was complemented with computer simulations to show how this network controllability might be determined by specific network properties. Overall, these results support aDFC as a better candidate for therapeutic neurostimulation and provide new insights regarding the controllability of neuronal systems.
... A possible approach to facilitate the fitting procedures could be to develop bidirectional intracortical devices able to record the neuronal activity in response to electrical stimulation and use the recorded neural activity to optimize the stimulation parameters (Rotermund et al., 2019). Another possibility could be to use machine learning to find optimal stimulation settings (Kumar et al., 2016). In any case, more studies are still needed. ...
The restoration of a useful visual sense in a profoundly blind person by direct electrical stimulation of the visual cortex has been a subject of study for many years. However, the field of cortically based sight restoration has made few advances in the last few decades, and many problems remain. In this context, the scientific and technological problems associated with safe and effective communication with the brain are very complex, and there are still many unresolved issues delaying its development. In this work, we review some of the biological and technical issues that still remain to be solved, including long-term biotolerability, the number of electrodes required to provide useful vision, and the delivery of information to the implants. Furthermore, we emphasize the possible role of the neuroplastic changes that follow vision loss in the success of this approach. We propose that increased collaborations among clinicians, basic researchers, and neural engineers will enhance our ability to send meaningful information to the brain and restore a limited but useful sense of vision to many blind individuals.
... In an in vitro study by Kumar et al., Q-learning learning was used to learn the optimal stimulation time for maximizing the intensity of induced spikes in cultures of cortical neurons. [4] Another approach developed to learn new, energy efficient, stimulation patterns for Parkinson's disease used a biophysical surrogate model. By testing different atypical patterns of stimulation against the model, they were able to identify a novel pattern that was more energy efficient than the standard ~130Hz stimulation of the subthalamic nucleus. ...
Neural modulation is becoming a fundamental tool for understanding and treating neurological diseases and their implicated neural circuits. Given that neural modulation interventions have high dimensional parameter spaces, one of the challenges is selecting the stimulation parameters that induce the desired effect. Moreover, the effect of a given set of stimulation parameters may change depending on the underlying neural state. In this study, we investigate and address the state-dependent effect of medial septum optogenetic stimulation on the hippocampus. We found that pre-stimulation hippocampal gamma (33-50Hz) power influences the effect of medial septum optogenetic stimulation on during-stimulation hippocampal gamma power. We then construct a simulation platform that models this phenomenon for testing optimization approaches. We then compare the performance of a standard implementation of Bayesian optimization, along with an extension to the algorithm that incorporates pre-stimulation state to learn a state-dependent policy. The state-dependent algorithm outperformed the standard approach, suggesting that incorporating pre-stimulation can improve neural modulation interventions.
... Yet, this would be sufficient to initiate SBEs only if the output of this local network is well connected to recruit large parts of the network. Conversely, recurrent input from highly excitable regions to the BIZ must not be too strong to avoid lasting depression of excitability in the BIZ by SBEs (Weihberger et al., 2013;Kumar et al., 2016). A moderately connected position with locally recurrent connectivity would fulfill these prerequisites. ...
The mesoscale architecture of neuronal networks strongly influences the initiation of spontaneous activity and its pathways of propagation. Spontaneous activity has been studied extensively in networks of cultured cortical neurons that generate complex yet reproducible patterns of synchronous bursting events that resemble the activity dynamics in developing neuronal networks in vivo. Synchronous bursts are mostly thought to be triggered at burst initiation sites due to build-up of noise or by highly active neurons, or to reflect reverberating activity that circulates within larger networks, although neither of these has been observed directly. Inferring such collective dynamics in neuronal populations from electrophysiological recordings crucially depends on the spatial resolution and sampling ratio relative to the size of the networks assessed. Using large-scale microelectrode arrays with 1024 electrodes at 0.3 mm pitch that covered the full extent of in vitro networks on about 1 cm², we investigated where bursts of spontaneous activity arise and how their propagation patterns relate to the regions of origin, the network’s structure, and to the overall distribution of activity. A set of alternating burst initiation zones (BIZ) dominated the initiation of distinct bursting events and triggered specific propagation patterns. Moreover, BIZs were typically located in areas with moderate activity levels, i.e., at transitions between hot and cold spots. The activity-dependent alternation between these zones suggests that the local networks forming the dominating BIZ enter a transient depressed state after several cycles (similar to Eytan et al., 2003), allowing other BIZs to take over temporarily. We propose that inhomogeneities in the network structure define such BIZs and that the depletion of local synaptic resources limit repetitive burst initiation.
... Desired response features of the neural network can be also achieved with reinforcement-learning using phenomenological model based on Markov Decision Process. The group of Egert (Kumar et al., 2016) developed a controller which autonomously optimized low-frequency stimulation settings and evaluated control strategy in real-time. Statistics of the burst magnitudes and spontaneous events were used to predict and to optimize an optimal inter-stimulus intervals maximizing the response efficacy for each individual network. ...
... In order to optimize the performance of such system, advanced signal processing techniques need to be used, able to rapidly and reliably compute and extract useful information from the recorded signals. In this context, Machine Learning algorithms and Information Theoretic quantities are rapidly taking ground respectively for autonomously adjusting system parameters (Kumar et al., 2016), and for extracting relevant features from neural signals (Panzeri et al., 2017). ...
One of the main limitations preventing the realization of a successful dialogue between the brain and a putative enabling device is the intricacy of brain signals. In this perspective, closed-loop in vitro systems can be used to investigate the interactions between a network of neurons and an external system, such as an interacting environment or an artificial device. In this chapter, we provide an overview of closed-loop in vitro systems, which have been developed for investigating potential neuroprosthetic applications. In particular, we first explore how to modify or set a target dynamical behavior in a network of neurons. We then analyze the behavior of in vitro systems connected to artificial devices, such as robots. Finally, we provide an overview of biological neuronal networks interacting with artificial neuronal networks, a configuration currently offering a promising solution for clinical applications.
... Desired response features of the neural network can be also achieved with reinforcement learning using phenomenological model based on Markov decision process. The group of Egert (Kumar et al. 2016) developed a controller which autonomously optimized low-frequency stimulation settings and evaluated control strategy in real time. Statistics of the burst magnitudes and spontaneous events were used to predict and to optimize an optimal inter-stimulus intervals maximizing the response efficacy for each individual network. ...
... In order to optimize the performance of such system, advanced signal processing techniques need to be used, able to rapidly and reliably compute and extract useful information from the recorded signals. In this context, machine learning algorithms and information theoretic quantities are rapidly taking ground respectively for autonomously adjusting system parameters (Kumar et al. 2016) and for extracting relevant features from neural signals (Panzeri et al. 2017). ...
... In a high throughput manner, machine learning has also been used to optimize combinations of DBS parameters and medications [37] and to determine which DBS parameters lead to desired changes in brain activity [38]. Last, there is a push towards the development of neuromodulation systems that use machine learning to monitor stimulation evoked dopamine signaling and adjusting stimulation parameters to optimize treatment [39] or to determine the time and brain state during which stimulation has the largest effect [40]. With the exception of Kumar et al.'s work, all of these implementations of machine learning used population-based datasets (i.e. ...
The ventral striatum (VS) is a central node within a distributed network that controls appetitive behavior, and neuromodulation of the VS has demonstrated therapeutic potential for appetitive disorders. Local field potential (LFP) oscillations recorded from deep brain stimulation (DBS) electrodes within the VS are a pragmatic source of neural systems-level information about appetitive behavior that could be used in responsive neuromodulation systems. Here, we recorded LFPs from the bilateral nucleus accumbens core and shell (subregions of the VS) during limited access to palatable food across varying conditions of hunger and food palatability in male rats. We used standard statistical methods (logistic regression) as well as the machine learning algorithm lasso to predict aspects of feeding behavior using VS LFPs. We were able to predict the amount of food eaten, the increase in consumption following food deprivation, and the type of food eaten. Further, we were able to predict whether the initiation of feeding was imminent up to 42.5 seconds before feeding began and classify current behavior as either feeding or not-feeding. In classifying feeding behavior, we found an optimal balance between model complexity and performance with models using 3 LFP features primarily from the alpha and high gamma frequencies. As shown here, unbiased methods can identify systems-level neural activity linked to domains of mental illness with potential application to the development and personalization of novel treatments.
... Electrical stimulation (Rattay, 1999;Merrill, 2010) is a consolidated technique that has been widely used to study neuronal networks (Kumar et al., 2016;Wülfing et al., 2018), to treat brain diseases (Perlmutter and Mink, 2006;Benabid et al., 2009) and somatosensory dysfunctions (Brindley and Lewin, 1968;Shannon, 1983Shannon, , 1985Sekirnjak et al., 2008;Tsai et al., 2012;Grosberg et al., 2017;Greenberg et al., 2018;Fan et al., 2019), and to enhance moto-rehabilitation (Raspopovic et al., 2014;Armenta Salas et al., 2018). Electrical stimulation was combined with prosthetic implants in a variety of in vivo applications (Woodson et al., 2009;Dagnelie, 2012). ...
Non-invasive electrical stimulation can be used to study and control neural activity in the brain or to alleviate somatosensory dysfunctions. One intriguing prospect is to precisely stimulate individual targeted neurons. Here, we investigated single-neuron current and voltage stimulation in vitro using high-density microelectrode arrays featuring 26,400 bidirectional electrodes at a pitch of 17.5 μm and an electrode area of 5 × 9 μm². We determined optimal waveforms, amplitudes and durations for both stimulation modes. Owing to the high spatial resolution of our arrays and the close proximity of the electrodes to the respective neurons, we were able to stimulate the axon initial segments (AIS) with charges of less than 2 pC. This resulted in minimal artifact production and reliable readout of stimulation efficiency directly at the soma of the stimulated cell. Stimulation signals as low as 70 mV or 100 nA, with pulse durations as short as 18 μs, yielded measurable action potential initiation and propagation. We found that the required stimulation signal amplitudes decreased with cell growth and development and that stimulation efficiency did not improve at higher electric fields generated by simultaneous multi-electrode stimulation.
... Black lines indicate model residuals. stimulus latency relative to the previous SB, and can be described by a saturating exponential model [8,14] ( Fig. 3 (B)). However, we found that this dependency was non-stationary when observed over long time scales. ...
