Conference PaperPDF Available

Pattern Regeneration in Coupled Networks

Authors:
Pattern Regeneration in Coupled Networks
Douglas G. Moore1, Sara I. Walker1and Michael Levin2
1BEYOND Center, Arizona State University, Tempe, AZ 85281
2Allen Discovery Center, Tufts University, Medford, MA 02155
douglas.g.moore@asu.edu
Abstract
Many organisms such as planaria, axolotls and deer exhibit
prodigious regenerative abilities, being capable of regenerat-
ing complex organs or entire body plans. An understanding
of how these organisms store and modify their morphological
patterning information is necessary to identify modes of con-
trol and intervention. Insight into this process is key to the
development of novel biomedical applications. In this work,
we present the CANN(k) model: an abstract computational
model of pattern regeneration which couples an artificial neu-
ral network (ANN) with a k-color cellular automaton (CA).
The ANN provides a global information processing system
which generates state-dependent update rules for the CA. The
CANN(k) models are constructed to generate target patterns
which are stable under perturbations of the pattern. We gen-
erate ensembles of CANN(4) models for each of the 4-color
patterns, assess their sensitivity to changes of the ANN struc-
ture. This provides a novel model for understanding the im-
portant biological phenomenon of neural control of cellular
morphogenesis in development or regeneration.
Introduction
Many animals are capable of regenerating complex struc-
tures after amputation (Birnbaum and S´
anchez Alvarado,
2008). For example, planaria can regenerate their entire
body from a fragment as small as 1/279 of the original an-
imal (Handberg-Thorsager et al., 2008). Control of cellular
activity toward the creation and repair of complex anatom-
ical patterns is a central aspect of evolutionary biology as
well as birth defects, traumatic injury and cancer. The abil-
ity to intervene upon and effectively control these processes
is key to developing novel biomedical applications (Baddour
et al., 2012; Levin, 2011). Modes of control over the process
of large-scale, complex homeostasis are still lacking, despite
significant development in our understanding of the molec-
ular mechanisms necessary for these processes (Stocum and
Cameron, 2011). Specifically, while it is known that the ner-
vous system can guide complex morphogenesis (Herrera-
Rincon et al., 2017), the control dynamics of this process
are very poorly understood. An important open question is
whether regenerative processes require large-scale informa-
tion about the current state of the organism, or if information
local to each cell or tissue is sufficient. The answer to this
question will suggest the ideal level and method of interven-
tion necessary to control regenerative processes.
This work presents an abstract model of pattern regener-
ation that combines both global information processing and
local update rules to generate stable target patterns.
Methods and Results
The model of pattern regeneration considered here, referred
to as a CANN(k) model, is composed of two parts: a k-color
cellular automaton (CA), and a feed-forward artificial neu-
ral network (ANN). The k-color CA is a one-dimensional
array of cells, each of which can be in one of kstates (or col-
ors). The state of each cell is updated according to a nearest-
neighbor rule with the same rule applied to each cell concur-
rently. Traditionally, a fixed rule is applied at each time step,
and the process is iterated to generate a trajectory of states
that terminates in an attractor cycle, a sequence of states that
repeats indefinitely. The distinguishing characteristic of a
CANN(k) model is that the rule used at each time step is not
fixed. Instead, the state of the CA is provided as input to
the CANN(k)’s ANN which outputs the CA update rule to
use at the current time step. Another important choice for
these lattice-type models is the boundary conditions for the
ends of the CA lattice. Here we employ a fixed, open bound-
ary condition where each boundary cell uses an unchanging
“white” cell for its missing neighbor’s state. See Figure 1a
for a schematic representation of a CANN(k) model.
The objective is to better understand how morphological
patterns can be faithfully regenerated when the pattern is
perturbed. We say that the pattern, or particular sequence of
colored CA states, regenerates if it is recovered as a fixed-
point attractor under the model’s dynamics. If the pattern is
perturbed, will the system ultimately return to and retain the
desired pattern? In particular, given a desired target pattern
and a set of perturbations, we wish to construct and analyze
a CANN(k) model with the following three properties:
1. The target pattern is a fixed-point attractor of the dynamic.
2. Every admissible perturbation of the target pattern ulti-
mately converges to that target pattern.
3. No cell that is colored becomes white in the next time step
along any of the perturbed trajectories.
We employed a simulated annealing (SA) algorithm (Kirk-
patrick et al., 1983) to construct CANN(k) models which
satisfy all three criteria. The decision to employ SA for
training was based primarily on the constraint-based spec-
ification of the problem. A deterministic algorithm, such
as back-propagation, is ill-suited here as we are primarily
interested in the final pattern generated rather than the par-
ticular sequence of states in the time series, and we do no
have a well-defined training set. One advantage of the SA
approach is each model is endowed with an energy which
quantifies how well it solves the problem at hand. This pro-
vides a way of quantifying the difference in effectiveness
between two models. This energy is, in our case, defined in
terms of three factors which mirror the constraints above:
1. δis the fraction of cells of the target pattern that are not
fixed by the CANN(k) update rule.
2. κis the fraction of observed states that do not transition
into the target state after some number of time steps.
3. τis the fraction of observed states which introduce new
white cells.
We then define the energy as
E=αδ +βκ +γτ (1)
with 0α, β, γ 1and δ+κ+γ= 1. For this work,
we chose α=β= 4/9, and γ= 1/9, though these can
be adjusted freely. There is no strict guarantee that the SA
algorithm will find a solution; however, the energy in eq. (1)
ensures that the degenerate global minimum satisfies our de-
sired heuristic properties.
We generated ensembles of 100 CANN(4) models for
each of the 729 possible 6cell, 4-color ( , , and )
target patterns with no white cells. We limited the set of pat-
tern perturbations which must converge to the target pattern
to amputations of the desired pattern. An amputation is the
removal of a contiguous region from either end of the target
pattern, i.e. setting cells on either end of the array to white.
This limited type of perturbation roughly models a common
type of intervention biologists perform, and from which re-
generative organisms should recover. An example of an am-
putation and regeneration process is depicted in Figure 1b.
We then assessed the sensitivity of the resulting CANN(4)
models to small perturbations of the underlying ANN. This is
defined as the average change in energy of the model when
either one weight or one threshold of the ANN is modified,
and is distinct from the types of perturbations applied to the
CA state discussed above. We find that 6-cell CANN(4)
models exhibit an average sensitivity of 0.020 ±0.009. This
suggests that the target pattern remains a fixed-point under
small perturbations of the ANN, but that the system looses
the ability to properly regenerate from all amputations.
Subsequent work will include improving the training al-
(a) A CANN Model (b) Trajectory
Figure 1: (a) A 3-cell CANN(k) model consists of a cellular au-
tomaton (CA) and a feed-forward, artificial neural network. (i.) At
each time step, the state of the CA is provided as input into the
ANN. (ii.) The ANN processes that input and generates k3outputs
with values {0,...,k 1}. (iii.) The outputs are assembled into
a CA rule and (iv.) used to update the state of the CA. (b) An Ex-
ample 6-cell CANN(4) Trajectory. The top-most row represents
an extreme amputation (setting colored cells to white) of the target
pattern (bottom row). Each row from top to bottom represents suc-
cessive updates of the state according to the underlying CANN(4)
model. The final row is the fully regenerated, stable target pattern.
gorithms, either by choosing an alternative approach or
modifying the constraint-based energy function, assessing
the scalability of this approach, and performing a detailed
sensitivity analyses of generalized models.
Acknowledgements
This work was supported by the Templeton Foundation
(TWCF0089/AB55 and TWCF0140). M.L. gratefully ac-
knowledges support by the Allen Discovery Center program
through The Paul G. Allen Frontiers Group (12171).
References
Baddour, J. A., Sousounis, K., and Tsonis, P. A. (2012). Organ
repair and regeneration: an overview. Birth defects research.
Part C, Embryo today: reviews, 96(1):1–29.
Birnbaum, K. D. and S ´
anchez Alvarado, A. (2008). Slicing
across kingdoms: regeneration in plants and animals. Cell,
132(4):697–710.
Handberg-Thorsager, M., Fernandez, E., and Salo, E. (2008). Stem
cells and regeneration in planarians. Frontiers in bioscience:
a journal and virtual library, 13:6374–6394.
Herrera-Rincon, C., Pai, V. P., Moran, K. M., Lemire, J. M., and
Levin, M. (2017). The brain is required for normal muscle
and nerve patterning during early xenopus development. Na-
ture communications, 8(1):587.
Kirkpatrick, S., Gelatt, Jr, C. D., and Vecchi, M. P. (1983). Op-
timization by simulated annealing. Science, 220(4598):671–
680.
Levin, M. (2011). The wisdom of the body: future techniques and
approaches to morphogenetic fields in regenerative medicine,
developmental biology and cancer. Regenerative medicine,
6(6):667–673.
Stocum, D. L. and Cameron, J. A. (2011). Looking proximally and
distally: 100 years of limb regeneration and beyond. Devel-
opmental dynamics: an official publication of the American
Association of Anatomists, 240(5):943–968.
... Are there larger or smaller effective iScience Article boundaries cutting across clusters of cells and genes that are more meaningful for the pattern discrimination problem? It has been hypothesized, in the context of cancer and morphogenesis, that when active biological units such as cells come together in informationally connected groups, [161][162][163] the computational boundary demarcating a coherent ''individual'' could scale up from single units to the collective. Large-scale phenomena such as morphogenesis have been described as behaviors of a collective intelligence in anatomical morphospace. ...
Article
Full-text available
Spatiotemporal patterns of cellular resting potential regulate several aspects of development. One key aspect of the bioelectric code is that transcriptional and morphogenetic states are determined not by local, single-cell, voltage levels but by specific distributions of voltage across cell sheets. We constructed and analyzed a minimal dynamical model of collective gene expression in cells based on inputs of multicellular voltage patterns. Causal integration analysis revealed a higher-order mechanism by which information about the voltage pattern was spatiotemporally integrated into gene activity, as well as a division of labor among and between the bioelectric and genetic components. We tested and confirmed predictions of this model in a system in which bioelectric control of morphogenesis regulates gene expression and organogenesis: the embryonic brain of the frog Xenopus laevis. This study demonstrates that machine learning and computational integration approaches can advance our understanding of the information-processing underlying morphogenetic decision-making, with a potential for other applications in developmental biology and regenerative medicine.
... How can we develop ways of manipulating biological systems (such as morphogenesis) via communication and motivation, rather than micromanagement? • Can we develop artificial neural networks whose output is not specific solutions to specific problems, but rather seeds that determine machines that solve problems (Moore et al., 2018)? Can we imitate the learning and generalization capacity of evolution in our machine learning architectures, such that each level of the output specifies a flexible design for an agent that functions in the problem space? ...
Article
Full-text available
The rich variety of biological forms and behaviours results from one evolutionary history on Earth, via frozen accidents and selection in specific environments. This ubiquitous baggage in natural, familiar model species obscures the plasticity and swarm intelligence of cellular collectives. Significant gaps exist in our understanding of the origin of anatomical novelty, of the relationship between genome and form, and of strategies for control of large-scale structure and function in regenerative medicine and bioengineering. Analysis of living forms that have never existed before is necessary to reveal deep design principles of life as it can be. We briefly review existing examples of chimaeras, cyborgs, hybrots and other beings along the spectrum containing evolved and designed systems. To drive experimental progress in multicellular synthetic morphology, we propose teleonomic (goal-seeking, problem-solving) behaviour in diverse problem spaces as a powerful invariant across possible beings regardless of composition or origin. Cybernetic perspectives on chimaeric morphogenesis erase artificial distinctions established by past limitations of technology and imagination. We suggest that a multi-scale competency architecture facilitates evolution of robust problem-solving, living machines. Creation and analysis of novel living forms will be an essential testbed for the emerging field of diverse intelligence, with numerous implications across regenerative medicine, robotics and ethics.
... shape of the cellular automaton and a pre-specified target shape [8,18,28,31,37,40]. ...
Preprint
Full-text available
Information-theoretic fitness functions are becoming increasingly popular to produce generally useful, task-independent behaviors. One such universal function, dubbed empowerment, measures the amount of control an agent exerts on its environment via its sensorimotor system. Specifically, empowerment attempts to maximize the mutual information between an agent's actions and its received sensor states at a later point in time. Traditionally, empowerment has been applied to a conventional sensorimotor apparatus, such as a robot. Here, we expand the approach to a distributed, multi-agent sensorimotor system embodied by a neural cellular automaton (NCA). We show that the addition of empowerment as a secondary objective in the evolution of NCA to perform the task of morphogenesis, growing and maintaining a pre-specified shape, results in higher fitness compared to evolving for morphogenesis alone. Results suggest there may be a synergistic relationship between morphogenesis and empowerment. That is, indirectly selecting for coordination between neighboring cells over the duration of development is beneficial to the developmental process itself. Such a finding may have applications in developmental biology by providing potential mechanisms of communication between cells during growth from a single cell to a multicellular, target morphology. Source code for the experiments in this paper can be found at: \url{https://github.com/caitlingrasso/empowered-nca}.
Article
Full-text available
Possible roles of brain-derived signals in the regulation of embryogenesis are unknown. Here we use an amputation assay in Xenopus laevis to show that absence of brain alters subsequent muscle and peripheral nerve patterning during early development. The muscle phenotype can be rescued by an antagonist of muscarinic acetylcholine receptors. The observed defects occur at considerable distances from the head, suggesting that the brain provides long-range cues for other tissue systems during development. The presence of brain also protects embryos from otherwise-teratogenic agents. Overexpression of a hyperpolarization-activated cyclic nucleotide-gated ion channel rescues the muscle phenotype and the neural mispatterning that occur in brainless embryos, even when expressed far from the muscle or neural cells that mispattern. We identify a previously undescribed developmental role for the brain and reveal a non-local input into the control of early morphogenesis that is mediated by neurotransmitters and ion channel activity.
Article
A number of organs have the intrinsic ability to regenerate, a distinctive feature that varies among organisms. Organ regeneration is a process not fully yet understood. However, when its underlying mechanisms are unraveled, it holds tremendous therapeutic potential for humans. In this review, we chose to summarize the repair and regenerative potential of the following organs and organ systems: thymus, adrenal gland, thyroid gland, intestine, lungs, heart, liver, blood vessels, germ cells, nervous system, eye tissues, hair cells, kidney and bladder, skin, hair follicles, pancreas, bone, and cartilage. For each organ, a review of the following is presented: (a) factors, pathways, and cells that are involved in the organ's intrinsic regenerative ability, (b) contribution of exogenous cells - such as progenitor cells, embryonic stem cells, induced pluripotent stem cells, and bone marrow-, adipose- and umbilical cord blood-derived stem cells - in repairing and regenerating organs in the absence of an innate intrinsic regenerative capability, (c) and the progress made in engineering bio-artificial scaffolds, tissues, and organs. Organ regeneration is a promising therapy that can alleviate humans from diseases that have not been yet cured. It is also superior to already existing treatments that utilize exogenous sources to substitute for the organ's lost structure and/or function(s).
Article
"The restoration of shape is a central goal of regenerative medicine - rebuilding a complex structure such as a hand or eye requires more than stem cells driven towards individual tissues - the placement of all components in the right arrangement is crucial and cannot be micromanaged by direct bioengineering."
Article
The experimental study of amphibian limb regeneration spans most of the 20th century and the first decade of the 21st century. We first review the major questions investigated over this time span: (1) the origin of regeneration blastema cells, the mechanism of tissue breakdown that liberates cells from their tissue organization to participate in blastema formation, (3) the mechanism of dedifferentiation of these cells, (4) how the blastema grows, (5) how the blastema is patterned to restore the missing limb structures, and (6) why adult anurans, birds and mammals do not have the regenerative powers of urodele salamanders. We then look forward in a perspective to discuss the many unanswered questions raised by investigations of the past century, what new approaches can be taken to answer them, and what the prospects are for translation of basic research on limb regeneration into clinical means to regenerate human appendages.
Article
Multicellular organisms possessing relatively long life spans are subjected to diverse, constant, and often intense intrinsic and extrinsic challenges to their survival. Animal and plant tissues wear out as part of normal physiological functions and can be lost to predators, disease, and injury. Both kingdoms survive this wide variety of insults by strategies that include the maintenance of adult stem cells or the induction of stem cell potential in differentiated cells. Repatterning mechanisms often deploy embryonic genes, but the question remains in both plants and animals whether regeneration invokes embryogenesis, generic patterning mechanisms, or unique circuitry comprised of well-established patterning genes.
Article
Understanding stem cells is a major goal of current research because of its potential medical applications. Although great advances have been made, such as the culturing and differentiation of embryonic stem cells and reprogramming of cell fates, many basic questions remain unanswered. Describing the mechanisms underlying regeneration will help to understand the biology of stem cells and therefore to control their behavior. While regeneration is being studied in a variety of models, the planarian is particularly noteworthy. In this model system a fragment as small as 1/279 of the animal can regenerate completely within a few weeks. These animals can also grow and degrow--specifically degenerating certain tissues--according to environmental conditions, thus demonstrating a complete control of their stem cell dynamics. However, one of the most interesting aspects of the planarian model system is the presence of a unique type of stem cell that can differentiate into all cell types found in the organism, including the germ line. This represents a simple, extremely powerful, and accessible stem cell system in which to address a variety of important questions. In the last ten years, molecular, cellular, and bioinformatics tools have been established for use in this model, making it ideally placed for in vivo analysis of stem cells in their natural environment without ethical complications.