No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Hourglass Effect in Hierarchical Dependency Networks
Abstract
Many hierarchically modular systems are structured in a way that resembles a bow-tie or hourglass. This "hourglass effect" means that the system generates many outputs from many inputs through a relatively small number of intermediate modules that are critical for the operation of the entire system (the waist of the hourglass). We investigate the hourglass effect in general (not necessarily layered) hierarchical dependency networks. Our analysis focuses on the number of source-to-target dependency paths that traverse each vertex, and it identifies the core of a dependency network as the smallest set of vertices that collectively cover almost all dependency paths. We then examine if a given network exhibits the hourglass property or not, comparing its core size with a "flat" (i.e., non-hierarchical) network that preserves the source dependencies of each target in the original network. As a possible explanation for the hourglass effect, we propose the Reuse Preference (RP) model that captures the bias of new modules for reusing intermediate modules of high complexity instead of connecting directly to sources. We have applied the proposed framework in a diverse set of dependency networks from technological, natural and information systems, showing that all these networks exhibit the general hourglass property but to a varying degree and with different waist characteristics.
... For the first purpose of the weighted hourglass analysis, we develop a method named multi-edge transformation (MET) which involves performing hourglass analysis on a weight-based transformation of the original network. An important concept in identifying the "waist" or core nodes in the hourglass analysis is the path centrality metric [19,25], which was earlier defined only for unweighted networks. The MET method presented in this paper redefines the path centrality of a node by taking into account the weights of edges along the source-target (sensory-motor) paths that traverse through the node. ...
... More recently frameworks characterizing the bow-tie structure have been developed to study networks that are organized in a hierarchically modular fashion and facilitate distributed information processing [16][17][18]. In the same line, [19] developed the hourglass analysis framework to study hierarchical dependency networks, especially for networks with a relatively higher number of inputs and outputs mediated through a much smaller set of intermediate modules. The hourglass effect has been observed in networks from various domains in biology including metabolism [16,20,21], neuronal structure for visual-cognitive tasks [22]. ...
... The hourglass effect has been observed in networks from various domains in biology including metabolism [16,20,21], neuronal structure for visual-cognitive tasks [22]. The hourglass framework identifies a set of core nodes (known as the τ-core) in a source-target dependency network based on the path centrality metric [19], returning a small set of critical nodes through which most input to output pathways pass. Additionally, [19] also developed the H-score metric to quantify the extent to which a given network shows the hourglass effect. ...
Understanding hierarchy and modularity in natural as well as technological networks is of utmost importance. A major aspect of such analysis involves identifying the nodes that are crucial to the overall processing structure of the network. More recently, the approach of hourglass analysis has been developed for the purpose of quantitatively analyzing whether only a few intermediate nodes mediate the information processing between a large number of inputs and outputs of a network. We develop a new framework for hourglass analysis that takes network weights into account while identifying the core nodes and the extent of hourglass effect in a given weighted network. We use this framework to study the structural connectome of the C. elegans and identify intermediate neurons that form the core of sensori-motor pathways in the organism. Our results show that the neurons forming the core of the connectome show significant differences across the male and hermaphrodite sexes, with most core nodes in the male concentrated in sex-organs while they are located in the head for the hermaphrodite. Our work demonstrates that taking weights into account for network analysis framework leads to emergence of different network patterns in terms of identification of core nodes and hourglass structure in the network, which otherwise would be missed by unweighted approaches.
... More recently frameworks characterizing the bow-tie structure have been developed to study networks that are organized in a hierarchically modular fashion and facilitate distributed information processing (Zhao et al., 2007;Friedlander et al., 2015;Mattie et al., 2018). In the same line, Sabrin and Dovrolis (2017) developed the hourglass analysis framework to study hierarchical dependency networks, especially for networks with a relatively higher number of inputs and outputs mediated through a much smaller set of intermediate modules. The hourglass effect has been observed in networks from various domains in biology including metabolism (Zhao et al., 2006(Zhao et al., , 2007Tanaka et al., 2005), neuronal structure for visual-cognitive tasks (Quiroga et al., 2005). ...
... The hourglass effect has been observed in networks from various domains in biology including metabolism (Zhao et al., 2006(Zhao et al., , 2007Tanaka et al., 2005), neuronal structure for visual-cognitive tasks (Quiroga et al., 2005). The hourglass framework identifies a set of core nodes (known as the τ-core) in a source-target dependency network based on the path centrality metric (Sabrin and Dovrolis, 2017), returning a small set of critical nodes through which most input to output pathways pass. Additionally, Sabrin and Dovrolis (2017) also developed the H-score metric to quantify the extent to which a given network shows the hourglass effect. ...
... The hourglass framework identifies a set of core nodes (known as the τ-core) in a source-target dependency network based on the path centrality metric (Sabrin and Dovrolis, 2017), returning a small set of critical nodes through which most input to output pathways pass. Additionally, Sabrin and Dovrolis (2017) also developed the H-score metric to quantify the extent to which a given network shows the hourglass effect. ...
Understanding hierarchy and modularity in natural as well as technological networks is of utmost importance. A major aspect of such analysis involves identifying the nodes that are crucial to the overall processing structure of the network. More recently, the approach of hourglass analysis has been developed for the purpose of quantitatively analyzing whether only a few intermediate nodes mediate the information processing between a large number of inputs and outputs of a network. We develop a new framework for hourglass analysis that takes network weights into account while identifying the core nodes and the extent of hourglass effect in a given weighted network. We use this framework to study the structural connectome of the \textit{C. elegans} and identify intermediate neurons that form the core of sensori-motor pathways in the organism. Our results show that the neurons forming the core of the connectome show significant differences across the male and hermaphrodite sexes, with most core nodes in the male concentrated in sex-organs while they are located in the head for the hermaphrodite. Our work demonstrates that taking weights into account for network analysis framework leads to emergence of different network patterns in terms of identification of core nodes and hourglass structure in the network, which otherwise would be missed by unweighted approaches.
... Introduction the analysis framework of [23] on C. elegans is that the former assumes that the network from a given set of input nodes (sources) to a given set of output nodes (targets) is a Directed Acyclic Graph (DAG). On the contrary, the C. elegans connectome includes many nested feedback loops between all three types of neurons. ...
... On the contrary, the C. elegans connectome includes many nested feedback loops between all three types of neurons. For this reason, we extend the methods of [23] in networks that may include cycles as long as we are given a set of sources and a set of targets. The key idea is to identify the set of feedforward paths from each source towards targets, and to apply the hourglass analysis framework on the union of such paths, across all sources. ...
... After utilizing one of the previous routing schemes to compute all paths from a sensory neuron to a motor neuron, we analyze these "source-target" paths based on the hourglass framework, developed in [23]. The objective of this analysis is to examine whether there is a small set of nodes through which almost all source-target paths go through. ...
We approach the C. elegans connectome as an information processing network that receives input from about 90 sensory neurons, processes that information through a highly recurrent network of about 80 interneurons, and it produces a coordinated output from about 120 motor neurons that control the nematode’s muscles. We focus on the feedforward flow of information from sensory neurons to motor neurons, and apply a recently developed network analysis framework referred to as the “hourglass effect”. The analysis reveals that this feedforward flow traverses a small core (“hourglass waist”) that consists of 10-15 interneurons. These are mostly the same interneurons that were previously shown (using a different analytical approach) to constitute the “rich-club” of the C. elegans connectome. This result is robust to the methodology that separates the feedforward from the feedback flow of information. The set of core interneurons remains mostly the same when we consider only chemical synapses or the combination of chemical synapses and gap junctions. The hourglass organization of the connectome suggests that C. elegans has some similarities with encoder-decoder artificial neural networks in which the input is first compressed and integrated in a low-dimensional latent space that encodes the given data in a more efficient manner, followed by a decoding network through which intermediate-level sub-functions are combined in different ways to compute the correlated outputs of the network. The core neurons at the hourglass waist represent the information bottleneck of the system, balancing the representation accuracy and compactness (complexity) of the given sensory information.
... Hierarchically modular designs enhance evolvability in natural systems [15,16,19], make the maintenance easier in technological systems, and provide agility and better abstraction of the system design [9,18]. ...
... Informally, an hourglass architecture means that the system of interest produces many outputs from many inputs through a relatively small number of highly central intermediate modules, referred to as the "waist" of the hourglass. It has been observed that hierarchically modular systems often exhibit the architecture of an hourglass; for reference, in fields like computer networking [2], neural networks [11,10], embryogenesis [5], metabolism [8,23], and many others [19,22], this phenomena is observed. A comprehensive survey of the literature on hierarchical systems evolution, and the hourglass effect is presented in [19]. ...
... It has been observed that hierarchically modular systems often exhibit the architecture of an hourglass; for reference, in fields like computer networking [2], neural networks [11,10], embryogenesis [5], metabolism [8,23], and many others [19,22], this phenomena is observed. A comprehensive survey of the literature on hierarchical systems evolution, and the hourglass effect is presented in [19]. ...
Many complex systems, both in technology and nature, exhibit hierarchical modularity: smaller modules, each of them providing a certain function, are used within larger modules that perform more complex functions. Previously, we have proposed a modeling framework, referred to as Evo-Lexis, that provides insight to some fundamental questions about evolving hierarchical systems. The predictions of the Evo-Lexis model should be tested using real data from evolving systems in which the outputs can be well represented by sequences. In this paper, we investigate the time series of iGEM synthetic DNA dataset sequences, and whether the resulting iGEM hierarchies exhibit the qualitative properties predicted by the Evo-Lexis framework. Contrary to Evo-Lexis, in iGEM the amount of reuse decreases during the timeline of the dataset. Although this results in development of less cost-efficient and less deep Lexis-DAGs, the dataset exhibits a bias in reusing specific nodes more often than others. This results in the Lexis-DAGs to take the shape of an hourglass with relatively high H-score values and stable set of core nodes. Despite the reuse bias and stability of the core set, the dataset presents a high amount of diversity among the targets which is in line with modeling of Evo-Lexis.
... It has been observed across several disciplines that hierarchically modular systems are often structured in a way that resembles a bow-tie or hourglass (depending on whether that structure is viewed horizontally or vertically) [25,1]. This structure enables the system to generate many outputs from many inputs through a relatively small number of intermediate modules, referred to as the "knot" of the bow-tie or the "waist" of the hourglass. ...
... This structure enables the system to generate many outputs from many inputs through a relatively small number of intermediate modules, referred to as the "knot" of the bow-tie or the "waist" of the hourglass. 1 This "hourglass effect" has been observed in systems of embryogenesis [26,27], metabolism [28,29,30], immunology [31,32], signaling networks [33], vision and cognition [34,35], deep neural networks [36], computer networking [37], manufacturing [38], as well as in the context of general core-periphery complex networks [39,40]. The few intermediate modules in the hourglass waist are critical for the operation of the entire system, and so they are also more conserved during the evolution of the system compared to modules that are closer to inputs or outputs [37,41,42]. ...
... In this paper, we apply the hourglass analysis framework of [1] on the C. elegans connectome [44]. The C. elegans connectome can be thought of as an information processing network that transforms stimuli received by the environment, through sensory neurons, into coordinated bodily activities (such as locomotion) controlled by motor neurons [44]. ...
We approach the C. elegans connectome as an information processing network that receives input from about 90 sensory neurons, processes that information through a highly recurrent network of about 80 interneurons, and it produces a coordinated output from about 120 motor neurons that control the nematode’s muscles. We focus on the feedforward flow of information from sensory neurons to motor neurons, and apply a recently developed network analysis framework referred to as the “hourglass effect”. The analysis reveals that this feedforward flow traverses a small core (“hourglass waist”) that consists of 10-15 interneurons. These are mostly the same interneurons that were previously shown (using a different analytical approach) to constitute the “rich-club” of the C. elegans connectome. This result is robust to the methodology that separates the feedforward from the feedback flow of information. The set of core interneurons remains mostly the same when we consider only chemical synapses or the combination of chemical synapses and gap junctions. The hourglass organization of the connectome suggests that C. elegans has some similarities with encoder-decoder artificial neural networks in which the input is first compressed and integrated in a low-dimensional latent space that encodes the given data in a more efficient manner, followed by a decoding network through which intermediate-level sub-functions are combined in different ways to compute the correlated outputs of the network. The core neurons at the hourglass waist represent the information bottleneck of the system, balancing the representation accuracy and compactness (complexity) of the given sensory information.
Author Summary
The C. elegans nematode is the only species for which the complete wiring diagram (“connectome”) of its neural system has been mapped. The connectome provides architectural constraints that limit the scope of possible functions of a neural system. In this work, we identify one such architectural constraint: the C. elegans connectome includes a small set (10-15) of neurons that compress and integrate the information provided by the much larger set of sensory neurons. These intermediate-level neurons encode few sub-functions that are combined and re-used in different ways to activate the circuits of motor neurons, which drive all higher-level complex functions of the organism such as feeding or locomotion. We refer to this encoding-decoding structure as “hourglass architecture” and identify the core neurons at the “waist” of the hourglass. We also discuss the similarities between this property of the C. elegans connectome and artificial neural networks. The hourglass architecture opens a new way to think about, and experiment with, intermediate-level neurons between input and output neural circuits.
... Additionally, modular systems are also organized in a hierarchical way: smaller modules are re-used within larger modules recursively [30]. Examples of such systems exist in a wide range of environments: in natural systems, it is believed that hierarchical modularity enhances evolvability (the ability of the system to adapt to new environments with minimal changes) and robustness (the ability to maintain the current status in the presence of internal or external variations) [26,32]. In the technological world, hierarchically modular designs are preferred in terms of design and development cost, easier maintenance and agility (e.g. less effort in producing future versions of a software), and better abstraction of the system design [28]. ...
... An additional focus of our work is the hourglass effect in hierarchical systems. Across many fields, such as in computer networking [1], deep neural networks [19], embryogenesis [13], metabolism [36], and many others [32], it has been observed that hierarchically modular systems often exhibit the architecture of an hourglass. Informally, an hourglass architecture means that the system of interest produces many outputs from many inputs through a relatively small number of highly central intermediate modules, referred to as the "waist" of the hourglass (Fig. 1). ...
... Informally, an hourglass architecture means that the system of interest produces many outputs from many inputs through a relatively small number of highly central intermediate modules, referred to as the "waist" of the hourglass (Fig. 1). The waist of the hourglass (also referred to as "core" in [32] as well as in this paper) includes critical modules of the system that are also sometimes more conserved during the evolution of the system compared to other modules [1,32]. Despite recent research on the hourglass effect in different types of hierarchical systems [2,1,17,32], one of the questions that is still open is to identify the conditions under which the hourglass effect emerges in hierarchies that are produced when the objective is to minimize the cost of interconnections. ...
It is well known that many complex systems, both in technology and nature, exhibit hierarchical modularity. What is not well understood however is how this hierarchical structure (which is fundamentally a network property) emerges, and how it evolves over time. Further, the norm is that hierarchical systems are designed incrementally over time to provide support for new outputs and potentially to new inputs. This is very different than re-designing a new system "from scratch" after a change in the outputs or inputs. We propose a modeling framework, referred to as Evo-Lexis, that provides insight to some general and fundamental queries about evolving hierarchical systems. Evo-Lexis models the system inputs as symbols ("sources") and the outputs as sequences of those symbols ("targets"). Evo-Lexis computes the optimized adjustment of a given hierarchy when the set of targets changes over time by additions and removals ("incremental design"). Additionally, Evo-Lexis computes the optimized hierarchy that generates a given set of targets from a set of sources in a static (non-evolving) setting ("clean-slate design"). The questions we focus on are: 1. How do some key properties of this hierarchy, e.g. depth of the network, reuse or centrality of each module, complexity (or sequence length) of intermediate modules, etc., depend on the evolutionary process generating the new targets? 2. Under what conditions do the emergent hierarchies exhibit the so called "hourglass effect"? Why are few intermediate modules reused much more than others? 3. Do intermediate modules persist during the evolution of hierarchies? Or are there "punctuated equilibria" where the highly reused modules change significantly? 4. Which are the differences in terms of cost and structure between the incrementally designed and the corresponding clean-slate designed hierarchies?
... Demand and supply networks are often described as having an hourglass structure. [170][171][172] Demand can be conceived as the lower bulb in an hourglass "pulling" supply. This structural observation can also serve as an operational analogy: many contemporary supply networks are organized to deliver just in time-just when the consumer is ready and able to buy. ...
... Considering a wide array of hourglass structures, one recent study notes, "The presence of these critical modules at the waist (the 'constraints') limit the space of all possible outputs that the system can generate." [172] During the 2017 Hurricane Season, the real capacity of crucial networks was again and again reduced to what was happening-or too often not happening-at the neck of these perceived hourglass structures. ...
The 2017 Atlantic Hurricane Season (particularly Hurricanes Harvey, Irma, and Maria) was remarkable for its power and the extent of damage to communities in the path of its storms. The ability to deliver lifeline commodities (food, fuel, public water supply, medical goods) to affected communities is a core capability of the public sector "relief channel." The larger and more isolated the population, the more dependent affected communities are on private sector supply chains to deliver sufficient quantities of these commodities in a timely manner. This study consists of case studies which examine the resilience of, challenges to, and interactions among private sector supply chains and the relief channel in Texas, Florida, Puerto Rico and the US Virgin Islands. The evidence presented in these case studies reveals new insights and opportunities for intervention which may help speed the restoration of lifeline supply chains during and after disasters-to prevent catastrophe.
... We have explained the rise of hierarchical modularity in networks with a biphasic (bow-tie) theory of module emergence (Mittenthal et al., 2012), which relates to things that grow (Caetano-Anollés et al., 2018). The theory is compatible with modeling frameworks that reveal hierarchical modularity induces an "hourglass" effect in which networks channel many inputs to produce many outputs through a core of intermediate nodes (Sabrin and Dovrolis, 2017). We used chronologies to test the rise of hierarchical modularity in evolutionary time . ...
... During the process of growth and selection, a number of structural motifs emerge that are useful for robust function of the adult connectome. In C. elegans, the hierarchical nature of the connectome reveals a number of higher-order organization principles such as rich-club connectivity [21] and the hourglass effect [22]. These structural aspects have their origins in neural development, and in fact are the primary basis for facilitating functions such as developmental plasticity and learning. ...
Connecting brain and behavior is a longstanding issue in the areas of behavioral science, artificial intelligence, and neurobiology. As is standard among models of artificial and biological neural networks, an analogue of the fully mature brain is presented as a blank slate. However, this does not consider the realities of biological development and developmental learning. Our purpose is to model the development of an artificial organism that exhibits complex behaviors. We introduce three alternate approaches to demonstrate how developmental embodied agents can be implemented. The resulting developmental Braitenberg vehicles (dBVs) will generate behaviors ranging from stimulus responses to group behavior that resembles collective motion. We will situate this work in the domain of artificial brain networks along with broader themes such as embodied cognition, feedback, and emergence. Our perspective is exemplified by three software instantiations that demonstrate how a BV-genetic algorithm hybrid model, a multisensory Hebbian learning model, and multi-agent approaches can be used to approach BV development. We introduce use cases such as optimized spatial cognition (vehicle-genetic algorithm hybrid model), hinges connecting behavioral and neural models (multisensory Hebbian learning model), and cumulative classification (multi-agent approaches). In conclusion, we consider future applications of the developmental neurosimulation approach.
... Alternatively, non-adaptive drivers that approach 'neutrality' can arise from patterns of network duplications and differentiations that generate modularity 'for free' as a phase transition [75]. Similarly, hierarchy may simply arise by a preference to reuse modules of similar complexity [76]. Finally, simulations have shown that decreasing connection costs in a network produces modularity, hierarchy, and evolvability when systems are poised to maximize performance [77,78]. ...
RNA evolves by adding substructural parts to growing molecules. Molecular accretion history can be dissected with phylogenetic methods that exploit structural and functional evidence. Here, we explore the statistical behaviors of lengths of double-stranded and single-stranded segments of growing tRNA, 5S rRNA, RNase P RNA, and rRNA molecules. The reconstruction of character state changes along branches of phylogenetic trees of molecules and trees of substructures revealed strong pushes towards an economy of scale. In addition, statistically significant negative correlations and strong associations between the average lengths of helical double-stranded stems and their time of origin (age) were identified with the Pierson’s correlation and Spearman’s rho methods. The ages of substructures were derived directly from published rooted trees of substructures. A similar negative correlation was detected in unpaired segments of rRNA but not for the other molecules studied. These results suggest a principle of diminishing returns in RNA accretion history. We show this principle follows a tendency of substructural parts to decrease their size when molecular systems enlarge that follows the Menzerath–Altmann’s law of language in full generality and without interference from the details of molecular growth.
... A related aspect of interest is how particular properties of networks evolve, such as modularity, hierarchy or structures like the hourglass shape [10][11][12][13]. There are many different measures and characteristics of networks that can be used to study evolvability. ...
Network approaches have become pervasive in many research fields. They allow for a more comprehensive understanding of complex relationships between entities as well as their group-level properties and dynamics. Many networks change over time, be it within seconds or millions of years, depending on the nature of the network. Our focus will be on comparative network analyses in life sciences, where deciphering temporal network changes is a core interest of molecular, ecological, neuropsychological and evolutionary biologists. Further, we will take a journey through different disciplines, such as social sciences, finance and computational gastronomy, to present commonalities and differences in how networks change and can be analysed. Finally, we envision how borrowing ideas from these disciplines could enrich the future of life science research.
... During the process of growth and selection, a number of structural motifs emerge that are useful for robust function of the adult connectome. In C. elegans, the hierarchical nature of the connectome reveals a number of higher-order organization principles such as rich-club connectivity [21] and the hourglass effect [22]. These structural aspects have their origins in neural development, and in fact are the primary basis for facilitating functions such as developmental plasticity and learning. ...
The connection between brain and behavior is a longstanding issue in the areas of behavioral science, artificial intelligence, and neurobiology. Particularly in artificial intelligence research, behavior is generated by a black box approximating the brain. As is standard among models of artificial and biological neural networks, an analogue of the fully mature brain is presented as a blank slate. This model generates outputs and behaviors from a priori associations, yet this does not consider the realities of biological development and developmental learning. Our purpose is to model the development of an artificial organism that exhibits complex behaviors. We will introduce our approach, which is to use Braitenberg Vehicles (BVs) to model the development of an artificial nervous system. The resulting developmental BVs will generate behaviors that range from stimulus responses to group behavior that resembles collective motion. Next, we will situate this work in the domain of artificial brain networks. Then we will focus on broader themes such as embodied cognition, feedback, and emergence. Our perspective will then be exemplified by three software instantiations that demonstrate how a BV-genetic algorithm hybrid model, multisensory Hebbian learning model, and multi-agent approaches can be used to approach BV development. We introduce use cases such as optimized spatial cognition (vehicle-genetic algorithm hybrid model), hinges connecting behavioral and neural models (multisensory Hebbian learning model), and cumulative classification (multi-agent approaches). In conclusion, we will revisit concepts related to our approach and how they might guide future development.
... The bow-tie can be explained by a preference to "reuse" modules of similar complexity instead of connecting to less complex modules. 123 It also decomposes nodes into 4 sets: input and output components, a central knot that may contain strongly connected components, and disconnected components known as tendrils. 60 The bottleneck of the central knot limits the flow of information and/or time duration in evolving networks as long as tendril connectivity remains constrained. ...
Networks describe how parts associate with each other to form integrated systems which often have modular and hierarchical structure. In biology, network growth involves two processes, one that unifies and the other that diversifies. Here, we propose a biphasic (bow-tie) theory of module emergence. In the first phase, parts are at first weakly linked and associate variously. As they diversify, they compete with each other and are often selected for performance. The emerging interactions constrain their structure and associations. This causes parts to self-organize into modules with tight linkage. In the second phase, variants of the modules diversify and become new parts for a new generative cycle of higher level organization. The paradigm predicts the rise of hierarchical modularity in evolving networks at different timescales and complexity levels. Remarkably, phylogenomic analyses uncover this emergence in the rewiring of metabolomic and transcriptome-informed metabolic networks, the nanosecond dynamics of proteins, and evolving networks of metabolism, elementary functionomes, and protein domain organization.
... We use layer active probability to represent active degree of nodes in each layer. Thus the active probability p ðnÞ I in layer I can be defined as smile curve as shown in Fig 3 [46,47]. Here ...
Building evolution model of supply chain networks could be helpful to understand its development law. However, specific characteristics and attributes of real supply chains are often neglected in existing evolution models. This work proposes a new evolution model of supply chain with manufactures as the core, based on external market demand and internal competition-cooperation. The evolution model assumes the external market environment is relatively stable, considers several factors, including specific topology of supply chain, external market demand, ecological growth and flow conservation. The simulation results suggest that the networks evolved by our model have similar structures as real supply chains. Meanwhile, the influences of external market demand and internal competition-cooperation to network evolution are analyzed. Additionally, 38 benchmark data sets are applied to validate the rationality of our evolution model, in which, nine manufacturing supply chains match the features of the networks constructed by our model.
Author Summary
Having a structural network representation of connectivity in the brain is instrumental in analyzing communication dynamics and neural information processing. In this work, we make steps towards understanding multisensory information flow and integration using a network diffusion approach. In particular, we model the flow of evoked activity, initiated by stimuli at primary sensory regions, using the asynchronous linear threshold (ALT) diffusion model. The ALT model captures how evoked activity that originates at a given region of the cortex “ripples through” other brain regions (referred to as an activation cascade). We apply the ALT model to the mouse connectome provided by the Allen Institute for Brain Science. A first result, using functional datasets based on voltage-sensitive dye (VSD) imaging, is that the ALT model, despite its simplicity, predicts the temporal ordering of each sensory activation cascade quite accurately. We further apply this model to study multisensory integration and find that a small number of brain regionsthe claustrum and the parietal temporal cortex being at the top of the listare involved in almost all cortical sensory streams. This suggests that the cortex relies on an hourglass architecture to first integrate and compress multisensory information from multiple sensory regions, before utilizing that lower dimensionality representation in higher level association regions and more complex cognitive tasks.
Having a structural network representation of connectivity in the brain is instrumental in analyzing communication dynamics and information processing in the brain. In this work, we make steps towards understanding multi-sensory information flow and integration using a network diffusion approach. In particular, we model the flow of evoked activity, initiated by stimuli at primary sensory regions, using the Asynchronous Linear Threshold (ALT) diffusion model. The ALT model captures how evoked activity that originates at a given region of the cortex ripples through other brain regions (referred to as an activation cascade). By comparing the model results to functional datasets based on Voltage Sensitive Dye (VSD) imaging, we find that in most cases the ALT model predicts the temporal ordering of an activation cascade correctly. Our results on the Mouse Connectivity Atlas from the Allen Institute for Brain Science show that a small number of brain regions are involved in many primary sensory streams -- the claustrum and the parietal temporal cortex being at the top of the list. This suggests that the cortex relies on an hourglass architecture to first integrate and compress multi-sensory information from multiple sensory regions, before utilizing that lower-dimensionality representation in higher-level association regions and more complex cognitive tasks.
Many complex systems, both in technology and nature, exhibit hierarchical modularity: smaller modules, each of them providing a certain function, are used within larger modules that perform more complex functions. Previously, we have proposed a modeling framework, referred to as Evo-Lexis [21], that provides insight to some fundamental questions about evolving hierarchical systems. The predictions of the Evo-Lexis model should be tested using real data from evolving systems in which the outputs can be well represented by sequences. In this paper, we investigate the time series of iGEM synthetic DNA dataset sequences, and whether the resulting iGEM hierarchies exhibit the qualitative properties predicted by the Evo-Lexis framework. Contrary to Evo-Lexis, in iGEM the amount of reuse decreases during the timeline of the dataset. Although this results in development of less cost-efficient and less deep Lexis-DAGs, the dataset exhibits a bias in reusing specific nodes more often than others. This results in the Lexis-DAGs to take the shape of an hourglass with relatively high H-score values and stable set of core nodes. Despite the reuse bias and stability of the core set, the dataset presents a high amount of diversity among the targets which is in line with modeling of Evo-Lexis.
It is well known that many complex systems, both in technology and nature, exhibit hierarchical modularity: smaller modules, each of them providing a certain function, are used within larger modules that perform more complex functions. What is not well understood however is how this hierarchical structure (which is fundamentally a network property) emerges, and how it evolves over time.We propose a modeling framework, referred to as Evo-Lexis, that provides insight to some fundamental questions about evolving hierarchical systems. Evo-Lexis models the most elementary modules of the system as symbols (“sources”) and the modules at the highest level of the hierarchy as sequences of those symbols (“targets”). Evo-Lexis computes the optimized adjustment of a given hierarchy when the set of targets changes over time by additions and removals (a process referred to as “incremental design”).In this paper we use computation modeling to show that:
Low-cost and deep hierarchies emerge when the population of target sequences evolves through tinkering and mutation.
Strong selection on the cost of new candidate targets results in reuse of more complex (longer) nodes in an optimized hierarchy.
The bias towards reuse of complex nodes results in an “hourglass architecture” (i.e., few intermediate nodes that cover almost all source–target paths).
With such bias, the core nodes are conserved for relatively long time periods although still being vulnerable to major transitions and punctuated equilibria.
Finally, we analyze the differences in terms of cost and structure between incrementally designed hierarchies and the corresponding “clean-slate” hierarchies which result when the system is designed from scratch after a change.
KeywordsComplex systemsHierarchical structure evolutionNetwork scienceOptimization
Data represented as strings abounds in biology, linguistics, document mining, web search and many other fields. Such data often have a hierarchical structure, either because they were artificially designed and composed in a hierarchical manner or because there is an underlying evolutionary process that creates repeatedly more complex strings from simpler substrings. We propose a framework, referred to as "Lexis", that produces an optimized hierarchical representation of a given set of "target" strings. The resulting hierarchy, "Lexis-DAG", shows how to construct each target through the concatenation of intermediate substrings, minimizing the total number of such concatenations or DAG edges. The Lexis optimization problem is related to the smallest grammar problem. After we prove its NP-Hardness for two cost formulations, we propose an efficient greedy algorithm for the construction of Lexis-DAGs. We also consider the problem of identifying the set of intermediate nodes (substrings) that collectively form the "core" of a Lexis-DAG, which is important in the analysis of Lexis-DAGs. We show that the Lexis framework can be applied in diverse applications such as optimized synthesis of DNA fragments in genomic libraries, hierarchical structure discovery in protein sequences, dictionary-based text compression, and feature extraction from a set of documents.
Hierarchical organization -- the recursive composition of sub-modules -- is
ubiquitous in biological networks, including neural, metabolic, ecological, and
genetic regulatory networks, and in human-made systems, such as large
organizations and the Internet. To date, most research on hierarchy in networks
has been limited to quantifying this property. However, an open, important
question in evolutionary biology is why hierarchical organization evolves in
the first place. It has recently been shown that modularity evolves because of
the presence of a cost for network connections. Here we investigate whether
such connection costs also tend to cause a hierarchical organization of such
modules. In computational simulations, we find that networks without a
connection cost do not evolve to be hierarchical, even when the task has a
hierarchical structure. However, with a connection cost, networks evolve to be
both modular and hierarchical, and these networks exhibit higher overall
performance and evolvability (i.e. faster adaptation to new environments).
Additional analyses confirm that hierarchy independently improves adaptability
after controlling for modularity. Overall, our results suggest that the same
force--the cost of connections--promotes the evolution of both hierarchy and
modularity, and that these properties are important drivers of network
performance and adaptability. In addition to shedding light on the emergence of
hierarchy across the many domains in which it appears, these findings will also
accelerate future research into evolving more complex, intelligent
computational brains in the fields of artificial intelligence and robotics.
Bow-tie or hourglass structure is a common architectural feature found in biological and technological networks. A bow-tie in a multi-layered structure occurs when intermediate layers have much fewer components than the input and output layers. Examples include metabolism where a handful of building blocks mediate between multiple input nutrients and multiple output biomass components, and signaling networks where information from numerous receptor types passes through a small set of signaling pathways to regulate multiple output genes. Little is known, however, about how bow-tie architectures evolve. Here, we address the evolution of bow-tie architectures using simulations of multi-layered systems evolving to fulfill a given input-output goal. We find that bow-ties spontaneously evolve when two conditions are met: (i) the evolutionary goal is rank deficient, where the rank corresponds to the minimal number of input features on which the outputs depend, and (ii) The effects of mutations on interaction intensities between components are described by product rule - namely the mutated element is multiplied by a random number. Product-rule mutations are more biologically realistic than the commonly used sum-rule mutations that add a random number to the mutated element. These conditions robustly lead to bow-tie structures. The minimal width of the intermediate network layers (the waist or knot of the bow-tie) equals the rank of the evolutionary goal. These findings can help explain the presence of bow-ties in diverse biological systems, and can also be relevant for machine learning applications that employ multi-layered networks.
In the hierarchy of data, information and knowledge, computational methods play a major role in the initial processing of
data to extract information, but they alone become less effective to compile knowledge from information. The Kyoto Encyclopedia
of Genes and Genomes (KEGG) resource (http://www.kegg.jp/ or http://www.genome.jp/kegg/) has been developed as a reference knowledge base to assist this latter process. In particular, the KEGG pathway maps are
widely used for biological interpretation of genome sequences and other high-throughput data. The link from genomes to pathways
is made through the KEGG Orthology system, a collection of manually defined ortholog groups identified by K numbers. To better
automate this interpretation process the KEGG modules defined by Boolean expressions of K numbers have been expanded and improved.
Once genes in a genome are annotated with K numbers, the KEGG modules can be computationally evaluated revealing metabolic
capacities and other phenotypic features. The reaction modules, which represent chemical units of reactions, have been used
to analyze design principles of metabolic networks and also to improve the definition of K numbers and associated annotations.
For translational bioinformatics, the KEGG MEDICUS resource has been developed by integrating drug labels (package inserts)
used in society.
Network-analysis literature is rich in node-centrality mea- sures that quantify the centrality of a node as a function of the (shortest) paths of the network that go through it. Existing work focuses on defining instances of such mea- sures and designing algorithms for the specific combinato- rial problems that arise for each instance. In this work, we propose a unifying definition of centrality that subsumes all path-counting based centrality definitions: e.g., stress, be- Tweenness or paths centrality. We also define a generic algo- rithm for computing this generalized centrality measure for every node and every group of nodes in the network. Next, we define two optimization problems: k-Group Central- ity Maximization and k-Edge Centrality Boosting. In the former, the task is to identify the subset of k nodes that have the largest group centrality. In the latter, the goal is to identify up to k edges to add to the network so that the centrality of a node is maximized. We show that both of these problems can be solved efficiently for arbitrary central- ity definitions using our general framework. In a thorough experimental evaluation we show the practical utility of our framework and the efficacy of our algorithms. Copyright
Recent studies uncovered important core/periphery network structures
characterizing complex sets of cooperative and competitive interactions between
network nodes, be they proteins, cells, species or humans. Better
characterization of the structure, dynamics and function of core/periphery
networks is a key step of our understanding cellular functions, species
adaptation, social and market changes. Here we summarize the current knowledge
of the structure and dynamics of "traditional" core/periphery networks,
rich-clubs, nested, bow-tie and onion networks. Comparing core/periphery
structures with network modules, we discriminate between global and local
cores. The core/periphery network organization lies in the middle of several
extreme properties, such as random/condensed structures, clique/star
configurations, network symmetry/asymmetry, network
assortativity/disassortativity, as well as network hierarchy/anti-hierarchy.
These properties of high complexity together with the large degeneracy of core
pathways ensuring cooperation and providing multiple options of network flow
re-channelling greatly contribute to the high robustness of complex systems.
Core processes enable a coordinated response to various stimuli, decrease
noise, and evolve slowly. The integrative function of network cores is an
important step in the development of a large variety of complex organisms and
organizations. In addition to these important features and several decades of
research interest, studies on core/periphery networks still have a number of
unexplored areas.
The "developmental hourglass'' describes a pattern of increasing morphological divergence towards earlier and later embryonic development, separated by a period of significant conservation across distant species (the "phylotypic stage''). Recent studies have found evidence in support of the hourglass effect at the genomic level. For instance, the phylotypic stage expresses the oldest and most conserved transcriptomes. However, the regulatory mechanism that causes the hourglass pattern remains an open question. Here, we use an evolutionary model of regulatory gene interactions during development to identify the conditions under which the hourglass effect can emerge in a general setting. The model focuses on the hierarchical gene regulatory network that controls the developmental process, and on the evolution of a population under random perturbations in the structure of that network. The model predicts, under fairly general assumptions, the emergence of an hourglass pattern in the structure of a temporal representation of the underlying gene regulatory network. The evolutionary age of the corresponding genes also follows an hourglass pattern, with the oldest genes concentrated at the hourglass waist. The key behind the hourglass effect is that developmental regulators should have an increasingly specific function as development progresses. Analysis of developmental gene expression profiles from
Drosophila melanogaster and
Arabidopsis thaliana provide consistent results with our theoretical predictions.
Hierarchy seems to pervade complexity in both living and artificial systems. Despite its relevance, no general theory that captures all features of hierarchy and its origins has been proposed yet. Here we present a formal approach resulting from the convergence of theoretical morphology and network theory that allows constructing a 3D morphospace of hierarchies and hence comparing the hierarchical organization of ecological, cellular, technological, and social networks. Embedded within large voids in the morphospace of all possible hierarchies, four major groups are identified. Two of them match the expected from random networks with similar connectivity, thus suggesting that nonadaptive factors are at work. Ecological and gene networks define the other two, indicating that their topological order is the result of functional constraints. These results are consistent with an exploration of the morphospace, using in silico evolved networks.
LetN be a finite set andz be a real-valued function defined on the set of subsets ofN that satisfies z(S)+z(T)≥z(S⋃T)+z(S⋂T) for allS, T inN. Such a function is called submodular. We consider the problem maxS⊂N{a(S):|S|≤K,z(S) submodular}. Several hard combinatorial optimization problems can be posed in this framework. For example, the problem of finding a maximum weight independent set in a matroid, when the elements of the matroid are colored and the elements of the independent set can have no more thanK colors, is in this class. The uncapacitated location problem is a special case of this matroid optimization problem. We analyze greedy and local improvement heuristics and a linear programming relaxation for this problem. Our results are worst case bounds on the quality of the approximations. For example, whenz(S) is nondecreasing andz(0) = 0, we show that a “greedy” heuristic always produces a solution whose value is at least 1 −[(K − 1)/K] K times the optimal value. This bound can be achieved for eachK and has a limiting value of (e − 1)/e, where e is the base of the natural logarithm.
Many systems migrate towards increasing or decreasing modularity, yet no explicit causal models exist to explain this process. This paper builds a general theory of modular systems, drawing on systems research from many disciplines. This general theory is then used to derive a model of inter-firm product modularity, including testable research propositions. The latter model provides a valuable tool for predicting technological trajectories, and demonstrates how the general theory may be applied to specific systems.
A central biological question is how natural organisms are so evolvable (capable of quickly adapting to new environments). A key driver of evolvability is the widespread modularity of biological networks-their organization as functional, sparsely connected subunits-but there is no consensus regarding why modularity itself evolved. Although most hypotheses assume indirect selection for evolvability, here we demonstrate that the ubiquitous, direct selection pressure to reduce the cost of connections between network nodes causes the emergence of modular networks. Computational evolution experiments with selection pressures to maximize network performance and minimize connection costs yield networks that are significantly more modular and more evolvable than control experiments that only select for performance. These results will catalyse research in numerous disciplines, such as neuroscience and genetics, and enhance our ability to harness evolution for engineering purposes.
Animal and plant development starts with a constituting phase called embryogenesis, which evolved independently in both lineages. Comparative anatomy of vertebrate development--based on the Meckel-Serrès law and von Baer's laws of embryology from the early nineteenth century--shows that embryos from various taxa appear different in early stages, converge to a similar form during mid-embryogenesis, and again diverge in later stages. This morphogenetic series is known as the embryonic 'hourglass', and its bottleneck of high conservation in mid-embryogenesis is referred to as the phylotypic stage. Recent analyses in zebrafish and Drosophila embryos provided convincing molecular support for the hourglass model, because during the phylotypic stage the transcriptome was dominated by ancient genes and global gene expression profiles were reported to be most conserved. Although extensively explored in animals, an embryonic hourglass has not been reported in plants, which represent the second major kingdom in the tree of life that evolved embryogenesis. Here we provide phylotranscriptomic evidence for a molecular embryonic hourglass in Arabidopsis thaliana, using two complementary approaches. This is particularly significant because the possible absence of an hourglass based on morphological features in plants suggests that morphological and molecular patterns might be uncoupled. Together with the reported developmental hourglass patterns in animals, these findings indicate convergent evolution of the molecular hourglass and a conserved logic of embryogenesis across kingdoms.
A common but informal notion in social network analysis and other fields is the concept of a core/periphery structure. The intuitive conception entails a dense, cohesive core and a sparse, unconnected periphery. This paper seeks to formalize the intuitive notion of a core/periphery structure and suggests algorithms for detecting this structure, along with statistical tests for testing a priori hypotheses. Different models are presented for different kinds of graphs (directed and undirected, valued and nonvalued). In addition, the close relation of the continuous models developed to certain centrality measures is discussed.
The pages and hyperlinks of the World-Wide Web may be viewed as nodes and edges in a directed graph. This graph has about a billion nodes today, several billion links, and appears to grow exponentially with time. There are many reasons—mathematical, sociological, and commercial—for studying the evolution of this graph. We first review a set of algorithms that operate on the Web graph, addressing problems from Web search, automatic community discovery, and classification. We then recall a number of measurements and properties of the Web graph. Noting that traditional random graph models do not explain these observations, we propose a new family of random graph models.
The Internet protocol stack has a layered architecture that resembles an hourglass. The lower and higher layers tend to see frequent innovations, while the protocols at the waist of the hourglass appear to be "ossified". We propose EvoArch, an abstract model for studying protocol stacks and their evolution. EvoArch is based on a few principles about layered network architectures and their evolution in a competitive environment where protocols acquire value based on their higher layer applications and compete with other protocols at the same layer. EvoArch produces an hourglass structure that is similar to the Internet architecture from general initial conditions and in a robust manner. It also suggests a plausible explanation why some protocols, such as TCP or IP, managed to survive much longer than most other protocols at the same layers. Furthermore, it suggests ways to design more competitive new protocols and more evolvable future Internet architectures.
Organizations engage in search whenever they perform nonroutine tasks, such as the definition and validation of a new strategy, the acquisition of new capabilities, or new product development. Previous work on search and organizational hierarchy has discovered that a hierarchy with a central decision maker at the top can speed up problem solving, but possibly at the cost of solution quality compared with results of a decentralized search. Our study uses a formal model and simulations to explore the effect of an organizational hierarchy on solution stability, solution quality, and search speed. Three insights arise on how a hierarchy can improve organizational search: (1) assigning a lead function that "anchors" a solution speeds up problem solving; (2) local solution choice should be delegated to the lowest level; and (3) structure matters little at the middle management level, but it matters at the front line; front-line groups should be kept small. These results highlight the importance for every organization of adapting its hierarchical structure to its search requirements.
We present an analysis of the statistical properties and growth of the free
on-line encyclopedia Wikipedia. By describing topics by vertices and hyperlinks
between them as edges, we can represent this encyclopedia as a directed graph.
The topological properties of this graph are in close analogy with that of the
World Wide Web, despite the very different growth mechanism. In particular we
measure a scale--invariant distribution of the in-- and out-- degree and we are
able to reproduce these features by means of a simple statistical model. As a
major consequence, Wikipedia growth can be described by local rules such as the
preferential attachment mechanism, though users can act globally on the
network.
Should researchers focus on designing new network architectures or improving the current Internet?
The exploration of the structural topology and the organizing principles of genome-based large-scale metabolic networks is essential for studying possible relations between structure and functionality of metabolic networks. Topological analysis of graph models has often been applied to study the structural characteristics of complex metabolic networks.
In this work, metabolic networks of 75 organisms were investigated from a topological point of view. Network decomposition of three microbes (Escherichia coli, Aeropyrum pernix and Saccharomyces cerevisiae) shows that almost all of the sub-networks exhibit a highly modularized bow-tie topological pattern similar to that of the global metabolic networks. Moreover, these small bow-ties are hierarchically nested into larger ones and collectively integrated into a large metabolic network, and important features of this modularity are not observed in the random shuffled network. In addition, such a bow-tie pattern appears to be present in certain chemically isolated functional modules and spatially separated modules including carbohydrate metabolism, cytosol and mitochondrion respectively.
The highly modularized bow-tie pattern is present at different levels and scales, and in different chemical and spatial modules of metabolic networks, which is likely the result of the evolutionary process rather than a random accident. Identification and analysis of such a pattern is helpful for understanding the design principles and facilitate the modelling of metabolic networks.
This first volume of the Metabolic Pathway Engineering Handbook provides an overview of metabolic pathway engineering with a look towards the future. It discusses cellular metabolism, including transport processes inside the cell and energy generating reactions, as well as rare metabolic conversions. This volume also explores balances and reaction models, the regulation of metabolic pathways, and genome scale and multiscale modeling tools. It also covers developing appropriate hosts for metabolic engineering including the use of Escherichia coli, yeast, Bacillus Subtilis, Streptomyces, filamentous fungi, and mammalian cells using cell culture. Christine Smolke, who recently developed a novel way to churn out large quantities of drugs from genetically modified brewer’s yeast, is regarded as one of the most brilliant new minds in biomedical engineering. In this handbook, she brings together pioneering scientists from dozens of disciplines to provide a complete record of accomplishment in metabolic pathway engineering. With a wealth of cutting edge research and analysis, this work also serves as an invaluable resource for those seeking to add their own contributions. Organized by topic, this 3000 page reference is available as two volumes that can be purchased individually or as a set. For information on the complete handbook visit Cat No. 3923.
Many real networks in nature and society share two generic properties: they are scale-free and they display a high degree of clustering. We show that these two features are the consequence of a hierarchical organization, implying that small groups of nodes organize in a hierarchical manner into increasingly large groups, while maintaining a scale-free topology. In hierarchical networks, the degree of clustering characterizing the different groups follows a strict scaling law, which can be used to identify the presence of a hierarchical organization in real networks. We find that several real networks, such as the Worldwideweb, actor network, the Internet at the domain level, and the semantic web obey this scaling law, indicating that hierarchy is a fundamental characteristic of many complex systems.
Robustness, the ability to maintain performance in the face of perturbations and uncertainty, is a long-recognized key property of living systems. Owing to intimate links to cellular complexity, however, its molecular and cellular basis has only recently begun to be understood. Theoretical approaches to complex engineered systems can provide guidelines for investigating cellular robustness because biology and engineering employ a common set of basic mechanisms in different combinations. Robustness may be a key to understanding cellular complexity, elucidating design principles, and fostering closer interactions between experimentation and theory.
Data represented as strings abounds in biology, linguistics, document mining, web search and many other fields. Such data often have a hierarchical structure, either because they were artificially designed and composed in a hierarchical manner or because there is an underlying evolutionary process that creates repeatedly more complex strings from simpler substrings. We propose a framework, referred to as Lexis, that produces an optimized hierarchical representation of a given set of "target" strings. The resulting hierarchy, "Lexis-DAG", shows how to construct each target through the concatenation of intermediate substrings, minimizing the total number of such concatenations or DAG edges. The Lexis optimization problem is related to the smallest grammar problem. After we prove its NP-hardness for two cost formulations, we propose an efficient greedy algorithm for the construction of Lexis-DAGs. We also consider the problem of identifying the set of intermediate nodes (substrings) that collectively form the "core" of a Lexis-DAG, which is important in the analysis of Lexis-DAGs. We show that the Lexis framework can be applied in diverse applications such as optimized synthesis of DNA fragments in genomic libraries, hierarchical structure discovery in protein sequences, dictionary-based text compression, and feature extraction from a set of documents.
Recent technological advances have enabled comprehensive determination of the molecular composition of living cells. The chemical interactions between many of these molecules are known, giving rise to genome-scale reconstructed biochemical reaction networks underlying cellular functions. Mathematical descriptions of the totality of these chemical interactions lead to genome-scale models that allow the computation of physiological functions. Reflecting these recent developments, this textbook explains how such quantitative and computable genotype-phenotype relationships are built using a genome-wide basis of information about the gene portfolio of a target organism. It describes how biological knowledge is assembled to reconstruct biochemical reaction networks, the formulation of computational models of biological functions, and how these models can be used to address key biological questions and enable predictive biology. Developed through extensive classroom use, the book is designed to provide students with a solid conceptual framework and an invaluable set of modeling tools and computational approaches.
Systems as diverse as genetic networks or the World Wide Web are best described as networks with complex topology. A common property of many large networks is that the vertex connectivities follow a scale-free power-law distribution. This feature was found to be a consequence of two generic mech-anisms: (i) networks expand continuously by the addition of new vertices, and (ii) new vertices attach preferentially to sites that are already well connected. A model based on these two ingredients reproduces the observed stationary scale-free distributions, which indicates that the development of large networks is governed by robust self-organizing phenomena that go beyond the particulars of the individual systems.
This chapter is based on the paper (Vitali et al 2011) and partly on (Vitali 2010). The copyright of some of the images belongs to PLoS ONE. Note that in order to make the chapter self-consistent and self-supporting, some redundancies with Chaps. 1 and 2 are taken into account.
Evolvability is an organism's capacity to generate heritable phenotypic variation. Metazoan evolution is marked by great morphological and physiological diversification, although the core genetic, cell biological, and developmental processes are largely conserved. Metazoan diversification has entailed the evolution of various regulatory processes controlling the time, place, and conditions of use of the conserved core processes. These regulatory processes, and certain of the core processes, have special properties relevant to evolutionary change. The properties of versatile protein elements, weak linkage, compartmentation, redundancy, and exploratory behavior reduce the interdependence of components and confer robustness and flexibility on processes during embryonic development and in adult physiology, They also confer evolvability on the organism by reducing constraints on change and allowing the accumulation of nonlethal variation. Evolvability may have been generally selected in the course of selection for robust, flexible processes suitable for complex development and physiology and specifically selected in lineages undergoing repeated radiations.
The scientific study of networks, including computer networks, social networks, and biological networks, has received an enormous amount of interest in the last few years. The rise of the Internet and the wide availability of inexpensive computers have made it possible to gather and analyze network data on a large scale, and the development of a variety of new theoretical tools has allowed us to extract new knowledge from many different kinds of networks. The study of networks is broadly interdisciplinary and important developments have occurred in many fields, including mathematics, physics, computer and information sciences, biology, and the social sciences. This book brings together the most important breakthroughs in each of these fields and presents them in a coherent fashion, highlighting the strong interconnections between work in different areas. Subjects covered include the measurement and structure of networks in many branches of science, methods for analyzing network data, including methods developed in physics, statistics, and sociology, the fundamentals of graph theory, computer algorithms, and spectral methods, mathematical models of networks, including random graph models and generative models, and theories of dynamical processes taking place on networks.
This paper discusses the organization of software that is inherently complex because of very many arbitrary details that must be precisely right for the software to be correct. We show how the software design technique known as information hiding, or abstraction, can be supplemented by a hierarchically structured document, which we call a module guide. The guide is intended to allow both designers and maintainers to identify easily the parts of the software that they must understand, without reading irrelevant details about other parts of the software. The paper includes an extract from a software module guide to illustrate our proposals.
We exploit recent advances in analysis of graph topology to better understand software evolution, and to construct predictors that facilitate software development and maintenance. Managing an evolving, collaborative software system is a complex and expensive process, which still cannot ensure software reliability. Emerging techniques in graph mining have revolutionized the modeling of many complex systems and processes. We show how we can use a graph-based characterization of a software system to capture its evolution and facilitate development, by helping us estimate bug severity, prioritize refactoring efforts, and predict defect-prone releases. Our work consists of three main thrusts. First, we construct graphs that capture software structure at two different levels: (a) the product, i.e., source code and module level, and (b) the process, i.e., developer collaboration level. We identify a set of graph metrics that capture interesting properties of these graphs. Second, we study the evolution of eleven open source programs, including Firefox, Eclipse, MySQL, over the lifespan of the programs, typically a decade or more. Third, we show how our graph metrics can be used to construct predictors for bug severity, high-maintenance software parts, and failure-prone releases. Our work strongly suggests that using graph topology analysis concepts can open many actionable avenues in software engineering research and practice.
A number of proposals have been advanced in recent years for the development of “general systems theory” which, abstracting from properties peculiar to physical, biological, or social systems, would be applicable to all of them. We might well feel that, while the goal is laudable, systems of such diverse kinds could hardly be expected to have any nontrivial properties in common. Metaphor and analogy can be helpful, or they can be misleading. All depends on whether the similarities the metaphor captures are significant or superficial.
The pages and hyperlinks of the World-Wide Web may be viewed as nodes and edges in a directed graph. This graph is a fascinating object of study: it has several hundred million nodes today, over a billion links, and appears to grow exponentially with time. There are many reasons -- mathematical, sociological, and commercial -- for studying the evolution of this graph. In this paper we begin by describing two algorithms that operate on the Web graph, addressing problems from Web search and automatic community discovery. We then report a number of measurements and properties of this graph that manifested themselves as we ran these algorithms on the Web. Finally, we observe that traditional random graph models do not explain these observations, and we propose a new family of random graph models. These models point to a rich new sub-field of the study of random graphs, and raise questions about the analysis of graph algorithms on the Web.
The value of depth-first search or "backtracking" as a technique for solving graph problems is illustrated by two examples. An algorithm for finding the biconnected components of an undirected graph and an improved version of an algorithm for finding the strongly connected components of a directed graph are presented. The space and time requirements of both algorithms are bounded by k1V + k2E + k3 for some constants k1, k2, and k3, where V is the number of vertices and E is the number of edges of the graph being examined.
Robustness is a ubiquitously observed property of biological systems. It is considered to be a fundamental feature of complex evolvable systems. It is attained by several underlying principles that are universal to both biological organisms and sophisticated engineering systems. Robustness facilitates evolvability and robust traits are often selected by evolution. Such a mutually beneficial process is made possible by specific architectural features observed in robust systems. But there are trade-offs between robustness, fragility, performance and resource demands, which explain system behaviour, including the patterns of failure. Insights into inherent properties of robust systems will provide us with a better understanding of complex diseases and a guiding principle for therapy design.
The study of the Web as a graph is not only fascinating in its own right, but also yields valuable insight into Web algorithms for crawling, searching and community discovery, and the sociological phenomena which characterize its evolution. We report on experiments on local and global properties of the Web graph using two AltaVista crawls each with over 200 million pages and 1.5 billion links. Our study indicates that the macroscopic structure of the Web is considerably more intricate than suggested by earlier experiments on a smaller scale.
We live in a dynamic economic and commercial world, surrounded by objects of remarkable complexity and power. In many industries, changes in products and technologies have brought with them new kinds of firms and forms of organization. We are discovering news ways of structuring work, of bringing buyers and sellers together, and of creating and using market information. Although our fast-moving economy often seems to be outside of our influence or control, human beings create the things that create the market forces. Devices, software programs, production processes, contracts, firms, and market are all the fruit of purposeful action: they are designed. Using the computer industry as an example, Carliss Y. Baldwin and Kim B. Clark develop a powerful theory of design and industrial evolution. They argue that the industry has experienced previously unimaginable levels of innovation and growth because it embraced the concept of modularity, building complex products from smaller subsystems that can be designed independently yet function together as a whole. Modularity freed designers to experiment with different approaches, as long as they obeyed the established design rules. Drawing upon the literatures of industrial organization, real options, and computer architecture, the authors provide insight into the forces of change that drive today's economy.
We construct the complete network of 30,288 majority opinions written by the U.S. Supreme Court and the cases they cite from 1754 to 2002 in the United States Reports. Data from this network demonstrates quantitatively the evolution of the norm of stare decisis in the 19th Century and a significant deviation from this norm by the activist Warren Court. We further describe a method for creating authority scores using the network data to identify the most important court precedents. This method yields rankings that conform closely to evaluations by legal experts, and even predicts which cases they will identify as important in the future. An analysis of these scores over time allows us to test several hypotheses about the rise and fall of precedent. We show that reversed cases tend to be much more important than other decisions, and the cases that overrule them quickly become and remain even more important as the reversed decisions decline. We also show that the Court is careful to ground overruling decisions in past precedent, and the care it exercises is increasing in the importance of the decision that is overruled. Finally, authority scores corroborate qualitative assessments of which issues and cases the Court prioritizes and how these change over time.
Intermediate-scale (or 'meso-scale') structures in networks have received
considerable attention, as the algorithmic detection of such structures makes
it possible to discover network features that are not apparent either at the
local scale of nodes and edges or at the global scale of summary statistics.
Numerous types of meso-scale structures can occur in networks, but
investigations of meso-scale network features have focused predominantly on the
identification and study of community structure. In this paper, we develop a
new method to investigate the meso-scale feature known as coreperiphery
structure, which consists of an identification of a network's nodes into a
densely connected core and a sparsely connected periphery. In contrast to
traditional network communities, the nodes in a core are also reasonably
well-connected to those in the periphery. Our new method of computing
core-periphery structure can identify multiple cores in a network and takes
different possible cores into account, thereby enabling a detailed description
of core-periphery structure. We illustrate the differences between our method
and existing methods for identifying which nodes belong to a core, and we use
it to classify the most important nodes using examples of friendship,
collaboration, transportation, and voting networks.
Link spam refers to attempts to promote the ranking of spammers' web sites by deceiving link-based ranking algo- rithms in search engines. Spammers often create densely connected link structure of sites so called \link farm". In this paper, we study the overall structure and distribution of link farms in a large-scale graph of the Japanese Web with 5.8 million sites and 283 million links. To examine the spam structure, we apply three graph algorithms to the web graph. First, the web graph is decomposed into strongly connected components (SCC). Beside the largest SCC (core) in the center of the web, we have observed that most of large com- ponents consist of link farms. Next, to extract spam sites in the core, we enumerate maximal cliques as seeds of link farms. Finally, we expand these link farms as a reliable spam seed set by a minimum cut technique that separates links among spam and non-spam sites. We found about 0.6 million spam sites in SCCs around the core, and extracted additional 8 thousand and 49 thousand sites as spams with high precision in the core by the maximal clique enumera- tion and by the minimum cut technique, respectively.
Are all film stars linked to Kevin Bacon? Why do the stock markets rise and fall sharply on the strength of a vague rumour? How does gossip spread so quickly? Are we all related through six degrees of separation? There is a growing awareness of the complex networks that pervade modern society. We see them in the rapid growth of the Internet, the ease of global communication, the swift spread of news and information, and in the way epidemics and financial crises develop with startling speed and intensity. This introductory book on the new science of networks takes an interdisciplinary approach, using economics, sociology, computing, information science and applied mathematics to address fundamental questions about the links that connect us, and the ways that our decisions can have consequences for others.
Modularity refers to the use of common units to create product variants. This paper aims at the development of models and solution approaches to the modularity problem for mechanical, electrical, and mixed process products (e.g., electromechanical products). To interpret various types of modularity, e.g., component-swapping, component-sharing and bus modularity, a matrix representation of the modularity problem is presented. The decomposition approach is used to determine modules for different products. The representation and solution approaches presented are illustrated with numerous examples. The paper presents a formal approach to modularity allowing for optimal forming of modules even in the situation of insufficient availability of information. The modules determined may be shared across different products