No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Contact Tracing and Epidemic Intervention via Deep Reinforcement Learning
Abstract
The recent outbreak of COVID-19 poses a serious threat to people’s lives. Epidemic control strategies have also caused damage to the economy by cutting off humans’ daily commute. In this paper, we develop an Individual-based Reinforcement Learning Epidemic Control Agent (IDRLECA) to search for smart epidemic control strategies that can simultaneously minimize infections and the cost of mobility intervention. IDRLECA first hires an infection probability model to calculate the current infection probability of each individual. Then, the infection probabilities together with individuals’ health status and movement information are fed to a novel GNN to estimate the spread of the virus through human contacts. The estimated risks are used to further support an RL agent to select individual-level epidemic-control actions. The training of IDRLECA is guided by a specially designed reward function considering both the cost of mobility intervention and the effectiveness of epidemic control. Moreover, we design a constraint for control-action selection that eases its difficulty and further improve exploring efficiency. Extensive experimental results demonstrate that IDRLECA can suppress infections at a very low level and retain more than of human mobility.
... However, they exhibit limitations in addressing the highly dynamic and complex nonlinear nature of epidemic spread. Reinforcement Learning (RL) (Rao, 2024;Song et al., 2020;Feng et al., 2023), as an adaptive intelligent optimization method, obtains the optimal strategies based on environmental feedback, particularly demonstrating significant advantages in dynamic multi-objective epidemic control. Nonetheless, RL in mobility restriction still faces challenges, in achieving fine-grained collaborative restrictions at the township-level administrative division (AD) and addressing the adaptability to cities with varying scales. ...
The necessity of achieving an effective balance between minimizing the losses associated with restricting human mobility and ensuring hospital capacity has gained significant attention in the aftermath of COVID-19. Reinforcement learning (RL)-based strategies for human mobility management have recently advanced in addressing the dynamic evolution of cities and epidemics; however, they still face challenges in achieving coordinated control at the township level and adapting to cities of varying scales. To address the above issues, we propose a multi-agent RL approach that achieves Pareto optimality in managing hospital capacity and human mobility (H2-MARL), applicable across cities of different scales. We first develop a township-level infection model with online-updatable parameters to simulate disease transmission and construct a city-wide dynamic spatiotemporal epidemic simulator. On this basis, H2-MARL is designed to treat each division as an agent, with a trade-off dual-objective reward function formulated and an experience replay buffer enriched with expert knowledge built. To evaluate the effectiveness of the model, we construct a township-level human mobility dataset containing over one billion records from four representative cities of varying scales. Extensive experiments demonstrate that H2-MARL has the optimal dual-objective trade-off capability, which can minimize hospital capacity strain while minimizing human mobility restriction loss. Meanwhile, the applicability of the proposed model to epidemic control in cities of varying scales is verified, which showcases its feasibility and versatility in practical applications.
... These features may contain information about location and identity that are engaged in the data stream (i.e. among sources and destinations) as well as information about the quality, presence direction, frequency, absence, and the length of stream's occurrences [21]. The Man-in-middle attack is a type of active attack that happens when a malicious entity interferes in an ongoing communication between two authenticated entities to conceal information, intercept, or even compromise the information that is being passed from one entity to another. ...
Graph neural networks (GNNs) have revolutionised the processing of information by facilitating the transmission of messages between graph nodes. Graph neural networks operate on graph‐structured data, which makes them suitable for a wide variety of computer vision problems, such as link prediction, node classification, and graph classification. The authors explore in depth the applications of GNNs in computer vision, including their design considerations, architectural challenges, applications, and implementation concerns. While conventional convolutional neural networks (CNNs) excel at object recognition in images and videos, GNN architectures offer a novel method for addressing various image and video comprehension challenges. A novel deep neural network‐based model for image and video analysis is proposed, which combines a neural network with fully connected layers on a graph. The proposed architecture extracts highly discriminative information from images and videos by leveraging the graph structure. Also, the investigation focuses on the enhancement of underlying connection network estimation using cutting‐edge graph learning algorithms. Experimental results on real‐world datasets demonstrate that the proposed GNN model is preferable to existing state‐of‐the‐art methods. It obtains a remarkable 96.63% accuracy on the ImageNet dataset, outperforming heuristic approaches, artificial neural networks, and conventional CNN techniques. From the results, we can see that GNNs are a potent instrument for graph data analysis and pave the way for machines to achieve human‐level visual intuition.
... For example, the authors in [65] develop Bayesian inference methods to estimate the probabilistic infection risk for epidemic control. IDRLECA [71] employs a GNN model with deep reinforcement learning to compute the current infection probability of each individual. The work in [62] proposes a risk-scoring algorithm and associates the risk score with the probability of infection for a contact tracing app. ...
The rapid global spread of the coronavirus disease (COVID-19) has severely impacted daily life worldwide. As potential solutions, various digital contact tracing (DCT) strategies have emerged to mitigate the virus’s spread while maintaining economic and social activities. The computational epidemiology problems of DCT often involve parameter optimization through learning processes, making it crucial to understand how to apply machine learning techniques for effective DCT optimization. While numerous research studies on DCT have emerged recently, most existing reviews primarily focus on DCT application design and implementation. This paper offers a comprehensive overview of privacy-preserving machine learning-based DCT in preparation for future pandemics. We propose a new taxonomy to classify existing DCT strategies into forward, backward, and proactive contact tracing. We then categorize several DCT apps developed during the COVID-19 pandemic based on their tracing strategies. Furthermore, we derive three research questions related to computational epidemiology for DCT and provide a detailed description of machine learning techniques to address these problems. We discuss the challenges of learning-based DCT and suggest potential solutions. Additionally, we include a case study demonstrating the review’s insights into the pandemic response. Finally, we summarize the study’s limitations and highlight promising future research directions in DCT.
Accurately predicting individual-level infection state is of great value since its essential role in reducing the damage of the epidemic. However, there exists an inescapable risk of privacy leakage in the fine-grained user mobility trajectories required by individual-level infection prediction. In this paper, we focus on developing a framework of privacy-preserving individual-level infection prediction based on federated learning (FL) and graph neural networks (GNN). We propose Falcon , a F ederated gr A ph L earning method for privacy-preserving individual-level infe C tion predicti ON . It utilizes a novel hypergraph structure with spatio-temporal hyperedges to describe the complex interactions between individuals and locations in the contagion process. By organically combining the FL framework with hypergraph neural networks, the information propagation process of the graph machine learning is able to be divided into two stages distributed on the server and the clients, respectively, so as to effectively protect user privacy while transmitting high-level information. Furthermore, it elaborately designs a differential privacy perturbation mechanism as well as a plausible pseudo location generation approach to preserve user privacy in the graph structure. Besides, it introduces a cooperative coupling mechanism between the individual-level prediction model and an additional region-level model to mitigate the detrimental impacts caused by the injected obfuscation mechanisms. Extensive experimental results show that our methodology outperforms state-of-the-art algorithms and is able to protect user privacy against actual privacy attacks. Our code and datasets are available at the link: https://github.com/wjfu99/FL-epidemic.
Combating an epidemic entails finding a plan that describes when and how to apply different interventions, such as mask-wearing mandates, vaccinations, school or workplace closures. An optimal plan will curb an epidemic with minimal loss of life, disease burden, and economic cost. Finding an optimal plan is an intractable computational problem in realistic settings. Policy-makers, however, would greatly benefit from tools that can efficiently search for plans that minimize disease and economic costs especially when considering multiple possible interventions over a continuous and complex action space given a continuous and equally complex state space. We formulate this problem as a Markov decision process. Our formulation is unique in its ability to represent multiple continuous interventions over any disease model defined by ordinary differential equations. We illustrate how to effectively apply state-of-the-art actor-critic reinforcement learning algorithms (PPO and SAC) to search for plans that minimize overall costs. We empirically evaluate the learning performance of these algorithms and compare their performance to hand-crafted baselines that mimic plans constructed by policy-makers. Our method outperforms baselines. Our work confirms the viability of a computational approach to support policy-makers
(1) Background: The implementation of effective control measures in a timely fashion is crucial to control the epidemic outbreak of COVID-19. In this study, we aimed to analyze the control measures implemented during the COVID-19 outbreak, as well as evaluating the responses and outcomes at different phases for epidemic control in Taiwan. (2) Methods: This case study reviewed responses to COVID-19 and the effectiveness of a range of control measures implemented for epidemic control in Taiwan and assessed all laboratory-confirmed cases between 11 January until 20 December 2020, inclusive of these dates. The confirmation of COVID-19 infection was defined as the positive result of a reverse-transcriptase–polymerase-chain-reaction test taken from a nasopharyngeal swab. Test results were reported by the Taiwan Centers for Disease Control. The incidence rate, mortality rate, and testing rate were compiled, and the risk ratio was provided to gain insights into the effectiveness of prevention measures. (3) Results and Discussion: This study presents retrospective data on the COVID-19 incidence rate in Taiwan, combined with the vital preventive control measures, in a timeline of the early stage of the epidemic that occurred in Taiwan. The implementation of multiple strategy control measures and the assistance of technologies to control the COVID-19 epidemic in Taiwan led to a relatively slower trend in the outbreak compared to the neighboring countries. In Taiwan, 766 confirmed patients were included, comprised of 88.1% imported cases and 7.2% local transmission cases, within the studied period. The incidence rate of COVID-19 in Taiwan during the studied period was 32 per million people, with a mortality rate of 0.3 per million people. Our analysis showed a significantly raised incidence risk ratio in the countries of interest in comparison to Taiwan during the study period; in the range of 1.9 to 947.5. The outbreak was brought under control through epidemic policies and hospital strategies implemented by the Taiwan Government. (4) Conclusion: Taiwan’s preventive strategies resulted in a drastically lower risk for Taiwan nationals of contracting COVID-19 when new pharmaceutical drug or vaccines were not yet available. The preventive strategies employed by Taiwan could serve as a guide and reference for future epidemic control strategies.
In real-world problems, heterogeneous entities are often related to each other through multiple interactions, forming a Heterogeneous Interaction Graph (HIG). While modeling HIGs to deal with fundamental tasks, graph neural networks present an attractive opportunity that can make full use of the heterogeneity and rich semantic information by aggregating and propagating information from different types of neighborhoods. However, learning on such complex graphs, often with millions or billions of nodes, edges, and various attributes, could suffer from expensive time cost and high memory consumption. In this article, we attempt to accelerate representation learning on large-scale HIGs by adopting the importance sampling of heterogeneous neighborhoods in a batch-wise manner, which naturally fits with most batch-based optimizations. Distinct from traditional homogeneous strategies neglecting semantic types of nodes and edges, to handle the rich heterogeneous semantics within HIGs, we devise both type-dependent and type-fusion samplers where the former respectively samples neighborhoods of each type and the latter jointly samples from candidates of all types. Furthermore, to overcome the imbalance between the down-sampled and the original information, we respectively propose heterogeneous estimators including the self-normalized and the adaptive estimators to improve the robustness of our sampling strategies.
Finally, we evaluate the performance of our models for node classification and link prediction on five real-world datasets, respectively. The empirical results demonstrate that our approach performs significantly better than other state-of-the-art alternatives, and is able to reduce the number of edges in computation by up to 93%, the memory cost by up to 92% and the time cost by up to 86%.
Due to complexity of social phenomena, it is a big challenge to predict the curves of epidemics that spread via social contacts and to control such epidemics. Misguided policies to mitigate epidemics may result in catastrophic consequences such as financial crisis, massive unemployment, and the surge of the number of critically ill patients exceeding the capacity of hospitals. In particular, under/overestimation of efficacy of interventions can mislead policymakers about perception of evolving situations. To avoid such pitfalls, we propose Expert-in-the-Loop (EITL) prescriptive analytics using mobility intervention for epidemics. Rather than employing a purely data-driven approach, the key advantage of our approach is to leverage experts' best knowledge in estimating disease spreading and the efficacy of interventions which allows us to efficiently narrow down factors and the scope of combinatorial possible worlds. We introduce our experience to develop Expert-in-the-Loop simulations during the Challenge on Mobility Intervention for Epidemics. We demonstrate that misconceptions about the causality can be corrected in the iterations of consulting with experts, developing simulations, and experimentation.
Graph representation learning resurges as a trending research subject owing to the widespread use of deep learning for Euclidean data, which inspire various creative designs of neural networks in the non-Euclidean domain, particularly graphs. With the success of these graph neural networks (GNN) in the static setting, we approach further practical scenarios where the graph dynamically evolves. Existing approaches typically resort to node embeddings and use a recurrent neural network (RNN, broadly speaking) to regulate the embeddings and learn the temporal dynamics. These methods require the knowledge of a node in the full time span (including both training and testing) and are less applicable to the frequent change of the node set. In some extreme scenarios, the node sets at different time steps may completely differ. To resolve this challenge, we propose EvolveGCN, which adapts the graph convolutional network (GCN) model along the temporal dimension without resorting to node embeddings. The proposed approach captures the dynamism of the graph sequence through using an RNN to evolve the GCN parameters. Two architectures are considered for the parameter evolution. We evaluate the proposed approach on tasks including link prediction, edge classification, and node classification. The experimental results indicate a generally higher performance of EvolveGCN compared with related approaches. The code is available at https://github.com/IBM/EvolveGCN.
Significance
This paper presents a large-scale analysis of the impact of lockdown measures introduced in response to the spread of novel coronavirus disease 2019 (COVID-19) on socioeconomic conditions of Italian citizens. We leverage a massive near–real-time dataset of human mobility and we model mobility restrictions as an exogenous shock to the economy, similar to a natural disaster. We find that lockdown measures have a twofold effect: First, their impact on mobility is stronger in municipalities with higher fiscal capacity; second, they induce a segregation effect: mobility contraction is stronger in municipalities where inequality is higher and income per capita is lower. We highlight the necessity of fiscal measures that account for these effects, targeting poverty and inequality mitigation.
Background:
The coronavirus disease 2019 (COVID-19) outbreak originating in Wuhan, Hubei province, China, coincided with chunyun, the period of mass migration for the annual Spring Festival. To contain its spread, China adopted unprecedented nationwide interventions on January 23 2020. These policies included large-scale quarantine, strict controls on travel and extensive monitoring of suspected cases. However, it is unknown whether these policies have had an impact on the epidemic. We sought to show how these control measures impacted the containment of the epidemic.
Methods:
We integrated population migration data before and after January 23 and most updated COVID-19 epidemiological data into the Susceptible-Exposed-Infectious-Removed (SEIR) model to derive the epidemic curve. We also used an artificial intelligence (AI) approach, trained on the 2003 SARS data, to predict the epidemic.
Results:
We found that the epidemic of China should peak by late February, showing gradual decline by end of April. A five-day delay in implementation would have increased epidemic size in mainland China three-fold. Lifting the Hubei quarantine would lead to a second epidemic peak in Hubei province in mid-March and extend the epidemic to late April, a result corroborated by the machine learning prediction.
Conclusions:
Our dynamic SEIR model was effective in predicting the COVID-19 epidemic peaks and sizes. The implementation of control measures on January 23 2020 was indispensable in reducing the eventual COVID-19 epidemic size.
Accurate and real-time traffic forecasting plays an important role in the intelligent traffic system and is of great significance for urban traffic planning, traffic management, and traffic control. However, traffic forecasting has always been considered an "open" scientific issue, owing to the constraints of urban road network topological structure and the law of dynamic change with time. To capture the spatial and temporal dependences simultaneously, we propose a novel neural network-based traffic forecasting method, the temporal graph convolutional network (T-GCN) model, which is combined with the graph convolutional network (GCN) and the gated recurrent unit (GRU). Specifically, the GCN is used to learn complex topological structures for capturing spatial dependence and the gated recurrent unit is used to learn dynamic changes of traffic data for capturing temporal dependence. Then, the T-GCN model is employed to traffic forecasting based on the urban road network. Experiments demonstrate that our T-GCN model can obtain the spatio-temporal correlation from traffic data and the predictions outperform state-of-art baselines on real-world traffic datasets. Our tensorflow implementation of the T-GCN is available at https://github.com/lehaifeng/T-GCN.
We introduce the Contextual Graph Markov Model, an approach combining ideas from gen-erative models and neural networks for the processing of graph data. It founds on a constructive methodology to build a deep architecture comprising layers of probabilistic models that learn to encode the structured information in an incre-mental fashion. Context is diffused in an efficient and scalable way across the graph vertexes and edges. The resulting graph encoding is used in combination with discriminative models to address structure classification benchmarks.
We present, GEM, the first heterogeneous graph neural network approach for detecting malicious accounts at Alipay, one of the world's leading mobile cashless payment platform. Our approach, inspired from a connected subgraph approach, adaptively learns discriminative embeddings from heterogeneous account-device graphs based on two fundamental weaknesses of attackers, i.e. device aggregation and activity aggregation. For the heterogeneous graph consists of various types of nodes, we propose an attention mechanism to learn the importance of different types of nodes, while using the sum operator for modeling the aggregation patterns of nodes in each type. Experiments show that our approaches consistently perform promising results compared with competitive methods over time.
Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.
Traditional methods for link prediction can be categorized into three main types: graph structure feature-based, latent feature-based, and explicit feature-based. Graph structure feature methods leverage some handcrafted node proximity scores, e.g., common neighbors, to estimate the likelihood of links. Latent feature methods rely on factorizing networks' matrix representations to learn an embedding for each node. Explicit feature methods train a machine learning model on two nodes' explicit attributes. Each of the three types of methods has its unique merits. In this paper, we propose SEAL (learning from Subgraphs, Embeddings, and Attributes for Link prediction), a new framework for link prediction which combines the power of all the three types into a single graph neural network (GNN). GNN is a new type of neural network which directly accepts graphs as input and outputs their labels. In SEAL, the input to the GNN is a local subgraph around each target link. We prove theoretically that our local subgraphs also reserve a great deal of high-order graph structure features related to link existence. Another key feature is that our GNN can naturally incorporate latent features and explicit features. It is achieved by concatenating node embeddings (latent features) and node attributes (explicit features) in the node information matrix for each subgraph, thus combining the three types of features to enhance GNN learning. Through extensive experiments, SEAL shows unprecedentedly strong performance against a wide range of baseline methods, including various link prediction heuristics and network embedding methods.
Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.
The dynamics of contact networks and epidemics of infectious diseases often
occur on comparable time scales. Ignoring one of these time scales may provide
an incomplete understanding of the population dynamics of the infection
process. We develop an individual-based approximation for the
susceptible-infected-recovered epidemic model applicable to arbitrary dynamic
networks. Our framework provides, at the individual-level, the probability flow
over time associated with the infection dynamics. This computationally
efficient framework discards the correlation between the states of different
nodes, yet provides accurate results in approximating direct numerical
simulations. It naturally captures the temporal heterogeneities and
correlations of contact sequences, fundamental ingredients regulating the
timing and size of an epidemic outbreak. Using real-life data, we show that the
static network model overestimates the reproduction number but underestimates
the infection potential of super-spreading individuals. The high accuracy of
our approximation further allows us to detect the index individual of an
epidemic outbreak.
Large scale realistic epidemic simulations have recently become an increasingly important application of high-performance computing. We propose a parallel algorithm, EpiFast, based on a novel interpretation of the stochastic disease propagation in a contact network. We implement it using a master-slave computation model which allows scalability on distributed memory systems. EpiFast runs extremely fast for realistic simulations that involve: (i) large populations consisting of millions of individuals and their heterogeneous details, (ii) dynamic interactions between the disease propagation, the individual behaviors, and the exogenous interventions, as well as (iii) large number of replicated runs necessary for statistically sound estimates about the stochastic epidemic evolution. We find that EpiFast runs several magnitude faster than another comparable simulation tool while delivering similar results. EpiFast has been tested on commodity clusters as well as SGI shared memory machines. For a fixed experiment, if given more computing resources, it scales automatically and runs faster. Finally, EpiFast has been used as the major simulation engine in real studies with rather sophisticated settings to evaluate various dynamic interventions and to provide decision support for public health policy makers.
The outbreak of influenza A pandemic (H1N1) 2009 prompted many countries in Asia, previously strongly affected by severe acute respiratory syndrome (SARS), to respond with stringent measures, particularly in preventing outbreaks in hospitals. We studied actual direct costs and cost-effectiveness of different response measures from a hospital perspective in tertiary hospitals in Singapore by simulating outbreaks of SARS, pandemic (H1N1) 2009, and 1918 Spanish influenza. Protection measures targeting only infected patients yielded lowest incremental cost/death averted of 23,000 (US dollars) for pandemic (H1N1) 2009. Enforced protection in high-risk areas (Yellow Alert) and full protection throughout the hospital (Orange Alert) averted deaths but came at an incremental cost of up to $2.5 million/death averted. SARS and Spanish influenza favored more stringent measures. High case-fatality rates, virulence, and high proportion of atypical manifestations impacted cost-effectiveness the most. A calibrated approach in accordance with viral characteristics and community risks may help refine responses to future epidemics.
In the absence of other evidence, modelling has been used extensively to help policy makers plan for a potential future influenza pandemic.
We have constructed an individual based model of a small community in the developed world with detail down to exact household structure obtained from census collection datasets and precise simulation of household demographics, movement within the community and individual contact patterns. We modelled the spread of pandemic influenza in this community and the effect on daily and final attack rates of four social distancing measures: school closure, increased case isolation, workplace non-attendance and community contact reduction. We compared the modelled results of final attack rates in the absence of any interventions and the effect of school closure as a single intervention with other published individual based models of pandemic influenza in the developed world.
We showed that published individual based models estimate similar final attack rates over a range of values for R(0) in a pandemic where no interventions have been implemented; that multiple social distancing measures applied early and continuously can be very effective in interrupting transmission of the pandemic virus for R(0) values up to 2.5; and that different conclusions reached on the simulated benefit of school closure in published models appear to result from differences in assumptions about the timing and duration of school closure and flow-on effects on other social contacts resulting from school closure.
Models of the spread and control of pandemic influenza have the potential to assist policy makers with decisions about which control strategies to adopt. However, attention needs to be given by policy makers to the assumptions underpinning both the models and the control strategies examined.
Most mathematical models for the spread of disease use differential equations based on uniform mixing assumptions or ad hoc models for the contact process. Here we explore the use of dynamic bipartite graphs to model the physical contact patterns that result from movements of individuals between specific locations. The graphs are generated by large-scale individual-based urban traffic simulations built on actual census, land-use and population-mobility data. We find that the contact network among people is a strongly connected small-world-like graph with a well-defined scale for the degree distribution. However, the locations graph is scale-free, which allows highly efficient outbreak detection by placing sensors in the hubs of the locations network. Within this large-scale simulation framework, we then analyse the relative merits of several proposed mitigation strategies for smallpox spread. Our results suggest that outbreaks can be contained by a strategy of targeted vaccination combined with early detection without resorting to mass vaccination of a population.
There has been intense debate about lockdown policies in the context of Covid-19 for limiting damage both to health and to the economy. We present an AI-driven approach for generating optimal lockdown policies that control the spread of the disease while balancing both health and economic costs. Furthermore, the proposed reinforcement learning approach automatically learns those policies, as a function of disease and population parameters. The approach accounts for imperfect lockdowns, can be used to explore a range of policies using tunable parameters, and can be easily extended to fine-grained lockdown strictness. The control approach can be used with any compatible disease and network simulation models.
Traffic flow prediction is crucial for public safety and traffic management, and remains a big challenge because of many complicated factors, e.g., multiple spatio-temporal dependencies, holidays, and weather. Some work leveraged 2D convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) to explore spatial relations and temporal relations, respectively, which outperformed the classical approaches. However, it is hard for these work to model spatio-temporal relations jointly. To tackle this, some studies utilized LSTMs to connect high-level layers of CNNs, but left the spatio-temporal correlations not fully exploited in low-level layers. In this work, we propose novel spatio-temporal CNNs to extract spatio-temporal features simultaneously from low-level to high-level layers, and propose a novel gated scheme to control the spatio-temporal features that should be propagated through the hierarchy of layers. Based on these, we propose an end-to-end framework, multiple gated spatio-temporal CNNs (MGSTC), for citywide traffic flow prediction. MGSTC can explore multiple spatio-temporal dependencies through multiple gated spatio-temporal CNN branches, and combine the spatio-temporal features with external factors dynamically. Extensive experiments on two real traffic datasets demonstrates that MGSTC outperforms other state-of-the-art baselines.
We introduce an individual-based model with dynamical equations for susceptible-infected-susceptible (SIS) epidemics on clustered networks. Linking the mean-field and quenched mean-field models, a general method for deriving a cluster approximation for three-node loops in complex networks is proposed. The underlying epidemic threshold condition is derived by using the quasi-static approximation. Our method thus extends the pair quenched mean-field (pQMF) approach for SIS disease spreading in unclustered networks to the scenario of epidemic outbreaks in clustered systems with abundant transitive relationships.We found that clustering can significantly alter the epidemic threshold, depending nontrivially on topological details of the underlying population structure. The validity of our method is verified through the existence of bounded solutions to the clustered pQMF model equations, and is further attested via stochastic simulations on homogeneous small-world artificial networks and growing scale-free synthetic networks with tunable clustering, as well as on real-world complex networked systems. Our method has vital implications for the future policy development and implementation of intervention measures in highly clustered networks, especially in the early stages of an epidemic in which clustering can decisively alter the growth of a contagious outbreak.
Public health measures were decisive in controlling the SARS epidemic in 2003. Isolation is the separation of ill persons from non-infected persons. Quarantine is movement restriction, often with fever surveillance, of contacts when it is not evident whether they have been infected but are not yet symptomatic or have not been infected. Community containment includes measures that range from increasing social distancing to community-wide quarantine. Whether these measures will be sufficient to control 2019-nCoV depends on addressing some unanswered questions.
Graph-structured data arise naturally in many different application domains. By representing data as graphs, we can capture entities (i.e., nodes) as well as their relationships (i.e., edges) with each other. Many useful insights can be derived from graph-structured data as demonstrated by an ever-growing body of work focused on graph mining. However, in the real-world, graphs can be both large—with many complex patterns—and noisy, which can pose a problem for effective graph mining. An effective way to deal with this issue is to incorporate “attention” into graph mining solutions. An attention mechanism allows a method to focus on task-relevant parts of the graph, helping it to make better decisions. In this work, we conduct a comprehensive and focused survey of the literature on the emerging field of graph attention models. We introduce three intuitive taxonomies to group existing work. These are based on problem setting (type of input and output), the type of attention mechanism used, and the task (e.g., graph classification, link prediction). We motivate our taxonomies through detailed examples and use each to survey competing approaches from a unique standpoint. Finally, we highlight several challenges in the area and discuss promising directions for future work.
Region-level demand forecasting is an essential task in ridehailing services. Accurate ride-hailing demand forecasting can guide vehicle dispatching, improve vehicle utilization, reduce the wait-time, and mitigate traffic congestion. This task is challenging due to the complicated spatiotemporal dependencies among regions. Existing approaches mainly focus on modeling the Euclidean correlations among spatially adjacent regions while we observe that non-Euclidean pair-wise correlations among possibly distant regions are also critical for accurate forecasting. In this paper, we propose the spatiotemporal multi-graph convolution network (ST-MGCN), a novel deep learning model for ride-hailing demand forecasting. We first encode the non-Euclidean pair-wise correlations among regions into multiple graphs and then explicitly model these correlations using multi-graph convolution. To utilize the global contextual information in modeling the temporal correlation, we further propose contextual gated recurrent neural network which augments recurrent neural network with a contextual-aware gating mechanism to re-weights different historical observations. We evaluate the proposed model on two real-world large scale ride-hailing demand datasets and observe consistent improvement of more than 10% over stateof-the-art baselines.
The strength of long-range ties
It seems reasonable that we would have the closest, strongest ties with people in our immediate social network and that the ties between networks would be weaker. However, Park et al. discovered strong ties that spanned extreme network (not geographic) distances in 11 culturally diverse population-scale networks on four continents—encompassing 56 million Twitter users and 58 million mobile phone subscribers. Although they are fairly rare, strong ties between networks could be important for the spreading of ideas or disease.
Science , this issue p. 1410
Online social networks offer the opportunity to collect a huge amount of valuable information about billions of users. The analysis of this data by service providers and unintended third parties are posing serious treats to user privacy. In particular, recent work has shown that users participating in more than one online social network can be identified based only on the structure of their links to other users. An effective tool to de-anonymize social network users is represented by graph matching algorithms. Indeed, by exploiting a sufficiently large set of seed nodes, a percolation process can correctly match almost all nodes across the different social networks. In this article, we show the crucial role of clustering, which is a relevant feature of social network graphs (and many other systems). Clustering has both the effect of making matching algorithms more prone to errors, and the potential to greatly reduce the number of seeds needed to trigger percolation. We show these facts by considering a fairly general class of random geometric graphs with variable clustering level. We assume that seeds can be identified in particular sub-regions of the network graph, while no a priori knowledge about the location of the other nodes is required. Under these conditions, we show how clever algorithms can achieve surprisingly good performance while limiting the number of matching errors.
Graphs are popularly used to represent objects with shared dependency relationships. To date, all existing graph clustering algorithms consider each node as a single attribute or a set of independent attributes, without realizing that content inside each node may also have complex structures. In this article, we formulate a new networked graph clustering task where a network contains a set of inter-connected (or networked) super-nodes, each of which is a single-attribute graph. The new super-node representation is applicable to many real-world applications, such as a citation network where each node denotes a paper whose content can be described as a graph, and citation relationships between papers form a networked graph (i.e., a supergraph). Networked graph clustering aims to find similar node groups, each of which contains nodes with similar content and structure information. The main challenge is to properly calculate the similarity between super-nodes for clustering. To solve the problem, we propose to characterize node similarity by integrating structure and content information of each super-node. To measure node content similarity, we use cosine distance by considering overlapped attributes between two super-nodes. To measure structure similarity, we propose an Attributed Random Walk Kernel (ARWK) to calculate the similarity between super-nodes. Detailed node content analysis is also included to build relationships between super-nodes with shared internal structure information, so the structure similarity can be calculated in a precise way. By integrating the structure similarity and content similarity as one matrix, the spectral clustering is used to achieve networked graph clustering. Our method enjoys sound theoretical properties, including bounded similarities and better structure similarity assessment than traditional graph clustering methods. Experiments on real-world applications demonstrate that our method significantly outperforms baseline approaches.
We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.
Given a graph, like a social/computer network or the blogosphere, in which an infection (or meme or virus) has been spreading for some time, how to select the k best nodes for immunization/quarantining immediately? Most previous works for controlling propagation (say via immunization) have concentrated on developing strategies for vaccination preemptively before the start of the epidemic. While very useful to provide insights in to which baseline policies can best control an infection, they may not be ideal to make real-time decisions as the infection is progressing.
In this paper, we study how to immunize healthy nodes, in the presence of already infected nodes. Efficient algorithms for such a problem can help public-health experts make more informed choices, tailoring their decisions to the actual distribution of the epidemic on the ground. First we formulate the Data-Aware Vaccination problem, and prove it is NP-hard and also that it is hard to approximate. Secondly, we propose three effective polynomial-time heuristics DAVA, DAVA-prune and DAVA-fast, of varying degrees of efficiency and performance. Finally, we also demonstrate the scalability and effectiveness of our algorithms through extensive experiments on multiple real networks including large epidemiology datasets (containing millions of interactions). Our algorithms show substantial gains of up to ten times more healthy nodes at the end against many other intuitive and nontrivial competitors.
Here we present the Global Epidemic and Mobility (GLEaM) model that integrates sociodemographic and population mobility data in a spatially structured stochastic disease approach to simulate the spread of epidemics at the worldwide scale. We discuss the flexible structure of the model that is open to the inclusion of different disease structures and local intervention policies. This makes GLEaM suitable for the computational modeling and anticipation of the spatio-temporal patterns of global epidemic spreading, the understanding of historical epidemics, the assessment of the role of human mobility in shaping global epidemics, and the analysis of mitigation and containment scenarios.
Networks of coupled dynamical systems have been used to model biological oscillators, Josephson junction arrays, excitable media, neural networks, spatial games, genetic control networks and many other self-organizing systems. Ordinarily, the connection topology is assumed to be either completely regular or completely random. But many biological, technological and social networks lie somewhere between these two extremes. Here we explore simple models of networks that can be tuned through this middle ground: regular networks 'rewired' to introduce increasing amounts of disorder. We find that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs. We call them 'small-world' networks, by analogy with the small-world phenomenon (popularly known as six degrees of separation. The neural network of the worm Caenorhabditis elegans, the power grid of the western United States, and the collaboration graph of film actors are shown to be small-world networks. Models of dynamical systems with small-world coupling display enhanced signal-propagation speed, computational power, and synchronizability. In particular, infectious diseases spread more easily in small-world networks than in regular lattices.
Inferring networks of diffusion and influence
- Manuel Gomez-Rodriguez
- Jure Leskovec
- Andreas Krause
William Ogilvy Kermack and Anderson G McKendrick. 1927. A contribution to the mathematical theory of epidemics
- Ogilvy William
- Anderson G Kermack
- Mckendrick
- Kermack William Ogilvy
Balancing Precision and Recall for Cost-effective Epidemic Containment
- Stefano Giovanni Rizzo
- Rizzo Stefano Giovanni
Dota 2 with large scale deep reinforcement learning
- Christopher Berner
- Greg Brockman
- Brooke Chan
- Vicki Cheung
- Przemysław Dębiak
- Christy Dennison
- David Farhi
- Quirin Fischer
- Shariq Hashme
- Chris Hesse
- Berner Christopher
Yinzhao Dong Chao Yu and Lijun Xia. 2020. Hierarchical Reinforcement Learning for Epidemics Intervention
- Yinzhao Dong
- Chao Yu
- Lijun Xia
Toby Phillips, and Samuel Webster. 2020. Variation in government responses to COVID-19. Blavatnik school of government working paper
- Thomas Hale
- Anna Petherick
- Toby Phillips
- Samuel Webster
- Hale Thomas
Cost-effectiveness analysis of hospital infection control response to an epidemic respiratory virus threat
- Yock Young Dan
- A Paul
- Joe Tambyah
- Jeremy Sim
- Li Yang Lim
- Wai Leng Hsu
- Chow
- A Dale
- Yue Sie Fisher
- Khek Yu Wong
- Ho
- Dan Yock Young
Fabian Perez-Sanjines, Niel Hens, Philippe Lemey, and Ann Nowé. 2020. Deep reinforcement learning for large-scale epidemic control
- J K Pieter
- Arno Libin
- Timothy Moonens
- Fabian Verstraeten
- Perez-Sanjines
- Libin JK
Mastering complex control in MOBA games with deep reinforcement learning
- Deheng Ye
- Zhao Liu
- Mingfei Sun
- Bei Shi
- Peilin Zhao
- Hao Wu
- Hongsheng Yu
- Shaojie Yang
- Xipeng Wu
- Qingwei Guo
- Ye Deheng
Hierarchical reinforcement learning for epidemics intervention
- Yinzhao Dong
- Chao Yu
- Lijun Xia
Niel Hens, Philippe Lemey, and Ann Nowé. 2020. Deep reinforcement learning for large-scale epidemic control
- J K Pieter
- Arno Libin
- Timothy Moonens
- Fabian Verstraeten
- Perez-Sanjines
- Libin Pieter JK
Variation in government responses to COVID-19
- Thomas Hale
- Anna Petherick
- Toby Phillips
- Samuel Webster
- Hale Thomas