Andreas Loukas

Andreas Loukas
Roche

About

64
Publications
5,585
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,196
Citations
Introduction
Andreas Loukas is a research scientist at École Polytechnique Fédérale de Lausanne. He is interested in learning problems involving graphs.
Additional affiliations
September 2018 - July 2020
École Polytechnique Fédérale de Lausanne
Position
  • Fellow
April 2016 - September 2018
École Polytechnique Fédérale de Lausanne
Position
  • PostDoc Position
April 2015 - April 2016
Technische Universität Berlin
Position
  • PostDoc Position

Publications

Publications (64)
Preprint
Integrating functions on discrete domains into neural networks is key to developing their capability to reason about discrete objects. But, discrete domains are (1) not naturally amenable to gradient-based optimization, and (2) incompatible with deep learning architectures that rely on representations in high-dimensional vector spaces. In this work...
Preprint
Full-text available
We approach the graph generation problem from a spectral perspective by first generating the dominant parts of the graph Laplacian spectrum and then building a graph matching these eigenvalues and eigenvectors. Spectral conditioning allows for direct modeling of the global and local graph structure and helps to overcome the expressivity and mode co...
Article
Proteins are typically represented by discrete atomic coordinates providing an accessible framework to describe different conformations. However, in some fields proteins are more accurately represented as near-continuous surfaces, as these are imprinted with geometric (shape) and chemical (electrostatics) features of the underlying protein structur...
Preprint
State-of-the-art approaches to reasoning and question answering over knowledge graphs (KGs) usually scale with the number of edges and can only be applied effectively on small instance-dependent subgraphs. In this paper, we address this issue by showing that multi-hop and more complex logical reasoning can be accomplished separately without losing...
Preprint
Can we use machine learning to compress graph data? The absence of ordering in graphs poses a significant challenge to conventional compression algorithms, limiting their attainable gains as well as their ability to discover relevant patterns. On the other hand, most graph compression approaches rely on domain-dependent handcrafted representations...
Preprint
Full-text available
Proteins are typically represented by discrete atomic coordinates providing an accessible framework to describe different conformations. However, in some fields proteins are more accurately represented as near-continuous surfaces, as these are imprinted with geometric (shape) and chemical (electrostatics) features of the underlying protein structur...
Preprint
This work explores the hypothesis that the complexity of the function a deep neural network (NN) is learning can be deduced by how fast its weights change during training. Our analysis provides evidence for this supposition by relating the network's distribution of Lipschitz constants (i.e., the norm of the gradient at different regions of the inpu...
Preprint
Full-text available
Attention-based architectures have become ubiquitous in machine learning, yet our understanding of the reasons for their effectiveness remains limited. This work proposes a new way to understand self-attention networks: we show that their output can be decomposed into a sum of smaller terms, each involving the operation of a sequence of attention h...
Preprint
Attention layers are widely used in natural language processing (NLP) and are beginning to influence computer vision architectures. However, they suffer from over-parameterization. For instance, it was shown that the majority of attention heads could be pruned without impacting accuracy. This work aims to enhance current understanding on how multip...
Preprint
Message-passing has proved to be an effective way to design graph neural networks, as it is able to leverage both permutation equivariance and an inductive bias towards learning local structures to achieve good generalization. However, current message-passing architectures have a limited representation power and fail to learn basic topological prop...
Preprint
Combinatorial optimization problems are notoriously challenging for neural networks, especially in the absence of labeled instances. This work proposes an unsupervised learning framework for CO problems on graphs that can provide integral solutions of certified quality. Inspired by Erdos' probabilistic method, we use a neural network to parametrize...
Preprint
A hallmark of graph neural networks is their ability to distinguish the isomorphism class of their inputs. This study derives the first hardness results for graph isomorphism in the message-passing model (MPNN). MPNN encompasses the majority of graph neural networks used today and is universal in the limit when nodes are given unique features. The...
Chapter
Spectral clustering refers to a family of well-known unsupervised learning algorithms. Rather than attempting to cluster points in their native domain, one constructs a (usually sparse) similarity graph and computes the principal eigenvectors of its Laplacian. The eigenvectors are then interpreted as transformed points and fed into a k-means cluste...
Conference Paper
This paper studies the expressive power of graph neural networks falling within the message-passing framework (GNNmp). Two results are presented. First, GNN_mp are shown to be Turing universal under sufficient conditions on their depth, width, node attributes, and layer expressiveness. Second, it is discovered that GNNmp can lose a significant port...
Preprint
Full-text available
Recent trends of incorporating attention mechanisms in vision have led researchers to reconsider the supremacy of convolutional layers as a primary building block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed that attention can completely replace convolution and achieve state-of-the-art performance on vis...
Article
Full-text available
The goal of this paper is to improve learning for multivariate processes whose structure is dependent on some known graph topology. Typically, the graph information is incorporated to the learning process via a smoothness assumption postulating that the values supported on well connected vertices exhibit small variations. We argue that smoothness i...
Conference Paper
We consider the problem of path inference: given a path prefix, i.e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix. In particular, we focus on natural paths occurring as a by-product of the interaction of an agent with a network---a driver on the transportation network, an information s...
Article
Graph-based techniques emerged as a choice to deal with the dimensionality issues in modeling multivariate time series. However, there is yet no complete understanding of how the underlying structure could be exploited to ease this task. This work provides contributions in this direction by considering the forecasting of a process evolving over a g...
Preprint
Full-text available
This paper studies the capacity limits of graph neural networks (GNN). Rather than focusing on a specific architecture, the networks considered here are those that fall within the message-passing framework, a model that encompasses several state-of-the-art networks. Two main results are presented. First, GNN are shown to be Turing universal under s...
Preprint
Full-text available
This paper focuses on the discrimination capacity of aggregation functions: these are the permutation invariant functions used by graph neural networks to combine the features of nodes. Realizing that the most powerful aggregation functions suffer from a dimensionality curse, we consider a restricted setting. In particular, we show that the standar...
Preprint
Full-text available
Deep convolutional neural networks have been shown to be able to fit a labeling over random data while still being able to generalize well on normal datasets. Describing deep convolutional neural network capacity through the measure of spectral complexity has been recently proposed to tackle this apparent paradox. Spectral complexity correlates wit...
Preprint
Full-text available
We consider the problem of path inference: given a path prefix, i.e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix. In particular, we focus on natural paths occurring as a by-product of the interaction of an agent with a network---a driver on the transportation network, an information s...
Preprint
Full-text available
Spectral clustering refers to a family of unsupervised learning algorithms that compute a spectral embedding of the original data based on the eigenvectors of a similarity graph. This non-linear transformation of the data is both the key of these algorithms' success and their Achilles heel: forming a graph and computing its dominant eigenvectors ca...
Preprint
Full-text available
Graph-based techniques emerged as a choice to deal with the dimensionality issues in modeling multivariate time series. However, there is yet no complete understanding of how the underlying structure could be exploited to ease this task. This work provides contributions in this direction by considering the forecasting of a process evolving over a g...
Preprint
Full-text available
How can we reduce the size of a graph without significantly altering its basic properties? We approach the graph reduction problem from the perspective of restricted similarity, a modification of a well-known measure for graph approximation. Our choice is motivated by the observation that restricted similarity implies strong spectral guarantees and...
Conference Paper
Full-text available
How does coarsening affect the spectrum of a general graph? We provide conditions such that the principal eigenvalues and eigenspaces of a coarsened and original graph Laplacian matrices are close. The achieved approximation is shown to depend on standard graph-theoretic properties, such as the degree and eigenvalue distributions, as well as on the...
Article
We currently witness the emergence of interesting new network topologies optimized towards the traffic matrices they serve, such as demand-aware datacenter interconnects (e.g., ProjecToR) and demand-aware peer-to-peer overlay networks (e.g., SplayNets). This paper introduces a formal framework and approach to reason about and design robust demand-a...
Conference Paper
Full-text available
How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove that the inner product between...
Conference Paper
Full-text available
Spectral clustering is a widely studied problem, yet its complexity is prohibitive for dynamic graphs of even modest size. We claim that it is possible to reuse information of past cluster assignments to expedite computation. Our approach builds on a recent idea of sidestepping the main bottleneck of spectral clustering, i.e., computing the graph e...
Article
Full-text available
We currently witness the emergence of interesting new network topologies optimized towards the traffic matrices they serve, such as demand-aware datacenter interconnects (e.g., ProjecToR) and demand-aware overlay networks (e.g., SplayNets). This paper introduces a formal framework and approach to reason about and design such topologies. We leverage...
Article
Full-text available
An emerging way to deal with high-dimensional non-euclidean data is to assume that the underlying structure can be captured by a graph. Recently, ideas have begun to emerge related to the analysis of time-varying graph signals. This work aims to elevate the notion of joint harmonic analysis to a full-fledged framework denoted as Time-Vertex Signal...
Article
Full-text available
Graph filters play a key role in processing the graph spectra of signals supported on the vertices of a graph. However, despite their widespread use, graph filters have been analyzed only in the deterministic setting, ignoring the impact of stochastic- ity in both the graph topology as well as the signal itself. To bridge this gap, we examine the s...
Conference Paper
This paper focuses on the harmonic analysis of graph signals that evolve with time. Our goal is to generalize and, in fact, unify the familiar concepts from time- and graph-frequency analysis. To this end, we study the properties of a joint time and graph Fourier transform (JFT) and the associated notion of variation. We build on our results to cre...
Conference Paper
Full-text available
Opportunistic routing protocols tackle the problem of efficient data collection in dynamic wireless sensor networks, where the radio is duty-cycled to save energy and the topology changes unpredictably due to node mobility and/or link dynamics. Unlike protocols that maintain a routing structure, in opportunistic protocols nodes forward packets to a...
Article
One of the cornerstones of the field of signal processing on graphs are graph filters, direct analogues of classical filters, but intended for signals defined on graphs. This work brings forth new insights on the distributed graph filtering problem. We design a family of autoregressive moving average (ARMA) recursions, which (i) are able to approxi...
Conference Paper
Distributed cloud applications, including batch processing, streaming, and scale-out databases, generate a significant amount of network traffic and a considerable fraction of their runtime is due to network activity. This paper initiates the study of deterministic algorithms for collocating frequently communicating nodes in a distributed networked...
Conference Paper
Despite their widespread use for the analysis of graph data, current graph filters are designed for graph signals that do not change over time, and thus they cannot simultaneously process time and graph frequency content in an adequate manner. This work presents ARMA 2D , an autoregressive moving average graph-temporal filter that captures jointly...
Article
Full-text available
An emerging way of tackling the dimensionality issues arising in the modeling of a multivariate process is to assume that the inherent data structure can be captured by a graph. Nevertheless, though state-of-the-art graph-based methods have been successful for many learning tasks, they do not consider time-evolving signals and thus are not suitable...
Article
Full-text available
Graph-based methods for signal processing have shown promise for the analysis of data exhibiting irregular structure, such as those found in social, transportation, and sensor networks. Yet, though these systems are often dynamic, state-of-the-art methods for signal processing on graphs ignore the dimension of time, treating successive graph signal...
Article
Full-text available
This letter extends the concept of graph-frequency to graph signals that evolve with time. Our goal is to generalize and, in fact, unify the familiar concepts from time- and graph-frequency analysis. To this end, we study a joint temporal and graph Fourier transform (JFT) and demonstrate its attractive properties. We build on our results to create...
Article
Full-text available
One of the cornerstones of the field of signal processing on graphs are graph filters, direct analogues of classical filters, but intended for signals defined on graphs. This work brings forth new insights on the distributed graph filtering problem. We design a family of autoregressive moving average (ARMA) recursions, which (i) are able to approxi...
Conference Paper
We have recently seen a surge of work on distributed graph filters, extending classical results to the graph setting. State of the art filters have however only been examined from a deterministic standpoint, ignoring the impact of stochasticity in the computation (e.g., temporal fluctuation of links) and input (e.g., the value of each node is a ran...
Article
This paper initiates the study of a fundamental online problem called online balanced repartitioning. Unlike the classic graph partitioning problem, our input is an arbitrary sequence of communication requests between nodes, with patterns that may change over time. The objective is to dynamically repartition the $n$ nodes into $\ell$ clusters, each...
Article
Full-text available
We introduce the concept of autoregressive moving average (ARMA) filters on a graph and show how they can be implemented in a distributed fashion. Our graph filter design philosophy is independent of the particular graph, meaning that the filter coefficients are derived irrespective of the graph. In contrast to finite-impulse response (FIR) graph f...
Conference Paper
Full-text available
Graph filters are a recent and powerful tool to process information in graphs. Yet despite their advantages, graph filters are limited. The limitation is exposed in a filtering task that is common but not fully solved in sensor networks: the identification of a signal's peaks and pits. Choosing the correct filter necessitates a-priori information a...
Thesis
We have recently seen a surge of research focusing on the processing of graph data. The emerging field of signal processing on graphs focuses on the extension of classical discrete signal processing techniques to the graph setting. Arguably, the greatest breakthrough of the field has been the extension of the Fourier transform from time signals and...
Article
Full-text available
Cooperation is the foundation of wireless ad hocnetworks with nodes forwarding their neighbors' packets forthe common good. However, energy and bandwidth constraints combined with selfish behaviour lead to collapsed networks where all nodes defect. Researchers have tried to incentivize or enforce the nodes for cooperation in various ways. However,...
Conference Paper
Full-text available
We address the problem of estimating the neighborhood cardinality of nodes in dynamic wireless networks. Different from previous studies, we consider networks with high densities (a hundred neighbors per node) and where all nodes estimate cardinality concurrently. Performing concurrent estimations on dense mobile networks is hard; we need estimator...
Conference Paper
Full-text available
The decentralized detection of event regions is a fundamental building block for monitoring and reasoning about spatial phenomena. However, so far the problem has been studied almost exclusively for static networks. This study proposes a theoretical framework with which we can analyze event detection algorithms suitable for large-scale mobile netwo...
Conference Paper
Full-text available
Fair rate allocation deals with the fundamental problem of sharing the channel efficiently and fairly. In wireless networks, several notable works have proposed optimal solutions to this problem. These approaches work well for static networks, but rely on an assumption that renders them sub-optimal when nodes are mobile: at each computation step, n...
Conference Paper
Full-text available
In large-scale resource-constrained systems, such as wireless sensor networks, global objectives should be ideally achieved through inexpensive local interactions. A technique satisfying these requirements is information potentials, in which distributed functions disseminate information about the process monitored by the network. Information potent...
Article
Full-text available
A common task of mobile wireless ad-hoc networks is to distributedly extract information from a monitored process. We define process information as a measure that is sensed and computed by each mobile node in a network. For complex tasks, such as searching in a network and coordination of robotic swarms, we are typically interested in the spatial d...
Article
Full-text available
Wireless Sensor Network (WSN) software is typically de-veloped in one of the two prominent WSN operating sys-tems: TinyOS or Contiki. Both of these operating systems are open-source projects and basically frameworks for WSN developers. In this paper, we study the software reposito-ries of these two projects. Software repositories provide a wealth o...
Conference Paper
Full-text available
In this paper we present a platform for developing mobile, locative and collaborative distributed games comprised of small programmable object technologies (e.g., wireless sensor networks) and traditional networked processors. The platform is implemented using a combination of JAVA Standard and Mobile editions, targeting also mobile phones that hav...

Network

Cited By

Projects

Projects (2)
Archived project
The D2S2 project aims at developing a framework for programming and operating distributed sensor systems that can be depended on in practical application scenarios. To make an experimental approach feasible, the project focuses on localization and tracking systems in two scenarios that are very relevant to the Dutch society: traffic monitoring and control (static setup) and rescue operations by firefighters and policemen (dynamic setup). A key, innovative feature of the project is the development and use of an advanced miniaturized radar sensor that can operate under a wide range of difficult environmental conditions (smoke, fog, etc.) that cannot be handled by typical localization systems in operation today.
Archived project
EWiDS is concentrating on extreme wireless distributed systems. In EWiDS, we aim at a better understanding of using wireless, user-centric sensor technology to monitor and manage the behavior of people. To this end, we use crowd management as our application domain. Our goal is to: Use body-worn sensors to measure proximity between people, leading to a so-called proximity graph. Automatically extract a proximity graph from a mobile sensor network consisting of thousands of people, to process the extracted data in real time and offline. The extracted data consists of a series of evolving graphs. We aim at developing new techniques for large-scale dynamic network analysis. Once analysis has been done, we will use the obtained information to provide feedback to a crowd of people, thus leading to actual crowd management. To get a better idea of what a proximity graph actually is, take a look at this description. Proximity graphs may be enriched with additional information such as sensed audio signals.