No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Graph Kernel Recursive Least-Squares Algorithms
Abstract
This paper presents graph kernel adaptive filters that model nonlinear input-output relationships of streaming graph signals. To this end, we propose centralized and distributed graph kernel recursive least-squares (GKRLS) algorithms utilizing the random Fourier features (RFF) map. Compared with solutions based on the traditional kernel trick, the proposed RFF approach presents two significant advantages. First, it sidesteps the need to maintain a high-dimensional dictionary, whose dimension increases with the number of graph nodes and time, which renders prohibitive computational and storage costs, especially when considering least-squares algorithms involving matrix inverses. Second, the distributed algorithm developed in this paper, referred to here as the graph diffusion kernel recursive least-squares (GDKRLS) algorithm, does not require centralized dictionary training, making it ideal for distributed learning in dynamic environments. To examine the performance of the proposed algorithms, we analyze the mean convergence of the GDKRLS algorithm and conduct numerical experiments. Results confirm the superiority of the proposed RFF-based GKRLS and GDKRLS over their LMS counterparts.
In recent years, graph signal processing has attracted much attention due to its ability to model irregular and interactive data generated by wireless sensor networks (WSNs). However, there is no practical method to deal with the problem of modeling nonlinear systems in non-Gaussian noise environments. Given that the maximum correntropy criterion (MCC) exhibits robustness to impulsive and non-Gaussian noise, this paper introduces it into the graph kernel adaptive filtering algorithm and develops a graph diffusion kernel MCC (GDKMCC) algorithm. To suppress the problem of the infinite growth of the filter coefficient vector of the proposed algorithm, a pre-trained dictionary (PD) method is used in this paper. In addition, the mean-square transient behavior and the convergence condition of the GDKMCC-PD algorithm are also provided under some assumptions. Finally, the simulation results verify the superiority of the proposed algorithm in modeling nonlinear systems under non-Gaussian noise environments and the correctness of the theoretical model.
This paper generalizes the proportionate-type adaptive algorithm to the graph signal processing and proposes two proportionate-type adaptive graph signal recovery algorithms. The gain matrix of the proportionate algorithm leads to faster convergence than least mean squares (LMS) algorithm. In this paper, the gain matrix is obtained in a closed-form by minimizing the gradient of the mean-square deviation (GMSD). The first algorithm is the proportionate-type graph LMS (Pt-GLMS) algorithm which simply uses a gain matrix in the recursion process of the LMS algorithm and accelerates the convergence of the Pt-GLMS algorithm compared to the LMS algorithm. The second algorithm is the proportionate-type graph extended LMS (Pt-GELMS) algorithm, which uses the previous signal vectors alongside the signal of the current iteration. The Pt-GELMS algorithm utilizes two gain matrices to control the effect of the signal of the previous iterations. The stability analyses of the algorithms are also provided. Simulation results demonstrate the efficacy of the two proposed proportionate-type LMS algorithms.
Numerous real-life systems exhibit complex nonlinear input-output relationships. Kernel adaptive filters, a popular class of nonlinear adaptive filters, can efficiently model these nonlinear input-output relationships. Their growing network structure, however, poses considerable challenges in terms of their hardware implementation, making them inefficient for real-time applications. Random Fourier features (RFF) facilitate the development of kernel adaptive filters with a fixed network structure. For the first time, this paper attempts to implement the RFF-based kernel least mean square (RFF-KLMS) algorithm on hardware. To this end, we propose several reformulations of the feature functions (FFs) that are computationally expensive in their native form so that they can be implemented in real-time VLSI. Specifically, we reformulate inner product evaluation, cosine, and exponential functions that appear in the implementation of FFs. With these reformulations, the proposed delayed RFF-KLMS (DRFF-KLMS) is then synthesized using
$45$
-nm CMOS technology with
$16$
-bit fixed-point representations. According to the synthesis results, pipelined DRFF-KLMS architectures require minimal hardware increase over the state-of-the-art conventional delayed LMS architecture while significantly improving estimation performance for the nonlinear model. Our results suggest that the cosine feature function-based DRFF-KLMS is appropriate for applications requiring high accuracy, whereas the exponential function-based DRFF-KLMS may be well suited for resource-constrained applications.
This paper presents communication-efficient approaches to federated learning for resource-constrained devices with access to streaming data. In particular, we first propose a partial-sharing-based framework for online federated learning, called PSO-Fed, wherein clients update local models from a stream of data and exchange tiny fractions of the model with the server, reducing the communication overhead. In contrast to classical federated learning approaches, the proposed strategy provides clients who are not part of a global iteration with the freedom to update local models whenever new data arrives. Furthermore, by devising a client-side innovation check, we also propose an event-triggered PSO-Fed (ETPSO-Fed) that further reduces the computational burden of clients while enhancing communication efficiency. We implement the abovementioned frameworks in the context of kernel regression, where clients perform local learning employing random Fourier features-based kernel least mean squares. In addition, we examine the mean and mean-square convergence of the proposed PSO-Fed. Finally, we conduct experiments to determine the efficacy of the proposed frameworks. Our results show that PSO-Fed and ETPSO-Fed can compete with Online-Fed while requiring significantly less communication overhead. Simulations demonstrate an 80% reduction in PSO-Fed and an 84.5% reduction in ETPSO-Fed communication overhead compared to Online-Fed. Notably, the proposed partial-sharing-based online FL strategies show good resilience against model-poisoning attacks without involving additional mechanisms.
Nonlinear spline adaptive filters are a class of adaptive filters for modelling nonlinear systems. To improve the convergence performance of existing nonlinear spline adaptive filters (SAFs), in this paper, we propose a low rank approximation for different SAF models by incorporating the technique of nearest Kronecker product decomposition. We consider the Wiener and Hammerstein SAF models for developing the proposed algorithms, and simulation studies carried out show that improved convergence and tracking performance can be achieved compared to traditional SAFs.
This paper develops adaptive graph filters that operate in reproducing kernel Hilbert spaces. We consider both centralized and fully distributed implementations. We first define nonlinear graph filters that operate on graph-shifted versions of the input signal. We then propose a centralized graph kernel least mean squares (GKLMS) algorithm to identify nonlinear graph filters' model parameters. To reduce the dictionary size of the centralized GKLMS, we apply the principles of coherence check and random Fourier features (RFF). The resulting algorithms have performance close to that of the GKLMS algorithm. Additionally, we leverage the graph structure to derive the distributed graph diffusion KLMS (GDKLMS) algorithms. We show that, unlike the coherence check-based approach, the GDKLMS based on RFF avoids the use of a pre-trained dictionary through its data-independent fixed structure. We conduct a detailed performance study of the proposed RFF-based GDKLMS, and the conditions for its convergence both in mean and mean-squared senses are derived. Extensive numerical simulations show that the GKLMS and GDKLMS can successfully identify nonlinear graph filters and adapt to model changes. Furthermore, RFF-based strategies show faster convergence for model identification and exhibit better tracking performance in model-changing scenarios.
The goal of this paper is to propose novel strategies for adaptive learning of signals defined over graphs, which are observed over a (randomly time-varying) subset of vertices. We recast two classical adaptive algorithms in the graph signal processing framework, namely, the least mean squares (LMS) and the recursive least squares (RLS) adaptive estimation strategies. For both methods, a detailed mean-square analysis illustrates the effect of random sampling on the adaptive reconstruction capability and the steady-state performance. Then, several probabilistic sampling strategies are proposed to design the sampling probability at each node in the graph, with the aim of optimizing the tradeoff between steady-state performance, graph sampling rate, and convergence rate of the adaptive algorithms. Finally, a distributed RLS strategy is derived and is shown to be convergent to its centralized counterpart. Numerical simulations carried out over both synthetic and real data illustrate the good performance of the proposed sampling and reconstruction strategies for (possibly distributed) adaptive learning of signals defined over graphs.
We present a novel diffusion scheme for online kernel-based learning over networks. So far, a major drawback of any online learning algorithm, operating in a reproducing kernel Hilbert space (RKHS), is the need for updating a growing number of parameters as time iterations evolve. Besides complexity, this leads to an increased need of communication resources, in a distributed setting. In contrast, the proposed method approximates the solution as a fixed-size vector (of larger dimension than the input space) using Random Fourier Features. This paves the way to use standard linear combine-then-adapt techniques. To the best of our knowledge, this is the first time that a complete protocol for distributed online learning in RKHS is presented. Conditions for asymptotic convergence and boundness of the networkwise regret are also provided. The simulated tests illustrate the performance of the proposed scheme.
We introduce the concept of autoregressive moving average (ARMA) filters on a
graph and show how they can be implemented in a distributed fashion. Our graph
filter design philosophy is independent of the particular graph, meaning that
the filter coefficients are derived irrespective of the graph. In contrast to
finite-impulse response (FIR) graph filters, ARMA graph filters are robust
against changes in the signal and/or graph. In addition, when time-varying
signals are considered, we prove that the proposed graph filters behave as ARMA
filters in the graph domain and, depending on the implementation, as first or
higher ARMA filters in the time domain.
We propose a sampling theory for signals that are supported on either
directed or undirected graphs. The theory follows the same paradigm as
classical sampling theory. We show that the perfect recovery is possible for
graph signals bandlimited under the graph Fourier transform, and the sampled
signal coefficients form a new graph signal, whose corresponding graph
structure is constructed from the original graph structure, preserving
frequency contents. By imposing a specific structure on the graph, graph
signals reduce to finite discrete-time signals and the proposed sampling theory
works reduces to classical signal processing. We further establish the
connection to frames with maximal robustness to erasures as well as compressed
sensing, and show how to choose the optimal sampling operator, how random
sampling works on circulant graphs and Erd\H{o}s-R\'enyi graphs, and how to
handle full-band graph signals by using graph filter banks. We validate the
proposed sampling theory on the simulated datasets of Erd\H{o}s-R\'enyi graphs
and small-world graphs, and a real-world dataset of online blogs. We show that
for each case, the proposed sampling theory achieves perfect recovery with high
probability. Finally, we apply the proposed sampling theory to semi-supervised
classification of online blogs and digit images, where we achieve similar or
better performance with fewer labeled samples compared to the previous work.
Many signal processing problems in wireless sensor networks can be solved by graph filtering techniques. Finite impulse response (FIR) graph filters (GFs) have received more attention in the literature because they enable distributed computation by the sensors. However, FIR GFs are limited in their ability to represent the global information of the network. This letter proposes a family of GFs with infinite impulse response (IIR) and provides algorithms for their distributed realization in wireless sensor networks. IIR GFs bring more flexibility to GF designers, as they can be designed and realized even when the graph spectrum is unknown. Numerical results show that IIR GFs are more accurate in approximating ideal GFs and more robust against network variation than FIR GFs.
In applications such as social, energy, transportation, sensor, and neuronal
networks, high-dimensional data naturally reside on the vertices of weighted
graphs. The emerging field of signal processing on graphs merges algebraic and
spectral graph theoretic concepts with computational harmonic analysis to
process such signals on graphs. In this tutorial overview, we outline the main
challenges of the area, discuss different ways to define graph spectral
domains, which are the analogues to the classical frequency domain, and
highlight the importance of incorporating the irregular structures of graph
data domains when processing signals on graphs. We then review methods to
generalize fundamental operations such as filtering, translation, modulation,
dilation, and downsampling to the graph setting, and survey the localized,
multiscale transforms that have been proposed to efficiently extract
information from high-dimensional data on graphs. We conclude with a brief
discussion of open issues and possible extensions.
In social settings, individuals interact through webs of relationships. Each
individual is a node in a complex network (or graph) of interdependencies and
generates data, lots of data. We label the data by its source, or formally
stated, we index the data by the nodes of the graph. The resulting signals
(data indexed by the nodes) are far removed from time or image signals indexed
by well ordered time samples or pixels. DSP, discrete signal processing,
provides a comprehensive, elegant, and efficient methodology to describe,
represent, transform, analyze, process, or synthesize these well ordered time
or image signals. This paper extends to signals on graphs DSP and its basic
tenets, including filters, convolution, z-transform, impulse response, spectral
representation, Fourier transform, frequency response, and illustrates DSP on
graphs by classifying blogs, linear predicting and compressing data from
irregularly located weather stations, or predicting behavior of customers of a
mobile service provider.
Kernel-based algorithms have been a topic of considerable interest in the machine learning community over the last ten years. Their attractiveness resides in their elegant treatment of nonlinear problems. They have been successfully applied to pattern recognition, regression and density estimation. A common characteristic of kernel-based methods is that they deal with kernel expansions whose number of terms equals the number of input data, making them unsuitable for online applications. Recently, several solutions have been proposed to circumvent this computational burden in time series prediction problems. Nevertheless, most of them require excessively elaborate and costly operations. In this paper, we investigate a new model reduction criterion that makes computationally demanding sparsification procedures unnecessary. The increase in the number of variables is controlled by the coherence parameter, a fundamental quantity that characterizes the behavior of dictionaries in sparse approximation problems. We incorporate the coherence criterion into a new kernel-based affine projection algorithm for time series prediction. We also derive the kernel-based normalized LMS algorithm as a particular case. Finally, experiments are conducted to compare our approach to existing methods.
The combination of the famed kernel trick and the least-mean-square (LMS) algorithm provides an interesting sample-by-sample update for an adaptive filter in reproducing kernel Hilbert spaces (RKHS), which is named in this paper the KLMS. Unlike the accepted view in kernel methods, this paper shows that in the finite training data case, the KLMS algorithm is well posed in RKHS without the addition of an extra regularization term to penalize solution norms as was suggested by Kivinen [Kivinen, Smola and Williamson, ldquoOnline Learning With Kernels,rdquo IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2165-2176, Aug. 2004] and Smale [Smale and Yao, ldquoOnline Learning Algorithms,rdquo Foundations in Computational Mathematics, vol. 6, no. 2, pp. 145-176, 2006]. This result is the main contribution of the paper and enhances the present understanding of the LMS algorithm with a machine learning perspective. The effect of the KLMS step size is also studied from the viewpoint of regularization. Two experiments are presented to support our conclusion that with finite data the KLMS algorithm can be readily used in high dimensional spaces and particularly in RKHS to derive nonlinear, stable algorithms with comparable performance to batch, regularized solutions.
Network data can be conveniently modeled as a graph signal, where data values are assigned to nodes of a graph that describes the underlying network topology. Successful learning from network data is built upon methods that effectively exploit this graph structure. In this article, we leverage graph signal processing (GSP) to characterize the representation space of graph neural networks (GNNs). We discuss the role of graph convolutional filters in GNNs and show that any architecture built with such filters has the fundamental properties of permutation equivariance and stability to changes in the topology. These two properties offer insight about the workings of GNNs and help explain their scalability and transferability properties, which, coupled with their local and distributed nature, make GNNs powerful tools for learning in physical networks. We also introduce GNN extensions using edge-varying and autoregressive moving average (ARMA) graph filters and discuss their properties. Finally, we study the use of GNNs in recommender systems and learning decentralized controllers for robot swarms.
The effective representation, processing, analysis, and visualization of large-scale structured data, especially those related to complex domains, such as networks and graphs, are one of the key questions in modern machine learning. Graph signal processing (GSP), a vibrant branch of signal processing models and algorithms that aims at handling data supported on graphs, opens new paths of research to address this challenge. In this article, we review a few important contributions made by GSP concepts and tools, such as graph filters and transforms, to the development of novel machine learning algorithms. In particular, our discussion focuses on the following three aspects: exploiting data structure and relational priors, improving data and computational efficiency, and enhancing model interpretability. Furthermore, we provide new perspectives on the future development of GSP techniques that may serve as a bridge between applied mathematics and signal processing on one side and machine learning and network science on the other. Cross-fertilization across these different disciplines may help unlock the numerous challenges of complex data analysis in the modern age.
This paper develops nonlinear kernel adaptive filtering algorithms based on the set-membership filtering (SMF) framework. The set-membership-based filtering approach is distinct from the conventional adaptive filtering approaches in that it aims for the filtering error being bounded in magnitude, as opposed to seeking to minimize the time average or ensemble average of the squared errors. The proposed kernel SMF algorithms feature selective updates of their parameter estimates by making discerning use of the input data, and selective increase of the dimension in the kernel expansion. These result in less computational cost and faster tracking without compromising the mean-squared error performance. We show, through convergence analysis, that the sequences of parameter estimates of our proposed algorithms are convergent, and the filtering error is asymptotically upper bounded in magnitude. Simulations are performed which show clearly the advantages of the proposed algorithms in terms of lower computational complexity, reduced dictionary size, and steady-state mean-squared errors comparable to existing algorithms.
We propose a kernel regression method to predict a target signal lying over a graph when an input observation is given. The input and the output could be two different physical quantities. In particular, the input may not be a graph signal at all or it could be agnostic to an underlying graph. We use a training dataset to learn the proposed regression model by formulating it as a convex optimization problem, where we use a graph-Laplacian based regularization to enforce that the predicted target is a graph signal. Once the model is learnt, it can be directly used on a large number of test data points one-by-one independently to predict the corresponding targets. Our approach employs kernels between the various input observations, and as a result the kernels are not restricted to be functions of the graph adjacency/Laplacian matrix. We show that the proposed kernel regression exhibits a smoothing effect, while simultaneously achieving noise-reduction and graph-smoothness. We then extend our method to the case when the underlying graph may not be known apriori, by simultaneously learning an underlying graph and the regression coefficients. Using extensive experiments, we show that our method provides a good prediction performance in adverse conditions, particularly when the training data is limited in size and is noisy. In graph signal reconstruction experiments, our method is shown to provide a good performance even for a highly under-determined subsampling.
Research in graph signal processing (GSP) aims to develop tools for processing data defined on irregular graph domains. In this paper, we first provide an overview of core ideas in GSP and their connection to conventional digital signal processing, along with a brief historical perspective to highlight how concepts recently developed in GSP build on top of prior research in other areas. We then summarize recent advances in developing basic GSP tools, including methods for sampling, filtering, or graph learning. Next, we review progress in several application areas using GSP, including processing and analysis of sensor network data, biological data, and applications to image processing and machine learning.
Graph-based methods pervade the inference toolkits of numerous disciplines including sociology, biology, neuroscience, physics, chemistry, and engineering. A challenging problem encountered in this context pertains to determining the attributes of a set of vertices given those of another subset at possibly different time instants. Leveraging spatiotemporal dynamics can drastically reduce the number of observed vertices, and hence the cost of sampling. Alleviating the limited flexibility of existing approaches, the present paper broadens the existing kernel-based graph function reconstruction framework to accommodate time-evolving functions over possibly time-evolving topologies. This approach inherits the versatility and generality of kernel-based methods, for which no knowledge on distributions or second-order statistics is required. Systematic guidelines are provided to construct two families of space-time kernels with complementary strengths. The first facilitates judicious control of regularization on a space-time frequency plane, whereas the second can afford time-varying topologies. Batch and online estimators are also put forth, and a novel kernel Kalman filter is developed to obtain these estimates at affordable computational cost. Numerical tests with real data sets corroborate the merits of the proposed methods relative to competing alternatives.
Many signal processing problems involve data whose underlying structure is non-Euclidean, but may be modeled as a manifold or (combinatorial) graph. For instance, in social networks, the characteristics of users can be modeled as signals on the vertices of the social graph. Sensor networks are graph models of distributed interconnected sensors, whose readings are modelled as time-dependent signals on the vertices. In genetics, gene expression data are modeled as signals defined on the regulatory network. In neuroscience, graph models are used to represent anatomical and functional structures of the brain. Modeling data given as points in a high-dimensional Euclidean space using nearest neighbor graphs is an increasingly popular trend in data science, allowing practitioners access to the intrinsic structure of the data. In computer graphics and vision, 3D objects are modeled as Riemannian manifolds (surfaces) endowed with properties such as color texture. Even more complex examples include networks of operators, e.g., functional correspondences or difference operators in a collection of 3D shapes, or orientations of overlapping cameras in multi-view vision ("structure from motion") problems. The complexity of geometric data and the availability of very large datasets (in the case of social networks, on the scale of billions) suggest the use of machine learning techniques. In particular, deep learning has recently proven to be a powerful tool for problems with large datasets with underlying Euclidean structure. The purpose of this paper is to overview the problems arising in relation to geometric deep learning and present solutions existing today for this class of problems, as well as key difficulties and future research directions.
One of the cornerstones of the field of signal processing on graphs are graph filters, direct analogues of classical filters, but intended for signals defined on graphs. This work brings forth new insights on the distributed graph filtering problem. We design a family of autoregressive moving average (ARMA) recursions, which (i) are able to approximate any desired graph frequency response, and (ii) give exact solutions for specific graph signal denoising and interpolation problems. The philosophy, to design the ARMA coefficients independently from the underlying graph, renders the ARMA graph filters suitable in static and, particularly, time-varying settings. The latter occur when the graph signal and/or graph topology are changing over time. We show that in case of a time-varying graph signal our approach extends naturally to a two-dimensional filter, operating concurrently in the graph and regular time domain. We also derive the graph filter behavior, as well as sufficient conditions for filter stability when the graph and signal are time-varying. The analytical and numerical results presented in this paper illustrate that ARMA graph filters are practically appealing for static and time-varying settings, as predicted by theoretical derivations.
The aim of this paper is to propose distributed strategies for adaptive learning of signals defined over graphs. Assuming the graph signal to be band-limited, the method enables distributed reconstruction, with guaranteed performance in terms of mean-square error, and tracking from a limited number of sampled observations taken from a subset of vertices. A detailed mean square analysis is carried out and illustrates the role played by the sampling strategy on the performance of the proposed method. Finally, some useful strategies for distributed selection of the sampling set are provided. Several numerical results validate our theoretical findings, and illustrate the performance of the proposed method for distributed adaptive learning of signals defined over graphs.
Defining a sound shift operator for signals existing on a certain graphical
structure, similar to the well-defined shift operator in classical signal
processing, is a crucial problem in graph signal processing. Since almost all
operations, such as filtering, transformation, prediction, etc., are directly
related to the graph shift operator. We define a unique shift operator that
satisfies all properties the shift operator as in the classical signal
processing, especially this shift operator preserves the energy of a graph
signal. Our definition of graph shift operator negates the shift operator
defined in the literature as the graph adjacency matrix, which generally does
not preserve the energy of a graph signal. We show that any graph shift
invariant graph filter can be written as a polynomial function of the graph
shift operator and that the adjacency matrix of a graph is indeed a linear
shift invariant graph filter with respect to the graph shift operator.
Furthermore, we introduce the concepts of the finite impulse response (FIR) and
infinite impulse response (IIR) filters similar to the classical signal
processing counterparts and obtain an explicit form of such filters. Based on
the defined shift operator, we obtain the optimal filtering on graphs, i.e.,
the corresponding Wiener filtering on graph, and elaborate on the structure of
such filters for any arbitrary graph structure. We specially treat the directed
cyclic graph and show that the optimal linear shift invariant filter is indeed
the well-known Wiener filter in classical signal processing. This result show
that, optimal linear time invariant filters for time series data is a subset of
optimal graph filters. We also elaborate on the best linear predictor graph
filters, optimal filters filter for product graphs.
We study the design of graph filters to implement arbitrary linear
transformations between graph signals. Graph filters can be represented by
matrix polynomials of the graph-shift operator, which captures the structure of
the graph and is assumed to be given. Thus, graph-filter design consists in
choosing the coefficients of these polynomials (known as filter coefficients)
to resemble desired linear transformations. Due to the local structure of the
graph-shift operator, graph filters can be implemented distributedly across
nodes, making them suitable for networked settings. We determine spectral
conditions under which a specific linear transformation can be implemented
perfectly using graph filters. Furthermore, for the cases where perfect
implementation is infeasible, the design of optimal approximations for
different error metrics is analyzed. We introduce the notion of a node-variant
graph filter, which allows the simultaneous implementation of multiple
(regular) graph filters in different nodes of the graph. This additional
flexibility enables the design of more general operators without undermining
the locality in implementation. Perfect and approximate implementation of
network operators is also studied for node-variant graph filters. We
demonstrate the practical relevance of the developed framework by studying in
detail the application of graph filters to the problems of finite-time
consensus and analog network coding. Finally, we present additional numerical
experiments comparing the performance of node-invariant and node-variant
filters when approximating arbitrary linear network operators.
Analysis and processing of very large data sets, or big data, poses a significant challenge. Massive data sets are collected and studied in numerous domains, from engineering sciences to social networks, biomolecular research, commerce, and security. Extracting valuable information from big data requires innovative approaches that efficiently process large amounts of data as well as handle and, moreover, utilize their structure. This article discusses a paradigm for large-scale data analysis based on the discrete signal processing (DSP) on graphs (DSPG). DSPG extends signal processing concepts and methodologies from the classical signal processing theory to data indexed by general graphs. Big data analysis presents several challenges to DSPG, in particular, in filtering and frequency analysis of very large data sets. We review fundamental concepts of DSPG, including graph signals and graph filters, graph Fourier transform, graph frequency, and spectrum ordering, and compare them with their counterparts from the classical signal processing theory. We then consider product graphs as a graph model that helps extend the application of DSPG methods to large data sets through efficient implementation based on parallelization and vectorization. We relate the presented framework to existing methods for large-scale data processing and illustrate it with an application to data compression.
We propose a novel discrete signal processing framework for the representation and analysis of datasets with complex structure. Such datasets arise in many social, economic, biological, and physical networks. Our framework extends traditional discrete signal processing theory to structured datasets by viewing them as signals represented by graphs, so that signal coefficients are indexed by graph nodes and relations between them are represented by weighted graph edges. We discuss the notions of signals and filters on graphs, and define the concepts of the spectrum and Fourier transform for graph signals. We demonstrate their relation to the generalized eigenvector basis of the graph adjacency matrix and study their properties. As a potential application of the graph Fourier transform, we consider the efficient representation of structured data that utilizes the sparseness of graph signals in the frequency domain.
We propose a novel discrete signal processing framework for structured datasets that arise from social, economic, biological, and physical networks. Our framework extends traditional discrete signal processing theory to datasets with complex structure that can be represented by graphs, so that data elements are indexed by graph nodes and relations between elements are represented by weighted graph edges. We interpret such datasets as signals on graphs, introduce the concept of graph filters for processing such signals, and discuss important properties of graph filters, including linearity, shift-invariance, and invertibility. We then demonstrate the application of graph filters to data classification by demonstrating that a classifier can be interpreted as an adaptive graph filter. Our experiments demonstrate that the proposed approach achieves high classification accuracy.
A distributed least-squares estimation strategy is developed by appealing to collaboration techniques that exploit the space-time structure of the data, achieving an exact recursive solution that is fully distributed. Each node is allowed to communicate with its immediate neighbor in order to exploit the spatial dimension, while it evolves locally to account for the time dimension as well. In applications where communication and energy resources are scarce, an approximate RLS scheme that is also fully distributed is proposed in order to decrease the communication burden necessary to implement distributed collaborative solution. The performance of the resulting algorithm tends to its exact counterpart in the mean-square sense as the forgetting factor lambda tends to unity. A spatial-temporal energy conservation argument is used to evaluate the steady-state performance of the individual nodes across the adaptive distributed network for the low communications RLS implementation. Computer simulations illustrate the results.
The linear least mean squares (LMS) algorithm has been recently extended to a reproducing kernel Hilbert space, resulting in an adaptive filter built from a weighted sum of kernel functions evaluated at each incoming data sample. With time, the size of the filter as well as the computation and memory requirements increase. In this paper, we shall propose a new efficient methodology for constraining the increase in length of a radial basis function (RBF) network resulting from the kernel LMS algorithm without significant sacrifice on performance. The method involves sequential Gaussian elimination steps on the Gram matrix to test the linear dependency of the feature vector corresponding to each new input with all the previous feature vectors. This gives an efficient method of continuing the learning as well as restricting the number of kernel functions used.
Distributed diffusion adaptation over graph signals
- R Nassif
- C Richard
- J Chen
- A H Sayed
R. Nassif, C. Richard, J. Chen, and A. H. Sayed, "Distributed diffusion
adaptation over graph signals," in Proc. IEEE Int. Conf. Acoust., Speech,
Signal Process., 2018, pp. 4129-4133.
Kernels and regularization on graphs
- A J Smola
- R Kondor
A. J. Smola and R. Kondor, "Kernels and regularization on graphs," in
Learning theory and kernel machines. Lecture Notes in Computer Science,
vol. 2777, pp. 144-158, Springer, 2003.