Large deviations of condition numbers and extremal eigenvalues of random matrices
ResearchGate has not been able to resolve any citations for this publication.
We investigate the random eigenvalues coming from the beta-Laguerre ensemble
with parameter p, which is a generalization of the real, complex and quaternion
Wishart matrices of parameter (n,p). In the case that the sample size n is much
smaller than the dimension of the population distribution p, a common situation
in modern data, we approximate the beta-Laguerre ensemble by a beta-Hermite
ensemble which is a generalization of the real, complex and quaternion Wigner
matrices. As corollaries, when n is much smaller than p, we show that the
largest and smallest eigenvalues of the complex Wishart matrix are
asymptotically independent; we obtain the limiting distribution of the
condition numbers as a sum of two i.i.d. random variables with a Tracy-Widom
distribution, which is much different from the exact square case that n=p by
Edelman (1988); we propose a test procedure for a spherical hypothesis test. By
the same approximation tool, we obtain the asymptotic distribution of the
smallest eigenvalue of the beta-Laguerre ensemble. In the second part of the
paper, under the assumption that n is much smaller than p in a certain scale,
we prove the large deviation principles for three basic statistics: the largest
eigenvalue, the smallest eigenvalue and the empirical distribution of
eigenvalues, where the last large deviation is derived by using a non-standard
method.
In this paper the authors show that the largest eigenvalue of the sample covariance matrix tends to a limit under certain conditions when both the number of variables and the sample size tend to infinity. The above result is proved under the mild restriction that the fourth moment of the elements of the sample sums of squares and cross products (SP) matrix exist.
The focus of this paper is on the probability,E
(O;J), that a setJ consisting of a finite union of intervals contains no eigenvalues for the finiteN Gaussian Orthogonal (=1) and Gaussian Symplectic (=4) Ensembles and their respective scaling limits both in the bulk and at the edge of the spectrum. We show how these probabilities can be expressed in terms of quantities arising in the corresponding unitary (=2) ensembles. Our most explicit new results concern the distribution of the largest eigenvalue in each of these ensembles. In the edge scaling limit we show that these largest eigenvalue distributions are given in terms of a particular Painlev II function.
This article proves that the spectral distribution of the random matrix , where and has iid entries with , tends to the semicircle law as , a.s.
In this paper, the authors show that the smallest (if ) or the -th smallest (if ) eigenvalue of a sample covariance matrix of the form tends almost surely to the limit as and , where X is a matrix with iid entries with mean zero, variance 1 and fourth moment finite. Also, as a by-product, it is shown that the almost sure limit of the largest eigenvalue is , a known result obtained by Yin, Bai and Krishnaiah. The present approach gives a unified treatment for both the extreme eigenvalues of large sample covariance matrices.
Random matrix theory is a maturing discipline with decades of research in multiple fields now beginning to converge. Experience has shown that many exact formulas are available for certain matrices with real, complex, or quaternion entries. In random matrix jargon, these are the cases β = 1, 2 and 4 respectively. This thesis explores the general P > 0 case mathematically and with symbolic software. We focus on generalizations of the Hermite distributions originating in physics (the "Gaussian" ensembles) and the Laguerre distributions of statistics (the "Wishart" matrices). One of our main contributions is the construction of tridiagonal matrix models for the general (β > 0) 0 β-Hermite and (β > 0, a > β(m - 1)/2) β-Laguerre ensembles of parameter a and size m, and investigate applications of these new ensembles, particularly in the areas of eigenvalue statistics. The new models are symmetric tridiagonal, and with entries from real distributions, regardless of the value of β. The entry distributions are either normal or X, so "classical", and the independence pattern is maximal, in the sense that the only constraints arise from the symmetric/semi-definite condition. The β-ensemble distributions have been studied for the particular 1, 2, 4 values of p as joint eigenvalue densities for full random matrix ensembles (Gaussian, or Hermite, and Wishart, or Laguerre) with real, complex, and quaternion entries (for references, see [66] and [70]). In addition, general -ensembles were considered and studied as theoretical distributions ([8, 51, 50, 55, 56]), with applications in lattice gas theory and statistical mechanics (the parameter being interpreted as an arbitrary inverse temperature of a Coulomb gas with logarithmic potential).
Many natural and social systems develop complex networks that are usually modeled as random graphs. The eigenvalue spectrum of these graphs provides information about their structural properties. While the semicircle law is known to describe the spectral densities of uncorrelated random graphs, much less is known about the spectra of real-world graphs, describing such complex systems as the Internet, metabolic pathways, networks of power stations, scientific collaborations, or movie actors, which are inherently correlated and usually very sparse. An important limitation in addressing the spectra of these systems is that the numerical determination of the spectra for systems with more than a few thousand nodes is prohibitively time and memory consuming. Making use of recent advances in algorithms for spectral characterization, here we develop methods to determine the eigenvalues of networks comparable in size to real systems, obtaining several surprising results on the spectra of adjacency matrices corresponding to models of real-world graphs. We find that when the number of links grows as the number of nodes, the spectral density of uncorrelated random matrices does not converge to the semicircle law. Furthermore, the spectra of real-world graphs have specific features, depending on the details of the corresponding models. In particular, scale-free graphs develop a trianglelike spectral density with a power-law tail, while small-world graphs have a complex spectral density consisting of several sharp peaks. These and further results indicate that the spectra of correlated graphs represent a practical tool for graph classification and can provide useful insight into the relevant structural properties of real networks.
This paper brings into play elements of the spectral theory of
such matrices and demonstrates their relevance to source detection and
bearing estimation in problems with sizable arrays. These results are
applied to the sample spatial covariance matrix, R ˆ, of
the sensed data. It is seen that detection can be achieved with a sample
size considerably less than that required by conventional approaches. It
is argued that more accurate estimates of direction of arrival can be
obtained by constraining R ˆ to be consistent with various
a priori constraints including those arising from large dimensional
random matrix theory. A set theoretic formalism is used for this
feasibility problem. Unsolved issues are discussed
Classical β \beta -Laguerre ensembles consist of three special matrix models taking the form X X T \mathbf {X}\mathbf {X}^T with X \mathbf {X} denoting a random matrix having i.i.d. entries being real ( β = 1 \beta =1 ), complex ( β = 2 \beta =2 ) or quaternion ( β = 4 \beta =4 ) normal distribution. It had been actually believed that no other choice of β > 0 \beta >0 (besides 1 , 2 1,2 and 4 4 ) would correspond to a matrix model X β X β T \mathbf {X}_\beta \mathbf {X}_\beta ^T which can be constructed with entries from a classical distribution until the work done by Dumitriu and Edelman in 2002. Since then the spectral properties of general β \beta -Laguerre ensembles have been extensively studied dealing with both the bulk case (involving all the eigenvalues) and the extremal case (addressing the (first few) largest and smallest eigenvalues). However, the ratio of the extremal eigenvalues (equivalently the condition number of X β \mathbf {X}_\beta ) has not been well explored in the literature. In this paper, we study such ratio in terms of large deviations.
Large deviations of the largest and smallest eigenvalues of are studied in this note, where is a random matrix with independent and identically distributed (i.i.d.) sub-Gaussian entries. The assumption imposed on the dimension size p and the sample size n is with . This study generalizes one result obtained in [3].
Let be a random matrix whose entries are independent and identically distributed real random variables with zero mean and unit variance. We study the limiting behaviors of the 2-normal condition number k ( p , n ) of in terms of large deviations for large n , with p being fixed or with p(n)=o(n) . We propose two main ingredients: (i) to relate the large-deviation probabilities of k ( p , n ) to those involving n independent and identically distributed random variables, which enables us to consider a quite general distribution of the entries (namely the sub-Gaussian distribution), and (ii) to control, for standard normal entries, the upper tail of k ( p , n ) using the upper tails of ratios of two independent random variables, which enables us to establish an application in statistical inference.
This is a translation of Harald Cram\'er's 1938 article, 'On a new limit theorem in probability theory', published in French in 1938 and deriving what is considered by mathematicians to be the first large deviation result. My hope is that this translation will help disseminate this historically important work, 80 years after its publication.
Karl Gustafson is the creator of the theory of antieigenvalue analysis. Its applications spread through fields asdiverse as numerical analysis, wavelets, statistics, quantum mechanics, and finance.Antieigenvalue analysis, with itsoperator trigonometry, is a unifying language which enables new and deeper geometrical understanding of essentially every result in operator theory and matrix theory, together withtheir applications. This book will open up its methods to a wide range of specialists © 2012 by World Scientific Publishing Co. Pte. Ltd. All rights reserved.
We study sample covariance matrices of the form W = (1 / n ) C C T , where C is a k x n matrix with independent and identically distributed (i.i.d.) mean 0 entries. This is a generalization of the so-called Wishart matrices, where the entries of C are i.i.d. standard normal random variables. Such matrices arise in statistics as sample covariance matrices, and the high-dimensional case, when k is large, arises in the analysis of DNA experiments. We investigate the large deviation properties of the largest and smallest eigenvalues of W when either k is fixed and n → ∞ or k n → ∞ with k n = o ( n / log log n ), in the case where the squares of the i.i.d. entries have finite exponential moments. Previous results, proving almost sure limits of the eigenvalues, require only finite fourth moments. Our most explicit results for large k are for the case where the entries of C are ∓ 1 with equal probability. We relate the large deviation rate functions of the smallest and largest eigenvalues to the rate functions for i.i.d. standard normal entries of C . This case is of particular interest since it is related to the problem of decoding of a signal in a code-division multiple-access (CDMA) system arising in mobile communication systems. In this example, k is the number of users in the system and n is the length of the coding sequence of each of the users. Each user transmits at the same time and uses the same frequency; the codes are used to distinguish the signals of the separate users. The results imply large deviation bounds for the probability of a bit error due to the interference of the various users.
Introduction.- LDP for Finite Dimensional Space.- Applications - The Finite Dimensional Case.- General Principles.- Sample Path Large Deviations.- The LDP for Abstract Empirical Measures.- Applications of Empirical Measures LDP.
Random matrix theory has found many applications in physics, statistics and engineering since its inception. In this paper we find new results and new connections between random matrices, information theory and physics. The contributions of this paper are: 1) Recursive formula to evaluate the expectation of the first moment (correlation function) of the characteristic polynomial of non-Hermitian matrix 2) It is shown that expectation of an expression of the form E H det(I + αH) with Gaussian Unitary Ensemble (GUE) satisfies KP (Kadomtsev-Petviashvili) equations. Same is true for Wishart matrices 3) Connection between expected value of the expression of form, E W det(I + αW), (where W is Wishart matrix) and Toda lattice equation. Besides, we discuss some beautiful connections established in [31,32,33] between random matrices and Painleve differential equations and Toda lattice equations.
We give an overview of random matrix theory (RMT) with the objective of highlighting the results and concepts that have a growing impact in the formulation and inference of statistical models and methodologies. This paper focuses on a number of application areas especially within the field of high-dimensional statistics and describes how the development of the theory and practice in high-dimensional statistical inference has been influenced by the corresponding developments in the field of RMT.
The distribution function of the level spacings for a random matrix in the limit of large dimensions is expressed by means of a rapidly converging infinite product which has been used for a numerical calculation. Comparison with Wigner's hypothesis gives a very good agreement.
Numerous problems in numerical analysis, including matrix inversion, eigenvalue calculations and polynomial zerofinding, share the following property: The difficulty of solving a given problem is large when the distance from that problem to the nearest "ill-posed" one is small. For example, the closer a matrix is to the set of noninvertible matrices, the larger its condition number with respect to inversion. We show that the sets of ill-posed problems for matrix inversion, eigenproblems, and polynomial zerofinding all have a common algebraic and geometric structure which lets us compute the probability distribution of the distance from a "random" problem to the set. From this probability distribution we derive, for example, the distribution of the condition number of a random matrix. We examine the relevance of this theory to the analysis and construction of numerical algorithms destined to be run in finite precision arithmetic.
The size distribution of level spacings in the region of compound nucleus excitation energies of the order of the neutron binding energy is considered. By analyzing available data derived by neutron spectroscopy, it is shown that the actual size distribution of level spacings qualitatively differs from random distribution. The relative number of near-lying levels is considerably smaller than for a random distribution. The conclusion is drawn that nuclear levels “repel” each other with a distribution approaching equidistance. This conclusion is based on experimental data relating mainly to odd-mass target nuclei. Assuming naturally that only equal spin levels interact, the observed “repulsion” may prove to be less pronounced owing to overlapping of the two sets of levels.
This paper is twofold. On the one hand, a short introduction is given to noncommutative random variables and a concise review of some areas of D. Voiculescu’s analysis is presented, especially concentrating on the relation to random matrices [Common. Math. Phys. 155, No. 1, 71-92 (1993; Zbl 0781.60006); Invent. Math. 118, No. 3, 411-440 (1994; Zbl 0820.60001)]. The main goal is to show that the negative logarithmic energy shares some properties with classical entropy functionals. On the other hand, we present a large deviation theorem for the empirical eigenvalue distribution of some not necessarily selfadjoint Gaussian random matrices with a full proof. This extends the first large deviation result due to G. Ben Arous and A. Guionnet for selfadjoint Gaussian matrices [Large deviations for Wigner’s law and Voiculescu’s non-commutative entropy. Probab. Theory Relat. Fields 108, No. 4, 517-542 (1997; Zbl 0970.58197)].
This self-contained account of the main results in large deviation theory includes recent developments and emphasizes the Freidlin-Wentzell results on small random perturbations. Metastability is described on physical grounds, followed by the development of more exacting approaches to its description. The first part of the book then develops such pertinent tools as the theory of large deviations which is used to provide a physically relevant dynamical description of metastability. Written for graduate students, this book affords an excellent route into contemporary research as well.
The author shows how an easy-to-state problem in combinatorics is accessible by methods in the subject of explicitly solvable differential equations. In so doing, he indicates connections among partitions, Painlevé’s equations, random matrices, the zeros of the Riemann zeta-function, and the method of steepest descent.
New kinds of statistical ensemble are defined, representing a mathematical idealization of the notion of ``all physical systems with equal probability.'' Three such ensembles are studied in detail, based mathematically upon the orthogonal, unitary, and symplectic groups. The orthogonal ensemble is relevant in most practical circumstances, the unitary ensemble applies only when time‐reversal invariance is violated, and the symplectic ensemble applies only to odd‐spin systems without rotational symmetry. The probability‐distributions for the energy levels are calculated in the three cases. Repulsion between neighboring levels is strongest in the symplectic ensemble and weakest in the orthogonal ensemble. An exact mathematical correspondence is found between these eigenvalue distributions and the statistical mechanics of a one‐dimensional classical Coulomb gas at three different temperatures. An unproved conjecture is put forward, expressing the thermodynamic variables of the Coulomb gas in closed analytic form as functions of temperature. By means of general group‐theoretical arguments, the conjecture is proved for the three temperatures which are directly relevant to the eigenvalue distribution problem. The electrostatic analog is exploited in order to deduce precise statements concerning the entropy, or degree of irregularity, of the eigenvalue distributions. Comparison of the theory with experimental data will be made in a subsequent paper.
We investigate the use of multiple transmitting and/or receiving antennas for single user communications over the additive Gaussian channel with and without fading. We derive formulas for the capacities and error exponents of such channels, and describe computational procedures to evaluate such formulas. We show that the potential gains of such multi-antenna systems over single-antenna systems is rather large under independenceassumptions for the fades and noises at different receiving antennas.
This paper introduces a unified framework for the detection of a single source with a sensor array in the context where the noise variance and the channel between the source and the sensors are unknown at the receiver. The Generalized Maximum Likelihood Test is studied and yields the analysis of the ratio between the maximum eigenvalue of the sampled covariance matrix and its normalized trace. Using recent results from random matrix theory, a practical way to evaluate the threshold and the
p
-value of the test is provided in the asymptotic regime where the number
K
of sensors and the number
N
of observations per sensor are large but have the same order of magnitude. The theoretical performance of the test is then analyzed in terms of Receiver Operating Characteristic (ROC) curve. It is, in particular, proved that both Type I and Type II error probabilities converge to zero exponentially as the dimensions increase at the same rate, and closed-form expressions are provided for the error exponents. These theoretical results rely on a precise description of the large deviations of the largest eigenvalue of spiked random matrix models, and establish that the presented test asymptotically outperforms the popular test based on the condition number of the sampled covariance matrix.
Let K1, K2,... be a sequence of regular graphs with degree v⩾2 such that n(Xi)→∞ and ck(Xi)/n(Xi)→0 as i∞ for each k⩾3, where n(Xi) is the order of Xi, and ck(Xi) is the number of k- cycles in X1. We determine the limiting probability density f(x) for the eigenvalues of X>i as i→∞. It turns out that for ∦x∦⩽2, otherwise It is further shown that f(x) is the expected eigenvalue distribution for every large randomly chosen labeled regular graph with degree v.
Using mathematical tools developed by Hermann Weyl, the Wigner classification of group?representations and co?representations is clarified and extended. The three types of representation, and the three types of co?representation, are shown to be directly related to the three types of division algebra with real coefficients, namely, the real numbers, complex numbers, and quaternions. The author's theory of matrix ensembles, in which again three possible types were found, is shown to be in exact correspondence with the Wigner classification of co?representations. In particular, it is proved that the most general kind of matrix ensemble, defined with a symmetry group which may be completely arbitrary, reduces to a direct product of independent irreducible ensembles each of which belongs to one of the three known types.
This is a tutorial on some basic non-asymptotic methods and concepts in
random matrix theory. The reader will learn several tools for the analysis of
the extreme singular values of random matrices with independent rows or
columns. Many of these methods sprung off from the development of geometric
functional analysis since the 1970's. They have applications in several fields,
most notably in theoretical computer science, statistics and signal processing.
A few basic applications are covered in this text, particularly for the problem
of estimating covariance matrices in statistics and for validating
probabilistic constructions of measurement matrices in compressed sensing.
These notes are written particularly for graduate students and beginning
researchers in different areas, including functional analysts, probabilists,
theoretical statisticians, electrical engineers, and theoretical computer
scientists.
In this paper two closely related heuristic principles of test construction (to be explained in Section 3), called Type I and Type II methods, of which Type II is identified with the usual likelihood ratio method, are noticed as underlying most of the classical tests of hypotheses, simple or composite, on means of univariate normal populations, and on total or partial correlations or regressions in the case of multinormal variates. In these situations the two methods are found to lead to identical tests having properties which happen to be very good in certain cases and moderately good in others. For certain types of composite hypotheses an extension is then made of the Type I method which is applied to construct tests of three different classes of hypotheses on multinormal populations (so as to cover, between them, a very large area of multivariate analysis), yielding in each case a test in general different from the corresponding and current likelihood ratio test. In each case, however, the two tests happen to come out identical for some degenerate "degrees of freedom." In contrast to the likelihood ratio test it is found that in these cases, for general "degrees of freedom," the corresponding Type I test is much easier to use on small samples, because of the relatively greater simplicity of the corresponding small sample distribution problem under the null hypothesis. In each case a lower bound of the power function of the Type I test is also given (against all relevant alternatives), anything like which, so far as the author is aware, would be far more difficult to obtain for the Type II tests in these situations. In this paper the general approach to the two methods is entirely of a heuristic nature except that, under fairly wide conditions, a lower bound to the power functions for each of the two types of tests is indicated to be readily available, which, however, is much too crude or wide a bound in general.
The distribution of the latent roots of the covariance matrix calculated from a sample from a normal multivariate population, was found by Fisher [3], Hsu [6] and Roy [10] for the special, but important case when the population covariance matrix is a scalar matrix, . By use of the representation theory of the linear group, we are able to obtain the general distribution for arbitrary .
The paper is largely expository, but some new results are included to round out the paper and bring it up to date. The following distributions are quoted in Section 7. 1. Type , exponential: (i) , (ii) Wishart, (iii) latent roots of the covariance matrix. 2. Type , binomial series: (i) variance ratio, F, (ii) latent roots with unequal population covariance matrices. 3. Type , Bessel: (i) noncentral , (ii) noncentral Wishart, (iii) noncentral means with known covariance. 4. Type , confluent hypergeometric: (i) noncentral F, (ii) noncentral multivariate F, (iii) noncentral latent roots. 5. Type , Gaussian hypergeometric: (i) multiple correlation coefficient, (ii) canonical correlation coefficients. The modifications required for the corresponding distributions derived from the complex normal distribution are outlined in Section 8, and the distributions are listed. The hypergeometric functions of matrix argument which occur in the multivariate distributions are defined in Section 4 by their expansions in zonal polynomials as defined in Section 5. Important properties of zonal polynomials and hypergeometric functions are quoted in Section 6. Formulae and methods for the calculation of zonal polynomials are given in Section 9 and the zonal polynomials up to degree 6 are given in the appendix. The distribution of quadratic forms is discussed in Section 10, orthogonal expansions of and in Laguerre polynomials in Section 11 and the asymptotic expansion of in Section 12. Section 13 has some formulae for moments.
Let x(1) denote the square of the largest singular value of an n × p matrix X, all of whose entries are independent standard Gaussian variates. Equivalently, x(1) is the largest principal component variance of the covariance matrix , or the largest eigenvalue of a pvariate Wishart distribution on n degrees of freedom with identity covariance. ¶ Consider the limit of large p and n with . When centered by and scaled by , the distribution of x(1) approaches the Tracey-Widom law of order 1, which is defined in terms of the Painlevé II differential equation and can be numerically evaluated and tabulated in software. Simulations show the approximation to be informative for n and p as small as 5. ¶ The limit is derived via a corresponding result for complex Wishart matrices using methods from random matrix theory. The result suggests that some aspects of large p multivariate distribution theory may be easier to apply in practice than their fixed p counterparts.
Dynamic resource allocation is an important means to increase the
sum capacity of fading multiple-access channels (MACs). In this paper,
we consider vector multi-access channels (channels where each user has
multiple degrees of freedom) and study the effect of power allocation as
a function of the channel state on the sum capacity (or spectral
efficiency) defined as the maximum sum of rates of users per unit degree
of freedom at which the users can jointly transmit reliably, in an
information-theoretic sense, assuming random directions of received
signal. Direct-sequence code-division multiple-access (DS-CDMA) channels
and MACs with multiple antennas at the receiver are two systems that
fall under the model. Our main result is the identification of a simple
dynamic power-allocation scheme that is optimal in a large system, i.e.,
with a large number of users and a correspondingly large number of
degrees of freedom. A key feature of this policy is that, for any user,
it depends on the instantaneous amplitude of channel state of that user
alone and the structure of the policy is “water-filling.” In
the contest of DS-CDMA and in the special case of no fading, the
asymptotically optimal power policy of water-filling simplifies to
constant power allocation over all realizations of signature sequences;
this result verifies the conjecture made in Verdu and Shamai (1999). We
study the behavior of the asymptotically optimal water-filling policy in
various regimes of number of users per unit degree of freedom and
signal-to-noise ratio (SNR). We also generalize this result to multiple
classes, i.e., the situation when users in different classes have
different average power constraints
In this note, we give the exact distribution of a scaled condition number used by Demmel to model the probability that matrix inversion is difficult. Specifically, consider a random matrix A and the scaled condition number D (A) = kAk F Delta kA Gamma1 k: Demmel provided bounds for the condition number distribution when A has real or complex normally distributed elements. Here, we give the exact formula. Key words and phrases: condition number, ill-conditioning, multivariate statistics, numerical analysis, random matrix This work was started while the author was a Visiting Scholar in the Department of Mathematics, MIT, and completed at CERFACS with the support of the North Atlantic Treaty Organization under a Grant awarded in 1989. 1 Introduction and statement of results In [4], Demmel investigates the probability that numerical analysis problems are difficult by unifying the common algebraic and geometric structures underlying the notion of ill-conditioning. As an applicati...
The genus series for maps is the generating series for the number of rooted maps with a given number of vertices and faces of each degree, and a given number of edges. It captures topological information about surfaces, and appears in questions arising in statistical mechanics, topology, group rings, and certain aspects of free probability theory. An expression has been given previously for the genus series for maps in locally orientable surfaces in terms of zonal polynomials. The purpose of this paper is to derive an integral representation for the genus series. We then show how this can be used in conjunction with integration techniques to determine the genus series for monopoles in locally orientable surfaces. This complements the analogous result for monopoles in orientable surfaces previously obtained by Harer and Zagier. A conjecture, subsequently proved by Okounkov, is given for the evaluation of an expectation operator acting on the Jack symmetric function. it specialises to known results for Schur functions and zonal polynomials.
Multiple antennas can be used for increasing the amount of diversity or the number of degrees of freedom in wireless communication systems. In this paper, we propose the point of view that both types of gains can be simultaneously obtained for a given multiple antenna channel, but there is a fundamental tradeoff between how much of each any coding scheme can get. We give a simple characterization of the optimal tradeoff curve and use it to evaluate the performance of existing multiple antenna schemes.