No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Abstract
Several basic properties of tensor nuclear norms are established in [S. Friedland and L.-H. Lim, Math. Comp., 87 (2018), pp. 1255–1281]. In this work, we give further studies on tensor nuclear norms. We present some special cases of tensor nuclear decompositions. We list some examples to show basic relationships among tensor rank, orthogonal rank and nuclear rank. Spectral and nuclear norms of Hermitian tensors are studied. We show that spectral and nuclear norms of real Hermitian decomposable tensors do not depend on the choice of base field. At last, we extend matrix polar decompositions to the tensor case, which is the product of a Hermitian tensor and a tensor whose spectral norm equals one. That is, we establish a link between any tensor and a Hermitian tensor. Bounds of nuclear rank are given based on tensor polar decompositions.
Tensor spectral -norms are generalizations of matrix induced norms. Matrix induced norms are an important type of matrix norms, and tensor spectral -norms are also important in applications. We discuss some basic properties of tensor spectral -norms. We extend the submultiplicativity of the matrix spectral 2-norm to the tensor case, based on which we give a bound of the tensor spectral 2-norm and provide a fast method for computing spectral 2-norms of sum-of-squares tensors. To compute tensor spectral -norms, we propose a higher order power method. Experiments show the high efficiency of the proposed methods and numerical results on spectral -norms of random tensors are also given.
The orthogonal decomposition factorizes a tensor into a sum of an orthogonal list of rank-one tensors. The corresponding rank is called orthogonal rank. We present several properties of orthogonal rank, which are different from those of tensor rank in many aspects. For instance, a subtensor may have a larger orthogonal rank than the whole tensor. To fit the orthogonal decomposition, we propose an algorithm based on the augmented Lagrangian method. The gradient of the objective function has a nice structure, inspiring us to use gradient-based optimization methods to solve it. We guarantee the orthogonality by a novel orthogonalization process. Numerical experiments show that the proposed method has a great advantage over the existing methods for strongly orthogonal decompositions in terms of the approximation error.
As is well known, the smallest possible ratio between the spectral norm and the Frobenius norm of an matrix with is and is (up to scalar scaling) attained only by matrices having pairwise orthonormal rows. In the present paper, the smallest possible ratio between spectral and Frobenius norms of tensors of order d, also called the best rank-one approximation ratio in the literature, is investigated. The exact value is not known for most configurations of . Using a natural definition of orthogonal tensors over the real field (resp. unitary tensors over the complex field), it is shown that the obvious lower bound is attained if and only if a tensor is orthogonal (resp. unitary) up to scaling. Whether or not orthogonal or unitary tensors exist depends on the dimensions and the field. A connection between the (non)existence of real orthogonal tensors of order three and the classical Hurwitz problem on composition algebras can be established: existence of orthogonal tensors of size is equivalent to the admissibility of the triple to Hurwitz problem. Some implications for higher-order tensors are then given. For instance, real orthogonal tensors of order do exist, but only when . In the complex case, the situation is more drastic: unitary tensors of size with exist only when . Finally, some numerical illustrations for spectral norm computation are presented.
We show that the two notions of entanglement: the maximum of the geometric measure of entanglement and the maximum of the nuclear norm is attained for the same states. We affirm the conjecture of Higuchi-Sudberry on the maximum entangled state of four qubits. We introduce the notion of d-density tensor for mixed d-partite states. We show that d-density tensor is separable if and only if its nuclear norm is 1. We suggest an alternating method for computing the nuclear norm of tensors. We apply the above results to symmetric tensors. We give many numerical examples.
In this paper we study multivariate polynomial functions in complex variables
and the corresponding associated symmetric tensor representations. The focus is
on finding conditions under which such complex polynomials/tensors always take
real values. We introduce the notion of symmetric conjugate forms and general
conjugate forms, and present characteristic conditions for such complex
polynomials to be real-valued. As applications of our results, we discuss the
relation between nonnegative polynomials and sums of squares in the context of
complex polynomials. Moreover, new notions of eigenvalues/eigenvectors for
complex tensors are introduced, extending properties from the Hermitian
matrices. Finally, we discuss an important property for symmetric tensors,
which states that the largest absolute value of eigenvalue of a symmetric real
tensor is equal to its largest singular value; the result is known as Banach's
theorem. We show that a similar result holds in the complex case as well.
Many problems can be formulated as recovering a low-rank tensor. Although an
increasingly common task, tensor recovery remains a challenging problem because
of the delicacy associated with the decomposition of higher order tensors. To
overcome these difficulties, existing approaches often proceed by unfolding
tensors into matrices and then apply techniques for matrix completion. We show
here that such matricization fails to exploit the tensor structure and may lead
to suboptimal procedure. More specifically, we investigate a convex
optimization approach to tensor completion by directly minimizing a tensor
nuclear norm and prove that this leads to an improved sample size requirement.
To establish our results, we develop a series of algebraic and probabilistic
techniques such as characterization of subdifferetial for tensor nuclear norm
and concentration inequalities for tensor martingales, which may be of
independent interests and could be useful in other tensor related problems.
Finding the rank of a tensor is a problem that has many applications.
Unfortunately it is often very difficult to determine the rank of a given
tensor. Inspired by the heuristics of convex relaxation, we consider the
nuclear norm instead of the rank of a tensor. We determine the nuclear norm of
various tensors of interest. Along the way, we also do a systematic study
various measures of orthogonality in tensor product spaces and we give a new
generalization of the Singular Value Decomposition to higher order tensors.
The Schmidt-Eckart-Young theorem for matrices states that the optimal rank-r approximation to a matrix is obtained by retaining the first r terms from the singular value decomposition of that matrix. This work considers a generalization of this optimal truncation property to the CANDECOMP/PARAFAC decomposition of tensors and establishes a necessary orthogonality condition. We prove that this condition is not satisfied at least by an open set of positive Lebesgue measure in complex tensor spaces. It is proved moreover that for complex tensors of small rank this condition can only be satisfied by a set of tensors of Lebesgue measure zero.
We discuss a technique that allows blind recovery of signals or blind
identification of mixtures in instances where such recovery or identification
were previously thought to be impossible: (i) closely located or highly
correlated sources in antenna array processing, (ii) highly correlated
spreading codes in CDMA radio communication, (iii) nearly dependent spectra in
fluorescent spectroscopy. This has important implications --- in the case of
antenna array processing, it allows for joint localization and extraction of
multiple sources from the measurement of a noisy mixture recorded on multiple
sensors in an entirely deterministic manner. In the case of CDMA, it allows the
possibility of having a number of users larger than the spreading gain. In the
case of fluorescent spectroscopy, it allows for detection of nearly identical
chemical constituents. The proposed technique involves the solution of a
bounded coherence low-rank multilinear approximation problem. We show that
bounded coherence allows us to establish existence and uniqueness of the
recovered solution. We will provide some statistical motivation for the
approximation problem and discuss greedy approximation bounds. To provide the
theoretical underpinnings for this technique, we develop a corresponding theory
of sparse separable decompositions of functions, including notions of rank and
nuclear norm that specialize to the usual ones for matrices and operators but
apply to also hypermatrices and tensors.
We show that a best rank one approximation to a real symmetric tensor, which
in principle can be nonsymmetric, can be chosen symmetric.
Furthermore, a symmetric best rank one approximation to a symmetric tensor is
unique if the tensor does not lie on a certain real algebraic variety.
The CANDECOMP/PARAFAC (CP) model decomposes a three-way array into a prespecified number of R factors and a residual array by minimizing the sum of squares of the latter. It is well known that an optimal solution for CP need not exist. We show that if an optimal CP solution does not exist, then any sequence of CP factors monotonically decreasing the CP criterion value to its infimum will exhibit the features of a so-called "degeneracy". That is, the parameter matrices become nearly rank deficient and the Euclidean norm of some factors tends to infinity. We also show that the CP criterion function does attain its infimum if one of the parameter matrices is constrained to be column-wise orthonormal.
The affine rank minimization problem consists of finding a matrix of minimum rank that satisfies a given system of linear equality constraints. Such problems have appeared in the literature of a diverse set of fields including system identification and control, Euclidean embedding, and collaborative filtering. Although specific instances can often be solved with specialized algorithms, the general affine rank minimization problem is NP-hard because it contains vector cardinality minimization as a special case.
In this paper, we show that if a certain restricted isometry property holds for the linear transformation defining the constraints, the minimum-rank solution can be recovered by solving a convex optimization problem, namely, the minimization of the nuclear norm over the given affine space. We present several random ensembles of equations where the restricted isometry property holds with overwhelming probability, provided the codimension of the subspace is sufficiently large.
The techniques used in our analysis have strong parallels in the compressed sensing framework. We discuss how affine rank minimization generalizes this preexisting concept and outline a dictionary relating concepts from cardinality minimization to those of rank minimization. We also discuss several algorithmic approaches to minimizing the nuclear norm and illustrate our results with numerical examples.
We establish several mathematical and computational properties of the nuclear norm for higher-order tensors. We show that like tensor rank, tensor nuclear norm is dependent on the choice of base field --- the value of the nuclear norm of a real 3-tensor depends on whether we regard it as a real 3-tensor or a complex 3-tensor with real entries. We show that every tensor has a nuclear norm attaining decomposition and every symmetric tensor has a symmetric nuclear norm attaining decomposition. There is a corresponding notion of nuclear rank that, unlike tensor rank, is upper semicontinuous. We establish an analogue of Banach's theorem for tensor spectral norm and Comon's conjecture for tensor rank --- for a symmetric tensor, its symmetric nuclear norm always equals its nuclear norm. We show that computing tensor nuclear norm is NP-hard in several sense. Deciding weak membership in the nuclear norm unit ball of 3-tensors is NP-hard, as is finding an -approximation of nuclear norm for 3-tensors. In addition, the problem of computing spectral or nuclear norm of a 4-tensor is NP-hard, even if we restrict the 4-tensor to be bi-Hermitian, bisymmetric, positive semidefinite, nonnegative valued, or all of the above. We discuss some simple polynomial-time approximation bounds. As an aside, we show that the nuclear (p,q)-norm of a matrix is NP-hard in general but can be computed in polynomial-time if p=1, , or p=q=2, with closed-form expressions for the nuclear (1,q)- and (p,1)-norms.
It is known that computing the spectral norm and the nuclear norm of a tensor is NP-hard in general. In this paper, we provide neat bounds for the spectral norm and the nuclear norm of a tensor based on tensor partitions. The spectral norm (respectively, the nuclear norm) can be lower and upper bounded by manipulating the spectral norms (respectively, the nuclear norms) of its subtensors. The bounds are sharp in general. When a tensor is partitioned into its matrix slices, our inequalities provide polynomial-time worst-case approximation bounds for computing the spectral norm and the nuclear norm of the tensor.
This paper studies nuclear norms of symmetric tensors. As recently shown by Friedland and Lim, the nuclear norm of a symmetric tensor can be achieved at a symmetric decomposition. We discuss how to compute symmetric tensor nuclear norms, depending on the tensor order and the ground field. Lasserre relaxations are proposed for the computation. The theoretical properties of the relaxations are studied. For symmetric tensors, we can compute their nuclear norms, as well as the nuclear decompositions. The proposed methods can be extended to nonsymmetric tensors.
We propose a theory of eigenvalues, eigenvectors, singular values, and singular vectors for tensors based on a constrained variational approach much like the Rayleigh quotient for symmetric matrix eigenvalues. These notions are particularly useful in generalizing certain areas where the spectral theory of matrices has traditionally played an important role. For illustration, we will discuss a multilinear generalization of the Perron-Frobenius theorem.
Linear algebra and matrix theory are fundamental tools in mathematical and physical science, as well as fertile fields for research. This second edition of this acclaimed text presents results of both classic and recent matrix analysis using canonical forms as a unifying theme and demonstrates their importance in a variety of applications. This thoroughly revised and updated second edition is a text for a second course on linear algebra and has more than 1,100 problems and exercises, new sections on the singular value and CS decompositions and the Weyr canonical form, expanded treatments of inverse problems and of block matrices, and much more.
For a 3-tensor of dimensions , we show that the
nuclear norm of its every matrix flattening is a lower bound of the tensor
nuclear norm, and which in turn is upper bounded by times the nuclear norm of the matrix flattening in mode j for all
j=1,2,3. The results can be generalized to N-tensors with any .
Both the lower and upper bounds for the tensor nuclear norm are sharp in the
case N=3. A computable criterion for the lower bound being tight is given as
well.
The singular value decomposition (SVD) has been extensively used in engineering and statistical applications. This method was originally discovered by Eckart and Young in [Psychometrika, 1 (1936), pp. 211--218], where they considered the problem of low-rank approximation to a matrix. A natural generalization of the SVD is the problem of low-rank approximation to high order tensors, which we call the multidimensional SVD. In this paper, we investigate certain properties of this decomposition as well as numerical algorithms.
We consider a problem of considerable practical interest: the recovery of a data matrix from a sampling of its entries. Suppose that we observe m entries selected uniformly at random from a matrix M. Can we complete the matrix and recover the entries that we have not seen?
We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries. We prove that if the number m of sampled entries obeys m ≥ C n^(1.2)r log n for some positive numerical constant C, then with very high probability, most n×n matrices of rank r can be perfectly recovered by solving a simple convex optimization program. This program finds the matrix with minimum nuclear norm that fits the data. The condition above assumes that the rank is not too large. However, if one replaces the 1.2 exponent with 1.25, then the result holds for all values of the rank. Similar results hold for arbitrary rectangular matrices as well. Our results are connected with the recent literature on compressed sensing, and show that objects other than signals and images can be perfectly reconstructed from very limited information.
This paper is concerned with the problem of recovering an unknown matrix from a small fraction of its entries. This is known as the matrix completion problem, and comes up in a great number of applications, including the famous Netflix Prize and other similar questions in collaborative filtering. In general, accurate recovery of a matrix from a small number of entries is impossible, but the knowledge that the unknown matrix has low rank radically changes this premise, making the search for solutions meaningful. This paper presents optimality results quantifying the minimum number of entries needed to recover a matrix of rank r exactly by any method whatsoever (the information theoretic limit). More importantly, the paper shows that, under certain incoherence assumptions on the singular vectors of the matrix, recovery is possible by solving a convenient convex program as soon as the number of entries is on the order of the information theoretic limit (up to logarithmic factors). This convex program simply finds, among all matrices consistent with the observed entries, that with minimum nuclear norm. As an example, we show that on the order of nr log( n ) samples are needed to recover a random n x n matrix of rank r by any method, and to be sure, nuclear norm minimization succeeds as soon as the number of entries is of the form nr polylog( n ).
A three-way array X (or three-dimensional matrix) is an array of numbers xijk subscripted by three indices. A triad is a multiplicative array, xijk = aibjck. Analogous to the rank and the row rank of a matrix, we define rank (X) to be the minimum number of triads whose sum is X, and dim1(X) to be the dimensionality of the space of matrices generated by the 1-slabs of X. (Rank and dim1 may not be equal.) We prove several lower bounds on rank. For example, a special case of Theorem 1 is that , where U and W are matrices; this generalizes a matrix theorem of Frobenius. We define the triple product [A, B, C] of three matrices to be the three-way array whose (i, j, k) element is given by ⩞rairbjrckr; in other words, the triple product is the sum of triads formed from the columns of A, B, and C. We prove several sufficient conditions for the factors of a triple product to be essentially unique. For example (see Theorem 4a), suppose , and each of the matrices has R columns. Suppose every set of rank (A) columns of A are independent, and similar conditions hold for B and C. Suppose rank (A) + rank (B) + rank (C) ⩾ 2R + 2. Then there exist diagonal matrices Λ, M, N and a permutation matrix P such that . Our results have applications to arithmetic complexity theory and to statistical models used in three-way multidimensional scaling.
In this paper we define the best rank-one approximation ratio of a tensor space. It turns out that in the finite dimensional case this provides an upper bound for the quotient of the residual of the best rankone approximation of any tensor in that tensor space and the norm of that tensor. This upper bound is strictly less than one, and it gives a convergence rate for the greedy rank-one update algorithm. For finite dimensional general tensor spaces, third order finite dimensional symmetric tensor spaces, and finite biquadratic tensor spaces, we give positive lower bounds for the best rank-one approximation ratio. For finite symmetric tensor spaces and finite dimensional biquadratic tensor spaces, we give upper bounds for this ratio.
We propose a theory of eigenvalues, eigenvectors, singular values, and singular vectors for tensors based on a constrained variational approach much like the Rayleigh quotient for symmetric matrix eigenvalues. These notions are particularly useful in generalizing certain areas where the spectral theory of matrices has traditionally played an important role. For illustration, we will discuss a multilinear generalization of the Perron-Frobenius theorem
. We explore the orthogonal decomposition of tensors (also known as multidimensional arrays or n-way arrays) using two di#erent definitions of orthogonality. We present numerous examples to illustrate the di#culties in understanding such decompositions. We conclude with a counterexample to a tensor extension of the Eckart--Young SVD approximation theorem by Leibovici and Sabatier [Linear Algebra Appl., 269 (1998), pp. 307--329]. Key words. tensor decomposition, singular value decomposition, principal components analysis, multidimensional arrays AMS subject classifications. 15A69, 49M27, 62H25 PII. S0895479800368354 1.
There has been continued interest in seeking a theorem describing optimal low-rank approximations to tensors of order 3 or higher, that parallels the Eckart-Young theorem for matrices. In this paper, we argue that the naive approach to this problem is doomed to failure because, unlike matrices, tensors of order 3 or higher can fail to have best rank-r approximations. The phenomenon is much more widespread than one might suspect: examples of this failure can be constructed over a wide range of dimensions, orders and ranks, regardless of the choice of norm (or even Bregman divergence). Moreover, we show that in many instances these counterexamples have positive volume: they cannot be regarded as isolated phenomena. In one extreme case, we exhibit a tensor space in which no rank-3 tensor has an optimal rank-2 approximation. The notable exceptions to this misbehavior are rank-1 tensors and order-2 tensors. In a more positive spirit, we propose a natural way of overcoming the ill-posedness of the low-rank approximation problem, by using weak solutions when true solutions do not exist. In our work we emphasize the importance of closely studying concrete low-dimensional examples as a first step towards more general results. To this end, we present a detailed analysis of equivalence classes of 2-by-2-by-2 tensors, and we develop methods for extending results upwards to higher orders and dimensions. Finally, we link our work to existing studies of tensors from an algebraic geometric point of view. The rank of a tensor can in theory be given a semialgebraic description; i.e., can be determined by a system of polynomial inequalities. In particular we make extensive use of the 2-by-2-by-2 hyperdeterminant.
Hermitian tensor and quantum mixed state
- G Ni
On the uniqueness of multilinear decomposition of N-way arrays
- N D Sidiropoulos
- R Bro