Gene H. Golub's research while affiliated with Stanford University and other places

Publications (344)

Article
Although generalized cross-validation is a popular tool for calculating a regularization parameter, it has been rarely applied to large-scale problems until recently. A major difficulty lies in the evaluation of the cross-validation function that requires the calculation of the trace of an inverse matrix. In the last few years stochastic trace esti...
Article
Fifty years after the invention of the QR algorithm by John Francis and Vera Kublanovskaya we reconstruct the ideas and the influences that led to its genesis from the originators’ own recollections and their sources and give an account of some of its subsequent developments.
Article
Full-text available
A new method is proposed to solve an ellipsoid-constrained integer least squares (EILS) problem arising in communications. In this method, the LLL reduction, which is cast as a QRZ factorization of a matrix, is used to transform the original EILS problem to a reduced EILS problem, and then a search algorithm is proposed to solve the reduced EILS pr...
Article
We present a new fast algorithm for solving the generalized eigenvalue problem Tx = λSx, in which both T and S are real symmetric tridiagonal matrices and S is positive definite. A method for solving this problem is to compute a Cholesky factorization S = LLT and solve the equivalent symmetric standard eigenvalue problem L-1TL-T (L T x) = ?(LT x)....
Article
Generalized eigenvalue problems play a significant role in many applications. In this paper, continuous methods are presented to compute generalized eigenvalues and their corresponding eigenvectors for two real symmetric matrices. Our study only requires that the right-hand-side matrix is positive semi-definite. The main idea of our continuous meth...
Article
As an application of the symmetric-triangular (ST) decomposition given by Golub and Yuan (2001) and Strang (2003), three block ST preconditioners are discussed here for saddle point problems. All three preconditioners transform saddle point problems into a symmetric and positive definite system. The condition number of the three symmetric and posit...
Article
Full-text available
Xiao-Wen Chang and Chris Paige dedicate this in memory of their warm, generous and inspirational friend Gene Golub. Abstract. Given an approximate solution to a data least squares (DLS) problem, we would like to know its minimal backward error. Here we derive formulas for what we call an "extended" minimal backward error, which is at worst a lower...
Article
Full-text available
The simultaneous solution of Ax = b and AT y = g, where A is a non-singular matrix, is required in a number of situations. Darmofal and Lu have proposed a method based on the Quasi-Minimal Residual algo- rithm (QMR). We will introduce a technique for the same purpose based on the LSQR method and show how its performance can be improved when using t...
Article
We describe the use of a higher-order singular value decomposition (HOSVD) in transforming a data tensor of genes × “x-settings,” that is, different settings of the experimental variable x × “y-settings,” which tabulates DNA microarray data from different studies, to a “core tensor” of “eigenarrays” × “x-eigengenes” × “y-eigengenes.” Reformulating...
Article
We further generalize the technique for constructing the Hermitian/skew-Hermitian splitting (HSS) iteration method for solving large sparse non-Hermitian positive definite system of linear equations to the normal/skew-Hermitian (NS) splitting obtaining a class of normal/skew-Hermitian splitting (NSS) iteration methods. Theoretical analyses show tha...
Article
Full-text available
The generalized eigenvalue problem [(H)\tilde] y =lH y\widetilde H y \,{=}\, \lambda H y with H a Hankel matrix and [(H)\tilde]\widetilde H the corresponding shifted Hankel matrix occurs in number of applications such as the reconstruction of the shape of a polygon from its moments, the determination of abscissa of quadrature formulas, of poles of...
Article
The basic goal of an inverse eigenvalue problem is to reconstruct the physical parameters of a certain system from the knowledge or desire of its dynamical behavior. Depending on the application, inverse eigenvalue problems appear in many different forms. This book discusses the fundamental questions, some known results, many applications, mathemat...
Article
Full-text available
The three results on the PageRank vector are preliminary but shed light on the eigenstructure of a PageRank modified Markov chain and what happens when changing the teleportation parameter in the PageRank model. Computations with the derivative of the PageRank vector with respect to the teleportation parameter show predictive ability and identify a...
Article
The PageRank model pioneered by Google is the most common approach for generating web search results. We present a two-stage algorithm for computing the PageRank vector where the algorithm exploits the lumpability of the underlying Markov chain. We make three contributions. First, the algorithm speeds up the PageRank calculation significantly. With...
Article
Full-text available
Abstract For the non-Hermitian and positive semidefinite systems of linear equations, we derive sucient,and necessary conditions for guaranteeing the unconditional convergence of the preconditioned Hermitian and skew-Hermitian splitting iteration methods. These result is specifically applied to linear systems of block tridiagonal form to obtain con...
Article
Full-text available
The convergence features of a preconditioned algorithm for the convection-diffusion equation based on its diffusion part are considered. Analyses of the distribution of the eigenvalues of the preconditioned matrix in arbitrary dimensions and of the fundamental parameters of convergence are provided, showing the existence of a proper cluster of eige...
Article
One of the most powerful iterative schemes for solving symmetric, positive definite linear systems is the conjugate gradient algorithm of Hestenes and Stiefel [J. Res. Nat. Bur. Standards, 49 (1952), pp. 409-435], especially when it is combined with preconditioning (cf. [P. Concus, G.H. Golub, and D.P. O'Leary, in Proceedings of the Symposium on Sp...
Article
We consider the problem of computing PageRank. The matrix involved is large and cannot be factored, and hence techniques based on matrix-vector products must be applied. A variant of the restarted refined Arnoldi method is proposed, which does not involve Ritz value computations. Numerical examples illustrate the performance and convergence behavio...
Article
We describe the singular value decomposition (SVD) of yeast genome-scale mRNA lengths distribution data measured by DNA microarrays. SVD uncovers in the mRNA abundance levels data matrix of genes × arrays, i.e., electrophoretic gel migration lengths or mRNA lengths, mathematically unique decorrelated and decoupled “eigengenes.” The eigengenes are t...
Article
We establish a class of accelerated Hermitian and skew-Hermitian splitting (AHSS) iteration methods for large sparse saddle-point problems by making use of the Hermitian and skew-Hermitian splitting (HSS) iteration technique. These methods involve two iteration parameters whose special choices can recover the known preconditioned HSS iteration meth...
Article
In this paper, continuous methods are introduced to compute both the extreme and interior eigenvalues and their corresponding eigenvectors for real symmetric matrices. The main idea is to convert the extreme and interior eigenvalue problems into some optimization problems. Then a continuous method which includes both a merit function and an ordinar...
Article
In this paper we will adapt a known method for diagonal scaling of symmetric positive definite tridiagonal matrices towards the semiseparable case. Based on the fact that a symmetric, positive definite tridiagonal matrix TT satisfies property A, one can easily construct a diagonal matrix [^(D)]\hat{D} such that [^(D)]T[^(D)]\hat{D}T\hat{D} has the...
Article
In previous work we introduced a construction to produce biorthogonal multiresolutions from given subdivisions. The approach involved estimating the solution to a least squares problem by means of a number of smaller least squares approximations on local portions of the data. In this work we use a result by Dahlquist, et al. on the method of averag...
Article
We describe the use of the matrix eigenvalue decomposition (EVD) and pseudoinverse projection and a tensor higher-order EVD (HOEVD) in reconstructing the pathways that compose a cellular system from genome-scale nondirectional networks of correlations among the genes of the system. The EVD formulates a genes × genes network as a linear superpositio...
Article
Full-text available
For iterative solution of saddle point problems, a nonsymmetric preconditioning is studied which, with respect to the upper-left block of the system matrix, can be seen as a variant of SSOR. An idealized situation where the SSOR is taken with respect to the skew-symmetric part plus the diagonal part of the upper-left block is analyzed in detail. Si...
Article
Full-text available
Abstract The optimal parameter of the Hermitian/skew-Hermitian splitting (HSS) iteration method for a real 2-by-2 linear system is obtained. The result is used to determine the optimal parameters for linear systems associated with certain 2-by-2 block matrices, and to estimate the optimal parameters of the HSS iteration method for linear systems wi...
Article
In this paper a generalized eigenvalue problem for nonsquare pencils of the form A-λB with A, B ∈ Cm×n and m > n, which was proposed recently by Boutry, Elad, Golub, and Milanfar [SIAM J. Matrix Anal. Appl, 27 (2006), pp. 582-601], is studied. An algebraic characterization for the distance between the pair (A, B) and the pairs (A0,B0) with the prop...
Article
Full-text available
For the iterative solution of saddle point problems, a nonsymmetric preconditioner is studied which, with respect to the upper-left block of the system matrix, can be seen as a variant of SSOR. An idealized situation where SSOR is taken with respect to the skew-symmetric part plus the diagonal part of the upper-left block is analyzed in detail. Sin...
Article
Full-text available
We study the role of preconditioning strategies recently developed for coercive problems in connection with a two-step iterative method, based on the Hermitian skew-Hermitian splitting (HSS) of the coefficient matrix, proposed by Bai, Golub and Ng for the solution of nonsymmetric linear systems whose real part is coercive. As a model problem we con...
Article
Full-text available
Currently there is a growing interest in semiseparable matrices and generalized semiseparable matrices. To gain an appreciation of the historical evolution of this concept, we present in this paper an extensive list of publications related to the field of semiseparable matrices. It is interesting to see that semiseparable matrices were investigated...
Article
Full-text available
Large linear systems of saddle point type arise in a wide variety of applications throughout computational science and engineering. Due to their indefiniteness and often poor spectral properties, such linear systems represent a significant challenge for solver developers. In recent years there has been a surge of interest in saddle point problems,...
Article
Recent work on block-based compression for low bit-rate coding has shown that employing a block coder within a sampling scheme where the image is downsampled prior to coding (and upsampled after the decoding stage) results in superior performance compared to standard block coding. We explore the use of optimal decimation and interpolation filters i...
Article
Full-text available
Gene expression data often contain missing expression values. Effective missing value estimation methods are needed since many algorithms for gene expression data analysis require a complete matrix of gene array values. In this paper, imputation methods based on the least squares formulation are proposed to estimate missing values in the gene expre...
Article
Full-text available
We study the role of preconditioning strategies recently developed for coercive problems in connection with a two-step iterative method (HSS) proposed by Bai, Golub and Ng for the solution of nonsymmetric linear systems whose real part is coercive. As a model problem we consider Finite Dierences (FD) matrix sequences fAn(a; p)gn discretizing the el...
Article
Full-text available
This work focuses on nonsquare matrix pencils A - λB, where A, B ∈ M m×n and m > n. Traditional methods for solving such nonsquare generalized eigenvalue problems (A -λB)v = 0 are expected to lead to no solutions in most cases. In this paper we propose a different treatment: We search for the minimal perturbation to the pair (A, B) such that these...
Article
Quadratic eigenvalue problems involving large matrices arise frequently in areas such as the vibration analysis of structures, micro-electro-mechanical systems (MEMS) simulation, and the solution of quadratically constrained least squares problems. The typical approach is to solve the quadratic eigenvalue problem using a mathematically equivalent l...
Article
Full-text available
By further generalizing the concept of Hermitian (or normal) and skew-Hermitian splitting for a non-Hermitian and positive-definite matrix, we introduce a new splitting, called positive-definite and skew-Hermitian (PS) splitting, and then establish a class of positivedefinite and skew-Hermitian splitting (PSS) methods similar to the Hermitian (or n...
Article
Full-text available
In shape reconstruction, the celebrated Fourier slice theorem plays an essential role. It allows one to reconstruct the shape of a quite general object from the knowledge of its Radon transform (S. Helgason, The Radon Transform, Birkhauser Boston, Boston, 1980)—in other words from the knowledge of projections of the object. In case the object is a...
Article
Full-text available
We consider a generic sequence of matrices (the nonnormal case is of interest) showing a proper cluster at zero in the sense of the singular values. By a direct use of the notion of majorizations, we show that the uniform spectral boundedness is sufficient for the proper clustering at zero of the eigenvalues: if the assumption of boundedness is rem...
Article
We consider a positive definite block preconditioner for solving saddle point linear systems. An approach based on augmenting the (1,1) block while keeping its condition number small is described, and algebraic analysis is performed. Ways of selecting the parameters involved are discussed, and analytical and numerical observations are given.
Article
We describe an integrative data-driven mathematical framework that formulates any number of genome-scale molecular biological data sets in terms of one chosen set of data samples, or of profiles extracted mathematically from data samples, designated the “basis” set. By using pseudoinverse projection, the molecular biological profiles of the data sa...
Conference Paper
Full-text available
Most multivariate statistical methods for gene expression data require a complete matrix of gene array values. In this paper, an imputation method based on least squares formulation is proposed to estimate missing values. It exploits local similarity structures in the data as well as least squares optimization process. The proposed local least squa...
Article
Full-text available
This paper discusses the problem of recovering a planar polygon from its measured complex moments. These moments correspond to an indicator function defined over the polygon's support. Previous work on this problem gave necessary and sufficient conditions for such successful recovery process and focused mainly on the case of exact measurements bein...
Article
The convergence features of a preconditioned algorithm for the convection-diffusion equation based on its diffusion part are considered. An analysis of the distribution of the eigenvalues of the preconditioned matrix and of the fundamental parameters of convergence are provided, showing the existence of a proper cluster of eigenvalues and the super...
Article
This paper presents a new computational approach for solving the Regularized Total Least Squares problem. The problem is formulated by adding a quadratic constraint to the Total Least Square minimization problem. Starting from the fact that a quadratically constrained Least Squares problem can be solved via a quadratic eigenvalue problem, an iterat...
Article
This paper discusses the problem of recovering a planar polygon from its measured complex moments. These moments correspond to an indicator function defined over the polygon's support. Previous work on this problem gave necessary and sufficient conditions for such successful recovery process and focused mainly on the case of exact measurements bein...
Article
the PageRank [16] vector. Our algorithm exploits the observation that the homogeneous discrete-time Markov chain associated with PageRank is lumpable [13]; the lumpable subset of nodes are precisely the dangling nodes. As a result the algorithm can converge in a fraction of the time compared to the standard PageRank algorithm [16]. On data of 451,2...
Article
Full-text available
In a previous work [2] we introduced a construction to produce biorthogonal multiresolutions from given subdivisions. The approach we employed could be interpreted as estimating the solution to a least squares problem by means of a number of smaller least squares approximations on local portions of the data. In this work we use a result by Dahlquis...
Article
We perform an algebraic analysis of a generalization of the augmented Lagrangian method for solution of saddle point linear systems. It is shown that in cases where the (1,1) block is singular, specifically semidefinite, a low-rank perturbation that minimizes the condition number of the perturbed matrix while maintaining sparsity is an e#ective app...
Article
Recent work on block-based compression for low bit-rate coding has shown that employing a block coder within a sampling scheme where the image is downsampled prior to coding (and upsampled after the decoding stage) results in superior performance compared to standard block coding. In this paper, we explore the use of optimal decimation and interpol...
Article
We study the asymptotic rate of convergence of the alternating Hermitian/skewHermitian iteration for solving saddle-point problems arising in the discretization of elliptic partial di#erential equations. By a careful analysis of the iterative scheme at the continuous level we determine optimal convergence parameters for the model problem of the Poi...
Article
Motivation: Gene expression microarray data sets often contain missing expression values. Robust missing value estimation methods are needed since many algorithms for gene expression analysis require a complete matrix of gene array values. In this paper, imputation methods based on the least squares and cluster structure are proposed to estimate mi...
Article
Full-text available
In this preliminary work, left and right conjugate direction vectors are defined for nonsymmetric, nonsingular matrices A and some properties of these vectors are studied. A left conjugate direction (LCD) method for solving nonsymmetric systems of linear equations is proposed. The method has no breakdown for real positive definite systems. The meth...
Article
Recent work on block-based compression for low bit-rate coding has shown that employing a block coder within a sampling scheme where the image is downsampled prior to coding (and upsampled after the decoding stage) results in superior performance compared to standard block coding. In this paper, we explore the use of optimal decimation and interpol...
Article
Full-text available
In this paper we consider the solution of linear systems of saddle point type by preconditioned Krylov subspace methods. A preconditioning strategy based on the symmetric\slash skew-symmetric splitting of the coefficient matrix is proposed, and some useful properties of the preconditioned matrix are established. The potential of this approach is il...
Article
We present an iterative algorithm (BIN) for scaling all the rows and columns of a real symmetric matrix to unit 2-norm. We study the theoretical convergence properties and its relation to optimal conditioning. Numerical experiments show that BIN requires 2–4 matrix–vector multiplications to obtain an adequate scaling, and in many cases significantl...
Article
Full-text available
Registration using the least-squares cost function is sensitive to the intensity fluctuations caused by the blood oxygen level dependent (BOLD) signal in functional MRI (fMRI) experiments, resulting in stimulus-correlated motion errors. These errors are severe enough to cause false-positive clusters in the activation maps of datasets acquired from...
Conference Paper
Block coders are among the most common compression tools available for still images and video sequences. Their low computational complexity along with their good performance make them a popular choice for compression of natural images. Yet, at low bit-rates, block coders introduce visually annoying artifacts into the image. One approach that allevi...
Article
Full-text available
A preconditioner for the convection-diusion equation based on the diusion part is considered. The algorithm is compared with a strategy based on a two-step iterative method (HSS) for the solution of nonsymmetric linear systems whose real part is coercive and with other preconditioned algorithms based on incomplete factorization techniques. The anal...
Article
Suppose the spectrum of a symmetric definite linear pencil is known. This paper addresses the question of what can be said about the spectrum when scalar multiples of a rank-one update are added to each matrix in the pencil.The secular equation for this problem is derived, and from it, a certain separation property is found which gives insight into...
Article
We present a novel technique for speeding up the computation of PageRank, a hyperlink-based estimate of the "importance" of Web pages, based on the ideas presented in [7]. The original PageRank algorithm uses the Power Method to compute successive iterates that converge to the principal eigenvector of the Markov matrix representing the Web link gra...
Article
Recent work on JPEG compression for low bit-rate coding has shown that employing a JPEG coder within a sampling scheme where the image is downsampled prior to coding (and upsampled after the decoding stage) results in superior performance compared to standard JPEG coding. In this paper, we explore the use of optimal decimation and interpolation fil...
Article
This work focuses on non-square matrix pencils A #B where A,B and m > n. Traditional methods for solving such non-square generalized eigenvalue problems (A-#B)v = 0 are expected to lead to no solutions in most cases. In this paper we propose a di#erent treatment, we search for the minimal perturbation on the pair B} such that these solutions are in...
Article
We observe that the convergence patterns of pages in the PageRank algorithm have a nonuniform distribution. Specifically, many pages converge to their true PageRank quickly, while relatively few pages take a much longer time to converge. Furthermore, we observe that these slow-converging pages are generally those pages with high PageRank. We use th...
Article
Full-text available
The web link graph has a nested block structure: the vast majority of hyperlinks link pages on a host to other pages on the same host, and many of those that do not link pages within the same domain. We show how to exploit this structure to speed up the computation of PageRank by a 3-stage algorithm whereby (1) the local PageRanks of pages for each...
Article
Full-text available
We present a novel algorithm for the fast computation of PageRank, a hyperlink-based estimate of the "importance" of Web pages. The original PageRank algorithm uses the Power Method to compute successive iterates that converge to the principal eigenvector of the Markov matrix representing the Web link graph. The algorithm presented here, called Qua...
Article
In this preliminary work, left and right conjugate direction vectors are defined for nonsymmetric, nonsingular matrices A and some properties of these vectors are studied. A left conjugate direction (LCD) method for solving general systems of # The work of the first author was partially supported by CNPq, CAPES, Fundacao Araucaria, Brazil. The work...
Article
In this paper we review 30 years of developments and applications of the variable projection method for solving separable nonlinear least-squares problems. These are problems for which the model function is a linear combination of nonlinear functions. Taking advantage of this special structure, the method of variable projections eliminates the line...
Article
We study efficient iterative methods for the large sparse non-Hermitian positive definite system of linear equations based on the Hermitian and skew-Hermitian splitting of the coefficient matrix. These methods include a Hermitian/skew-Hermitian splitting (HSS) iteration and its inexact variant, the inexact Hermitian/skew-Hermitian splitting (IHSS)...
Article
Full-text available
In this paper we review 30 years of developments and applications of the variable projection method for solving separable nonlinear least-squares problems. These are problems for which the model function is a linear combination of nonlinear functions. Taking advantage of this special structure, the method of variable projections eliminates the line...
Article
We study ecient iterative methods for the large sparse non-Hermitian positive de nite system of linear equations based on the Hermitian and skew-Hermitian splitting of the coecient matrix. These methods include a Hermitian/skew-Hermitian splitting (HSS) iteration and its inexact variant, the inexact Hermitian/skew-Hermitian splitting (IHSS) iterati...
Conference Paper
We discuss the recovery of a planar polygon from its measured complex moments. Previous work on this problem gave necessary and sufficient conditions for such successful recovery and focused mainly on the case of exact measurements. This paper extended these results by treating the case where a longer than necessary series of noise corrupted moment...
Article
A new decomposition of a nonsingular matrix, the Symmetric times Triangular (ST) decomposition, is proposed. By this decomposition, every nonsingular matrix can be represented as a product of a symmetric matrix S and a triangular matrix T. Furthermore, S can be made positive definite. Two numerical algorithms for computing the ST decomposition with...