Page 1
arXiv:1102.1845v1 [math.NA] 9 Feb 2011
An improved method for the computation of
the MoorePenrose inverse matrix
Vasilios N. Katsikis
General Department of Mathematics
Technological Education Institute of Piraeus,
Aigaleo, 12244 Athens, Greece, vaskats@gmail.com
Dimitrios Pappas
Athanassios Petralias
Department of Statistics
Athens University of Economics and Business
76 Patission Str, 10434, Athens, Greece
pappdimitris@gmail.com, petralia@aueb.gr
February 10, 2011
Abstract
In this article we provide a fast computational method in order to
calculate the MoorePenrose inverse of singular square matrices and
of rectangular matrices. The proposed method proves to be much
faster and has significantly better accuracy than the already proposed
methods, while works for full and sparse matrices.
Keywords: MoorePenrose inverse matrix, tensorproduct matrix.
1Introduction
Let T be a n × n real matrix. It is known that when T is singular, then
its unique generalized inverse T†(known as the Moore Penrose inverse) is
1
Page 2
defined. In the case when T is a real m × n matrix, Penrose showed that
there is a unique matrix satisfying the four Penrose equations, called the
generalized inverse of T. A lot of work concerning generalized inverses has
been carried out, in finite and infinite dimension (e.g., [2, 8]).
In a recent article [6], the first two authors provided a new method for the
fast computation of the generalized inverse of full rank rectangular matrices
and of square matrices with at least one zero row or column. In order to reach
this goal, a special type of tensor product of two vectors was used, usually
defined in infinite dimensional Hilbert spaces. In this work we extend our
method so that it can be used in any kind of matrix, square or rectangular,
full rank or not. The numerical experiments show that the proposed method
is competitive in terms of accuracy and much faster than the commonly used
methods, and can be also used for large sparse matrices.
There are several methods for computing the MoorePenrose inverse matrix
(cf. [2]). One of the most commonly used methods is the Singular Value
Decomposition (SVD) method. This method is very accurate but also time
intensive since it requires a large amount of computational resources, espe
cially in the case of large matrices. On a recent work, Toutounian and Ataei
[11] presented an algorithm based on the conjugate GramSchmidt process
and the MoorePenrose inverse of partitioned matrices, the CGSMPi algo
rithm and they conclude that this algorithm is a robust and efficient tool
for computing the MoorePenrose inverse of large sparse and rank deficient
matrices.
In the present manuscript, we construct a very fast and reliable method
(see the qrginv function in the Appendix) in order to estimate the Moore
Penrose inverse matrix. The computational effort required for the qrginv
function (see Figure 1 and Tables 1, 4) in order to obtain the generalized
inverse is substantially lower, particularly for large matrices, compared to
those provided by the SVD and the method presented in [11] (CGSMPi al
gorithm). In addition, we obtain reliable and very accurate approximations
in each one of the tested cases (Tables 2, 3 and 4). In order to test this
algorithm, we have used random singular matrices (see subsection 4.1) as
well as a collection of singular test matrices (see subsection 4.2) with “large”
condition number (illconditioned matrices) from the Matrix Computation
Toolbox (see [9]). We also tested the proposed method on sparse matrices
from the Matrix Market collection [7] (see subsection 4.3). In what follows,
we make use of the highlevel language Matlab both for calculations of the
generalized inverse matrix, as well as for testing the reliability of the ob
2
Page 3
tained results. Specifically, the Matlab 7.4 (R2007a) Service Pack 3 version
of the software was used on an Intel Core i7 920 Processor system running
at 2.67GHz with 6 GB of RAM memory using the Windows XP Professional
64bit Operating System.
2 Preliminaries and notation
We shall denote by Rm×nthe linear space of all m × n real matrices. For
T ∈ Rm×n, the generalized inverse T†∈ Rn×m(known as the Moore Pen
rose inverse) is the unique matrix that satisfies the following four Penrose
equations:
TT†= (TT†)∗,T†T = (T†T)∗, TT†T = T,T†TT†= T†,
where T∗denotes the transpose matrix of T. The number r = dimR(T) is
called the rank of T and ? ,? denotes the usual innerproduct in Rn.
According to [10], for each x ∈ Rk, we consider the mapping
e ⊗ f : Rk→ Rkwith (e ⊗ f)(x) = ?x,e?f.
Assume that {e1,...,en} and {f1,...,fn} are two collections of orthonormal
vectors and linearly independent vectors of Rkwith n < k, respectively.
Then, every rankn operator T can be written in the form T =?n
We shall refer to this type of tensor product as the tensorproduct of the
collections {e1,...,en} and {f1,...,fn}. The adjoint operator T∗of T is the
rankn operator T∗=?n
The tensorproduct of two collections of vectors, as defined above, is a
linear operator, therefore, it has a corresponding matrix representation T.
We shall refer to this matrix T as the tensorproduct matrix of the given col
lections. In order to compute the MoorePenrose inverse of the corresponding
tensorproduct matrix, we use the following theorem,
Theorem 2.1 ([5], Theorem 3.2) Let H be a Hilbert space. If T =?n
fiis a rankn operator then its generalized inverse is also a rankn operator
and for each x ∈ H, it is defined by the relation
i=1ei⊗fi.
i=1fi⊗ ei.
i=1ei⊗
T†x =
n
?
i=1
λi(x)ei,
where the functions λiare the solution of an appropriately defined n×n linear
system.
3
Page 4
3 The computational method
In [6], based on theorem 1, the authors developed an algorithm (the ginv
function) for computing the generalized inverse of full rank matrices, and of
square matrices with at least one zero row or column and the rest of the
matrix full rank. In other words, our main concern was to calculate the
corresponding λiin the expansion
T†x =
n
?
i=1
λi(x)ei,
where {e1,...,en} are the first n vectors of the standard basis of Rk, in order
to provide the generalized inverse T†.
To extend this result for any kind of matrix, we will make use of the QR
factorization, as well as the reverse order law for generalized inverses. The
following proposition is a restatement of a part of R. Bouldin’s theorem [1]
which holds for operators and matrices (see also [3], [4]).
Proposition 3.1 Let A,B be bounded operators on H with closed range.
Then (AB)†= B†A†if and only if the following three conditions hold:
i) The range of AB is closed,
ii) A†A commutes with BB∗,
iii) BB†commutes with A∗A.
Using the above proposition, we have the following result, which can be found
also, in a similar form but with a different proof, in [2].
Proposition 3.2 Let A = QR be the QR factorization of A. Then, A†=
R†Q∗.
Proof: We must prove that the conditions of Bouldin’s theorem hold. The
first condition always holds, since in the case of matrices the range is always
closed. For the second condition, it is easy to see that since Q is a unitary
matrix, Q†= Q∗= Q−1and so
Q†QRR∗= Q−1QRR∗= IRR∗= RR∗I = RR∗Q†Q.
The third condition can be proved in a similar way.
Using the QR factorization, the matrix R is upper triangular but not nec
essarily of full rank. So a variant of the QR method must be used, the QR
with column pivoting as described in the following form from [12]:
4
Page 5
Theorem 3.3 ([12], Theorem 3.3.11) Let A ∈ Rn×mmatrix, with rank(A) =
r > 0. Then there exist matricesˆA,Q,R such thatˆA is obtained by A by
permuting its columns, Q ∈ Rn×nis orthogonal, R =
?
R11 R12
00
?
∈ Rn×m,
R11∈ Rr×ris nonsingular and upper triangular andˆA = QR.
Using the above theorem, we have that AP = QR, where P is a per
mutation matrix (therefore unitary). By proposition 3.2 we have that A†=
PR†Q∗.
To calculate the rank of R11, one needs only the number of its columns
that have at least one value above a tolerance level in absolute terms. This
tolerance is set equal to 10−5, which is also used by Toutounian and Ataei
[11], and turns out to provide accurate results. The implementation of all
the above ideas are presented in the qrginv function (see the Appendix).
4Numerical experiments
In this section we perform numerical experiments comparing Matlab’s pinv
function, Toutounian and Ataei’ s method [11] ( CGSMPi algorithm) and the
proposed method qrginv function. Testing of pinv, CGSMPi and qrginv
was performed separately for random singular matrices and for singular test
matrices with “large” condition number from the Matrix Computation Tool
box (see [9]). We also tested the proposed method in sparse matrices and we
obtained very fast and accurate results.
4.1 Random singular matrices
We are comparing the performance of the proposed method (qrginv)to that
of the SVD method used by Matlab (pinv) and the method proposed in
[11] (CGSMPi algorithm), on a series of random singular matrices with rank
2n, n = 8,9,10,11,12. In addition, the accuracy of the results is examined
with the matrix 2norm in error matrices corresponding to the four properties
characterizing the MoorePenrose inverse.
In Table 1 we can see the time efficiency for the same matrices of the
proposed method, the CGSMPi algorithm and the pinv command used by
Matlab. The qrginv method needs about 4% up to 11% the corresponding
time needed by the pinv function to calculate the MoorePenrose inverse,
5
Page 6
Table 1: Computational Time Results; Random singular matrices
pinvCGSMPi qrginv Rank
28
29
210
211
212
0.496
1.185
14.066
152.384
1081.81
1.281
15.183
448.172
11374.47

0.0247
0.104
1.556
9.259
45.061
Notes: Time is measured in seconds. CGSMPi
algorithm was not able to produce numerical
results for a rank 212matrix, even after 3 days.
depending on the matrix. On the other hand the CGSMPi turns to be com
putantionally demanding requiring from 50 times up to more than 1200 times
the corresponding time needed by qrginv. Furthermore, the larger the rank
of the matrix, the greater this difference, so that for a matrix with rank 212,
the CGSMPi algorithm was not able to produce a numerical result, even after
3 days running, while qrginv needs only up to 45 seconds.
In Table 2, the accuracy of the proposed method is tested based on the
2norm errors. It is evident that the proposed method (qrginv) produced a
reliable approximation in all the tests that were conducted. The associated
errors are greatly lower than the corresponding of the other methods, while in
certain (many) cases are equal to zero, under the 64bit IEEE double precision
arithmetic in Matlab. Therefore, the proposed method allows us for a both
fast and accurate computation of the MoorePenrose inverse matrix.
4.2 Singular test matrices
In this section we use a set of singular test matrices that includes 13 singular
matrices, of size 200×200, obtained from the function matrix in the Matrix
Computation Toolbox [9](which includes test matrices from Matlab itself).
The condition number of these matrices range from order 1015to 10135. For
comparative purpose we also apply as in the previous section, Matlab’s pinv
function which implements the SVD method and CGSMPi algorithm. Since
these matrices are of relatively small size and so as to measure the time
needed for each algorithm to compute the MoorePenrose inverse accurately,
each algorithm runs 100 distinct times. The reported time is the mean time
over these 100 replications. The error results are presented in Table 3, while
the time responses are shown in Figure 1.
6
Page 7
Table 2: Error Results; Random singular matrices
Rank?TT†T − T?2
?T†TT†− T†?2
Method?TT†− (TT†)∗?2
?T†T − (T†T)∗?2
pinv
CGSMPi
qrginv
1.21 × 10−12
2.13 × 10−8
1.44 × 10−13
5.58 × 10−13
7.44 × 10−7
0
2.98 × 10−13
2.01 × 10−11
0
3.46 × 10−13
5.29 × 10−7
0
28
pinv
CGSMPi
qrginv
3.07 × 10−12
1.62 × 10−8
3.62 × 10−13
2.31 × 10−12
9.79 × 10−9
0
1.49 × 10−12
9.68 × 10−11
0
7.33 × 10−13
5.18 × 10−9
0
29
pinv
CGSMPi
qrginv
8.24 × 10−12
1.57 × 10−8
7.47 × 10−13
3.18 × 10−12
3.70 × 10−7
0
1.63 × 10−12
2.90 × 10−10
0
1.62 × 10−12
1.65 × 10−7
0
210
pinv
CGSMPi
qrginv
4.11 × 10−11
6.99 × 10−7
6.31 × 10−12
2.90 × 10−11
4.19 × 10−5
0
9.32 × 10−12
5.66 × 10−9
0
9.45 × 10−12
1.15 × 10−5
0
211
pinv
CGSMPi
qrginv
1.19 × 10−10

1.01 × 10−11
7.52 × 10−10

0
2.43 × 10−11

0
5.69 × 10−11

0
212
Notes: Zeros (0) denote numbers close to zero under 64bit IEEE double precision arithmetic. The CGSMPi
algorithm was not able to produce numerical results for matrix with rank 212even after 3 days running.
The results show a clear ordering of the three methods for this set of
test problems, with qrginv in the first place, followed by pinv and then
by CGSMPi algorithm. The worst cases for pinv, with comparatively large
errors for the ?T†TT†− T†?2norm, are the lotkin, prolate, hilb, and vand
matrices. The CGSMPi algorithm has very large errors in the cases of Kahan,
lotkin, prolate, hilb, magic and vand matrices, for all tested norms. Moreover,
the algorithm failed to produce a numerical result for the chow matrix, since
the computed MoorePenrose inverse included only NaNs. Nevertheless, we
observe that there are also cases (in certain norms) that the CGSMPi algo
rithm has smaller error than pinv.On the other hand, the proposed qrginv
method gives very accurate results for all matrices and proves to be overally
more efficient than the other two methods.
7
Page 8
48 13 19 24 3041 4352
0
1
2
3
4
5
6
7x 10−3
Matrix number in Matrix Computational Toolbox
Time in seconds
o : CGS− MPi
x : pinv
* : qrginv
Figure 1: Time efficiency for the pinv, qrginv functions and the CGSMPi
algorithm
4.3 Matrix Market sparse matrices
In this section we test the proposed algorithm on sparse matrices, from the
Matrix Market collection [7]. We follow the same method and the same
matrices as in [11], while the matrices are taken as rank deficient: A Z =
[A Z], where A is one of the chosen matrices, Z is a zero matrix of order
m × 100 and m takes values shown in Table 4, as in [11].
The proposed algorithm is tested against the CGSMPi algorithm, since
this method is proposed by Toutounian and Ataei [11] as suitable for large
and sparse matrices. The pinv function of Matlab is not applicable in sparse
matrices and thus ommited. We observe that in sparse matrices as well, the
proposed method seems to greatly outperform the CGSMPi algorithm, both
in terms of speed and accuracy.
5 Concluding Remarks
We proposed a new method for calculating the MoorePenrose inverse of sin
gular square, rectangular, full or sparse matrices. It is apparent that the
proposed method provides a substantially faster numerical way for calculat
ing the MoorePenrose inverse of a given matrix (see Figure 1 and Tables 1,
8
Page 9
Table 3: Error Results; Singular test matrices
Condition number Method?TT†T − T?2
Matrix?T†TT†− T†?2
?TT†− (TT†)∗?2
?T†T − (T†T)∗?2
chow8.01849 × 10135
pinv
4.741 × 10−13

1.691 × 10−13
1.896 × 10−13

0
2.313 × 10−13

0
2.261 × 10−13

0
(r=199) CGSMPi
qrginv
cycol2.05 × 1048
pinv
1.539 × 10−13
3.507 × 10−14
1.262 × 10−15
1.637 × 10−16
6.365 × 10−17
0
4.285 × 10−15
6.691 × 10−16
0
3.996 × 10−15
1.236 × 10−15
0
(r=50)CGSMPi
qrginv
gearmat3.504 × 1017
pinv
1.899 × 10−14
2.379 × 10−13
1.923 × 10−14
3.033 × 10−13
3.392 × 10−13
0
8.063 × 10−14
6.113 × 10−14
0
7.561 × 10−14
5.478 × 10−13
0
(r = 199) CGSMPi
qrginv
kahan2.30018 × 1024
pinv
1.432 × 10−13
4.011 × 10+18
6.964 × 10−15
0
4.921 × 10+35
0
0
262.069
0
9.070 × 10−14
5.765 × 10+18
0
(r = 199) CGSMPi
qrginv
lotkin8.97733 × 1021
pinv
0.0001
29.59 × 10+7
3.546 × 10−11
103.5 × 10+6
1.252 × 10+14
0
0.001
0
0
0.0011
74.5 × 10+6
0
(r = 19)CGSMPi
qrginv
prolate5.61627 × 1017
pinv
0.0002
2.218 × 10+15
6.588 × 10−11
7.398 × 109
7.885 × 10+30
0
0.0096
0.625
0
0.0081
4.796 × 10+15
0
(r = 117) CGSMPi
qrginv
hilb1.17164 × 1019
pinv
0.0001
39.6 × 10+5
6.941 × 10−12
3.184 × 109
2.811 × 10+12
0
0.0033
5.533 × 10−10
0
0.0051
30.95 × 10+5
0
(r = 20)CGSMPi
qrginv
magic4.92358 × 1019
pinv
0
1.755 × 10+18
4.566 × 10−14
2.491 × 10−19
8.333 × 10+19
0
9.701 × 10−14
293.29
0
4.236 × 10−14
6.329 × 10+14
0
(r = 3)CGSMPi
qrginv
vand1.16262 × 1020
pinv
0.0003
6.498 × 10+11
5.553 × 10−11
179.8 × 10+6
5.299 × 10+20
0
0.0022
0
0
0.0019
2.39 × 10+11
0
(r = 34)CGSMPi
qrginv
Notes:
parenthesis is denoted the rank (r) of each matrix. The CGSMPi algorithm was not able to produce
numerical results for the chow matrix.
Zeros (0) denote numbers close to zero under 64bit IEEE double precision arithmetic.In
4). Also, it is evident, from Tables 2, 3 and 4 that the proposed function
(qrginv)provides a far more reliable approximation in all the tests, compared
to other existing methods.
Appendix: Matlab code of the qrginv function
function qrginv = qrginv(B) [N,M] = size(B); [Q,R,P] = qr(B);
r=sum(any(abs(R)>1e5,2)); R1 = R(1:r,:); R2 = ginv(R1); R3 = [R2
zeros(M,Nr)]; A = P*R3*Q’; qrginv = A;
9
Page 10
Table 4: Error and Computational Time Results; Matrix Market sparse ma
trices
Matrix Time Method?TT†T − T?2
?T†TT†− T†?2
?TT†− (TT†)∗?2
?T†T − (T†T)∗?2
WELL1033 Z
0.0716
0.0111
CGSMPi
2.801 × 10−11
1.142 × 10−12
0
0
9.307 × 10−12
0
1.303 × 10−10
0
(m = 1033) qrginv
WELL1850 Z
0.4626
0.0315
CGSMPi
2.184 × 10−11
1.89 × 10−12
1.136 × 10−10
0
7.135 × 10−12
0
7.236 × 10−11
0
(m = 1850) qrginv
ILCC1850 Z
0.4546
0.0315
CGSMPi
61.106 × 10+4
5.809 × 10−11
1.687 × 10+9
0
2.637 × 10−10
0
50.971 × 10+4
0
(m = 1850) qrginv
GR3030 Z
0.9618
0.0336
CGSMPi
4.765 × 10−10
7.321 × 10−12
5.91 × 10−11
0
2.206 × 10−11
0
6.746 × 10−10
0
(m = 900)qrginv
WATT1 Z
1.2687
0.0064
CGSMPi
8
0
101.6 × 10+6
0
7.743 × 10−16
0
27.783
0
(m = 1856)qrginv
Notes:
parenthesis is denoted the row size of each matrix (m × 100). Time is measured in seconds.
Zeros (0) denote numbers close to zero under 64bit IEEE double precision arithmetic.In
In case the matrix of interest is sparse, the third line of the code is replaced
with
[Q,R,P] = spqr(B);
since the function qr embeded in Matlab does not support sparse matrices
under this format. The function spqr is part of the SuiteSparse toolbox, built
by Professor Timothy A. Davis, University of Florida and can be downloaded
electronically from http://www.cise.ufl.edu/research/sparse/SuiteSparse/.
References
[1] R. Bouldin, The pseudoinverse of a product, SIAM Journal on Applied
Mathematics,24:(4)(1973), 489–495.
[2] A. BenIsrael and T. N. E. Grenville, Generalized Inverses: Theory and
Applications, SpringerVerlag, Berlin 2002.
[3] T. N. E. Greville, Note on the generalized inverse of a matrix product,
SIAM Review, 8(1966), 518–521.
[4] S. Izumino, The product of operators with closed range and an extension
of the reverse order law, Tohoku Mathematical Journal, 34(1982), 43–52.
10
Page 11
[5] S. Karanasios and D. Pappas, Generalized inverses and special type
operator Algebras, Facta Universitatis(NIS), Series Mathematics and
Informatics, 21(2006), 41–48.
[6] V. N. Katsikis, D. Pappas, Fast computing of the MoorePenrose inverse
matrix, Electronic Journal of Linear Algebra, 17(2008), 637650.
[7] Matrix
Technology,
http://arxiv.org///math.nist.gov/MatrixMarket
Market, National Institute
MD.
of Standards
on
and
fromGaithersburg, Availableline
[8] M.A. Rakha, On the MoorePenrose generalized inverse matrix, Applied
Mathematics and Computation, 158(2004), 185200.
[9] N.J. Higham The Matrix Computation Toolbox, http://www.ma.man.
ac.uk/ higham/mctoolbox
[10] J. Ringrose. Compact non self adjoint operators. Van Nostrand London
1971.
[11] F. Toutounian, A. Ataei, A new method for computing MoorePenrose
inverse matrices, Journal of Computational and applied Mathematics,
228(2009), 412 417.
[12] D. Watkins, Fundamentals of Matrix Computations, WileyInterscience
New York 2002.
11