Properties of cyclic subspace regression

Idaho State University, ID, USA
Journal of Multivariate Analysis (Impact Factor: 0.93). 03/2007; 98(3):625-637. DOI: 10.1016/j.jmva.2006.05.004
Source: RePEc


Various properties of the regression vector produced by cyclic subspace regression with regard to the meancentered linear regression equation are put forth. In particular, the subspace associated with the creation of is shown to contain a basis that maximizes certain covariances with respect to , the orthogonal projection of onto a specific subspace of the range of X. This basis is constructed. Moreover, this paper shows how the maximum covariance values effect the . Several alternative representations of are also developed. These representations show that is a modified version of the l-factor principal components regression vector , with the modification occurring by a nonorthogonal projection. Additionally, these representations enable prediction properties associated with to be explicitly identified. Finally, methods for choosing factors are spelled out.

Full-text preview

Available from:
  • Source
    • "as a new approach to the complex multivariate calibration 75 problem. CSR was shown to produce solutions of PCR, PLS, LS and 76 other related intermediate regressions with one algorithm [16] [17] [18]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This contribution introduces Elastic Component Regression (ECR) as an explorative data analysis method that utilizes a tuning parameter α ∈ [0,1] to supervise the X-matrix decomposition. It is demonstrated theoretically that the elastic component resulting from ECR coincides with principal components of PCA when α = 0 and also coincides with PLS components when α = 1. In this context, PCR and PLS occupy the two ends of ECR and α ∈ (0,1) will lead to an infinite number of transitional models which collectively uncover the model path from PCR to PLS. Therefore, the framework of ECR shows a natural progression from PCR to PLS and may help add some insight into their relationships in theory. The performance of ECR is investigated on a series of simulated datasets together with a real world near infrared dataset. (The source codes implementing ECR in MATLAB are freely available at
    Full-text · Article · Dec 2010 · Chemometrics and Intelligent Laboratory Systems
  • [Show abstract] [Hide abstract]
    ABSTRACT: In linear regression a relationship between a response variable y and a set of predictor variables x 1,…, x p is modeled as y = β0 + β1x 1 + … + βp x p + ε, where ε represents random error and the βi 's are constant coefficients to be estimated from n observations of y and the x i . Given n observations of these p + 1 variables, the method of cyclic subspace regression (CSR) can be used to provide a finite number of estimators of β in the linear model y = X β + ε. Among the estimators produced by CSR are the estimators produced by the methods of least squares (LS), principal components regression (PCR), and partial least squares (PLS). In this article, after careful consideration of the invariant subspaces of X t X, a new method of regression is developed. This method, which the authors call invariant subspace regression (ISR), uses a selection of subspaces within the invariant subspaces of X t X, to create estimators of β. These subspaces are identified and created by the use of a finite list of user supplied non zero constants. Since there are no constraints on these constants other than they be non zero, ISR is capable of producing an infinite number of β estimators. It is shown that by an identified choice of constants all estimators produced by CSR can be generated by ISR, thereby showing that ISR is a generalization of CSR. Moreover, it is shown that all ISR estimators can be produced by CSR applied to an identified modification of the y data. Finally, examples are given showing that there exist ISR estimators that perform better in predictions than the best of all CSR estimators. It is also shown that there exist constants that create ISR estimators which perform equal to or worse than all CSR estimators in predictions. However, in this situation the best ISR estimator, in terms of prediction, is the best PCR estimator.
    No preview · Article · Feb 2013 · Communication in Statistics- Theory and Methods