Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
A scree plot for choosing the number of factors. The y-axis shows the standardized singular valuesˆσvaluesˆ valuesˆσ k / √ N J, wherê σ k s are obtained from Step 7 of Algorithm 1. The data are simulated from an IFA model with K = 5, J = 200, and N = 4000. The input dimension is set to be 10 in Algorithm 1. A singular value gap can be found between the 5th and 6th singular values
Source publication
We revisit a singular value decomposition (SVD) algorithm given in Chen et al. (Psychometrika 84:124–146, 2019b) for exploratory item factor analysis (IFA). This algorithm estimates a multidimensional IFA model by SVD and was used to obtain a starting point for joint maximum likelihood estimation in Chen et al. (2019b). Thanks to the analytic and c...
Contexts in source publication
Context 1
... first run Algorithm 1, but replace the unknown K in Step 1 of the algorithm by a reasonably large number K † . Then, a scree plot can be obtained by plottingˆσplottingˆ plottingˆσ k in a descending order, forˆσforˆ forˆσ k s produced by Step 7 of Algorithm 1. Figure 1 shows such a scree plot, for which the data are generated from a five-factor model (K = 5) with J = 200 and N = 4000, and the input number of factors is set to be K † = 10 in Step 1 of the algorithm. Unsurprisingly, an obvious gap is observed betweenˆσbetweenˆ betweenˆσ 5 andˆσandˆ andˆσ 6 . ...
Context 2
... first run Algorithm 1, but replace the unknown K in Step 1 of the algorithm by a reasonably large number K † . Then, a scree plot can be obtained by plottingˆσplottingˆ plottingˆσ k in a descending order, forˆσforˆ forˆσ k s produced by Step 7 of Algorithm 1. Figure 1 shows such a scree plot, for which the data are generated from a five-factor model (K = 5) with J = 200 and N = 4000, and the input number of factors is set to be K † = 10 in Step 1 of the algorithm. Unsurprisingly, an obvious gap is observed betweenˆσbetweenˆ betweenˆσ 5 andˆσandˆ andˆσ 6 . ...
Similar publications
Singular Value Decomposition (SVD) and its close relative, Principal Component Analysis (PCA), are well-known linear matrix decomposition techniques that are widely used in applications such as dimension reduction and clustering. However, an important limitation of SVD/PCA is its sensitivity to noise in the input data. In this paper, we take anothe...
p>Feature extraction plays an important role in accurate preprocessing and real-world applications. High-dimensional features in the data have a significant impact on the machine learning classification system. Relevant feature extraction is a fundamental step not only to reduce the dimensionality but also to improve the performance of the classifi...
This study addresses the challenge of utilizing incomplete long-term discharge data when using gridded precipitation datasets and data-driven modeling in Iran’s Karkheh basin. The Multilayer Perceptron Neural Network (MLPNN), a rainfall-runoff (R-R) model, was applied, leveraging precipitation data from the Asian Precipitation—Highly Resolved Obser...
Nowadays, numerous attacks can be considered high risks in terms of the security of Wireless Sensor Networks (WSN). As a result, different applications are introduced to manage the data and information exchange and related security sides to be save in transmission of data. Recently, most of the security attacks are classified as cyber ones. These a...
This study presents the design and validation of a numerical method based on an AI-driven ROM framework for implementing stress virtual sensing. By leveraging Reduced-Order Models (ROMs), the research aims to develop a virtual stress transducer capable of the real-time monitoring of mechanical stresses in mechanical components previously analyzed w...
Citations
... In particular, variants of the Lanczos Algorithm (whose convergence also depends on gaps of subsequent singular values σ k+1 − σ k ) are found in standard numerical linear algebra programs in MATLAB, Mathematica, Scipy (python), and others through their dependency on Fortran's ARPACK library [24]. In particular, these methods and other variants are found in various implementations of principal component analysis (PCA) and singular value decomposition (SVD) algorithms [34,52], as well as neural network architectures and computations [26,36]. Power iteration also appears in recommendation engines from Twitter (X) and Google Search [16,20]. ...
We study the gaps between consecutive singular values of random rectangular matrices. Specifically, if M is an random matrix with independent and identically distributed entries and is a deterministic positive definite matrix, then under some technical assumptions we give lower bounds for the gaps between consecutive singular values of . As a consequence, we show that sample covariance matrices have simple spectrum with high probability. Our results resolve a conjecture of Vu [{\em Probab. Surv.}, 18:179--200, 2021]. We also discuss some applications, including a bound on the spacings of eigenvalues of the adjacency matrix of random bipartite graphs.
... We use the Adam optimizer (Kingma and Ba, 2014) configured with a variable "learning rate" parameter which is programmed to decrease from 0.05 to zero on a predetermined 30,000-step schedule. While SVI will often converge even if the posterior means of all variables are randomly initialized, we attempt to provide a more reasonable starting point for optimization by computing initial guesses of these using a fast IRT approximation (Zhang et al., 2020), resorting to random initialization only when this approach does not yield a solution. The posterior standard deviations of all variables are initialized to a fixed value of 0.1, which is the default value for mean-field normal posteriors in NumPyro. ...
We present an exploratory method for discovering likely misconceptions from multiple-choice concept test data, as well as preliminary evidence that this method recovers known misconceptions from real student responses. Our procedure is based on a Bayesian implementation of the Multidimensional Nominal Categories IRT model (MNCM) combined with standard factor-analytic rotation methods; by analyzing student responses at the level of individual distractors rather than at the level of entire questions, this approach is able to highlight multiple likely misconceptions for subsequent investigation without requiring any manual labeling of test content. We explore the performance of the Bayesian MNCM on synthetic data and find that it is able to recover multidimensional item parameters consistently at achievable sample sizes. These studies demonstrate the method's robustness to overfitting and ability to perform automatic dimensionality assessment and selection. The method also compares favorably to existing IRT software implementing marginal maximum likelihood estimation which we use as a validation benchmark. We then apply our method to approximately 10,000 students' responses to a research-designed concept test: the Force Concept Inventory. In addition to a broad first dimension strongly correlated with overall test score, we discover thirteen additional dimensions which load on smaller sets of distractors; we discuss two as examples, showing that these are consistent with already-known misconceptions in Newtonian mechanics. While work remains to validate our findings, our hope is that future applications of this method could aid in the refinement of existing concept inventories or the development of new ones, enable the discovery of previously-unknown student misconceptions across a variety of disciplines, and—by leveraging the method's ability to quantify the prevalence of particular misconceptions—provide opportunities for targeted instruction at both the individual and classroom level.
... share a similar spirit with those in Zhang et al. (2020), which proposed and evaluated an SVDbased approach for item factor analysis. In each simulation setting, we generate 100 independent replicates. ...
... When N ≥ 2000, the estimation accuracy of is slightly worse but comparable to JML. We point out that due to their non-iterative nature, SVD or eigendecomposition-based methods typically tend to give worse estimation compared to iterative methods that aim to find the MLE; for example, see the comparison of an SVD-based method and a joint MLE method for item factor analysis in Zhang et al. (2020). However, it turns out that given a fixed computational resource (i.e., the default maximum iteration number in the sirt package), the iterative method JML can give worse estimation accuracy than our spectral method. ...
Grade of membership (GoM) models are popular individual-level mixture models for multivariate categorical data. GoM allows each subject to have mixed memberships in multiple extreme latent profiles. Therefore, GoM models have a richer modeling capacity than latent class models that restrict each subject to belong to a single profile. The flexibility of GoM comes at the cost of more challenging identifiability and estimation problems. In this work, we propose a singular value decomposition (SVD)-based spectral approach to GoM analysis with multivariate binary responses. Our approach hinges on the observation that the expectation of the data matrix has a low-rank decomposition under a GoM model. For identifiability, we develop sufficient and almost necessary conditions for a notion of expectation identifiability. For estimation, we extract only a few leading singular vectors of the observed data matrix and exploit the simplex geometry of these vectors to estimate the mixed membership scores and other parameters. We also establish the consistency of our estimator in the double-asymptotic regime where both the number of subjects and the number of items grow to infinity. Our spectral method has a huge computational advantage over Bayesian or likelihood-based methods and is scalable to large-scale and high-dimensional data. Extensive simulation studies demonstrate the superior efficiency and accuracy of our method. We also illustrate our method by applying it to a personality test dataset.
... (1) The first stage obtains an "initial of initial" estimator (Z,α) as follows. A similar "double-SVD" idea has been introduced in Zhang, Chen and Li (2020). (a) Construct an estimatorE t for E ⋆ t by applying the universal singular value thresholding (Chatterjee, 2015) to each matrix A t . ...
We introduce a semiparametric latent space model for analyzing longitudinal network data. The model consists of a static latent space component and a time-varying node-specific baseline component. We develop a semiparametric efficient score equation for the latent space parameter by adjusting for the baseline nuisance component. Estimation is accomplished through a one-step update estimator and an appropriately penalized maximum likelihood estimator. We derive oracle error bounds for the two estimators and address identifiability concerns from a quotient manifold perspective. Our approach is demonstrated using the New York Citi Bike Dataset.
... We consider K ∈ {3, 8}, N ∈ {200, 1000, 2000, 3000, 4000, 5000}, and J = N/5, which correspond to large-scale and high-dimensional data scenarios. Such simulation regimes share a similar spirit with those in Zhang et al. (2020), which proposed and evaluated an SVD-based approach for item factor analysis. In each simulation setting, we generate 100 independent replicates. ...
... As shown in Figure accuracy of Π is slightly worse but comparable to JML. We point out that due to their noniterative nature, SVD or eigendecomposition-based methods typically tend to give worse estimation compared to iterative methods that aim to find the MLE; for example, see the comparison of an SVD-based method and a joint MLE method for item factor analysis in Zhang et al. (2020). However, it turns out that given a fixed computational resource (i.e., the default maximum iteration number in the sirt package), the iterative method JML can ...
... There are several future research directions worth exploring. First, building on the identifiability results, it would be interesting to derive statistical error bounds to show how the parameter estimation error explicitly depends on N , J, and K, similar in spirit toZhang et al. (2020) for item factor analysis. Second, this work has focused on binary responses, while in practice it is also of interest to perform GoM analysis of polytomous responses, such as Likert-scale responses in psychological questionnaires (see, e.g.Gu et al., 2023). ...
Grade of Membership (GoM) models are popular individual-level mixture models for multivariate categorical data. GoM allows each subject to have mixed memberships in multiple extreme latent profiles. Therefore GoM models have a richer modeling capacity than latent class models that restrict each subject to belong to a single profile. The flexibility of GoM comes at the cost of more challenging identifiability and estimation problems. In this work, we propose a singular value decomposition (SVD) based spectral approach to GoM analysis with multivariate binary responses. Our approach hinges on the observation that the expectation of the data matrix has a low-rank decomposition under a GoM model. For identifiability, we develop sufficient and almost necessary conditions for a notion of expectation identifiability. For estimation, we extract only a few leading singular vectors of the observed data matrix, and exploit the simplex geometry of these vectors to estimate the mixed membership scores and other parameters. Our spectral method has a huge computational advantage over Bayesian or likelihood-based methods and is scalable to large-scale and high-dimensional data. Extensive simulation studies demonstrate the superior efficiency and accuracy of our method. We also illustrate our method by applying it to a personality test dataset.
... The current theoretical results only consider a low-dimensional setting where the numbers of manifest variables and factors are fixed and the sample size goes to infinity. As factor analysis is commonly used by those analysing high-dimensional multivariate data, it is of interest to generalise the current results to a high-dimensional regime where the numbers of manifest variables, factors, and observations all grow to infinity (Chen & Li, 2022;Chen et al., 2019Chen et al., , 2020Zhang et al., 2020). In particular, it will be of interest to see how the rotation methods work with the joint maximum likelihood estimator for high-dimensional factor models (Chen et al., 2019. ...
Researchers have widely used exploratory factor analysis (EFA) to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that they often use to find interpretable loading matrices. In this paper, we propose a new family of oblique rotations based on component-wise [Formula: see text] loss functions [Formula: see text] that is closely related to an [Formula: see text] regularised estimator. We develop model selection and post-selection inference procedures based on the proposed rotation method. When the true loading matrix is sparse, the proposed method tends to outperform traditional rotation and regularised estimation methods in terms of statistical accuracy and computational cost. Since the proposed loss functions are nonsmooth, we develop an iteratively reweighted gradient projection algorithm for solving the optimisation problem. We also develop theoretical results that establish the statistical consistency of the estimation, model selection, and post-selection inference. We evaluate the proposed method and compare it with regularised estimation and traditional rotation methods via simulation studies. We further illustrate it using an application to the Big Five personality assessment.
... The EM algorithm is sensitive to the initial value, and an ideal initial value may enhance the estimation accuracy. Future research can explore how to optimize the setting of the initial value for the discrimination parameter in P-MCAT, like using the singular value decomposition (Zhang et al., 2020). ...
Online calibration is a key technology for item calibration in computerized adaptive testing (CAT) and has been widely used in various forms of CAT, including unidimensional CAT, multidimensional CAT (MCAT), CAT with polytomously scored items, and cognitive diagnostic CAT. However, as multidimensional and polytomous assessment data become more common, only a few published reports focus on online calibration in MCAT with polytomously scored items (P‐MCAT). Therefore, standing on the shoulders of the existing online calibration methods/designs, this study proposes four new P‐MCAT online calibration methods and two new P‐MCAT online calibration designs and conducts two simulation studies to evaluate their performance under varying conditions (i.e., different calibration sample sizes and correlations between dimensions). Results show that all of the newly proposed methods can accurately recover item parameters, and the adaptive designs outperform the random design in most cases. In the end, this paper provides practical guidance based on simulation results.
... Other estimators. Note that other F-consistent estimators may be available for GLFMs, such as SVD-based methods [Chatterjee, 2015, Zhang et al., 2020, regularized estimators [Klopp, 2014, Koltchinskii et al., 2011, Negahban and Wainwright, 2012, Robin et al., 2020, and methods based on a matrix factorization norm Zhou, 2013, 2016]. ...
Matrix completion is a class of machine learning methods that concerns the prediction of missing entries in a partially observed matrix. This paper studies matrix completion for mixed data, i.e., data involving mixed types of variables (e.g., continuous, binary, ordinal). We formulate it as a low-rank matrix estimation problem under a general family of non-linear factor models and then propose entrywise consistent estimators for estimating the low-rank matrix. Tight probabilistic error bounds are derived for the proposed estimators. The proposed methods are evaluated by simulation studies and real-data applications for collaborative filtering and large-scale educational assessment.
... In the latest survey performed in (Zhang et al., 2020) 19 European countries, children's estimated time being exposed online stretches from 134 minutes in Switzerland up to 219 minutes in Norway (Smahel, D., Machackova, H., Mascheroni, G., Dedkova, L., Staksrud, E., Ólafsson, K., Livingstone, S., and Hasebrink, U. (2020). Comparing with the same report from 2010 in some countries such as Germany, France, Italy and Spain, the average time children spend on the internet has multiplied. ...
... The current theoretical results only consider a low-dimensional setting where the numbers of manifest variables and factors are fixed and the sample size goes to infinity. As factor analysis is commonly used to analyse high-dimensional multivariate data, it is of interest to generalise the current results to a highdimensional regime where the numbers of manifest variables, factors, and observations all grow to infinity (Chen & Li, 2022;Chen et al., 2019Chen et al., , 2020Zhang et al., 2020). In particular, it will be of interest to see how the rotation methods work with the joint maximum likelihood estimator for high-dimensional factor models (Chen et al., 2019. ...
Exploratory factor analysis (EFA) has been widely used to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that are widely used to find interpretable loading matrices. This paper proposes a new family of oblique rotations based on component-wise loss functions that is closely related to an regularised estimator. Model selection and post-selection inference procedures are developed based on the proposed rotation. When the true loading matrix is sparse, the proposed method tends to outperform traditional rotation and regularised estimation methods, in terms of statistical accuracy and computational cost. Since the proposed loss functions are non-smooth, an iteratively reweighted gradient projection algorithm is developed for solving the optimisation problem. Theoretical results are developed that establish the statistical consistency of the estimation, model selection, and post-selection inference. The proposed method is evaluated and compared with regularised estimation and traditional rotation methods via simulation studies. It is further illustrated by an application to big-five personality assessment.
