Table 4 - uploaded by Chunzheng Cao
Content may be subject to copyright.
Percentages of equations failing the specification tests when the correct specification of Model 2 is S1 and the sample size 1,000. The significance
Source publication
The polychoric instrumental variable (PIV) approach is a recently proposed method to fit a confirmatory factor analysis model with ordinal data. In this paper, we first examine the small-sample properties of the specification tests for testing the validity of instrumental variables (IVs). Second, we investigate the effects of using different number...
Contexts in source publication
Context 1
... the other hand, PIV1 may lead to misleading results in Model 2. Table 4 displays the empirical rejection rates if the correct model specification is S1 for Model 2 when n = 1,000. It is seen that PIV1 and PIV2 are not likely to detect the misspecified equations, whereas PIVall always rejects the misspecified equations. ...
Similar publications
Relational inference leverages relationships between entities and links in a network to infer information about the network from a small sample. This method is often used when global information about the network is not available or difficult to obtain. However, how reliable is inference from a small labelled sample? How should the network be sampl...
Citations
... Kirby and Bollen (2009) investigated the Sargan (1958) test when all indicators are continuous. Jin and Cao (2018) proposed alternative overidentification tests for ordinal variables. Jin et al. (2021) further generalized the tests to a mixture of different types of indicators. ...
... All these studies suggested to use as many MIIVs as possible when performing the overidentification tests. In particular, the simulation study in Jin and Cao (2018) showed that the overidentification test can have a very low power when the degrees of freedom of the overidentification test is one. ...
... These conditions will help researchers to understand the risks of using overidentification tests to test the validity of IVs when the MIIV approach is used. In the context of confirmatory factor analysis (CFA), Table 4 in Jin and Cao (2018) has shown that the power can converge to the size. However, they did not investigate the implications behind. ...
In the context of structural equation modeling, the model-implied instrumental variable (MIIV) approach has been shown to be more robust against model misspecification than the systemwide approaches (e.g., maximum likelihood and least squares). Besides the goodness-of-fit tests that test the fit of the entire hypothesized covariance structure, the overidentification tests for MIIV can be used to test model specification on an equation-by-equation basis. However, it is known in the econometrics literature that the overidentification tests are inconsistent against general misspecification, if it is used to test a zero correlation between the instrumental variables and the error terms. In this paper, we show that such inconsistency can also occur for the MIIV approach. Numerical examples where the powers of the tests converge to the size are presented. Theoretical results are proved to support the numerical findings. Implications on when the overidentification tests are consistent are also presented.
... Polychoric correlation considers and sorts variables into a series of categories; it is an alternative to the Pearson coefficient, specifically for cases where instruments yield ordinal data from categorical variables (Jin & Cao, 2018). The Pearson CFA uses the maximum likelihood (ML), which assumes that observed data follow a continuous MVN. ...
... Various scholars (e.g., Finney & DiStefano, 2006;Holgado-Tello et al., 2010) have argued that polychoric correlations are more accurate and offer more robust estimations when handling ordinal data originating from Likert scales. More recently, Jin and Cao (2018) also recommended using polychoric correlations for Likert data. However, despite these researchers' strong arguments, few applied resources currently use this approach in nursing research. ...
... The use of a relatively large sample size, 800 family members, denotes a particular strength of the study, as it provides extensive data that are likely to answer the research questions. The use of polychoric correlation is well suited to ordinal data drawn from a Likert scale; thus, we are in a good position to examine the assertion raised by other scholars (e.g., Jin & Cao, 2018) that polychoric correlations are more accurate and offer more robust estimations when handling ordinal data originating from Likert scales. Another strength of the study is the inclusion of family members from units where the illness experience at the time of admission of a patient provides substantial family perceived support measures from nurses. ...
Background:
Measures in nursing research frequently use Likert scales that yield ordinal data. Confirmatory factor analysis using Pearson correlations commonly applies to such data, although this violates ordinal scale assumptions.
Objectives:
To illustrate the application of polychoric correlations and polychoric confirmatory factor analysis as a valid alternative statistical approach using data on family members' perceived support from nurses as an exemplar.
Methods:
A primary analysis of cross-sectional data from a sample of 800 participants using data collected with the Icelandic-Family Perceived Support Questionnaire was conducted using polychoric versus Pearson correlations, analysis of variance, and confirmatory factor analysis.
Results:
A two-factor measurement model was compatible with data from family members in the Ugandan care settings. Two contextual factors (cognitive and emotional support) constituted the family support measurement model. A factor correlation indicated that the two factors reflected distinct but closely related aspects of family support. Polychoric correlation revealed 13.8% (range 5.5%-25.2%) higher correlations compared to Pearson correlations. Moreover, the polychoric agreed with the data, while the Pearson confirmatory factor analysis did not fit based on multiple statistical criteria. Analyses indicated a difference in emotional and cognitive support perception across two family characteristics: education and relationship to the patient.
Discussion:
A polychoric correlation suggests stronger associations, and consequently, the approach can be more credible with an ordinal Likert scale than Pearson correlations. Hence, polychoric confirmatory factor analysis can address a larger proportion of variance. In nursing research, polychoric confirmatory factor analysis can confidently be utilized when conducting confirmatory factor analysis of ordinal variables in Likert scales. Further, when a Pearson confirmatory factor analysis is used for ordinal Likert scales, the researcher should carefully evaluate the difference between the two approaches and justify their methodological choice. Even though we do not suggest dispensing with Pearson correlations entirely, we recommend using polychoric correlation for ordinal Likert scales.
... This is relevant because multivariate normality is usually less likely in the former. in addition, the absence of multivariate normality is very common in social science research (Jin & cao, 2018;Li, 2016). This results in the incorrect use of statistical tests during the validation processes, which do not correspond to the nature of the items or the assumption of multivariate normal distribution (sullivan & Artino, 2013). ...
... cFA is generally calculated with the maximum Likelihood estimation method (mL) (Li, 2016), which assumes that the observed indicators (items) follow a continuous and multivariate normal distribution (myung, 2003). in the case of psychological tests, this is not the most suitable method, as items usually have an ordinal nature (Gitta & Bengt, 2009) and continuous multivariate normal distribution is unlikely (holtmann et al., 2016). Therefore, cFA requires estimators appropriate to these characteristics such as the Diagonally Weighted Least squares (DWLs) method or robust estimations such as robust maximum Likelihood (mLr) or Weighted Least squares with Adjusted mean and Variance (WLsmV) (Jin & cao, 2018). These methods, especially mLr, are recommended, as they reduce biases compared to mL. ...
... The second block corresponded to the cFA with the rmL estimator, which is reported as the most appropriate estimator considering the continuous nature of the variables and the absence of multivariate normality (holtmann et al., 2016;Jin & cao, 2018). Three models have been tested: a) an oblique two-factor model; b) an orthogonal two-factor model; and c) a bifactor model with two specific factors (sF) and a general factor (GF). ...
Background. Emotion Regulation comprises a set of strategies (cognitive, emotional, and physiological) that allow individuals faced with internal or external stimuli to manage their emotional response, to adapt to the environment, and to achieve goals. The Emotion Regulation Questionnaire (ERQ) is used to assess Emotion Regulation. It has been translated into several languages (including Spanish) and has been adapted around the world, but its psychometric properties have not been tested in Ecuador.
Objective. To confirm the bifactor structure of the Emotion Regulation
Questionnaire and its reliability in a sample of Ecuadorian college students.
Design. A quantitative and instrumental study using Confirmatory Factor Analysis with Robust Maximum Likelihood estimation. The sample consisted of 400 participants (62.5% women), aged 18 to 25 (M = 21.1; SD = 1.95) from two universities in Ecuador and seven different undergraduate courses.
Results. The bifactor model of the test is confirmed with an adequate adjustment ꭓ2 = 35.99; p > .001; ꭓ2 /df = 1.43; CFI = .98; TLI = .96; SRMR = .034; and RMSEA = .033 CI95%: [.033–.052]; ωH = .70; ωHs1 = .23; ωHs2 = .35. Reliability is high with ω = .86 CI95%: [.81–.88]. Conclusion. The bifactor model of the ERQ is an adequate and reliable test to assess Emotion Regulation among Ecuadorian college students.
... Second, model specification tests such as the Sargan test offer equation-by-equation tests of model fit (Sargan, 1958). As we will demonstrate, it is possible to modify the Sargan test to create a multilevel overidentification test in a MSEM context [see Jin and Cao (2018) for an example of modification of Sargan for categorical indicators, and Jin et al. (2021) for an example of modification of Sargan for indicators of different types]. Comparing to the goodness-of-fit tests based on ML, the Sargan test is a local test in the sense that each overidentified equation is tested by a Sargan test. ...
... Comparing to the goodness-of-fit tests based on ML, the Sargan test is a local test in the sense that each overidentified equation is tested by a Sargan test. Hence, one can detect multiple structural misspecifications after testing all equations (Jin & Cao, 2018), before fitting the whole model. In contrast, the ML-based goodness-of-fit tests are performed sequentially with the modification index, one modification after another. ...
... and g r ð Þ ¼ R zy À R zx h: The test statistic T av 2 is the same as the test statistics in Jin and Cao (2018), who proposed a test statistic for ordinal indicators, and Jin et al. (2021), who extended the test statistic to different types of indicators. Even though their test statistics were developed for single level data, T av 2 remains applicable to the current context, since MIIV-2SLS is applied to the within level and the between level separately. ...
This study develops a new limited information estimator for random intercept Multilevel Structural Equation Models (MSEM). It is based on the Model Implied Instrumental Variable Two-Stage Least Squares (MIIV-2SLS) estimator, which has been shown to be an excellent alternative or supplement to maximum likelihood (ML) in SEMs (Bollen, 1996 Bollen, K. A. (1996). An alternative two stage least squares (2SLS) estimator for latent variable equations. Psychometrika, 61, 109–121.[Crossref], [Web of Science ®] , [Google Scholar]). We also develop a multilevel overidentification test statistic that applies to equations at the within or between levels. Our Monte Carlo simulation analysis suggests that MIIV-2SLS is more robust than ML to misspecification at within or between levels, performs well given fewer than 100 clusters, and shows that our multilevel overidentification test for equations performs well at both levels of the model.
... En el segundo bloque consta el análisis de invarianza factorial, que en primera se analizan los Análisis Factoriales Confirmatorios (AFC), con método de Mínimos Cuadrados Ponderados con Media y Varianza Ajustada (WLSMV), que es el método más adecuado para variables categóricas y con ausencia de normalidad multivariante (Holtmann et al., 2016). La línea base independiente de los AFC es de 201 hombres y 234 mujeres (los tamaños de los grupos (Jin & Cao, 2018) se encuentran dentro de los parámetros de validez y aceptación). Los AFC muestran los valores del modelo Chi Cuadrado (χ 2 ) y el Chi cuadrado normado (χ 2 /gl). ...
El objetivo de este estudio fue investigar la invarianza factorial y la fiabilidad con respecto al sexo del Cuestionario de Regulación Emocional (ERQ) en una muestra de jóvenes ecuatorianos. Se trata de un estudio instrumental de la equivalencia de medida o invarianza factorial del ERQ y de la fiabilidad. Participaron 435 estudiantes de bachillerato (53,8% mujeres), con edades comprendidas entre 14 y 20 (X̅= 16.7; s= 1.4). El modelo de dos factores es confirmado con χ²= 145.4 gl (68); p< .001;χ²/gl= 2.1; GFI= .94; CFI= .91 y RMSEA= .05 [.03 - .06]. Las restricciones al modelo en la equivalencia de medida muestran que este es invariante con respecto al sexo. Además, la fiabilidad muestra que las magnitudes son aceptables con ωSE= .636; ωRC= .762 en hombres y ωSE= .694; ωRC= .742 en mujeres. Se evidencia que el modelo del ERQ es invariante con respecto al sexo y que, a su vez, es fiable.
... Furthermore, a number of recent analytic developments have extended diagnostic tests for the PIV estimator 14 BOLLEN ET AL. originally proposed by Bollen and Maydeu-Olivares (2007). For example, Jin and Cao (2018) proposed an equation-by-equation overidentification test compatible with the PIV framework. ...
... Oczkowski and Farrell (1998) proposed a MIIV method to discriminate between different measurement scales of the same construct with a nonnested test. Jin and Cao (2018) propose a chi square overidentification test that applies to equations with categorical dependent variables when using MIIVs. ...
Structural equation models (SEMs) are widely used to handle multiequation systems that involve latent variables, multiple indicators, and measurement error. Maximum likelihood (ML) and diagonally weighted least squares (DWLS) dominate the estimation of SEMs with continuous or categorical endogenous variables, respectively. When a model is correctly specified, ML and DWLS function well. But, in the face of incorrect structures or nonconvergence, their performance can seriously deteriorate. Model implied instrumental variable, two stage least squares (MIIV-2SLS) estimates and tests individual equations, is more robust to misspecifications, and is noniterative, thus avoiding nonconvergence. This article is an overview and tutorial on MIIV-2SLS. It reviews the six major steps in using MIIV-2SLS: (a) model specification; (b) model identification; (c) latent to observed (L2O) variable transformation; (d) finding MIIVs; (e) using 2SLS; and (f) tests of overidentified equations. Each step is illustrated using a running empirical example from Reisenzein's (1986) randomized experiment on helping behavior. We also explain and illustrate the analytic conditions under which an equation estimated with MIIV-2SLS is robust to structural misspecifications. We include additional sections on MIIV approaches using a covariance matrix and mean vector as data input, conducting multilevel SEM, analyzing categorical endogenous variables, causal inference, and extensions and applications. Online supplemental material illustrates input code for all examples and simulations using the R package MIIVsem. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
... Thus, the ML method does not adequately estimate item saturation and factor configuration. That is why, in these cases, the use of robust estimates such as robust ML, diagonally weighted least squares, or adjusted mean and variance weighted least squares (WLSMV; DiStefano and Morgan, 2014; Li, 2016;Jin and Cao, 2018) are preferred. These estimates are more appropriate to the characteristics of Likert scale items with and without multivariate normality. ...
... Thus, the calculation of the CFA with robust estimation (WLSMV) can be considered a better estimator for the AUDIT due to the type of items it has (Likert type), especially because its multivariate distribution is not normal (Sullivan and Artino, 2013;Holtmann et al., 2016;Li, 2016). This allows for estimation with reduced bias in the results, compared with other estimates (Jin and Cao, 2018), which better represents the distance between the latent variables (Hazardous Alcohol Use, Dependence Symptoms, and Harmful Alcohol Use) and the instrument (AUDIT) itself. A more realistic estimation of results can translate into a better understanding of the problem during the diagnostic process and future intervention. ...
... It is also worth noting that the relatively low variability in alcohol consumption we found should not be a problem for our analyses of the factorial configuration of the AUDIT since there is variability between the results to conduct the analyses. Also, the use of robust estimates such as the WLSMV can help correct significant biases derived from non-normal multivariate distributions (Li, 2016), which is the case of the items from the AUDIT, since it is very rare to find multivariate normality in Likert scale items (Jin and Cao, 2018). ...
Objective: Confirm the three correlated factors model of the Alcohol Use Disorders Identification Test (AUDIT) using robust estimations and evaluate its internal consistency with a sample of Ecuadorian adolescents.
Method: Descriptive and instrumental analysis that includes confirmatory factor analysis with robust estimation and the calculation of its internal consistency.
Participants: A total of 1113 adolescents in which 56.1% are men and 43.9% are women), and they were between 11 and 19 years old (X= 14.9 years; s = 1.67). Students from eight educational centres
in Cotopaxi (54.1%) and Tungurahua (45.9%) in Ecuador were also included.
Results: The three correlated factors model from the AUDIT is confirmed with χ2 = 95.67; P < 0.001; df = 32; χ2/df = 2.98; comparative adjustment index = 0.93; Tucker-Lewis index = 0.90; standardized root mean square residual = 0.046; root mean square error of approximation = 0.042; 95% confidence interval [0.033–0.052].
Conclusions: The three correlated factors model from the AUDIT using robust estimations has an adequate fit and is also reliable in a sample of Ecuadorian adolescents.
... Otro aspecto que se ha observado en estudios que utilizan el PWBS-E con muestras latinoamericanas es la tendencia a verificar sus propiedades con técnicas que hoy en día están siendo cuestionadas tales como el coeficiente momento-producto de Cronbach (α) para el cálculo de la fiabilidad, o estimadores de menor robustez como la máxima verosimilitud (Elosua Oliden & Zumbo, 2008;Jin & Cao, 2018;Li, 2016). De esta forma, al no contar con parámetros referenciales de los análisis instrumentales utilizados para la verificación, los resultados expuestos pueden presentar errores de precisión en las puntuaciones y contener un nivel considerable de sesgo. ...
... De esta forma, al no contar con parámetros referenciales de los análisis instrumentales utilizados para la verificación, los resultados expuestos pueden presentar errores de precisión en las puntuaciones y contener un nivel considerable de sesgo. Esto sucede, por ejemplo, en los análisis factoriales cuando se usan estimaciones específicas sin verificar los requisitos previos de normalidad multivariante y de naturaleza del ítem (Domínguez-Lara & Navarro-Loli, 2018;Jin & Cao, 2018), por lo que la estructuración factorial pueda variar o no ajustarse al modelo original. En la actualidad, resulta cada vez más frecuente en investigación instrumental el uso de medidas alternativas como el coeficiente omega (ω; McDonald, 1999) que ha sido escasamente utilizado en la medida de fiabilidad del PWBS-E (Domínguez- Lara et al., 2018;Pineda-Roa et al., 2018). ...
... En la actualidad, resulta cada vez más frecuente en investigación instrumental el uso de medidas alternativas como el coeficiente omega (ω; McDonald, 1999) que ha sido escasamente utilizado en la medida de fiabilidad del PWBS-E (Domínguez- Lara et al., 2018;Pineda-Roa et al., 2018). Asimismo, la estimación factorial se está realizando principalmente con base en correlaciones policóricas o tetracóricas debido a la naturaleza ordinal de los ítems y al método de estimación robusta (Domínguez- Lara, 2014;Domínguez-Lara et al., 2018;Ferrando & Anguiano-Carrasco, 2010;Jin & Cao, 2018;Li, 2016) que se ajustan a las particularidades propias de medición en ciencias sociales. ...
The study explores the factorial structure of the Ryff’s Psychological Wellbeing Questionnaire (PWBS) in university students in Ecuador, configuring itself as an instrumental study of factorial validity, internal consistency type and temporal stability and temporal validity of the test. Four hundred forty-one university students participated (73% women and 27% men), between 17 and 39 years old (M = 20.9; SD = 2.36), from two universities in Ambato, Ecuador. At the level of results, the model of six correlated factors is confirmed in a version of 28 items (withdrawal of item 13) with adjustment indicators of x2 = 971.1; p < .001; gl = 335; x2/gl = 2.9; CFI = .93; TLI = .92; SRMR = .08; RMSEA = .066 [.061 - .071]. The reliability of the factors is acceptable, between ω = .58 in personal growth and ω = .79 for self-acceptance and purpose in life. Furthermore, it has temporal stability validity in two-week intervals with r = .92, p < .001; and t = 2.14; p < .05. It is concluded that the six-factor adjustment model of the Ryff Scale is adapted to the university population of Ecuador, in line with previous studies in similar populations.
... Bollen (1996, pages 117-118) and Kirby and Bollen (2009) recommend the Sargan (Sargan, 1958) test for equation-by-equation overidentification tests of misspecification. Jin and Cao (2018) showed that these tests are not suitable for ordinal endogenous variables and proposed alternative overidentification tests for such variables. ...
... In contrast, we will also develop goodness-of-fit tests for the whole model and overidentification tests for individual equations. Similarly to the standard error estimators, one set of overidentification tests for equations matches Kirby and Bollen (2009) for observed continuous variables and the other set matches Jin and Cao (2018) for observed ordinal variables. ...
... Bollen (1996;2001), , and Kirby and Bollen (2009) have investigated the case where y is continuous. In contrast, Bollen and Maydeu-Olivares (2007), Jin et al. (2016), Cao (2018), andNestler (2013) have investigated the case where y is ordinal. Fisher and Bollen (2020) is the only study we know of that considered indicators of different types. ...
The model-implied instrumental variable (MIIV) estimator is an equation-by-equation estimator of structural equation models that is more robust to structural misspecifications than full information estimators. Previous studies have concentrated on endogenous variables that are all continuous (MIIV-2SLS) or all ordinal . We develop a unified MIIV approach that applies to a mixture of binary, ordinal, censored, or continuous endogenous observed variables. We include estimates of factor loadings, regression coefficients, variances, and covariances along with their asymptotic standard errors. In addition, we create new goodness of fit tests of the model and overidentification tests of single equations. Our simulation study shows that the proposed MIIV approach is more robust to structural misspecifications than diagonally weighted least squares (DWLS) and that both the goodness of fit model tests and the overidentification equations tests can detect structural misspecifications. We also find that the bias in asymptotic standard errors for the MIIV estimators of factor loadings and regression coefficients are often lower than the DWLS ones, though the differences are small in large samples. Our analysis shows that scaling indicators with low reliability can adversely affect the MIIV estimators. Also, using a small subset of MIIVs reduces small sample bias of coefficient estimates, but can lower the power of overidentification tests of equations.
... Verification of multivariate normality is important to make decisions about the selection of statistical estimators with a multivariate non-normality characteristic (Cain et al., 2017). Furthermore, due to the categorical ordinal nature of the items, a matrix of polychoric correlations of the items was estimated (Jin & Cao, 2018). This specific analysis aims to identify the absence of multicollinearity, which is detected when there are intercorrelations greater than .90 ...
The aim of the study was to evaluate the construct validity based on the internal structure,
the relationship with other variables, and the internal consistency among items of the Fear of COVID-19 Scale (FCV-19S) in a sample of 743 Ecuadorians. The findings confirm the presence of a bifactor structure, which includes a general factor and two specific factors: one emotional and the other physiological. The general factor, and the specific factors presented adequate levels of internal consistency. Finally, the FCV-19S showed a highly significant relationship with GAD-7 at the latent level. The scale has adequate psychometric properties for its application.
