No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares
Abstract
In confirmatory factor analysis (CFA), the use of maximum likelihood (ML) assumes that the observed indicators follow a continuous and multivariate normal distribution, which is not appropriate for ordinal observed variables. Robust ML (MLR) has been introduced into CFA models when this normality assumption is slightly or moderately violated. Diagonally weighted least squares (WLSMV), on the other hand, is specifically designed for ordinal data. Although WLSMV makes no distributional assumptions about the observed variables, a normal latent distribution underlying each observed categorical variable is instead assumed. A Monte Carlo simulation was carried out to compare the effects of different configurations of latent response distributions, numbers of categories, and sample sizes on model parameter estimates, standard errors, and chi-square test statistics in a correlated two-factor model. The results showed that WLSMV was less biased and more accurate than MLR in estimating the factor loadings across nearly every condition. However, WLSMV yielded moderate overestimation of the interfactor correlations when the sample size was small or/and when the latent distributions were moderately nonnormal. With respect to standard error estimates of the factor loadings and the interfactor correlations, MLR outperformed WLSMV when the latent distributions were nonnormal with a small sample size of N = 200. Finally, the proposed model tended to be over-rejected by chi-square test statistics under both MLR and WLSMV in the condition of small sample size N = 200.
... All models were evaluated with an oblimin rotation and using the mean and variance-adjusted weighted least squares (WLSMV) method, implemented in 'lavaan' (Rosseel, 2022). This estimator was chosen for its demonstrated effectiveness in handling ordinal data (Li, 2016). Additionally, a three-factor model was tested on a secondary dataset to evaluate its performance, as the original proposal suggested three factors (Silvera et al., 2001). ...
... In this regard, it is essential to create a legitimate, accurate, and succinct instrument to evaluate university students' aptitude for solving problems. An innovative approach using the WLSMV estimator was employed for the CFA, recognized for its effectiveness in analyzing ordinal variables (Li, 2016). This kind of tool is crucial for catching the essence of problem-solving and thus a perfect choice for research requiring exact and quick evaluations in learning environments. ...
Problem-solving is becoming more and more seen as an important skill for college students to learn to build metacognitive skills, critical thought, and the ability to learn on their own. Even though this skill is very important, there aren't many approved tools that can be used to test it in schools, especially in Peru. The goal of this study is to fill in that gap by creating and testing a short problem-solving scale based on the Rational Problem-Solving Style, which stresses taking a planned and organized approach to problems. 733 Peruvian college students (mean age: 21.56 years, standard deviation: 4.15 years; 59.89% female) took part. A 15-item Problem-Solving Questionnaire and used experimental (EFA) and confirmatory factor analysis (CFA) to test it. The scale's validity and reliability were checked, along with its link to academic self-efficacy. There were four parts to the Problem-Solving Questionnaire: Solution Analysis and Planning, Critical Evaluation of Solutions, Generation and Evaluation of Alternatives, and Prioritization and Review of Alternatives. Fit scores from CFA (like CFI = 0.98 and RMSEA = 0.062) and reliability coefficients (ω = 0.73-0.90) showed that it was a reliable educational tool. There was proof of concept validity in the form of correlations with academic self-efficacy (r = 0.36-0.80). The scale is a validity and effective way to test the problem-solving skills of university students in Peru. Due to its brevity and emphasis on logical methods, it is suitable for use in both education and research, aligning with global goals for quality education.
... Therefore, the community sample was randomly divided into two groups: an EFA was performed with Sample 1 (N = 712) and a CFA with Sample 2 (N = 712). In both the EFA and CFA, the items were defined as ordinal variables and the weighted least squares means and variances adjusted (WLSMV) estimation procedure was applied (Li, 2016). Several considerations were taken into account during the selection of WLSMV over other estimation methods, such as maximum likelihood robust to non-normality (MLR). ...
... Additionally, WLSMV accounts for the threshold structure inherent in ordinal data, a feature not explicitly modeled in MLR-based approaches. Moreover, WLSMV is a robust and reliable method for handling non-normal or highly skewed indicator distributions, ensuring more stable parameter estimates (Li, 2016). ...
The risk for non-medical use and dependence on benzodiazepines (BZDs) is high. However, there is no available validated psychometric instrument that assesses the motives for BZD use. Therefore, the aim of the present study was to develop a scale identifying the motives for BZD use, examine the factor structure, and corroborate the construct validity of the scale. Items for the scale were generated from previous data collection and from the empirical literature. Consequently, 82 motives were tested among a large community (N = 1424) and a clinical sample (N = 113). Medical and non-medical BZD use, other substance use, and several psychological constructs were assessed in both samples. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), as well as bivariate correlations and regression analyses, were performed. The EFA model included 48 items with four factors: “personal and interpersonal benefits”, “substance use regulation”, “coping”, and “sleep facilitation”. The four-factor CFA model demonstrated adequate levels of model fit. Members of the clinical sample had significantly higher rates of all four motives. The construct validity of the Motives for Benzodiazepine Use Questionnaire (MBUQ-48) was supported by positive correlations between the motivational factors and psychological constructs, different outcomes of BZD use, and other substance use. Coping motives had positive association with various outcomes of BZD use. Based on the results, the MBUQ-48 is a reliable and valid scale for assessing motives for BZD use. Exploring the motivations underlying BZD use can help clinicians in the recognition of the risk of BZD use disorder and in increasing the efficacy of therapeutic processes.
... (WLSMV) foi empregado na análise da PSS-10 devido à natureza ordinal dos dados. Este método é ideal para lidar com escalas Likert, uma vez que considera a distribuição não normal dos dados, reduzindo vieses e garantindo estimativas mais confiáveis (Li, 2016). Além disso, o WLSMV é amplamente recomendado para estudos com tamanhos amostrais pequenos a moderados, como o presente estudo. ...
... A partir dos índices de ajuste obtidos e das cargas fatoriais significativas, torna-se evidente que o modelo unifatorial é eficaz para capturar a experiência subjetiva de estresse percebido. Essa conclusão é sustentada por literatura robusta, como Byrne (Byrne, 2016;Li, 2016). ...
Mães solo de crianças com Transtorno do Espectro Autista (TEA) enfrentam elevados níveis de estresse e sobrecarga emocional, agravados pela ausência de redes de apoio consistentes. Este estudo teve como objetivo validar a Escala de Estresse Percebido (PSS-10) e investigar a influência do cuidado temporário no bem-estar dessas mães. Trata-se de uma pesquisa quantitativa com aplicação da Análise Fatorial Confirmatória (CFA) em uma amostra de mães solo, selecionadas por conveniência. A CFA indicou bons índices de ajuste para a estrutura bifatorial da PSS-10, confirmando sua validade psicométrica nesse grupo específico. Além disso, os resultados apontaram que a presença de mecanismos de cuidado temporário está associada à redução dos níveis de estresse percebido, sugerindo uma relação protetiva desse tipo de suporte. Conclui-se que a PSS-10 é uma ferramenta válida para mensurar o estresse nessa população, e que intervenções que promovam o cuidado compartilhado podem contribuir significativamente para a saúde mental de mães solo em contextos de cuidado contínuo de filhos com TEA.
... These values are commonly used in simulation studies involving unrestricted factor structures (e.g., Christensen et al., 2025;Garrido et al., 2016). Lastly, the sample sizes simulated ranged from 300 to 1,000, which are generally considered medium to large (Li, 2016a). These sample sizes fall within recommended guidelines (Goretzko et al., 2021), and are consistent with previous simulation studies of wording effects (Woods, 2006). ...
... Another limitation of this research is that MLR was used to estimate the ESEM models with categorical data. Even though MLR may provide good estimates for data with five or more categories and moderate skewness (Bandalos, 2014), it is inferior to categorical variable estimators, particularly with few response categories and/or increased level of skewness (Li, 2016a;Li, 2016b). MLR was used because the estimation of the RIIFA over polychoric correlations is not scale invariant and its use requires additional considerations that have not been properly tested in the literature. ...
Wording effects, the systematic method variance arising from the inconsistent responding to positively and negatively worded items of the same construct, are pervasive in the behavioral and health sciences. Although several factor modeling strategies have been proposed to mitigate their adverse effects, there is limited systematic research assessing their performance with exploratory structural equation models (ESEM). The present study evaluated the impact of different types of response bias related to wording effects (random and straight-line carelessness, acquiescence, item difficulty, and mixed) on ESEM models incorporating two popular method modeling strategies, the correlated traits-correlated methods minus one (CTC[M-1]) model and random intercept item factor analysis (RIIFA), as well as the “do nothing” approach. Five variables were manipulated using Monte Carlo methods: the type and magnitude of response bias, factor loadings, factor correlations, and sample size. Overall, the results showed that ignoring wording effects leads to poor model fit and serious distortions of the ESEM estimates. The RIIFA approach generally performed best at countering these adverse impacts and recovering unbiased factor structures, whereas the CTC(M-1) models struggled when biases affected both positively and negatively worded items. A straightforward guide is offered to applied researchers who wish to use ESEM with mixed-worded scales.
... The preliminary measurement model for the proposed instrument was tested using CFA. Indicators were treated as ordinal variables with a diagonally weighted least squares with mean and variance adjusted estimator, and missing data were handled with fullinformation maximum likelihood (DiStefano & Morgan, 2014;Enders & Bandalos, 2001;Li, 2016). Model fit was evaluated with absolute (i.e., root-mean-square error of approximation [RMSEA]) and relative (i.e., comparative fit index [CFI] and Tucker-Lewis Index [TLI]) fit indices. ...
... Finally, structural models evaluated the proposed instrument's construct validity, linking combined self-stigma to related health outcomes (i.e., psychological distress, active concealment, and healthrelated QOL). Indicators were treated as ordinal variables with a weighted least squares with mean and variance adjusted estimator, and missing data were handled with full-information maximum likelihood (DiStefano & Morgan, 2014;Enders & Bandalos, 2001;Li, 2016). The same fit indices as those used in the CFA were used to evaluate the SEM model fit, and acceptable internal consistency using McDonald's ω was set at ≥0.7 (Bagozzi & Yi, 2012). ...
Self-stigma is pervasive among adults with mental health, drug use, and carceral histories. Yet, existing instruments are limited because they are lengthy and capture a single type of self-stigma, imposing significant respondent burden and resulting in incomplete assessment. This study describes the development and psychometric evaluation of a brief instrument of criminal justice (CJ), opioid use disorder (OUD), and mental health disorder (MHD) combined self-stigma. Participants (N = 213) were justice-involved adults with OUD and MHD who completed a survey from August 2023 to January 2024 on sociodemographic characteristics; three validated measures of self-stigma; psychological distress; active concealment of CJ, OUD, and MHD; and health-related quality of life (QOL). Analyses included exploratory factor analysis, confirmatory factor analysis, and structural equation modeling. Four exploratory factor analysis models were compared. Fit indices, factor loadings, and the meaning of factor indicators were examined. MHD self-stigma was associated with greater psychological distress (p < .001), more active concealment (p < .001), and poorer mental health QOL (p = .001). CJ self-stigma was associated with more active concealment (p = .007) but also better mental health QOL (p < .001). While correlated with MHD and CJ self-stigma, OUD self-stigma was not related to the health outcomes (p > .05 for all outcomes). OUD, MHD, and CJ self-stigma are correlated yet distinct and can be measured with a single, brief 12-item instrument. Preliminary construct validity was supported through associations with poorer health, but further evaluation of the validity and reliability of the brief instrument is needed.
... A confirmatory factor analysis (CFA) was conducted (n = 127) to determine if the ULFI-PL maintained a unidimensional structure similar to the original questionnaire. The CFA utilized a polychoric matrix along with the robust diagonally weighted least squares (RDWLS) extraction method (factor loadings > 0.40) [38]. The model fit was evaluated using several indices: the chisquared test (χ 2 ), the root mean square error of approximation (RMSEA), the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the standardized root mean square residual (SRMR). ...
Background
The Upper Limb Functional Index (ULFI) is a robust, widely used tool for assessing the functional status of the upper limbs (ULs) and the effectiveness of interventions in patients with musculoskeletal disorders (MSDs). This study aimed to translate and culturally adapt the ULFI into Polish (ULFI-PL) and evaluate its psychometric properties and practical characteristics.
Methods
A total of 127 patients (54% female, = 45.07 ± 14.97 years) with various ULMSDs and symptom durations completed the ULFI-PL, the shortened Disabilities of the Arm, Shoulder, and Hand questionnaire—Polish version (QuickDASH-PL), the Polish version of the World Health Organization Quality of Life—BREF (WHOQOL-BREF-PL), Numeric Pain Rating Scale (NPRS), and the seven-point Global Rating of Change (GRC) scale. The internal consistency, construct validity, and factor structure were assessed in all the participants; the test–retest reliability and measurement error were evaluated in a subgroup (n = 112, 2–3-day interval); and the responsiveness and interpretability were evaluated in another subgroup (n = 56, 8-week interval, after physiotherapy).
Results
The ULFI-PL demonstrated a good internal consistency (α = 0.77) and high construct validity, supported by the confirmation of five out of six a priori hypotheses (83.33%). A confirmatory factor analysis (CFA) revealed a unidimensional structure. It also demonstrated a high test–retest reliability (r = 0.85). The measurement error was calculated using the standard error of measurement (SEM = 4.75%) and the minimal detectable change (MDC95 = 13.17%). The ULFI-PL showed a high responsiveness after physiotherapy, which was confirmed by the effect size (ES = 2.08) and the standardized response mean (SRM = 1.88). The minimal clinically important difference (MCID) for the ULFI-PL was 28.29% (95% CI: 24.96–31.63).
Conclusions
The ULFI-PL is a reliable, valid, and responsive tool for assessing the upper limb function in Polish-speaking patients with ULMSDs and is suitable for use in both clinical and research contexts. The results are consistent with previous studies on the original English version and other language adaptations.
... The normality of the variables' distribution was analyzed using the Kolmogorov-Smirnov and Shapiro-Wilk tests (Field, 2009), and two main analyzes were performed from this dataset. First, the models (to be shown in the results) were tested through a complete SEM using weighted least square mean and variance adjusted as the estimation method (DiStefano and Morgan, 2014;Li, 2016). ...
Consumer mobilization to access healthy and quality foods can positively impact the planet. This fact aligns with food citizenship, which recognizes practices, rights, and obligations related to accessing healthy and sustainable food by conscious, collaborative, and politically active individuals. Despite their growing relevance, few studies systematically focus on these consumers' profiles. This study aims to develop and test a measure of food citizenship attitude, which can be analyzed with food citizenship behaviours (declared or intended). Items were raised through literature analysis and refined via expert validation. An online survey among Brazilian consumers (n = 329) tested the food citizenship measure based on 11 declared and intentional behaviours. Exploratory factor analysis verified the validity of the internal structure, allowing the identification of the dimensions of food citizenship. Structural equation modeling and generalized estimating equations provided further insights. Findings suggest the potential of this measure to assess food citizenship behaviour. The measure also demonstrated its potential to sufficiently detect differences between declared and behavioural intentions over time, indicating its potential for replication and testing in longitudinal studies. The study thus provides insights into how food citizenship attitudes can be measured and linked to behaviours, offering a tool to refine and use to understand consumers, develop public policies, and drive practices that depend on or benefit from the emergence of food citizens.
... Parameters were estimated with the robust weighted least squares mean-and variance-adjusted (WLSMV) estimator. The WLSMV is specifically designed for categorical data in which the normality assumption is typically violated (Li, 2015;Nussbeck et al., 2006), which also applies to the DBC-T. To account for the clustered data structure (i.e., teachers rated students' behaviours), we used the complex sample option, a design-based approach which adjusts standard errors according to the clustered data structure (Asparouhov, 2005). ...
... However, since the maximum likelihood method assumes multivariate normality, it was considered unsuitable for our data. Thus, we used the diagonal weighted least squares method, which does not assume a specific distribution (Li, 2016;Mîndrilã, 2010). All statistical tests and models were performed using SPSS (v. ...
The groove experience has been defined as a pleasurable urge to move that is induced in humans while listening to music. In the past, the intensity of this experience has been measured in various ways. In 2020, the English language Experience of Groove Questionnaire (EGQ-EN), with two scales and six items, was proposed as a concise, valid, and reliable psychometric inventory for measuring listeners’ urge to move and pleasure, followed by a German translation (EGQ-DE) in 2021. This study presents a Japanese version of the Experience of Groove Questionnaire (EGQ-JA). The items were carefully translated from English into Japanese. A listening experiment was conducted to investigate the psychometric properties of the EGQ-JA. The results show that the EGQ-JA replicates the two-factor structure of the original EGQ-EN and the EGQ-DE. Both scales had high internal consistency and good criterion-related validity. The EGQ-JA facilitates groove research among Japanese-speaking populations. This study also established the convergent criterion validity of the EGQ-JA scales with existing measurement scales, such as the Multiple Mood Scale and a nori item. Its equivalence with the English and German versions makes cross-cultural comparisons possible, and, thus, promotes culturally diverse research on the groove experience and music perception.
... Wickham és mtsai, 2019), a megerősítő faktorelemzésekhez a lavaan csomagot használtuk (verzió: 0.6-15; Rosseel, 2012). Mivel a CTt tételeire adott válaszok típusa dichotóm (helyes/helytelen), a mérőeszköz egydimenziós természetének tesztelését robusztus, súlyozott legnagyobb valószínűség (WLSMV) módszerrel végeztük (Li, 2016). A CTS mérőeszköz válaszformátuma 1-7-ig terjedő értékelő skála, ezért a megerősítő faktorelemzésénél robusztus legnagyobb valószínűség (MLR) becslést alkalmaztunk (Brown, 2006). ...
Háttér és célkitűzések: Az algoritmikus gondolkodás (Computational Thinking; CT) a hatékony ember-számítógép interakciók alapját képező problémamegoldó készségek csoportját jelenti, amely egyre növekvő jelentőséggel bír az oktatásban és a mindennapi életben. A CT megbízható mérése különösen fontos az oktatásfejlesztés szempontjából. Tanulmányunk a CT fogalmának rövid áttekintését követően bemutatja a CT két standardizált mérőeszközének magyar nyelvű adaptációját. Módszer: A CT teljesítmény mérésére magyar nyelvre fordítottuk a 28 tételes algoritmikus gondolkodás tesztet (Computational Thinking test; CTt), valamint annak 5, felnőttek számára nehezített tételét. A CT önbevallásos mérésére elkészítettük az algoritmikus gondolkodás skálák (Computational Thinking Scales; CTS) magyar fordítását. A fordítási folyamatot követően az eszközöket 203 magyar egyetemi hallgató adatán, klasszikus tesztelméleti alapon, megerősítő faktorelemzéssel teszteltük. Eredmények: A magyar CTt egydimenziós, megfelelő diszkriminációs erővel bíró, megbízható mérési eszköznek bizonyult. Az ötdimenziós, 29 tételt tartalmazó CTS-t a szakirodalom és a faktorelemzés alapján 15 tételesre redukáltuk. A magyar CTS faktorstruktúrája megerősítést nyert, a teljes skála megbízhatónak bizonyult, ugyanakkor az alskálák külön használata megbízhatósági és diszkriminációs érvényességi elemzések alapján csak korlátozott mértékben javasolt. Következtetések: A standardizált CT-mérés kulcsfontosságú mind az oktatás, mind a CT és annak pszichológiai korrelátumai közötti összetett összefüggések feltérképezése szempontjából. A CTt és CTS magyar változatai alkalmazhatók felnőttek vizsgálatára a felnőttoktatásban, továbbképzésben és munkahelyi kiválasztásban. A CT vizsgálatát célzó kutatási programunk következő lépése a CT mérőeszközök alkalmazása és értékelése magyar középiskolás diákok körében.
... Estos análisis se realizaron con el fin de respaldar la selección del estimador adecuado para el análisis factorial confirmatorio de los ítems. En caso de que la hipótesis de normalidad multivariada fuera rechazada, se utilizaría el método de estimación Robust Weighted Least Square Mean and Variance (WLSMV) (Li, 2016). ...
La Escala de Efectos de la Musicoterapia en Grupo en la Dependencia Química es un cuestionario autoinforme de 20 ítems que evalúa los beneficios percibidos de la musicoterapia grupal en los procesos de cambio de adultos con dependencia química. En el primer estudio, se emplearon procedimientos teóricos para la construcción de instrumentos de evaluación, como el Análisis Semántico y el Análisis de Jueces. La MTDQ fue considerada pertinente y adecuada para la población objetivo, con ítems conectados a sus respectivos dominios. En el segundo estudio, se evaluó la validez estructural y la confiabilidad de la Escala de Efectos dela Musicoterapia en Grupo en la Dependencia Química mediante análisis factorial confirmatorio, probando tres modelos: unidimensional, dos factores correlacionados y bifactorial. La muestra incluyó 154 hombres (77,37%) y 45 mujeres, con una edad media de 44,7 años. Los resultados indicaron que la estructura bifactorial, compuesta por dos factores específicos, procesos cognitivos y conductuales, y un factor general, era la más adecuada. La investigación ofrece evidencia inicial de que la Escala de Efectos de la Musicoterapia en Grupo en la Dependencia Química es un instrumento adecuado para medir los efectos de la Musicoterapia en personas con Dependencia Química.
... Reliability of the three anchor questions was estimated separately for the three anchor questions and PSB solely for the MIC anchor question since it evaluates change over time [25]. Ordinal item CFA using weighted least square mean and variance-adjusted estimators was applied in all models [36]. ...
Purpose
To explore content validity, construct validity, and reliability of anchor questions used to determine minimal important change (MIC), patient acceptable symptom state (PASS) and treatment failure (TF) in patients undergoing knee or hip arthroplasty.
Methods
A mixed-methods study from one public hospital. Evaluation of content validity involved applying thematic analysis to data from think-aloud interviews. To ascertain construct validity and reliability, we focused on patients who underwent surgery between 2016 and 2022 and had responded to preoperative and either 3-, 12- or 24-month postoperative questionnaires. Confirmatory factor analysis (CFA) was employed to assess present state bias (PSB), model fit, and reliability of the anchor questions.
Results
We conducted 18 interviews with patients aged 52 to 84 (10 female). Based on seven emerging themes from the content validity analysis, MIC and PASS anchor questions were considered relevant and comprehensible, while the TF anchor question had several problems. Data from 1197 to 2207 patients, with 3-, 12-, or 24-month postoperative responses, were used to evaluate construct validity. The median age was 69-70 years (56-59% female). PSB for MIC was between 54 and 73%, and reliability for the anchor questions was between 0.52 and 0.80 for all time points. The CFA models varied between good and poor fit.
Conclusion
The MIC and PASS anchor questions demonstrated a high degree of content validity, while it was questionable for TF. Construct validity was considered good to poor for PASS, but patients may consider their present state more than their preoperative state when responding to the MIC. Reliability was considered acceptable in both MIC and PASS.
... Alternatively, the Unweighted Least Squares Mean-And-Variance Adjusted (ULSMV) method may also be preferred. It is widely accepted in the literature that considering the variables as categorical and using WLSM/ULSMV as the estimation method is more appropriate (Kılıç & Doğan, 2021;Lei, 2009;Li, 2016aLi, , 2016bLiang & Yang, 2014;Moshagen & Musch, 2014;Oranje, 2003). Similar to EFA, when the number of categories is two, the tetrachoric covariance matrix should be used, while for 3-6 categories, the polychoric covariance matrix should be used. ...
Bu çalışmanın amacı, geliştirilmiş ya da uyarlanmış bir ölçek kullanacak araştırmacılara rehberlik edecek kontrol listesi geliştirmektir. Bu amaçla araştırmacılar tarafından alanyazına dayalı olarak ölçek geliştirme ve uyarlama adımları hazırlanmış ve tüm adımların ağırlıkları tanımlanmıştır. Hazırlanmış olan adımlar ve ağırlıkları eğitimde ölçme ve değerlendirme alanında doktora derecesine sahip olan 10 akademisyene gönderilmiştir. Akademisyenlerden hem adımlara yönelik görüşleri hem de takdir ettikleri ağırlıklar istenmiştir. Gelen dönütlere göre düzenlemeler yapılarak ölçek geliştirme ve uyarlama adımlarının ağırlıkları tanımlanmış, adımların ağırlıklarına göre elde edilecek toplam puan göz önüne alınarak bir ölçüt belirlenmiştir. Hâlihazırda geliştirilmiş/uyarlanmış bir ölçeği seçecek araştırmacılara önerilen ölçüt temel alınarak bu ölçekleri seçip seçmeme konusunda bir referans sunulmuştur. Bu yönüyle çalışmanın geliştirilmiş ya da uyarlanmış ölçeklerin seçiminde rehber oluşturması açısından önemli katkıları olacağı düşünülmektedir.
... The items of the PCF-BS were subjected to CFA using the structural equation modeling to confirm its internal structure using the weighted least squares-mean and variance adjusted (WLSMV) estimator with the "JASP" program, which robustly deals with potentially non-nomal data and items are treated as ordinal (Li, 2016a(Li, , 2016b. To evaluate the fit of our measurement models, we applied a percentile-based approach to interpret the fit indices, as recommended by recent methodological advancements (e.g., ...
This study presents the development and validation of the Organizational Psychological Contract Fulfillment/Breach Scale (PCF-BS), a brief instrument designed to assess employees’ perceptions of whether their employer has fulfilled or breached implicit obligations. The PCF-BS consists of two 4-item subscales measuring fulfillment and breach independently. A sample of 384 employees from public and private organizations in Puerto Rico participated in the study. Confirmatory factor analyses supported a two-factor structure with excellent fit indices. The scale demonstrated strong internal consistency, convergent and discriminant validity, and was invariant across gender, age, job position, organizational type, and employment type. Fulfillment was positively correlated with job satisfaction, engagement, and commitment, and negatively associated with burnout and turnover intention. Conversely, breach was linked to negative work outcomes. The PCF-BS offers a psychometrically robust and contextually relevant tool for researchers and practitioners seeking to evaluate the psychological contract in Spanish-speaking organizational settings.
... In this segment, the CFA was performed on the two correlated factors model (Domínguez Lara et al., 2014) using the estimation of Unweighted Least Squares (ULS). This estimation can be used due to the ordinal nature of the items and in the absence of multivariate normality (Li, 2016). The fit statistics used included absolute fit indexes like Chi-Square (χ 2 ) and the Standardized Root Mean Square Residual (SRMR); relative adjustments that correspond to the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI); and the non-centrality-based index which is the Root Mean Square Error of Approximation (RMSEA). ...
Objective: Analyze the evidence of validity of scores of the Academic Procrastination Scale (APS), its measurement equivalence based on nationality, its reliability of the scores, and its validity in relation to other variables in university students from Ecuador, Venezuela, and Peru. Method: This paper involves a quantitative, descriptive, psychometric, and cross-sectional study. Participants: Seven hundred and sixty-two university students from Ecuador (n = 370), Peru (n = 202), and Venezuela (n = 190). Results: The two-factor oblique structure of the APS is confirmed. There is evidence of validity in the APS scores through its relationship with scores of other measures, such as Academic Self-Efficacy. It is identified that the APS has evidence of measurement invariance (strong) according to nationality, as well as adequate internal consistency in the scores of the items and discriminant validity. Conclusion: The scores of the APS are valid for analysis in university students from Ecuador, Peru, and Venezuela for the study of academic procrastination, showing resistance to the cultural variations of these three countries.
... Given that the Mardia coefficient (564.67) indicated significant multivariate non-normality, the robust diagonally weighted least squares (RDWLS) extraction method was selected because it is well suited for ordinal data with non-normal distributions (Li, 2016). To the best of our knowledge, there is no consensus regarding the criteria for item deletion or retention. ...
This study aimed to translate and culturally adapt the System Usability Scale for Gamified E-learning Courses (SUS-G scale) to the Spanish population and evaluate its factor structure. A cross-sectional study was conducted in two phases: translation and cultural adaptation of the SUS-G scale, followed by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), with 103 and 201 participants, respectively. A total of 304 students were enrolled in the study. The EFA revealed three correlated factors: User Experience (16 items), Educational Usability (nine items), and Usability (four items). The 29-item structure showed adequate-to-excellent goodness-of-fit indices and good internal consistency in CFA. The Spanish version of the SUS-G was validated and found to be reliable. The final instrument, with 29 items across three dimensions, can assess the usability of gamified e-learning courses among Spanish-speaking university students.
... (Martín-Fernández et al., 2018). Weighted least squares with meanand variance-adjusted were utilized as the estimation method because they adequately perform with categorical data (Li, 2016). The model's goodness of fit was assessed by a combination of fit indices: values of the comparative fit index (CFI) and the Tucker-Lewis index (TLI) above 0.95 were deemed as indicative of a good fit, and values of the root-mean-squared error of approximation (RMSEA) below 0.06 and 0.08 were considered an excellent fit or a mediocre fit, respectively (Hu & Bentler, 1999;McCallum et al., 1996). ...
Objective: Intimate partner violence (IPV) is prevalent in adolescence and often leads to future relationship problems. Understanding the factors that promote IPV is crucial for targeting prevention and intervention efforts in this developmental stage. Attitudes of acceptance and justification of this type of violence have been shown to play a significant role in increasing the risk of this behavior occurring, which underscores the need to accurately measure this construct. This study aims to adapt the Acceptability of Intimate Partner Violence scale, originally developed for adults, for use with adolescents; validate the adapted measure (A-ADV); develop a shortened version; and compare acceptability levels between adolescents and young adults. Method: Three samples were collected: two from adolescents (N = 824 and N = 406) and one from young adults (N = 347). The latent structure, internal consistency, and validity evidence of the A-ADV were assessed. Item discrimination and thresholds were examined using item response theory. Additionally, automatic test assembly was used to develop a shortened version of the scale using both quantitative and qualitative criteria. Results: Both the full-length and shortened A-ADV followed a one-factor model, presented adequate internal consistency and validity evidence, and mapped accurately moderate to very high acceptability levels. On average, adolescents presented higher levels of attitudes of acceptability than young adults. Conclusions: The validation of the A-ADV for adolescents is an important contribution to the field. This measure’s psychometric properties and validity evidence support its use for the assessment of attitudes of acceptability in the adolescent population.
... This allowed us to conduct an exploratory factor analysis (EFA) so as to test the factor structure of the IES-27 on the first half. We used diagonally weighted least squares (WLSMV) to estimate model parameters throughout our analyses because of the ordinal data structure [27]. EFA with oblique (geomin) rotation was conducted sequentially across models with one to four factors. ...
Background
In clinical practice and research, self-report instruments are frequently used for assessing the severity of borderline personality disorder (BPD) symptomatology experienced by men and women. Men with BPD are often underrepresented in samples used to evaluate self-report questionnaires. Measurement invariance (MI) is used to examine whether self-report questionnaires determine the same latent construct across groups or varying conditions (e.g., measurement occasions).
Methods
The present study investigated measurement invariance for two self-report measures of BPD features: the Borderline Symptom List (BSL-23) and the Impulsivity and Emotion Dysregulation Scale (IES-27). An inpatient sample of N = 3507 individuals (n = 560 males) was used to test for measurement variance between males and females, and over time from pre- to post-treatment.
Results
Confirmatory factor analysis results supported a unidimensional structure for the BSL-23 and a three-factor model for the IES-27. Both instruments were found to be measurement invariant with regard to sex and time.
Conclusions
The results suggest that the BSL-23 and IES-27 can be used to assess BPD symptoms in men and women, as well as to assess treatment effects at admission and at the end of treatment.
... Zur Überprüfung der zuvor gefundenen Faktorenstruktur wurde im Rahmen von Fragestellung 2 eine konfirmatiorische Faktorenanalyse (CFA) mit Weighted Least Square Means and Variances Schätzer (WLSMV) (Li 2016) an den Daten aus Teilstudie 2 durchgeführt. Um die Nestung der Schüler*innen in Klassen zu berücksichtigen, wurden clusterrobuste Standardfehler geschätzt. ...
Im Beitrag wird die Entwicklung eines Fragebogens zur multifaktoriellen Messung der selbstwahrgenommenen sozialen Integration (SSI 3–4) anhand von zwei Teilstudien mit insgesamt n = 1283 Schüler*innen aus n = 31 dritten und n = 31 vierten Klassen beschrieben. In Teilstudie 1 wurde den Schüler*innen ein auf Basis bestehender Messinstrumente entwickelter Itempool vorgelegt und explorativ analysiert. Es resultieren sechs trennscharfe und inhaltlich plausible Skalen mit jeweils vier Items: Anerkennung und Wertschätzung, negative Interaktion und sozialer Ausschluss, außerschulischer Kontakt, Selbstoffenbarung und emotionale Unterstützung, Sympathie und Wohlbefinden, unterrichtsbezogene Hilfe. Teilstudie 2 bestätigt die sechsfaktorielle Struktur des Fragebogens durch eine konfirmatorische Faktorenanalyse. Weitergehende Faktorenanalysen deuten auf eine gute Passung eines Bi-Faktormodells mit einem Globalfaktor und den sechs benannten Subfaktoren hin. Alle Skalen korrelieren im mittleren bis hohen Bereich mit zwei einschlägigen eindimensionalen Skalen zur selbstwahrgenommenen sozialen Integration und im schwachen bis mittleren Bereich mit der sozialen Akzeptanz durch die Mitschüler*innen. Der SSI 3–4 ist unter einer CC-BY Lizenz frei nutzbar.
... For the MC Scale, which has binary responses, we used the Diagonally Weighted Least Squares (DWLS) method. DWLS is ideal for ordinal or binary data as it does not assume continuous variables or normal distribution (Li, 2016;Muthén, 1993). ...
This paper explores the efficacy of AI-driven chatbots in accurately inferring personality traits compared to traditional psychometric tests within a real-world professional hiring context. The study is driven by the increasing integration of AI tools in recruitment processes, which necessitates a deeper understanding of their reliability and validity. Using a quasi-experimental design with propensity score matching, we analysed data from 159 candidates and other professionals from Serbian and Montenegrin regions who completed both traditional psychometric assessments and AI-based personality evaluations based on the Big Five Personality model. A novel one-question-per-facet approach was employed in the chatbot assessments with a goal of enabling more granular analysis of the chatbot’s psychometric properties. The findings indicate that the chatbot demonstrated good structural, substantive, and convergent validity for certain traits, particularly Extraversion and Conscientiousness, but not for Neuroticism, Agreeableness, and Openness. While robust regression confirmed that AI-inferred scores are less susceptible to social desirability bias than traditional tests, they did not significantly predict real-world outcomes, indicating issues with external validity, particularly predictive validity. The results suggest that AI-driven chatbots show promise for identifying certain personality traits and demonstrate resistance to social desirability bias. This paper contributes to the emerging field of AI and psychometrics by offering insights into the potential and limitations of AI tools in professional selection, while developing an approach for refining psychometric properties of AI-driven assessments.
... Confirmatory factor analysis (CFA) was used to evaluate the construct validity of the RI-5-BF by examining the fit between the data from the present sample and the four-factor DSM-5 model, which has been investigated in prior studies of the UCLA PTSD-RI (Elhai et al., 2013;Takada et al., 2018). Due to the Likert-type response pattern of the RI-5-BF, the data followed an ordinal structure; thus, a polychoric correlation matrix (Holgado-Tello et al., 2010;Li, 2016) and robust weighted least squares with mean and variance adjustment (WLSMV) estimator were used (consistent with established recommendations; Flora & Curran, 2004). ...
Objective: Natural disasters are a salient source of traumatic exposures for many youth, although not all youth exposed to these traumatic experiences go on to develop posttraumatic stress symptoms or other maladaptive reactions. Given the low resource context and high need for psychological support engendered by natural disasters globally, efficient, evidentiary, culturally appropriate tools for assessing risk are needed to effectively identify youth most in need of services. Method: The present study used a sample of 96,108 Puerto Rican youth collected in 2018, 5–9 months after Hurricane Maria in 2017, to examine the psychometric properties of a Spanish translation of the University of California Los Angeles Posttraumatic Stress Disorder Reaction Index–Brief Form (RI-5-BF). Participants completed measures of hurricane exposure, posttraumatic stress symptoms, and depressive symptoms. Results: Results indicated a strong fit between the data and the four-factor Diagnostic and Statistical Manual of Mental Disorders, fifth edition, model of posttraumatic stress disorder (root-mean-square error of approximation = .030, 90% confidence interval [.029, .031], Tucker–Lewis index = .976, comparative fit index = .983) and multigroup factor analysis indicated measurement invariance by gender at the scalar level. Normative values for this population by grade and gender were also estimated by examining the descriptive statistics of participants’ responses and convergent validity analysis indicated a strong relationship between the University of California Los Angeles Posttraumatic Stress Disorder Reaction Index–Brief Form total score and a measure of depressive symptoms (r = .64). Conclusions: The implications of psychometric research for improving global efforts to triage and respond to psychological distress in the aftermath of natural disasters are discussed, with an emphasis on the critical need for psychometric research to diversify through development and adaptation of instruments in global populations.
... The model estimation was conducted employing robust diagonally weighted least squares (DWLS) estimation with test statistics adjusted in terms of mean and variance, which is a suggested estimation method for ordinal data (DiStefano & Morgan, 2014;Li, 2016aLi, , 2016bLi, , 2021. Multiple fit indices were considered when evaluating the goodness of the fit of the model to the data. ...
The use of the internet has become an increasingly integral part of individuals’ daily lives, bringing negative consequences such as internet addiction. Understanding the motivations behind internet use is crucial for preventing addiction and developing effective intervention strategies. The objectives of this study are to test the validity and reliability of the Questionnaire of Internet Use Motives (MUI) among Turkish adults and to investigate the predictive effects of socio-demographic variables and internet use motivations on internet addiction. The study was conducted with a sample of 640 adults selected through convenience sampling at two different time points. The majority of participants were women, highly educated, and from a middle socioeconomic background. Data were collected using a socio-demographic questionnaire, the Questionnaire of Internet Use Motives (MUI), and the Internet Addiction Test (IAT). To evaluate the structural validity of the scale, a Confirmatory Factor Analysis (CFA) was performed. Additionally, measurement invariance across genders was examined, and Hierarchical Multiple Linear Regression Analyses were conducted to identify predictors of internet addiction. CFA confirmed the structural validity of the MUI, revealing a five-factor structure with a good fit to the data. The five identified motives were enhancement, coping, social, conformity, and utility. The analyses also demonstrated that the scale possesses convergent and discriminant validity, as well as high reliability. Furthermore, the instrument exhibited measurement invariance across genders. Significant predictors of internet addiction included educational level, socioeconomic status, and the enhancement, social, coping, and conformity motives. The validated MUI provides a robust tool for assessing internet use motives in Turkish adults, offering a foundation for future research and intervention development. Addressing psychological motives such as enhancement, social, coping, and conformity in prevention and treatment strategies may reinforce efforts to mitigate problematic internet use.
... Since each of the techniques discussed in the research performs its own calculations, instead of directly comparing the results, CFA was applied via the JASP software to the different models established, and each model's fit and error indices were compared. In CFA, diagonally weighted least squares estimation (DWLS), which is the most reliable parameter estimation method for ordinal data and generally when the variables are not normally distributed, was used (Brown, 2006;Flora & Curran, 2004;Li, 2016). ...
This study compares the psychometric properties of scales developed using Exploratory Factor Analysis (EFA), Self-Organizing Map (SOM), and Andrich's Rating Scale Model (RSM). Data for the research were collected by administering the "Statistical Attitude Scale" trial form, previously used in a separate study, to 808 individuals. First, EFA, SOM and RSM were applied to decide the number of dimensions of the scale, and to select items. Subsequently, Confirmatory Factor Analysis (CFA) was used to the forms obtained from different methods and their CFA fit indices were compared. The analysis revealed variations in the number of dimensions and item distribution across different methods. Results indicated that the form generated using SOM exhibited the highest fit indices. Furthermore, the CFA fit indices of the form created with RSM were found to be satisfactory, offering detailed insights into both items and individuals.
... p < .001), we used a robust maximum likelihood estimator which is less sensitive to violations of the multivariate normality assumption [34]. The invariance tests were performed following the steps proposed by Bowen and Masa [35]. ...
Background: There is evidence of growing racial and ethnic disparities in genomic healthcare and precision medicine. Validated survey instruments and measures are required to understand the needs of diverse populations to appropriately tailor person-centered approaches and end disparities in genomic healthcare and precision medicine. Methods: We aimed to examine the psychometric properties of a culturally adapted Spanish version of the Attitudes Toward Genomics and Precision Medicine (AGPM). First, we culturally adapted the AGPM. We then conducted a web-based evaluation of the Spanish AGPM in a cohort of 486 individuals identifying as Hispanic to establish the Spanish version’s reliability, factor structure, and measurement invariance relative to the English version. We also compared AGPM responses between Spanish- and English-speaking Hispanic individuals. Results: The Spanish version of the AGPM demonstrates robust internal consistency with Cronbach alpha ranging from 0.84-0.98 across domains. All AGPM items significantly loaded on their respective factor (p < 0.001). Configural, metric, strict, and residual invariance models all met absolute and relative fit criteria. Significant differences were observed between Spanish and English-speaking participants in some AGPM subscales. Conclusions: The Spanish version of the AGPM demonstrates sound psychometric properties and may be useful for informing culturally empowered approaches to genomic healthcare and precision medicine for people identifying as Hispanic.
... Fourth, all items selected from the EFA were subjected to confirmatory factor analyses (CFA) using the structural equation modeling to confirm the internal structure of the Turnover Intention Scale using the weighted least squares-mean and variance adjusted (WLSMV) TURNOVER INTENTION SCALE 1 13 estimator with the "lavaan" package, which robustly deals with potentially non-nomal data and items are treated as ordinal (Li, 2016a(Li, , 2016b. To evaluate the fit of our measurement model, we applied a percentile-based approach to interpret the fit indices, as recommended by recent methodological advancements (e.g., ...
... The Diagonally Weighted Least Squares (DWLS) estimator was selected because the EDS-E items use an ordinal Likert-type scale response format. DWLS is considered more robust than Maximum Likelihood (ML) for analyzing ordinal data, particularly when assumptions of multivariate normality might not be fully met, as is common with scale data [51]. ...
Background/Objectives: Discrimination is a phenomenon of special relevance in adolescence, as this is a key period in the development of young people, so measures that accurately and reliably assess it are essential. The aim of this research is to study the psychometric properties of the Spanish version of the Everyday Discrimination Scale in a sample of Spanish adolescents. Methods: The scale was applied to 1000 adolescents using Computer Assistance Web Interview (CAWI) methodology by means of a stratified random sampling by age, gender and territorial distribution. Results: The results reveal an unifactorial structure of the scale, with adequate measures of reliability and validity that confirm that it is a suitable instrument for assessing everyday discrimination in this population. Conclusions: This study has implications for understanding the experiences of discrimination in adolescents and for developing interventions to reduce discrimination and promote equality. Limitations and implications for the future are also discussed.
... Confirmatory factor analysis (CFA) using the weighted least squares mean and variance-adjusted estimator [32] was performed to evaluate whether the prehypothesized 4-factor model fit our observed data as evidence of construct validity [33]. The following goodness-of-fit indices were used: the χ 2 /df ratio, the P value, the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). ...
Background
Previous research has demonstrated a correlation between nursing students’ self-efficacy and their clinical performance, competence, and behavior during clinical practice placements. Assessing students’ self-efficacy in clinical performance could be a valuable method for identifying areas that need reinforcement and for recognizing students who may require additional support during clinical practice placements.
Objective
This study aimed to translate the Self-Efficacy in Clinical Performance Scale (SECP) from English into Norwegian and to evaluate the psychometric properties of the Norwegian version.
Methods
A cross-sectional study design was used. The SECP was translated into Norwegian following a 6-step process: forward translation, forward translation synthesis, backward translation, backward translation synthesis, cognitive debriefing, and psychometric testing. The validity and reliability of the translated version were assessed using confirmatory factor analysis (CFA), Cronbach α, McDonald ω, and composite reliability.
Results
A total of 399 nursing students completed the Norwegian version of the SECP. The CFA goodness-of-fit indices ( χ ² / df ratio=1.578, comparative fit index=0.98, Tucker-Lewis index=0.98, standardized root mean square residual=0.056, root mean square error of approximation=0.038) indicated an acceptable model fit. Reliability measures, including Cronbach ⍺, McDonald ω, and composite reliability, were high, with factor-level values ranging from 0.94 to 0.98.
Conclusion
The Norwegian version of the SECP demonstrated strong potential as an instrument for assessing self-efficacy in both current and required competencies among nursing students in clinical practice within nursing education. Future research should aim to confirm the factor structure of the SECP and evaluate its test-retest reliability.
... The next stage of the analysis involved conducting a confirmatory factor analysis (CFA) using maximum likelihood estimation to test the discriminant validity of the items of each construct (inclusive leadership, fulfilling the psychological contract, proactive work behavior, workplace well-being, and life-related well-being) at the individual level (n = 1000). The most common method used to estimate parameters in CFA models is Sustainability 2023, 15, 11059 9 of 16 maximum likelihood (ML) because of its attractive statistical properties (i.e., asymptotic unbiasedness, normality, consistency, and maximal efficiency) [69]. ...
Proactivity is a particularly important attribute of knowledge-intensive companies, where work that requires enhancing the potential of knowledge-intensive employees in a sustainable working environment is crucial. Another important challenge for these firms is to account for the increasing importance of the functioning of the cognitive mechanisms leading to the increased well-being of knowledge workers following the implementation of a psychological contract. The aim of this article is to identify the relationship between inclusive leadership, the fulfilment of a psychological contract, two dimensions of well-being (workplace and life-related well-being), and knowledge workers’ proactivity. Based on survey data collected using the CAWI method from 1000 knowledge workers employed in Polish companies in the business services sector, the research hypotheses proposed in this study were tested using a stepwise equation-modelling (SEM) technique, which resulted in a model containing all the main constructs. The results obtained indicate that inclusive leadership positively relates to the fulfilment of the psychological contract. Furthermore, the fulfilment of the psychological contract positively associates proactive working behavior with the wellbeing of knowledge workers. Along with proactive work behavior, two dimensions of well-being were examined as outcome variables. Our analysis also shows that knowledge-intensive organizations, intending to develop the proactivity of their employees and nurture a high level of well-being in their lives and in the workplace, should ensure that they fulfil the expectations and obligations of the psychological contract. One way to achieve this is for managers to employ an inclusive management style, which supports an atmosphere of a safe working environment in a diverse setting and allows employees to feel comfortable sharing their opinions and ideas. The study of inclusive leadership in the context of knowledge-intensive organizations provides human resource professionals and employee managers with important insights into how inclusive leadership can effectively contribute to the psychological contract, which, consequently, will lead to proactive work behavior and improve employees’ workplace and life-related well-being.
... The robust maximum likelihood estimator (MLR) was employed, as all observed variables were treated as ordinal, and parameters were calculated accordingly. Some studies recommend using MLR when data distributions are not normal (Rhemtulla et al., 2012) because it typically produces less biased standard error estimates and more accurate calculations of factor correlations (Li, 2016), which is why it has been the chosen estimator in this case. Following the original PIC-A manual (Artola González et al., 2012), a two-factor model was tested: (1) Narrative Creativity, which included scores from the Fluency, Flexibility, and Narrative Originality tests; and (2) Graphic Creativity, which included scores from the Originality, Elaboration, Special Details, and Title tests. ...
... The recommended sample size for conducting confirmatory factor analysis for assessment of structural validity is at least 150 individuals. 72 Therefore, confirmatory factor analysis was not conducted in this study due to the small sample size, which is used to test whether the data fit the hypothesized factor structure. 50 This study is a secondary analysis of previous studies and for one of those studies, the data of an instrument used for construct validity was not collected, which may limit external validity of our study. ...
... In this analysis, we aimed to evaluate and compare two different models of PAS: a one-factor model based on Shneidman's conceptualization of psychache as a single construct and a two-factor model proposed in existing literature that divides the scale into two dimensions: one measuring frequency and the other intensity of psychache. CFA was conducted using the Weighted Least Squares Mean and Variance adjusted (WSLMV) estimator ("lavaan" R package) for categorical data, which is also robust to nonnormality [21]. Model fit was assessed via the Comparative Fit Index (CFI), Tucker Lewis Index (TLI), and Root Mean Square Error of Approximation (RMSEA), with CFI > 0.95, TLI > 0.95, and RMSEA < 0.06 indicating good fit, and CFI > 0.90, TLI > 0.90, and RMSEA < 0.08 indicating acceptable fit [22]. ...
Purpose
This study aims to psychometrically validate the Psychache Scale (PAS) and investigate its prognostic value in predicting postoperative outcomes.
Methods
This is a prospective single-center study. Adults undergoing lumbar or thoracolumbar surgery were recruited. Participants completed PAS preoperatively and patient-reported outcome measures evaluating mental health, pain, physical function, and disability preoperatively and at one and six months postoperatively. PAS internal consistency was evaluated by Cronbach’s alpha coefficient, and factor structure was evaluated using confirmatory factor analysis. Construct validity was assessed by examining correlations between PAS and measures of mental and physical health. PAS prognostic utility was evaluated by assessing its association with short- and longer-term surgical outcomes.
Results
We included 166 patients. Mean (SD) age was 59.7 (12) years, with 55% females. PAS reliability was high (Cronbach’s alpha = 0.95), and factor analysis confirmed the hypothesized one-factor structure. PAS showed strong correlations with PHQ-9 (r = 0.64), PROMIS anxiety (r = 0.64), pain catastrophizing scale (PCS) (r = 0.7), and its helplessness (r = 0.72), magnification (r = 0.59), and rumination (r = 0.59) subscales. However, it shows weak to moderate correlations with non-mental health-related metrics (0.07 < r < 0.44). Preoperative PAS was moderately correlated with one-month pain interference, and six-month PHQ-9 and PROMIS anxiety scores. In predicting outcomes, the addition of PAS to models including baseline values improved the prediction of all outcomes except for PROMIS physical function.
Conclusions
Our study suggests PAS may be a valuable tool for assessing psychological distress in this patient population. Further research is needed to understand its relevance in spine surgery practice.
Level of Evidence
II.
... The SEM analyses are performed using the robust maximum likelihood estimation method (MLR), so that the violation of the normal distribution is taken into account in the calculation of standard errors and test statistics (Li, 2016). All the values were estimated using the full information maximum likelihood (FIML) method (Grimm & Wagner, 2020), whereby all cases with no valid values were removed. ...
The global digital transformation poses challenges for schools and teachers. The importance of digitalization-related professional development (PD) in overcoming these challenges is widely recognized, even if only few studies confirm the desired success. This article explores the relationship between the participation of teachers in digitalization-related PD and variables that are associated with successful PD in an international context. The structural equation modeling analyses are based on data from the International Computer and Information Literacy Study (ICILS 2018). Data from Chile (N = 1682 teachers), Denmark (N = 1108), Germany (N = 2303), the Republic of Korea (N = 2122), and the USA (N = 3174) are considered. These five countries are located on four different continents and have divergent conditions for working with ICT in schools and teacher PD. The results reveal significant relations between the participation in digitalization-related PD and positive views on using ICT in schools, the emphasis on promoting ICT-related skills among students, and the frequency of the use of ICT in the classroom across all countries. These findings demonstrate the importance of PD in the context of the digital transformation in schools, irrespective of international variations in school systems, digitalization processes, and conditions for PD.
... (Rosseel 2012). The models were estimated using diagonally weighted least squares, with mean and variance adjustment (WLSMV) to accommodate categorical variables Barger et al. 2020;Li 2015). Following Kline (2015), we estimated Models 1, 2, and 4 iteratively revising them by allowing items to covary until an acceptable fit was obtained. ...
Early warning indicator and intervention systems (EWS) have been promoted to identify students at risk of school underperformance or dropout. Current EWS systems typically include administrative data on attendance, behavior incidents requiring disciplinary action, and course performance. This study tested whether specific emotional and behavioral risk symptoms measured by a student self‐report universal screener administered in the fall can predict the three EWS indicators after controlling for fall behavioral incidents and whether they account for some of the variance in EWS attributed to demographic characteristics. Using data from 3307 middle school students, we found that after accounting for fall disciplinary issues, conduct problems predicted poorer student outcomes, but hyperactivity/inattention was predictive of better course performance. Peer problems predicted lower performance in some courses, while emotional problems predicted better performance as well as fewer behavioral issues. We also found that these symptoms accounted for some of the variance in EWS attributed to race and gender. The results suggest that student self‐report universal screening can complement existing EWS measures and potentially identify at‐risk students who may not otherwise be identified through traditional EWS indicators.
... Notes. Fit statistics were estimated using diagonally weighted least squares with robust standard errors (WLSMV) (Lei, 2009;Li, 2016). Values with asterisks satisfy the cutoff criteria suggested by Hu and Bentler (1999). ...
Despite growing recognition of the need for cross-national or cross-cultural validation of measures in social psychological research, tension persists between proponents of measurement invariance and practitioners frustrated with stringent standards and ambiguous recommendations. This article critiques common applications of measurement invariance standards and proposes an alternative method for establishing cross-group validity. We highlight how measurement invariance emerged from concerns about fairness in high-stakes individual selections and is based on meta-theoretical assumptions usually irrelevant for drawing cross-societal comparisons. Using the General System Justification Scale as an example, we demonstrate how reliance on a nomological network can ensure meaningful group differences without meeting invariance criteria and show how non-invariance is preferable to approximate (or partial) invariance. We recommend that psychologists interested in cross-group comparisons isolate construct-relevant factors from method bias. Doing so requires defining a priori the goal of scale use and what is “societal” or “cultural” about what is being measured.
... Notes. Fit statistics were estimated using diagonally weighted least squares with robust standard errors (WLSMV) (Lei, 2009;Li, 2016). Values with asterisks satisfy the cutoff criteria suggested by Hu and Bentler (1999). ...
Despite growing recognition of the need for cross-national or cross-cultural validation of measures in social psychological research, tension persists between proponents of measurement invariance and practitioners frustrated with stringent standards and ambiguous recommendations. This article critiques common applications of measurement invariance standards and proposes an alternative method for establishing cross-group validity. We highlight how measurement invariance emerged from concerns about fairness in high-stakes individual selections and is based on meta-theoretical assumptions usually irrelevant for drawing cross-societal comparisons. Using the General System Justification Scale as an example, we demonstrate how reliance on a nomological network can ensure meaningful group differences without meeting invariance criteria and show how non-invariance is preferable to approximate (or partial) invariance. We recommend that psychologists interested in cross-group comparisons isolate construct-relevant factors from method bias. Doing so requires defining a priori the goal of scale use and what is “societal” or “cultural” about what is being measured.
... A confirmatory factor analysis was run with the lavaan package version 0.6-18, fitting the EPSI's eight-factor structure. Similar to the original validation studies [1], weighted least squares mean and variance adjusted (WLSMV) was used as estimation method as it is a robust estimator for ordinal data [26,27]. Model fit was evaluated with the Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), and Tucker-Lewis Index (TLI; also called non-normed fit index) according to the guidelines by Schermelleh-Engel and colleagues [28], who recommend interpreting RMSEA values between 0.05 and 0.08 and CFI/TLI values between 0.95 and 0.97 as indicating acceptable fit, and RMSEA values ≤ 0.05 and CFI/TLI values ≥ 0.97 as indicating good fit. ...
Background
The Eating Pathology Symptoms Inventory (EPSI) is a multidimensional self-report measure for the assessment of eating pathology and related aspects: Body Dissatisfaction, Binge Eating, Cognitive Restraint, Purging, Restricting, Excessive Exercise, Negative Attitudes Toward Obesity, and Muscle Building. The aims of the current studies were to provide a German translation of the EPSI and replicate the original EPSI’s psychometric properties and correlates.
Methods
In two cross-sectional surveys using convenience samples (n = 361 and n = 178), participants completed the German EPSI along with other questionnaires.
Results
In both studies, the EPSI’s eight-factor structure, high internal consistencies, and differential associations with other instruments assessing eating disorder-specific and general psychopathology as well as gender differences on the EPSI’s scales were largely replicated.
Conclusions
The German EPSI had sound psychometric properties that allow for a reliable and valid, multidimensional assessment of eating-disorder psychopathology.
... As teachers were nested in schools, we further requested cluster-robust standard errors by specifying TYPE = COMPLEX. Hence, a sandwich estimator was employed to statistically correct standard errors for the nestedness of teachers in schools (Li, 2016). Without such an adjustment, standard errors of the regression coefficients would have been underestimated, resulting in an overestimation of the coefficients' significance (McNeish et al., 2017). ...
Despite a multitude of studies in organizational behavior research invested in understanding the interrelationship among personality traits, teacher-leader relationship (LMX), creativity, and innovative behaviors over the past decades, these concepts have not attracted much attention in education. The present research concerned how innovative teaching can be enhanced through teacher openness to experience, creativity and LMX. More specifically, we examined the relationship between teacher openness to experience and teachers' implementation of innovative teaching ideas through mediating and moderating roles of LMX and teacher creativity. Employing a stratified sampling strategy, the data was collected from 3016 teachers nested within 148 schools across different regions in Malaysia. A latent moderated mediation analysis was utilized to test ten hypotheses. Results showed that teachers’ openness to experience was a significant predictor of their creativity, the quality of their relationship with the school leader (LMX) and innovative teaching practices. A significant and indirect relationship between openness and the implementations of innovations in the classroom was also evident through the mediating roles of creativity and LMX. However, we found no evidence for the moderating role of LMX and creativity in the effect of both openness and creativity on the implementation of innovations. We conclude that creating space for teachers that support willingness for new experiences could help establishing better relationships with principals, which together could enhance the development and implementation of creative ideas in classrooms that might address issues with and enhance student learning.
... Con la segunda submuestra (n=640) se llevó a cabo un análisis factorial confirmatorio (AFC), mediante el programa EQS 6.1, sobre el modelo de dos dimensiones obtenido a partir del AFE y el unidimensional propuesto inicialmente por los autores. El AFC fue aplicado sobre la matriz de correlaciones y los parámetros fueron estimados con el método de Máxima Verosimilitud robusto, método recomendable cuando no es posible probar la normalidad multivariada (Jaccard, 2018) o son datos ordinales (Li, 2016 (Byrne, 2016). Sobre el modelo de medida que presentó mejores ajustes se efectuaron los análisis de consistencia interna y la validez discriminante y convergente, con el programa SmartPLS (Ringle et al., 2024). ...
Resumen El objetivo de la presente investigación fue poner a prueba la dimensionalidad de la Escala de Orientación a la Dominancia Social (SDO), la cual mide el grado en que los individuos respaldan los valores antiigualitarios y apoyan y perpetúan los sistemas jerárquicos de desigualdad basados en grupos, y evaluar sus propiedades psicométricas, al ser aplicado a una muestra de adolescentes chilenos. La muestra consistió en 1,280 estudiantes chilenos de 7º y 8º curso de educación básica y de 1º a 4º de educación media. Se observó un mejor ajuste bifactorial y las dos dimensiones del SDO obtuvieron buenos índices de fiabilidad y adecuada validez interna convergente y discriminante. Los resultados obtenidos indican que la escala SDO es un instrumento válido y confiable para medir la Orientación a la Dominancia Social en adolescentes chilenos. Abstract This study assessed the dimensionality and psychometric properties of the Social Dominance Orientation Scale (SDO), which measures individuals' endorsement of anti-egalitarian values, and support and perpetuate hierarchical group-based systems of inequality. The sample included of 1,280 Chilean students from 7th and 8th grade of basic education and from 1st to 4th grade of secondary education. The scale showed good reliability indices and adequate convergent and discriminant validity. A better adjustment of the bifactorial model was observed. The results indicate that the SDO scale is a reliable and structurally valid instrument to measure the Social Dominance Orientation in Chilean adolescents.
This study investigates the influence of digital skills, personal innovativeness, and attitudes toward smart home adoption on digital consumer behavior in Poland, as well as the relationship between digital activity and environmental awareness. In the context of growing interest in sustainable development and digital responsibility, the research aims to identify the psychosocial and technological determinants of conscious online behaviors. The study employs a structured survey (n = 1246) using validated scales, which were analyzed through Confirmatory Factor Analysis (CFA) and Structural Equation Modeling (SEM). The findings reveal that personal innovativeness is the strongest predictor of digital consumer behavior, followed by digital skills and, to a lesser extent, smart home adoption. Moreover, digitally active consumers demonstrate significantly higher levels of environmental awareness, suggesting that digital engagement fosters pro-environmental attitudes. Gender differences were observed in the strength of these relationships, with digital skills and innovativeness having a greater impact on women’s behavior, while smart home technology attitudes were more relevant among men. These results contribute to the understanding of digital consumer responsibility and its environmental implications, highlighting the importance of digital competence development and technological openness in promoting sustainable consumption patterns.
Introduction: Suicidality is prevalent in Iran; however, psychometric properties of the scales assessing suicide-related risk factors (i.e., interpersonal needs and capability for suicide) are unavailable in Iran. Thus, this study examined the factor structure, measurement invariance, and convergent validity of the Farsi versions of the Interpersonal Needs Questionnaire (F-INQ) and the Acquired Capability with Rehearsal for Suicide Scale (F-ACWRSS). Methods: Participants were community members (n = 773; 69.1% women) who filled out the F-INQ and F-ACWRSS, as well as scales assessing suicide ideation, suicide attempts, and hopelessness. Analyses focused on validating the factor structure, internal consistencies, convergent validity, and testing measurement invariance of the F-INQ and F-ACWRSS by sex assigned at birth. Results: The original two-factor structure of the F-INQ and a three-factor structure of the F-ACWRSS were supported. Additionally, both the F-INQ and F-ACWRSS were invariant across sex, indicating that the scales perform similarly by sex. Men reported a higher score than women on the pain tolerance subscale of the F-ACWRSS. Both the F-INQ and F-ACWRSS subscales were associated with suicide ideation, suicide attempts, and hopelessness, supporting the convergent validity of the scales. Discussion: Findings suggest that the F-INQ and F-ACWRSS can aid researchers examining suicide-related risk factors in Iran.
Background/Objectives: The complexity of modern healthcare requires teamwork. Healthcare teams must be cohesive to ensure efficient and quality care. The objective of this study was to validate the Spanish short version of the Group Environment Questionnaire (GEQ) in undergraduate nursing students undergoing clinical simulation training. Methods: The psychometric properties of the questionnaire were analyzed. We began with a statistical analysis of the items. Internal consistency was assessed using McDonald’s omega coefficient. Test-retest reliability was determined using Spearman’s correlation coefficient. An exploratory factor analysis was performed using the principal components analysis method with varimax rotation. Finally, a confirmatory factor analysis of the proposed theoretical models was performed to select the most appropriate one using the weighted least squares method adjusted for mean and variance (WLSMV) and goodness-of-fit indices. Results: The questionnaire items showed a standard deviation higher than 1 and a negative skewness lower than 0.5. Internal consistency and test-retest reliability were higher than 0.8. The item-total correlation coefficient values were above 0.44. The exploratory factor analysis confirmed the presence of four factors, each with three items. Confirmatory factor analysis determined that the four-factor cohesion model was the one that obtained the best fit. Conclusions: The Spanish short version of the Group Environment Questionnaire (GEQ) is a valid and reliable instrument for analyzing group cohesion in teams of nursing students undergoing clinical simulation training.
Background
Existent research examining perceptions of nicotine addiction use largely surface level items that fail to address the complexity of nicotine addiction. Additional investigation is needed to better understand people’s perceptions of nicotine addiction and whether these align with measures of nicotine dependence. Understanding these perceptions about nicotine addiction may help provide insight into vaping intentions and behavior. This study examines the validity of the Nicotine Addiction Perceptions (NAP) scale, a novel measure designed to assess perceptions of addictive vaping behavior that aligns with the clinical dimensions of tobacco use disorder.
Methods
Data were collected from U.S. adults via Prolific (n = 549). As part of scale development and validation a confirmatory factor analysis and psychometric evaluation was conducted. The NAP’s reliability, convergent, discriminant, and criterion validity were established.
Results
A five-factor solution returned acceptable fit on all model indices (RMSEA = 0.050; CFI = 0.994; TLI = 0.993). The NAP was significantly related to assessments of perceived risk, 6 month quit intentions, the number of quit intentions over the past year, and past 30-day e-cigarette use ( P’s < .05). Findings also indicate support for discriminant validity.
Conclusions
Findings suggest that for most, perceptions of nicotine addiction may not fully align with the clinical criteria of addiction, which may be due to the lack of education surrounding the clinical definition. Future research examining perceptions of nicotine addiction can utilize the NAP scale to better understand people’s understanding of addiction and its relationship to vaping-related behavior.
We present a novel approach for computing model scores for ordinal factor models, that is, graded response models (GRMs) fitted with a limited information (LI) estimator. The method makes it possible to compute score‐based tests for parameter instability for ordinal factor models. This way, rapid execution of numerous parameter instability tests for multidimensional item response theory (MIRT) models is facilitated. We present a comparative analysis of the performance of the proposed score‐based tests for ordinal factor models in comparison to tests for GRMs fitted with a full information (FI) estimator. The new method has a good Type I error rate, high power and is computationally faster than FI estimation. We further illustrate that the proposed method works well with complex models in real data applications. The method is implemented in the lavaan package in R.
Dysfunctional metacognitive beliefs have been established as a transdiagnostic correlate of anxiety and depression in adults and are considered a central mechanism of persistent emotional distress according to the metacognitive model of psychological disorders. However, the importance of metacognitions for distress and emotional disorder in adolescence is far less researched, and to investigate this further there is a need for reliable and valid assessment tools. The Metacognitions Questionnaire-30 (MCQ-30) might be a suitable and valid assessment tool in adolescents with the implication that it can be used to research the role of metacognitions in youth mental health and track how metacognitions change and relate to outcomes over time from adolescence to adulthood. We therefore aimed to examine the psychometric properties of the MCQ-30 in an Upper Secondary School-sample of 494 Norwegian adolescents, aged 16- to 18-years old. A confirmatory factor analysis indicated a good fit for the proposed five-factor structure and evidence for measurement invariance was supported across sex and groups of anxiety severity. The factors showed acceptable to good internal consistency and there was support for convergent validity. In conclusion, these findings indicate that the adult version of the MCQ-30 can be applied in Norwegian adolescents from 16-years old.
Purpose This study evaluated the validity and reliability of the Brazilian Screen Version of the Ritvo Autism and Asperger Diagnostic Scale-Revised (RAADS-R-BR), comparing the original instrument's response categories with a version adapted to the Brazilian context. Methods A total of 627 participants aged 16 to 66 took part, divided into autistic (N = 352) and non-autistic (N = 275) groups, with 72.28% self-identified as female. The adequacy of the factor structure was assessed via Confirmatory Factor Analysis (CFA), and sensitivity and specificity were examined through ROC Curve analysis. Results The bifactor Screen model with modified response categories demonstrated excellent fit [χ²(169) = 318.443, p < 0.001, χ²/df = 1.884, CFI = 0.956, TLI = 0.950, SRMR = 0.081, RMSEA = 0.058 (90% CI = 0.048–0.067)], with significant factor loadings (λ>|0.30|, p < 0.05) and high internal reliability (≥ 0.80). Conclusion The ROC Curve indicated 90.1% sensitivity and 87.9% specificity, showing that the RAADS-R-BR Screen version presents suitable psychometric parameters for the Brazilian context.
This study details the cross-cultural adaptation of the Ritvo Autism Asperger Diagnostic Scale-Revised (RAADS-R) to Brazilian Portuguese (RAADS-R-BR). The adaptation process involved translation, synthesis, expert review (n = 5), back-translation, validation by the original author, and a pilot study (n = 142) with autistic adults aged 19–60 (M = 33.49, SD = 8.44). Data collection was conducted via Google Meet by trained assistants. Confirmatory Factor Analysis (CFA) evaluated the instrument’s structure using indices like CFI, TLI, RMSEA, and SRMR. Internal consistency was measured through Cronbach’s α, McDonald’s ω, and CR, with item parameters analyzed using IRT. Semantic equivalence was maintained, showing satisfactory coefficients for CL (0.89), PP (0.87), and RL (0.88), and participant comprehension above 80%. CFA revealed 27 items with non-significant factor loadings in both four-factor and two-factor models, indicating structural limitations. The RAADS-14 screen model had excellent fit indices [χ²(74) = 88.072, p = 0.126], though items Q60 and Q30 were not significant. Results indicate weak performance in analyses using Classical Test Theory and Item Response Theory. Future research should revise problematic items and consider creating a shortened version of RAADS-R-BR with items showing satisfactory discrimination.
Monte Carlo computer simulations were used to investigate the performance of three χ–2 test statistics in confirmatory factor analysis (CFA). Normal theory maximum likelihood χ–2 (ML), Browne's asymptotic distribution free χ–2 (ADF), and the Satorra-Bentler rescaled χ–2 (SB) were examined under varying conditions of sample size, model specification, and multivariate distribution. For properly specified models, ML and SB showed no evidence of bias under normal distributions across all sample sizes, whereas ADF was biased at all but the largest sample sizes. ML was increasingly overestimated with increasing nonnormality, but both SB (at all sample sizes) and ADF (only at large sample sizes) showed no evidence of bias. For misspecified models, ML was again inflated with increasing nonnormality, but both SB and ADF were underestimated with increasing nonnormality. It appears that the power of the SB and ADF test statistics to detect a model misspecification is attenuated given nonnormally distributed data. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Ordinal variables are common in many empirical investigations in the social and behavioral sciences. Researchers often apply the maximum likelihood method to fit structural equation models to ordinal data. This assumes that the observed measures have normal distributions, which is not the case when the variables are ordinal. A better approach is to use polychoric correlations and fit the models using methods such as unweighted least squares (ULS), maximum likelihood (ML), weighted least squares (WLS), or diagonally weighted least squares (DWLS). In this simulation evaluation we study the behavior of these methods in combination with polychoric correlations when the models are misspecified. We also study the effect of model size and number of categories on the parameter estimates, their standard errors, and the common chi-square measures of fit when the models are both correct and misspecified. When used routinely, these methods give consistent parameter estimates but ULS, ML, and DWLS give incorrect standard errors. Correct standard errors can be obtained for these methods by robustification using an estimate of the asymptotic covariance matrix W of the polychoric correlations. When used in this way the methods are here called RULS, RML, and RDWLS.
In covariance structure modeling several estimation methods are available. The robustness of an estimator against specific violations of assumptions can be determined empirically by means of a Monte Carlo study. Many such studies in covariance structure analysis have been published, but the conclusions frequently seem to contradict each other An overview of robustness studies in covariance structure analysis is given, and an attempt is made to generalize their findings. Robustness studies are described and distinguished from each other systematically by means of certain characteristics. These characteristics serve as explanatory variables in a meta-analysis concerning the behavior of parameter estimators, standard error estimators, and goodness-of-fit statistics when the model is correctly specified.
The use of Monte Carlo simulations for the empirical assessment of statistical estimators is becoming more common in structural equation modeling research. Yet, there is little guidance for the researcher interested in using the technique. In this article we illustrate both the design and implementation of Monte Carlo simulations. We present 9 steps in planning and performing a Monte Carlo analysis: (1) developing a theoretically derived research question of interest, (2) creating a valid model, (3) designing specific experimental conditions, (4) choosing values of population parameters, (5) choosing an appropriate software package, (6) executing the simulations, (7) file storage, (8) troubleshooting and verification, and (9) summarizing results. Throughout the article, we use as a running example a Monte Carlo simulation that we performed to illustrate many of the relevant points with concrete information and detail.
This simulation study compared maximum likelihood (ML) estimation with weighted least squares means and variance adjusted (WLSMV) estimation. The study was based on confirmatory factor analyses with 1, 2, 4, and 8 factors, based on 250, 500, 750, and 1,000 cases, and on 5, 10, 20, and 40 variables with 2, 3, 4, 5, and 6 categories. There was no model misspecification. The most important results were that with 2 and 3 categories the rejection rates of the WLSMV chi-square test corresponded much more to the expected rejection rates according to an alpha level of. 05 than the rejection rates of the ML chi-square test. The magnitude of the loadings was more precisely estimated by means of WLSMV when the variables had only 2 or 3 categories. The sample size for WLSMV estimation needed not to be larger than the sample size for ML estimation.
Treating Likert rating scale data as continuous outcomes in confirmatory factor analysis violates the assumption of multivariate normality. Given certain requirements pertaining to the number of categories, skewness, size of the factor loadings, and so forth, it seems nevertheless possible to recover true parameter values if the data stem from a single homogeneous population. It is shown that, in a multigroup context, an analysis of Likert data under the assumption of multivariate normality may distort the factor structure differently across groups. In that case, investigations of measurement invariance (MI), which are necessary for meaningful group comparisons, are problematic. Analyzing subscale scores computed from Likert items does not seem to solve the problem.
Factor analysis models with ordinal indicators are often estimated using a 3-stage procedure where the last stage involves obtaining parameter estimates by least squares from the sample polychoric correlations. A simulation study involving 324 conditions (1,000 replications per condition) was performed to compare the performance of diagonally weighted least squares (DWLS) and unweighted least squares (ULS) in the procedure's third stage. Overall, both methods provided accurate and similar results. However, ULS was found to provide more accurate and less variable parameter estimates, as well as more precise standard errors and better coverage rates. Nevertheless, convergence rates for DWLS are higher. Our recommendation is therefore to use ULS, and, in the case of nonconvergence, to use DWLS, as this method might converge when ULS does not.
An investigation of the distributional characteristics of 440 large-sample achievement and psychometric measures found all to be significantly nonnormal at the alpha .01 significance level. Several classes of contamination were found, including tail weights from the uniform to the double exponential, exponential-level asymmetry, severe digit preferences, multimodalities, and modes external to the mean/median interval. Thus, the underlying tenets of normality-assuming statistics appear fallacious for these commonly used types of data. However, findings here also fail to support the types of distributions used in most prior robustness research suggesting the failure of such statistics under nonnormal conditions. A reevaluation of the statistical robustness literature appears appropriate in light of these findings. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
A simulation study compared the performance of robust normal theory maximum likelihood (ML) and robust categorical least squares (cat-LS) methodology for estimating confirmatory factor analysis models with ordinal variables. Data were generated from 2 models with 2-7 categories, 4 sample sizes, 2 latent distributions, and 5 patterns of category thresholds. Results revealed that factor loadings and robust standard errors were generally most accurately estimated using cat-LS, especially with fewer than 5 categories; however, factor correlations and model fit were assessed equally well with ML. Cat-LS was found to be more sensitive to sample size and to violations of the assumption of normality of the underlying continuous variables. Normal theory ML was found to be more sensitive to asymmetric category thresholds and was especially biased when estimating large factor loadings. Accordingly, we recommend cat-LS for data sets containing variables with fewer than 5 categories and ML when there are 5 or more categories, sample size is small, and category thresholds are approximately symmetric. With 6-7 categories, results were similar across methods for many conditions; in these cases, either method is acceptable. (PsycINFO Database Record (c) 2012 APA, all rights reserved).
Of the several measures of optimism presently available in the literature, the Life Orientation Test (LOT; Scheier & Carver, 1985) has been the most widely used in empirical research. This article explores, confirms, and cross-validates the factor structure of the Chinese version of the LOT with ordinal data by using robust weighted least squares (robust WLS) estimation within the Taiwanese cultural context. Results of exploratory and confirmatory factor analyses using 3 different samples (Ntotal = 1,119) show that the factor structure of the Chinese version of the LOT is better conceptualized as a correlated 2-factor model than a single-factor model. The composite reliability was 0.7 for the "disagreement on optimism" factor and 0.74 for the "agreement on optimism" factor. In addition, comparison results of the 2 estimators using empirical data and simulation data suggest that robust WLS is less biased than maximum likelihood (ML) for estimating factor loadings and interfactor correlations in the factor analytic model of the Chinese version of the LOT. (PsycINFO Database Record (c) 2012 APA, all rights reserved).
Maximum likelihood is the most common estimation method in structural equation modeling. Standard errors for maximum likelihood estimates are obtained from the associated information matrix, which can be estimated from the sample using either expected or observed information. It is known that, with complete data, estimates based on observed or expected information are consistent. The situation changes with incomplete data. When the data are missing at random (MAR), standard errors based on expected information are not consistent, and observed information should be used. A less known fact is that in the presence of nonnormality, the estimated information matrix also enters the robust computations (both standard errors and the test statistic). Thus, with MAR nonnormal data, the use of the expected information matrix can potentially lead to incorrect robust computations. This article summarizes the results of 2 simulation studies that investigated the effect of using observed versus expected information estimates of standard errors and test statistics with normal and nonnormal incomplete data. Observed information is preferred across all conditions. Recommendations to researchers and software developers are outlined.
According to Kenny and McCoach (2003), chi-square tests of structural equation models produce inflated Type I error rates when the degrees of freedom increase. So far, the amount of this bias in large models has not been quantified. In a Monte Carlo study of confirmatory factor models with a range of 48 to 960 degrees of freedom it was found that the traditional maximum likelihood ratio statistic, TML, overestimates nominal Type I error rates up to 70% under conditions of multivariate normality. Some alternative statistics for the correction of model-size effects were also investigated: the scaled Satorra-Bentler statistic, TSC; the adjusted Satorra-Bentler statistic, TAD (Satorra & Bentler, 1988, 1994); corresponding Bartlett corrections, TMLb, TSCb, and TADb (Bartlett, 1950); and corresponding Swain corrections, TMLs , TSCs , and TADs (Swain, 1975). The empirical findings indicate that the model test statistic TMLs should be applied when large structural equation models are analyzed and the observed variables have (approximately) a multivariate normal distribution.
The performance of parameter estimates and standard errors in estimating F. Samejima’s graded response model was examined across 324 conditions. Full information maximum likelihood (FIML) was compared with a 3-stage estimator for categorical item factor analysis (CIFA) when the unweighted least squares method was used in CIFA’s third stage. CIFA is much faster in estimating multidimensional models, particularly with correlated dimensions. Overall, CIFA yields slightly more accurate parameter estimates, and FIML yields slightly more accurate standard errors. Yet, across most conditions, differences between methods are negligible. FIML is the best election in small sample sizes (200 observations). CIFA is the best election in larger samples (on computational grounds). Both methods failed in a number of conditions, most of which involved 200 observations, few indicators per dimension, highly skewed items, or low factor loadings. These conditions are to be avoided in applications.
Reporting practices in 194 confirmatory factor analysis studies (1,409 factor models) published in American Psychological Association journals from 1998 to 2006 were reviewed and compared with established reporting guidelines. Three research questions were addressed: (a) how do actual reporting practices compare with published guidelines? (b) how do researchers report model fit in light of divergent perspectives on the use of ancillary fit indices (e.g., L.-T. Hu & P. M. Bentler, 1999; H. W. Marsh, K.-T., Hau, & Z. Wen, 2004)? and (c) are fit measures that support hypothesized models reported more often than fit measures that are less favorable? Results indicate some positive findings with respect to reporting practices including proposing multiple models a priori and near universal reporting of the chi-square significance test. However, many deficiencies were found such as lack of information regarding missing data and assessment of normality. Additionally, the authors found increases in reported values of some incremental fit statistics and no statistically significant evidence that researchers selectively report measures of fit that support their preferred model. Recommendations for reporting are summarized and a checklist is provided to help editors, reviewers, and authors improve reporting practices.
The polychoric correlation is discussed as a generalization of the tetrachoric correlation coefficient to more than two classes. Two estimation methods are discussed: Maximum likelihood estimation, and what may be called two-step maximum likelihood estimation. For the latter method, the thresholds are estimated in the first step. For both methods, asymptotic covariance matrices for estimates are derived, and the methods are illustrated and compared with artificial and real data.
We describe a general procedure by which any number of parameters of the factor analytic model can be held fixed at any values and the remaining free parameters estimated by the maximum likelihood method. The generality of the approach makes it possible to deal with all kinds of solutions: orthogonal, oblique and various mixtures of these. By choosing the fixed parameters appropriately, factors can be defined to have desired properties and make subsequent rotation unnecessary. The goodness of fit of the maximum likelihood solution under the hypothesis represented by the fixed parameters is tested by a large samplex
2 test based on the likelihood ratio technique. A by-product of the procedure is an estimate of the variance-covariance matrix of the estimated parameters. From this, approximate confidence intervals for the parameters can be obtained. Several examples illustrating the usefulness of the procedure are given.
Confirmatory factor analysis (CFA) is widely used for examining hypothesized relations among ordinal variables (e.g., Likert-type items). A theoretically appropriate method fits the CFA model to polychoric correlations using either weighted least squares (WLS) or robust WLS. Importantly, this approach assumes that a continuous, normal latent process determines each observed variable. The extent to which violations of this assumption undermine CFA estimation is not well-known. In this article, the authors empirically study this issue using a computer simulation study. The results suggest that estimation of polychoric correlations is robust to modest violations of underlying normality. Further, WLS performed adequately only at the largest sample size but led to substantial estimation difficulties with smaller samples. Finally, robust WLS performed well across all conditions.
We describe a general procedure by which any number of parameters of the factor analytic model can be held fixed at any values and the remaining free parameters estimated by the maximum likelihood method. The generality of the approach makes it possible to deal with all kinds of solutions: orthogonal, oblique and various mixtures of these. By choosing the fixed parameters appropriately, factors can be defined to have desired properties and make subsequent rotation unnecessary. The goodness of fit of the maximum likelihood solution under the hypothesis represented by the fixed parameters is tested by a large sample χ2 test based on the likelihood ratio technique. A by-product of the procedure is an estimate of the variance-covariance matrix of the estimated parameters. From this, approximate confidence intervals for the parameters can be obtained. Several examples illustrating the usefulness of the procedure are given.
Monte Carlo computer simulations were used to investigate the performance of three χ2 test statistics in confirmatory factor analysis (CFA). Normal theory maximum likelihood χ2 (ML), Browne's asymptotic distribution free χ2 (ADF), and the Satorra-Bentler rescaled χ2 (SB) were examined under varying conditions of sample size, model specification, and multivariate distribution. For properly specified models, ML and SB showed no evidence of bias under normal distributions across all sample sizes, whereas ADF was biased at all but the largest sample sizes. ML was increasingly overestimated with increasing nonnormality, but both SB (at all sample sizes) and ADF (only at large sample sizes) showed no evidence of bias. For misspecified models, ML was again inflated with increasing nonnormality, but both SB and ADF were underestimated with increasing nonnormality. It appears that the power of the SB and ADF test statistics to detect a model misspecification is attenuated given nonnormally distributed data.
Model Notation, Covariances, and Path Analysis. Causality and Causal Models. Structural Equation Models with Observed Variables. The Consequences of Measurement Error. Measurement Models: The Relation Between Latent and Observed Variables. Confirmatory Factor Analysis. The General Model, Part I: Latent Variable and Measurement Models Combined. The General Model, Part II: Extensions. Appendices. Distribution Theory. References. Index.
This paper expands on a recent study by Muthen & Kaplan (1985) by examining the impact of non-normal Likert variables on testing and estimation in factor analysis for models of various size. Normal theory GLS and the recently developed ADF estimator are compared for six cases of non-normality, two sample sizes, and four models of increasing size in a Monte Carlo framework with a large number of replications. Results show that GLS and ADF chi-square tests are increasingly sensitive to non-normality when the size of the model increases. No parameter estimate bias was observed for GLS and only slight parameter bias was found for ADF. A downward bias in estimated standard errors was found for GLS which remains constant across model size. For ADF, a downward bias in estimated standard errors was also found which became increasingly worse with the size of the model.
In practice, several measures of association are used when analyzing structural equation models with ordinal variables: ordinary Pearson correlations (PE approach), polychoric and polyserial correlations (PO approach), and conditional polychoric correlations (CPO approach). In the case of structural equation models without latent variables, the literature has shown that the PE approach is outperformed by the alternatives. In this article we report a Monte Carlo study showing the comparative performance of the aforementioned alternative approaches under deviations from their respective assumptions in the case of structural equation models with latent variables when attention is restricted to point estimates of model parameters. The CPO approach is shown to be the most robust against nonnormality. It is also robust to randomness of the exogenous variables, but not to the existence of measurement errors in them. The PO approach lacks robustness against nonnormality. The PE approach lacks robustness against transformation errors but otherwise it can perform about as well as the alternative approaches.
We evaluated whether "more is ever too much" for the number of indicators (p) per factor (p/f) in confirmatory factor analysis by varying sample size (N = 50-1000) and p/f (2-12 items per factor) in 35,000 Monte Carlo solutions. For all N's, solution behavior steadily improved (more proper solutions, more accurate parameter estimates, greater reliability) with increasing p/f. There was a compensatory relation between N and p/f: large p/f compensated for small N and large N compensated for small p/f, but large-N and large-p/f was best. A bias in the behavior of the χ2 was also demonstrated where apparent goodness of fit declined with increasing p/f ratios even though approximating models were "true". Fit was similar for proper and improper solutions, as were parameter estimates form improper solutions not involving offending estimates. We also used the 12-p/f data to construct 2, 3, 4, or 6 parcels of items (e.g., two parcels of 6 items per factor, three parcels of 4 items per factor, etc.), but the 12-indicator (nonparceled) solutions were somewhat better behaved. At least for conditions in our simulation study, traditional "rules" implying fewer indicators should be used for smaller N may be inappropriate and researchers should consider using more indicators per factor that is evident in current practice.
Using simulated data and a multiple indicator approach, examines the problems that surround categorization error. Results indicate that while categorization error does produce distortions in multiple indicator models, under most of the conditions explored, the bias was not sufficient to alter substantive interpretations and the estimates were efficient.-from Authors
A. Satorra and P. Bentler . . . developed an approach to the asymptotic behavior of covariance structure statistics that rather naturally yields corrections to the goodness-of-fit statistic of the scaling and Satterthwaite types / present these results and . . . illustrate how they improve upon the uncorrected statistics that are now implemented in the field of covariance structure analysis / [show] that the proposed corrections not only encompass the ones advocated by A. Shapiro and M. Browne (1987) in case of elliptical data but do not suffer from the drawback of Browne-Shapiro's corrections of lack of robustness against deviations from the assumption of an elliptical distribution / provides a theory for correcting the standard covariance matrix of the vector of parameter estimates (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In this paper, the authors develop test statistics that can be correctly applied to the normal theory maximum likelihood estimator. The authors propose three new asymptotically distribution-free test statistics (T
YB, T
C, and T
F) that technically must yield improved behaviour in samples of realistic size, and use Monte Carlo methods to study their actual finite sample behaviour. Among these three, T
YB performs most stably across different models and distribution conditions. The only drawback of T
YB as a general test statistic is the tendency slightly to overaccept the correct models when sample size is small. The test statistic T
F is very reliable for models with not so large degrees of freedom; however, its rejection is still too high when model degrees of freedom get larger. The rejection rate of the statistic T
C is also too high for small to moderate sample sizes, though it dramatically improves on T
B . A new index is proposed for evaluating whether a resealed statistic will be robust. Recommendations are given regarding the application of each test statistic. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This paper considers the problem of applying factor analysis to non‐normal categorical variables. A Monte Carlo study is conducted where five prototypical cases of non‐normal variables are generated. Two normal theory estimators, ML and GLS, are compared to Browne's (1982) ADF estimator. A categorical variable methodology (CVM) estimator of Muthén (1984) is also considered for the most severely skewed case. Results show that ML and GLS chi‐square tests are quite robust but obtain too large values for variables that arc severely skewed and kurtotic. ADF, however, performs well.
Parameter estimate bias appears non‐existent for all estimators. Results also show that ML and GLS estimated standard errors are biased downward. For ADF no such standard error bias was found. The CVM estimator appears to work well when applied to severely skewed variables that had been dichotomized. ML and GLS results for a kurtosis only case showed no distortion of chi‐square or parameter estimates and only a slight downward bias in estimated standard errors. The results are compared to those of other related studies.
A common question asked by researchers is, "What sample size do I need for my study?" Over the years, several rules of thumb have been proposed. In reality there is no rule of thumb that applies to all situations. The sample size needed for a study depends on many factors, including the size of the model, distribution of the variables, amount of missing data, reliability of the variables, and strength of the relations among the variables. The purpose of this article is to demonstrate how substantive researchers can use a Monte Carlo study to decide on sample size and determine power. Two models are used as examples, a confirmatory factor analysis (CFA) model and a growth model. The analyses are carried out using the Mplus program (Muthén& Muthén 1998).
This study examined the performance of two alternative estimation approaches in structural equation modeling for ordinal data
under different levels of model misspecification, score skewness, sample size, and model size. Both approaches involve analyzing
a polychoric correlation matrix as well as adjusting standard error estimates and model chi-squared, but one estimates model
parameters with maximum likelihood and the other with robust weighted least-squared. Relative bias in parameter estimates
and standard error estimates, Type I error rate, and empirical power of the model test, where appropriate, were evaluated
through Monte Carlo simulations. These alternative approaches generally provided unbiased parameter estimates when the model
was correctly specified. They also provided unbiased standard error estimates and adequate Type I error control in general
unless sample size was small and the measured variables were moderately skewed. Differences between the methods in convergence
problems and the evaluation criteria, especially under small sample and skewed variable conditions, were discussed.
KeywordsEstimation-Ordinal data-Model misspecification-Small sample structural equation modeling
In structural equation modeling the statistician needs assumptions inorder (1) to guarantee that the estimates are consistent for the parameters of interest, and (2) to evaluate precision of the estimates and significance level of test statistics. With respect to purpose (1), the typical type of analyses (ML and WLS) are robust against violation of distributional assumptions; i.e., estimates remain consistent or any type of WLS analysis and distribution of z. (It should be noted, however, that (1) is sensitive to structural misspecification.) A typical assumption used for purpose (2), is the assumption that the vector z of observable follows a multivariate normal distribution.In relation to purpose (2), distributional misspecification may have consequences for efficiency, as well as power of test statistics (see Satorra, 1989a); that is, some estimation methods may bemore precise than others for a given specific distribution of z. For instance, ADF-WLS is asymptotically optimal under a variety of distributions of z, while the asymptotic optimality of NT-WLS may be lost when the data is non-normalViolation of a distributional assumption may have consequences for purpose (2). However, recent theory, such as the one described in Sections 7 and 8, showes that asymptotic variances of estimates and asympttic null distributions of test statistics derived under the normality assumption may be correct even when z is non-normal provided certain model conditions hold (the conditions of Theorem 1). That is, in a specific application with z non-normally distributed, the assumption that z is normal play the role of a working device that facilitates calculation of the correct distribution of statistics of interest. This corresponds to what in Section 7 and 8 has been called asymptotic robustness.For most of the models considered in practice, replacing the assyumption uncorrelation for the assumption of independence implised reaching the properties of asymptotic robustness; in that case, in order to evaluate the asymptotic behavior of statistics of interest, a NT form for produces correct results even for non-normal data. This robustness result applies regardless of the type of fitting criterion used.Distinction between uncorrelation and independence becomes crucial when dealing with the asymptotic robustness issue. Statistical independence among variables of the model guarantee that the distribution of statistics of interest are asymptotically distribution-free of the non-normal variables; thus a NT form for applies. As an example of where such distinction is apparent, consider a simple regression model with a heteroskedastic disturbance term. Here the disturbance term is uncorrelated with the regressor, but the variance varies with the value of the regressor. For a study showing that ADF-WLS protects against heteroskedasticity of erros, while ML wil generally fail, see Mooijaart and Satorra (1987).In regresion analysis the usual method for detecting heteroskedasticity is by looking at residual plots. Presumably, alsi in structural equation modeling, the need to distinguish between uncorrelation and independence will force the researcher to go back to the row data in order to do a similar type of residuals inspection.In concluding, an importance consideration is to compute sampling variability for estimates and test statistics using appropriate formulae, without requiring that the estimation procedure be the best in some sense. We have seen that such computations can be carried out correctly using the wrong assumptions with respect to the distribution of the vector of observable variables, provided some additional model conditions hold. Roughly speaking, such additional model conditions amount to strengthen the usual assumption of uncorrelation among some random constituents of the model to the assumption of stochastic independecen.
A structural equation model is proposed with a generalized measurement part, allowing for dichotomous and ordered categorical variables (indicators) in addition to continuous ones. A computationally feasible three-stage estimator is proposed for any combination of observed variable types. This approach provides large-sample chi-square tests of fit and standard errors of estimates for situations not previously covered. Two multiple-indicator modeling examples are given. One is a simultaneous analysis of two groups with a structural equation model underlying skewed Likert variables. The second is a longitudinal model with a structural model for multivariate probit regressions.
Analysis of Ordinal Categorical Data Alan Agresti Statistical Science Now has its first coordinated manual of methods for analyzing ordered categorical data. This book discusses specialized models that, unlike standard methods underlying nominal categorical data, efficiently use the information on ordering. It begins with an introduction to basic descriptive and inferential methods for categorical data, and then gives thorough coverage of the most current developments, such as loglinear and logit models for ordinal data. Special emphasis is placed on interpretation and application of methods and contains an integrated comparison of the available strategies for analyzing ordinal data. This is a case study work with illuminating examples taken from across the wide spectrum of ordinal categorical applications. 1984 (0 471-89055-3) 287 pp. Regression Diagnostics Identifying Influential Data and Sources of Collinearity David A. Belsley, Edwin Kuh and Roy E. Welsch This book provides the practicing statistician and econometrician with new tools for assessing the quality and reliability of regression estimates. Diagnostic techniques are developed that aid in the systematic location of data points that are either unusual or inordinately influential; measure the presence and intensity of collinear relations among the regression data and help to identify the variables involved in each; and pinpoint the estimated coefficients that are potentially most adversely affected. The primary emphasis of these contributions is on diagnostics, but suggestions for remedial action are given and illustrated. 1980 (0 471-05856-4) 292 pp. Applied Regression Analysis Second Edition Norman Draper and Harry Smith Featuring a significant expansion of material reflecting recent advances, here is a complete and up-to-date introduction to the fundamentals of regression analysis, focusing on understanding the latest concepts and applications of these methods. The authors thoroughly explore the fitting and checking of both linear and nonlinear regression models, using small or large data sets and pocket or high-speed computing equipment. Features added to this Second Edition include the practical implications of linear regression; the Durbin-Watson test for serial correlation; families of transformations; inverse, ridge, latent root and robust regression; and nonlinear growth models. Includes many new exercises and worked examples.
In many areas, covariance structure analysis plays an important role in understanding how the relationship among observed variables might be generated by hypothesized latent variables. Once a model is established as relevant to a given data set, it is important to evaluate the significance of specific parameters, such as coefficients of regressions among latent variables, within the model. The popular z-test of a parameter is the estimator of the parameter divided by its standard error estimator. A valid z-statistic must be based on a high-quality standard error estimator. We focus on the quality of the standard error estimator from both MLE and ADF methods, which are the two most frequently used methods in covariance structure practice. For these two estimation methods, empirical evidence shows that classical formulae give “too optimistic” standard error estimators, with the result that the z-tests regularly give false conclusions. We review one and introduce another simple corrected standard error estimator. These substantially improve on the classical ones, depending on distribution and sample size. Two implications of this study are that significant parameters as printed in most statistical software may not be really significant, and that corrected standard errors should be direct output for the two most widely used methods. A comparison of the accuracy of the estimators based on these two methods is also made.
This study investigated the impact of categorization on confirmatory factor analysis (CFA) parameter estimates, standard errors, and 5 ad hoc fit indexes. Models were generated that represented empirical research situations in terms of model size, sample sizes, and loading values. CFA results obtained from analysis of normally distributed, continuous data were compared to results obtained from 5-category Likert-type data with normal distributions. The ordered categorical data were analyzed using the estimators: Weighted Least Squares (WLS; with polychoric correlation [PC] input) and Maximum Likelihood (ML; with Pearson Product-Moment [PPM] input). ML-PPM-based parameter estimates reported moderate levels of negative bias for all conditions, WLS-PC-based standard errors showed high amounts of bias, especially with a small sample size and moderate loading values. With nonnormally distributed, ordered categorical data, ML-PPM-based parameter estimates, standard errors, and factor intercorrelation showed high levels of bias. Bias levels in standard errors were reduced when the Satorra-Bentler (1988) rescaling correction was applied to nonnormal, ordered categorical data. Five ad hoc model fit indexes appeared robust to the majority of study conditions
With tropical deforestation a major contributor to greenhouse gas emissions and biodiversity loss, the land-use decisions of small-scale farmers at the forest margins have important implications for the global environment. Farmers’ incentives for maintaining forest fallow in a shifting cultivation agricultural system depend upon the market and non-market services it provides to them. This study estimates the value of those services, including hydrological externalities that may affect other farms downstream. The analysis uses cross-sectional farm-level survey data from the Zona Bragantina in the Eastern Brazilian Amazon to assess the value of forest fallow to farmers and test whether it provides local externalities. I estimate production functions for crops and forest products to determine the contributions of on-farm and off-farm forest fallow to income from these two activities. Instrumental variables and spatial econometric approaches help address issues of endogeneity and variation in unobservable factors over space. I use geographic information on the location of farms to obtain data on external forest fallow and to model the hydrological externality as an upstream-to-downstream process. The results indicate that fallow does contribute significantly to productivity both on-farm and downstream, boosting income from both crops and forest products. In addition, most farms appear to allocate sufficient land to fallow, accounting for both the value of hydrological spillovers and the opportunity cost of land left out of cultivation. These results suggest that farming communities may have some self-interest in preserving forest cover locally—a finding that may bolster policy efforts aimed at conserving tropical forests.
Methods for obtaining tests of fit of structural models for covariance matrices and estimator standard errors which are asymptotically distribution free are derived. Modifications to standard normal theory tests and standard errors which make them applicable to the wider class of elliptical distributions are provided. A random sampling experiment to investigate some of the proposed methods is described.
Prelis 2: User's reference guide: A program for multivariate data screening and data summarization
- K G Jöreskog
- D Sörbom
Jöreskog, K. G., & Sörbom, D. (1996). Prelis 2: User's reference guide: A
program for multivariate data screening and data summarization.
Chicago, IL: Scientific Software.