No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Parametric and nonparametric bootstrap methods for general MANOVA
Abstract
We develop parametric and nonparametric bootstrap methods for multi-factor multivariate data, without assuming normality, and allowing for covariance matrices that are heterogeneous between groups. The newly proposed, general procedure includes several situations as special cases, such as the multivariate Behrens–Fisher problem, the multivariate one-way layout, as well as crossed and hierarchically nested two-way layouts. We derive the asymptotic distribution of the bootstrap tests for general factorial designs and evaluate their performance in an extensive comparative simulation study. For moderate sample sizes, the bootstrap approach provides an improvement to existing methods in particular for situations with nonnormal data and heterogeneous covariance matrices in unbalanced designs. For balanced designs, less computationally intensive alternatives based on approximate sampling distributions of multivariate tests can be recommended.
... Classical repeated measures models, where hypotheses are tested with Hotelling's T 2 (Hotelling, 1931) or Wilks's Λ (Wilks, 1932), assume normally distributed observation vectors and a common covariance matrix for all groups, see e.g. the monograph of Davis (2002). In medical and biological research, however, the assumptions of equal covariance matrices and multivariate normally distributed outcomes are often not met and a violation of them may inflate the type-I error rates, see the comments in Xu and Cui (2008), Suo et al. (2013) or Konietschke et al. (2015). Therefore, other procedures have been developed for repeated measures which are based on certain approximation techniques (Geisser and Greenhouse (1958); Greenhouse and Geisser (1959); Huynh and Feldt (1976); Lecoutre (1991); Keselman et al. (2000); Werner (2004); Ahmad et al. (2008); Brunner (2009) ;Brunner et al. (2009); Kenward and Roger (2009) ;Brunner et al. (2012); Chi et al. (2012); Pauly et al. (2015b)). ...
... However, it is well known that it requires large sample sizes to keep the pre-assigned type-I error level, see e.g. Brunner (2001), and Konietschke et al. (2015). ...
... Although ϕ W T S possesses these nice asymptotic properties, it is well-known that very large sample sizes n i are necessary to maintain the pre-assigned level α using quantiles of the limiting χ 2 -distribution, see Konietschke et al. (2015), and Brunner (2001) as well as Table 2 below. This leads to a limited applicability of the WTS in practice. ...
For general repeated measures designs the Wald-type statistic (WTS) is an asymptotically valid procedure allowing for unequal covariance matrices and possibly non-normal multivariate observations. The drawback of this procedure is the poor performance for small to moderate samples, i.e. decisions based on the WTS may become quite liberal. It is the aim of the present paper to improve its small sample behavior by means of a novel permutation procedure. In particular, it is shown that a permutation version of the WTS inherits its good large sample properties while yielding a very accurate finite sample control of the type-I error as shown in extensive simulations. Moreover, the new permutation method is motivated by a practical data set of a split plot design with a factorial structure on the repeated measures.
... Specifically, for testing multivariate main-and interaction effects in one-, two-and higher-way MANOVA models, the MANOVA function provides the • Wald-type statistic (WTS) proposed by Konietschke et al. (2015) using a parametric bootstrap, a wild bootstrap or its asymptotic χ 2 -distribution for p-value computations, and ...
... In this set-up Konietschke et al. (2015) propose a Wald-type statistic (WTS) ...
... Denoting by c the corresponding (1 − α)-quantile of the (conditional) distribution of T N the test rejects H 0 if T N > c . The validity of this procedure (also named parametric bootstrap WTS) is proven in Konietschke et al. (2015). ...
The numerical availability of statistical inference methods for a modern and robust analysis of longitudinal- and multivariate data in factorial experiments is an essential element in research and education. While existing approaches that rely on specific distributional assumptions of the data (multivariate normality and/or characteristic covariance matrices) are implemented in statistical software packages, there is a need for user-friendly software that can be used for the analysis of data that do not fulfill the aforementioned assumptions and provide accurate p-value and confidence interval estimates. Therefore, newly developed statistical methods for the analysis of repeated measures designs and multivariate data that neither assume multivariate normality nor specific covariance matrices have been implemented in the freely available R-package MANOVA.RM. The package is equipped with a graphical user interface for plausible applications in academia and other educational purpose. Several motivating examples illustrate the application of the methods.
... We now study the parametric bootstrap procedure, which have applied with regard to coefficient alpha in the one-sample situation, see also Konietschke et al. (2015) for a recent application in the MANOVA context. For observed sample covariances Σ 1 and Σ 2 the resampling mechanisms is given by generating independent bootstrap variables ...
... On the Distributional Assumptions. We have consciously chosen to work under a general ADF framework since multivariate normality is a rather strong assumption that is usually violated for practical data at hand, see e.g. the discussion in Konietschke et al. (2015). This is especially the case when confronted with ties in the data and / or small sample sizes. ...
... In fact, our proposed parametric bootstrap procedure originally stems from this parametric model. The motivation to apply it with a slightly different covariance estimator also in the ADF case is due to the multivariate central limit theorem, see also the explanation in Konietschke et al. (2015). ...
The two-sample problem for Cronbach's coefficient , as an estimate of test or composite score reliability, has attracted little attention, compared to the extensive treatment of the one-sample case. It is necessary to compare the reliability of a test for different subgroups, for different tests or the short and long forms of a test. In this paper, we study statistically how to compare two coefficients and . The null hypothesis of interest is , which we test against one-or two-sided alternatives. For this purpose, resampling-based permutation and bootstrap tests are proposed. These statistical tests ensure a better control of the type I error, in finite or very small sample sizes, when the state-of-affairs \textit{asymptotically distribution-free} (ADF) large-sample test may fail to properly attain the nominal significance level. We introduce the permutation and bootstrap tests for the two-group multivariate non-normal models under the general ADF setting, thereby improving on the small sample properties of the well-known ADF asymptotic test. By proper choice of a studentized test statistic, the resampling tests are modified such that they are still asymptotically valid, if the data may not be exchangeable. The usefulness of the proposed resampling-based testing strategies is demonstrated in an extensive simulation study and illustrated by real data applications.
... In contrast to [39, p. 101], the covariance functions of the different groups may differ from each other, i.e., heteroscedasticity is explicitly allowed. As in classical (M)ANOVA settings [25,31], a more general factorial structure can be incorporated easily by splitting up the indices (see below). Since we put the greater importance on the accurate description of the multiple testing procedures and related standard examples, we merely focus on the one-way case. ...
... Here, using a Gaussian process for generating the parametric bootstrap sample seems natural regarding the limiting distribution in Theorem 1 since the mean function estimators are asymptotically Gaussian anyway. Moreover, using the estimators of sample covariance functions, we mimic the covariance structure of the given functional data, see [25] for a similar motivation in the MANOVA context. Note, that we still postulate our general model, i.e. we do not assume a parametric model for our functional data (x i1 , . . . ...
... The simulation setup is based on the simulations in [30]. We simulated k = 4 samples with sample sizes (n 1 , n 2 , n 3 , n 4 ) = K · (15,20,25,30), where K ∈ {1, 2, 4} under the null hypothesis settings and K = 1 under the alternative hypothesis settings, by ...
While there exists several inferential methods for analyzing functional data in factorial designs, there is a lack of statistical tests that are valid (i) in general designs, (ii) under non-restrictive assumptions on the data generating process and (iii) allow for coherent post-hoc analyses. In particular, most existing methods assume Gaussianity or equal covariance functions across groups (homoscedasticity) and are only applicable for specific study designs that do not allow for evaluation of interactions. Moreover, all available strategies are only designed for testing global hypotheses and do not directly allow a more in-depth analysis of multiple local hypotheses. To address the first two problems (i)-(ii), we propose flexible integral-type test statistics that are applicable in general factorial designs under minimal assumptions on the data generating process. In particular, we neither postulate homoscedasticity nor Gaussianity. To approximate the statistics' null distribution, we adopt a resampling approach and validate it methodologically. Finally, we use our flexible testing framework to (iii) infer several local null hypotheses simultaneously. To allow for powerful data analysis, we thereby take the complex dependencies of the different local test statistics into account. In extensive simulations we confirm that the new methods are flexibly applicable. Two illustrate data analyses complete our study. The new testing procedures are implemented in the R package multiFANOVA, which will be available on CRAN soon.
... However, classical MANOVA (Bartlett, 1939;Dempster, 1960;Lawley, 1938;Pillai, 1955;Wilks, 1946) relies on restrictive assumptions as normality and homogeneity of covariances. But the "normality assumption becomes quasi impossible to justify when moving from univariate to multivariate observations" (Konietschke et al., 2015) and, similarly, homogeneity is often implausible. To overcome these, several remedies have been suggested for tackling at least one of both issues. ...
... Thereby solutions have been developed for specific layouts, e.g. one-or two-way (Krishnamoorthy and Lu, 2010;Zhang, 2011;Bathke et al., 2018;Zhang et al., 2022) as well as for general factorial designs (Konietschke et al., 2015;Friedrich and Pauly, 2018). As common in statistical inference, all these proposals focus on the expectation (vector) and thus infer means or contrasts thereof. ...
... Higherway layouts and also hierarchically designs with nested factors can be incorporated in a similar way, see e.g. Pauly et al. (2015); Friedrich and Pauly (2018); Friedrich et al. (2019) for mean-based testing strategies. ...
Multivariate analysis-of-variance (MANOVA) is a well established tool to examine multivariate endpoints. While classical approaches depend on restrictive assumptions like normality and homogeneity, there is a recent trend to more general and flexible proce dures. In this paper, we proceed on this path, but do not follow the typical mean-focused perspective. Instead we consider general quantiles, in particular the median, for a more robust multivariate analysis. The resulting methodology is applicable for all kind of factorial designs and shown to be asymptotically valid. Our theoretical results are complemented by an extensive simulation study for small and moderate sample sizes. An illustrative data analysis is also presented.
... As traditional methods like Hotelling's T 2 or Wilk's Lambda tests [Johnson et al., 2002, e.g.] rely on (too) restrictive assumptions like normality and covariance equality, the last two decades have seen the development of alternative approaches to address these limitations. For example, Krishnamoorthy and Xia [2006] or Smaga [2017] proposed methods for the two sample case while Konietschke et al. [2015], Roy et al. [2015], Smaga [2015], Hu et al. [2017], Friedrich and Pauly [2018] or Sattler and Pauly [2018] studied one-way and more general hypotheses in complex MANOVA designs. An appropriate way to investigate hypotheses regarding vector-valued parameters, like expectation vectors, are quadratic forms. ...
... Thereby we consider a ≥ 2 groups and assume that for fixed i ∈ {1, ..., a} and j ∈ {1, ..., d} the random variables ǫ ikj are identical distributed with E(ǫ ikj ) = 0. It is important to note that by splitting up the indices, this setting implies factorial designs inside a group as in Konietschke et al. [2015]. We allow unbalanced sample sizes n 1 , ..., n a and denote with N = a i=1 n i the total sample size. ...
Comparing the mean vectors across different groups is a cornerstone in the realm of multivariate statistics, with quadratic forms commonly serving as test statistics. However, when the overall hypothesis is rejected, identifying specific vector components or determining the groups among which differences exist requires additional investigations. Conversely, employing multiple contrast tests (MCT) allows conclusions about which components or groups contribute to these differences. However, they come with a trade-off, as MCT lose some benefits inherent to quadratic forms. In this paper, we combine both approaches to get a quadratic form based multiple contrast test that leverages the advantages of both. To understand its theoretical properties, we investigate its asymptotic distribution in a semiparametric model. We thereby focus on two common quadratic forms - the Wald-type statistic and the Anova-type statistic - although our findings are applicable to any quadratic form. Furthermore, we employ Monte-Carlo and resampling techniques to enhance the test's performance in small sample scenarios. Through an extensive simulation study, we assess the performance of our proposed tests against existing alternatives, highlighting their advantages.
... This procedure proves specifically valuable against the background of normality violations due to skewed data (for more information, see Desgagné et al., 1998). Further, in the context of variance analysis, it mitigates the effect of heteroscedasticity in the predictors as emphasised in theoretical and applied studies (Konietschke et al., 2015;Krishnamoorthy et al., 2007;Zhou and Wong, 2011). ...
... Therefore, we apply the Box-Cox transformation to the metric covariates to correct the model and run the ANCOVA using a bootstrap algorithm. The robustness of this approach was often demonstrated in similar studies against the background of unequal variances (e.g., Konietschke et al., 2015;Krishnamoorthy et al., 2007;Zhou and Wong, 2011). ...
Emission reduction in the energy sector is built on two main pillars: a shift from fossil fuels to renewable sources and using available energy more efficiently. While renewable electricity production experienced a substantial acceleration in capacity building in the past ten years, energy demand-despite longstanding policy efforts-is still constantly on the rise worldwide. This paper combines both aspects analysing whether different (co-) ownership types in renewables in the residential sector are associated with an increased inclination to invest in energy-efficient appliances or to behave more energy consciously. To do so, we estimate an analysis of covari-ance using a sample with demographic and electricity consumption data from 1454 German households. Our results show that, in general, (co-)owners of renewable energy installations are more willing to invest in energy-efficient technologies than people who are not involved with renewables. However, there are differences between (co-)ownership types. People who have the choice between self-consumption and sale to third parties or the grid, i.e., fully-fledged prosumers, show the strongest inclination to invest in energy efficiency. Further, when analysing energy savings through adapted behaviour, solely this (co-)ownership group shows an increased propensity for conscious energy consumption behaviour.
... To obtain tests with good finite sample properties even for smaller sample sizes, we investigate different bootstrap and permutation procedures. These demonstrate good qualities in many models (see, for example, Amro et al., 2021;Ditzhaus et al., 2021;Konietschke et al., 2015;Konietschke and Pauly, 2014;Smaga and Zhang, 2020). However, this is not an invariable rule, as we also note in the extensive simulation study, where we check the control of type I error level and power of the considered tests in comparison with the tests of Martínez-Camblor and Corral (2011) and their modifications. ...
... The last method considered is the parametric bootstrap approach (the B 3 test for short), which uses bootstrap samples that are constructed in a much different way than in the above methods. The idea of this approach is similar to the parametric bootstrap proposed by Konietschke et al. (2015). In step 2, we generate the bootstrap samples Y b 1 (t), . . . ...
This paper is motivated by medical studies in which the same patients with multiple sclerosis are examined at several successive visits and described by fractional anisotropy tract profiles, which can be represented as functions. Since the observations for each patient are dependent random processes, they follow a repeated measures design for functional data. To compare the results for different visits, we thus consider functional repeated measures analysis of variance. For this purpose, a pointwise test statistic is constructed by adapting the classical test statistic for one-way repeated measures analysis of variance to the functional data framework. By integrating and taking the supremum of the pointwise test statistic, we create two global test statistics. Apart from verifying the general null hypothesis on the equality of mean functions corresponding to different objects, we also propose a simple method for post hoc analysis. We illustrate the finite sample properties of permutation and bootstrap testing procedures in an extensive simulation study. Finally, we analyze a motivating real data example in detail.
... In that sense, large sample sizes are then required to maintain the preassigned type I error rate. For example, both Konietschke et al. [15] and Pauly et al. [22] utilized Wald-type statistic. Several resampling-based improvements were proposed to tackle the small-sample issue of the Wald-type test under MANOVA settings [5,8,10,15,21,22,24,25]. ...
... For example, both Konietschke et al. [15] and Pauly et al. [22] utilized Wald-type statistic. Several resampling-based improvements were proposed to tackle the small-sample issue of the Wald-type test under MANOVA settings [5,8,10,15,21,22,24,25]. Other methods were also proposed to mitigate the effect of heterogeneity of covariance matrices in MANOVA by modifying the sums of squares and cross products matrices [12,32]. ...
Multivariate repeated measures data naturally arise in clinical trials and other fields such as biomedical science, public health, agriculture, social science and so on. For data of this type, the classical approach is to conduct multivariate analysis of variance (MANOVA) based on Wilks' Lambda and other multivariate statistics, which require the assumptions of multivariate normality and homogeneity of within-cell covariance matrices. However, data being analyzed nowadays show marked departure from multivariate normality and homogeneity. This paper proposes a finite-sample test by modifying the sums of squares matrices to make them insensitive to the heterogeneity in MANOVA. The proposed test is invariant to affine transformation and robust against nonnormality. The proposed method can be used in various experimental designs, for example, factorial design and crossover design. Under various simulation settings, the proposed method outperforms the classical Doubly Multivariate Model and Multivariate Mixed Model proposed elsewhere, especially for unbalanced sample sizes with heteroscedasticity. The applications of the proposed method are illustrated with ophthalmology data in factorial and crossover designs. The proposed method successfully identified and validated a significant main effect and demonstrated that univariate analysis could be oversensitive to small but clinically unimportant interactions.
... We use the same approach to evaluate critical points for Min-T, AMin-T, Max-T. The adoption of the parametric instead of the non-parametric bootstrap is motivated from previous research (e.g., Konietschke et al. 2015) and its natural evolution from our primary parametric considerations. ...
... For the validation of the bootstrap test we need to show that given the observations, the conditional distribution of the bootstrap statistic converges to the null distribution of the original statistic. (see, for example, Konietschke et al. (2015), Pauly et al. (2015)). ...
In a one-way design with unknown and unequal error variances, we are interested in testing the null hypothesis against the ordered alternative (with at least one strict inequality). We propose the likelihood ratio test (LRT) as well as two Min-T tests (Min-T and AMin-T) and study their asymptotic behaviour. For better finite sample properties, a parametric bootstrap procedure is used. Asymptotic accuracy of the parametric bootstrap is established. Simulation results show that all the tests control the nominal level in case of small, moderate, and highly unbalanced sample sizes. This even holds for combinations with large variances. However, the asymptotic likelihood ratio test (ALRT) achieves the nominal size only for large samples and k=3. Power comparisons between LRT and Min-T test show that the LRT has more power than Min-T. However, the Min-T test is easier to use as it does not require evaluation of MLEs. Similar observations are made while comparing ALRT and AMin-T test. The robustness of these tests under departure from normality is also investigated by considering five non-normal distributions (t-distribution, Laplace, exponential, Weibull, and lognormal). When high variances are combined with small or highly unbalanced sample sizes, the tests are observed to be robust only for symmetric distributions. If variances of groups are equal and sample sizes of each group are at least 10, the proposed tests perform satisfactorily for symmetric and skewed distributions. R packages are developed for all tests and a practical example is provided to illustrate their application.
... This one-way MANOVA type test was used by Rupasinghe Arachchige Don and Olive (2019), and a special case was used by Zhang and Liu (2013) and Konietschke et al. (2015) (2018). ...
... Large sample Wald type tests are fairly common, but need large sample sizes. See, for example, Zhang et al. (2016) for the two-way MANOVA model, Duchesne and Francq (2015), Konietschke et al. (2015), and Smaga (2017). ...
A Wald type test with the wrong dispersion matrix is used when the dispersion matrix is not a consistent estimator of the asymptotic covariance matrix of the test statistic. One class of such tests occurs when there are p groups and it is assumed that the population covariance matrices from the p groups are equal, but the common covariance matrix assumption does not hold. The pooled t test, one-way ANOVA F test, and one-way MANOVA F test are examples of this class. Another class of such tests is used for weighted least squares. Two bootstrap confidence regions are modified to obtain large sample Wald type tests with the wrong dispersion matrix.
... While there exist several promising approaches to adequately deal with the problem of covariance heterogeneity in the classical case with d < N (see e.g. [6,16,17,20,27,37,1,24,9,32,35,26,18,15]) most procedures for high-dimensional repeated measures designs rely on certain sparsity conditions (see e.g. [2,11,23,30,34,10,19] and the references cited therein). ...
Statisticians increasingly face the problem to reconsider the adaptability of classical inference techniques. In particular, divers types of high-dimensional data structures are observed in various research areas; disclosing the boundaries of conventional multivariate data analysis. Such situations occur, e.g., frequently in life sciences whenever it is easier or cheaper to repeatedly generate a large number d of observations per subject than recruiting many, say N, subjects. In this paper we discuss inference procedures for such situations in general heteroscedastic split-plot designs with a independent groups of repeated measurements. These will, e.g., be able to answer questions about the occurrence of certain time, group and interactions effects or about particular profiles. The test procedures are based on standardized quadratic forms involving suitably symmetrized U-statistics-type estimators which are robust against an increasing number of dimensions d and/or groups a. We then discuss its limit distributions in a general asymptotic framework and additionally propose improved small sample approximations. Finally its small sample performance is investigated in simulations and the applicability is illustrated by a real data analysis.
... In the context of the MANOVA model and relaxed conditions, Friedrich et al. (2017) and Bathke et al. (2018) discussed different parametric and nonparametric approaches to study longitudinal and multivariate data in factorial experiments and the use of different bootstrapping techniques to compute the respective p-values. One of the discussed approaches is Konietschke et al. (2015) Wald-type statistic (WTS) that follows an asymptotic chisquared distribution and bootstrap technique for p-value computation. ...
Package claims influence product perceptions. Specifically, prosocial information concerning sustainable organisational practices improves product evaluation. This research evaluates the effect of supporting cultural and artistic activities as a claim that influences taste evaluations. Taste is a metaphor for an emotional response to products. We conducted a mixed model experiment: The claim “Supports cultural/artistic activities” is the between-group variable, and four fruits are the within-subject variable (i.e., guava, pineapple, red apple, and green grapes). Descriptive statistics and a MANOVA analysis show that the sweet and umami taste is enhanced when fruits hold the cultural/artistic claim. This effect is not observed for the sour and bitter taste. Therefore, the results confirm the taste metaphor. Within the cause-related marketing literature, it is possible to interpret that supporting the culture/arts industry has a symbolic meaning for the consumer.
... However, in some problems, the permutation test can be applied under some conditions, even if the data are not exchangeable, because exchangeability holds approximately or asymptotically. See, for example, the permutation approach for paired samples studied by [46,110,111]. For example, Konietschke and Pauly [46] presented three simple conditions in which the permutation principle works to obtain an asymptotic p-value. ...
Today, permutation tests represent a powerful and increasingly widespread tool of statistical inference for hypothesis-testing problems. To the best of our knowledge, a review of the application of permutation tests for complex data in practical data analysis for hypothesis testing is missing. In particular, it is essential to review the application of permutation tests in two-sample or multi-sample problems and in regression analysis. The aim of this paper is to consider the main scientific contributions on the subject of permutation methods for hypothesis testing in the mentioned fields. Notes on their use to address the problem of missing data and, in particular, right-censored data, will also be included. This review also tries to highlight the limits and advantages of the works cited with a critical eye and also to provide practical indications to researchers and practitioners who need to identify flexible and distribution-free solutions for the most disparate hypothesis-testing problems.
... In order to validate the nonparametric bootstrap test NBT A , we need to show that the conditional distribution of the bootstrap statistic T * Alm given the observations converges to the null distribution of T A (see [34,35]). In the following theorem, we establish the asymptotic accuracy of NBT A . ...
The problem of combining information from several samples for estimating or testing a common mean for normal populations has been extensively studied in the statistical literature. In this paper, we take up this problem in the context of the common mean direction of several Fisher-von Mises-Langevin (FvML) distributions. The concentration parameters are taken to be unknown and heterogeneous. A non-iterative combined estimator is proposed and is seen to have substantially better risk performance than individual sample mean directions and a grand mean direction in a simulation study. Further , a test based on this non-iterative estimator is proposed, and nonparametric bootstrap and permutation resampling methods are developed for its implementation. Two more alternative tests are proposed and their implementation is carried out using nonpara-metric bootstrap resampling. A detailed simulation study shows that these test procedures achieve the nominal size and have good power performance. An 'R' package is developed for the implementation of the tests. A real data set is considered for illustrating the procedures. ARTICLE HISTORY
... Unfortunately, these assumptions are often not fulfilled in practical applications. To this end, many alternative procedures have been developed that address assumption violation problems and have been compared in extensive simulation studies (Konietschke et al. 2015;Bathke et al. 2018;McFarquhar et al. 2016;Friedrich et al. 2017;Livacic-Rojas et al. 2017). In line with the MANOVA results, standardisation was performed for each parameter and the appropriate pollinator for each variety was determined. ...
Within the scope of this study, the compatibility of some summer apple cultivars and the possibility of their use as pollinator to each other were investigated. In addition, the effects of factors on quality parameters were investigated under different statistical models on harvested fruits. It can be said that a suitable and sufficient number of pollinators should be used since the initial fruit set was below 20% in all examined varieties except ‘Summer Red’ (22.24%). As a result of standardization of data, ‘Vista Bella’ for ‘Jersey Mac’ and ‘Jersey Mac’ for ‘Vista Bella’ were the most successful pollinators to obtain high quality fruits. A similar reciprocal effect was also seen between ‘Williams Pride’ and ‘Summer Red’. However, the fact that open pollination gave the best results showed that at least one more suitable pollinator should be used to the proposed cultivars. The xenia effect was seen in many characteristics including fruit width, fruit length, fruit weight, red blush ratio and seed weight. While the number of seeds and stalk length positively affected the pomological characteristics of the fruits, the increase in fruit size negatively affected the coloration. Multivariate methods gave interpretations that are more accurate in statistical analysis, since the interaction effects of factors on the characteristics investigated are important and the characteristics are related to each other.
... Before starting the analysis of the data, the Researcher coded participants' gender role attitude scores of 95 and above as egalitarian gender role attitudes, and the scores below 95 as traditional gender role attitudes. In addition, the Researcher examined normality, equality of covariance matrices, and homogeneity of variances to investigate whether the assumptions required for the analysis of two-way MANOVA were met (Keselman et al., 1998;Konietschke et al., 2015). The Researcher analyzed the normality test of the data with Kolmogorov-Smirnov and Shapiro-Wilk. ...
The aim of this study is to examine the behaviors of married individuals to maintain relationships according to their gender and gender role attitudes. The study group of the study consisted of 177 (52.8%) females and 158 males (%) aged between 24 and 50 (x̄=30.2), residing in İzmir, and having a relationship period of 2 to 20 years (x̄=5.9). 47.2), a total of 335 heterosexual married people. As data collection tools in the research, “Relationship Maintenance Strategies Scale”, “Gender Roles Attitude Scale” and “Personal Information Form” were used. Two-way MANOVA method was used in the analysis of the data. Findings from the two-way MANOVA analysis show that gender and gender roles have an impact on relationship maintenance behaviors. In this context, it has been determined that gender roles have a higher degree of influence than biological sex in maintaining relationships. According to the findings of the study, it was determined that female participants exhibited relationship-maintaining behaviors more frequently than male participants. In terms of gender roles attitude, It has been determined that the participants who have an egalitarian gender role attitude exhibit more frequent relationship maintenance behaviors than the participants who have traditional gender roles attitudes. As a result, gender and gender roles have a significant effect on relationship maintenance strategies.
... There exist many robust or nonparametric variants of classical MANOVA as well (see citations in Finch & French, 2013;Konietschke, Bathke, Harrar & Pauly, 2015), but these approaches still have assumptions, related to homogenous variance (e.g., Anderson, 2001) or error distributions (e.g., Bathke et al., 2018), which are also violated by these data. Indeed, once data differ in so many ways, it may not be appropriate to statistically compare them based on a single value (group means or medians). ...
Despite considerable advances in knowledge tracing algorithms, educational technologies that use this technology typically continue to use older algorithms, such as Bayesian Knowledge Tracing. One key reason for this is that contemporary knowledge tracing algorithms primarily infer next-problem correctness in the learning system, but do not attempt to infer the knowledge the student can carry out of the system, information more useful for teachers. The ability of knowledge tracing algorithms to predict problem correctness using data from intelligent tutoring systems has been extensively researched, but data from outcomes other than next-problem correctness have received less attention. In addition, there has been limited use of knowledge tracing algorithms in games, because algorithms that do attempt to infer knowledge from answer correctness are often too simple to capture the more complex evidence of learning within games.
... However, the asymptotic validity of all these tests depends on the homoscedasticity of the errors. Konietschke et al. (2015) and Rupasinghe and Olive (2019) proposed nonparametric bootstrap tests that relax the homoscedasticity assumption, but both proposals are only asymptotically valid. Moreover, both these and other classical tests are only able to detect differences in location. ...
We propose a Bayesian hypothesis testing procedure for comparing the multivariate distributions of several treatment groups against a control group. This test is derived from a flexible model for the group distributions based on a random binary vector such that, if its jth element equals one, then the jth treatment group is merged with the control group. The group distributions’ flexibility comes from a dependent Dirichlet process, while the latent vector prior distribution ensures a multiplicity correction to the testing procedure. We explore the posterior consistency of the Bayes factor and provide a Monte Carlo simulation study comparing the performance of our procedure with state-of-the-art alternatives. Our results show that the presented method performs better than competing approaches. Finally, we apply our proposal to two classical experiments. The first one studies the effects of tuberculosis vaccines on multiple health outcomes for rabbits, and the second one analyzes the effects of two drugs on weight gain for rats. In both applications, we find relevant differences between the control group and at least one treatment group.
... However, inferences based on most robust covariance estimators may behave poorly when sample sizes are small, as Long and Ervin (2000) and others (e.g., Cao et al., 2020) have demonstrated using Monte Carlo simulations. A different kind of solution is to use simulationbased methods (Ananda & Weerahandi, 1997;Friedrich & Pauly, 2018;Gamage et al., 2004;Konietschke et al., 2015;Xu, 2015;Zimmermann et al., 2020). Although resampling methods do not make assumptions about the equality of covariance matrices, a drawback of Monte Carlo-based approaches is that they are often time-consuming, especially when the sample sizes are large. ...
Multivariate analysis of variance (MANOVA) is the analysis strategy usually chosen in Psychological and Educational research to test the effect of an intervention on a set of dependent variables that are related. Three aspects are common in applied research: First, intervention is determined by the combination of more than one variable and, therefore, the design structure is factorial. Second, either if research is framed in a quasi-experimental or an experimental methodology, it is often needed to control the effect of other present variables to reduce the error variance and increase the effect of the intervention variables and their interaction, or for both reasons; thus, increasing the strength of the test. In this case MANCOVA will be used. Third, when for different reasons, the size of the groups is not the same, the response of the subjects in the different intervention conditions is not homogeneous and data loss occurs, MANCOVA analysis does not provide valid statistical inferences. In the present research, a modification of MANCOVA is developed to test the hypotheses of a factorial model and obtain adequate information regardless of the degree of heterogeneity and the imbalance of groups size. It is assumed that the interrelated system of observed variables is the causal agent of the underlying (or emergent) construct, not the other way around. To deal with missing data, formulas are developed to group the results of the analysis into a single final estimate after multiple imputation has been done. Both developments are studied by simulation, and it is concluded that they can be used with full guarantees. An example is shown.
... [25] compared the performance of a parametric bootstrap test with the generalized F-test of [22]. [26] proposed a bootstrap method and [27] studied a permutation test for general ANOVA problems. A drawback of simulation-based approaches is that they are often time-consuming especially when the sample sizes are large. ...
... Here, we develop a parametric bootstrap procedure to evaluate critical points for T 1 for a general q and k, which is motivated by heteroscedastic MANOVA (Friedrich and Pauly 2018;Konietschke et al. 2015). In Sect. ...
Fisher–von Mises–Langevin distributions are widely used for modeling directional data. In this paper, the problem of testing homogeneity of mean directions of several Fisher–von Mises–Langevin populations is considered when the concentration parameters are unknown and heterogeneous. First, an adaptive test based on the likelihood ratio statistic is proposed. Critical points are evaluated using a parametric bootstrap. Second, a heuristic test statistic is considered based on pairwise group differences. A nonparametric bootstrap procedure is adapted for evaluating critical points. Finally, a permutation test is also proposed. An extensive simulation study is performed to compare the size and power values of these tests with those proposed earlier. It is observed that both parametric and nonparametric bootstrap based tests achieve size values quite close to the nominal size. Asymptotic tests and permutation tests have size values higher than the nominal size. Bootstrap tests are seen to have very good power performance. The robustness of tests is also studied by considering contamination in Fisher–von Mises–Langevin distributions. R packages are developed for the actual implementation of all tests. A real data set has been considered for illustrations.
... The last method is the parametric bootstrap approach PB, whose idea is similar to the parametric bootstrap proposed in Konietschke et al. (2015), for nonparametric ANOVA for random vectors. In that and present papers, it is shown that although the parametric bootstrap is typically applied for parametric models, it can also be successfully used in nonparametric ones (any specific distribution of the data is not assumed). ...
Dynamic cluster analysis is understood as partitioning of spatio-temporal objects into groups which can change their composition over time. The aim of the paper is to propose the measure of cluster stability for such groups. The measure takes values from [0;1] interval. Some of its characteristics are discussed on the basis of simple example. The distribution of the measure under random membership is estimated by Monte Carlo simulation. The real example from the analysis of European Union countries in 15-year period is also provided.KeywordsDynamic cluster analysisSpatio-temporal analysisCluster stabilityEU countries
... For both continuous and categorical variables, effected sizes are reported using the standardized mean difference (SMD). To obtain a single P value testing the difference of cytokine concentrations between the groups, parametric bootstrap MANOVA (multivariate ANOVA), defined by Konietschke et al. [13], was applied. This method is appropriate for unbalanced designs, nonnormal data, heterogeneity of within group covariance matrices, and allows for computing a single P value using all cytokine concentrations. ...
Objectives:
Female sexual and reproductive health is heavily influenced by the levels and ratios of Lactobacilli species and vaginal cytokines. Menopause marks a profound body change as it shifts to a natural and permanent non-reproductive state. Vulvovaginal diseases encompass a broad variety of sexual health conditions. Furthermore, both menopause and vulvovaginal diseases affect vaginal Lactobacilli and cytokine levels. Thus, this study aimed to investigate the correlation between menopause, vulvovaginal diseases, and vaginal Lactobacilli and cytokine levels.
Methods:
Vaginal swab samples were collected as part of a prospective data bank creation to study vaginal conditions as approved by the Institutional Review Board of Texas Tech University Health Sciences Center, Lubbock, USA. This study utilized 38 samples in this database, which were assigned to the pre-menopausal with no vulvovaginal conditions (n = 20) and post-menopausal with vulvovaginal conditions (n = 18) groups. A real-time polymerase chain reaction was conducted to determine the relative concentration of Lactobacilli species, while cytokine analysis was performed using multiplex enzyme-linked immunosorbent assay immunoassay. The standardized mean difference, multivariate analysis of variance, and permutational unequal variance t test were used for the statistical analysis.
Results:
Cytokines, interleukin (IL)-6, macrophage inflammatory protein-1α, IL-8, and Lactobacillus iners expression were significantly elevated in the control group compared to the study group (P = 0.03 for the cytokines, P = 0.0194 for Lactobacilli).
Conclusions:
The levels of vaginal cytokine and Lactobacillus profile were significantly different between the pre-menopausal and post-menopausal groups.
... This approach has been shown to provide more accurate inferential results compared to traditional methods, especially in controlling the Type-I error [30,31]. We adopt the modified ANOVA-type test statistic (MATS) with additional bootstrap for testing the main and interaction effects of factors [32]. MATS could offer the advantage of being invariant under scale transformations for multivariate data, which is an essential feature [33]. ...
The assessment of asphalt pavement structures with diversified designs and constructions has been a critical challenge due to the practical limitations in testing and methodology. Here, we propose a complex network approach to the evaluation of longitudinal asphalt pavement performances in the Research Institute of Highway Ministry of Transport track (RIOHTrack). We add to the growing field in which network approaches are employed and indicate the profound application in understanding pavement structures. Specifically, similar and disparate pavement performances are revealed regardless of their predefined design and construction categories. Evidence shows that short-term pavement performance depends on the strength of the subbase structure, whereas long-term performance relies on the thickness of the asphalt layer. In addition, our results indicate that recycling materials could be an important substitute for reducing the thickness of asphalt concrete and maintaining a healthy service life.
... Moreover the dimension of the pooled vector is denoted by D = a i=1 d i . This framework also allows a factorial structure by splitting up the indices, regarding time, group or both, as done in Konietschke et al. [2015] for example. In this work, the usual condition for designs with several groups n i /N → κ i ∈ (0, 1] for i = 1, ..., a is not necessary. ...
In repeated Measure Designs with multiple groups, the primary purpose is to compare different groups in various aspects. For several reasons, the number of measurements and therefore the dimension of the observation vectors can depend on the group, making the usage of existing approaches impossible. We develop an approach which can be used not only for a possibly increasing number of groups a, but also for group-depending dimension , which is allowed to go to infinity. This is a unique high-dimensional asymptotic framework impressing through its variety and do without usual conditions on the relation between sample size and dimension. It especially includes settings with fixed dimensions in some groups and increasing dimensions in other ones, which can be seen as semi-high-dimensional. To find a appropriate statistic test new and innovative estimators are developed, which can be used under these diverse settings on and without any adjustments. We investigated the asymptotic distribution of a quadratic-form-based test statistic and developed an asymptotic correct test. Finally, an extensive simulation study is conducted to investigate the role of the single group's dimension.
... Dado que, a primera vista, los gráficos de las distribuciones indicaban que, casi con total seguridad, no se cumplían los criterios de normalidad, optamos por usar modified ANOVA-type statistic (MATS), incluido en el paquete MANOVA.RM (Friedrich et al., 2019). MATS no asume términos de error normales, homogeneidad del tamaño muestral ni homocedasticidad (Bathke et al., 2018;Friedrich & Pauly, 2017;Konietschke et al., 2015), por lo que se ajustaba a los requerimientos del corpus. La prueba mostró que no había diferencias significativas en la frecuencia de Tanto la regresión logística como la prueba k-NN y la de SVN atribuyeron El divino Jasón a Mira de Amescua, y también obtuvieron resultados coincidentes con La universal redención, que se atribuyó en los tres casos a Lope de Vega. ...
Partiendo de dudas acerca de la autoría de El divino Jasón, el artículo examina la paternidad de dos autos sacramentales con aproximaciones estilométricas complementarias. En primer lugar, se realizó una ronda inicial de pruebas estilométricas tradicionales basadas en tokens. Seguidamente, se clasificaron los textos de acuerdo a las características métricas de sus romances. Las pruebas de ambas aproximaciones asociaron El divino Jasón con Mira de Amescua, mientras que clasificaron La universal redención entre las obras de Lope de Vega.
Por lo tanto, el autor parece imprimir en los patrones rítmicos de sus octosílabos una huella semejante a las que revelan los métodos estilométricos clásicos. De esta manera, junto a la contribución a la lingüística forense, este artículo presenta un nuevo método para el estudio de los aspectos sonoros del teatro aurisecular.
... For MANCOVA analyses, the R package "MANOVA.RM" including MATS statistics for multivariate data was used as it proved to be independent of distribution of the data and unequal dispersion of covariates between groups [51]. Furthermore, a bootstrap resampling approach (100 k) was used as proposed by Konietschke et al. [52] or Friedrich et al. [53] in case of small sample sizes. Effect-sizes were given in partial eta squared η 2 p and Cohen's d, the critical variance inflation was 5.0, and α was set to 0.05. ...
Background
Frailty is accompanied by limitations of activities of daily living (ADL) and frequently associated with reduced quality of life, institutionalization, and higher health care costs. Despite the importance of ADL performance for the consequence of frailty, movement analyses based on kinematic markers during the performance of complex upper extremity-based manual ADL tasks in frail elderly is still pending.
The main objective of this study was to evaluate if ADL task performance of two different tasks in frail elderlies can be assessed by an activity measurement based on an acceleration sensor integrated into a smartwatch, and further to what degree kinematic parameters would be task independent.
Methods
ADL data was obtained from twenty-seven elderly participants (mean age 81.6 ± 7.0 years) who performed two ADL tasks. Acceleration data of the dominant hand was collected using a smartwatch. Participants were split up in three groups, F (frail, n = 6), P (pre-frail, n = 13) and R (robust, n = 8) according to a frailty screening. A variety of kinematic measures were calculated from the vector product reflecting activity, agility, smoothness, energy, and intensity.
Results
Measures of agility, smoothness, and intensity revealed significant differences between the groups (effect sizes combined over tasks η ² p = 0.18 – 0.26). Smoothness was particularly affected by frailty in the tea making task, while activity, agility, a different smoothness parameter and two intensity measures were related to frailty in the gardening task. Four of nine parameters revealed good reliability over both tasks ( r = 0.44 – 0.69). Multiple linear regression for the data combined across tasks showed that only the variability of the magnitude of acceleration peaks (agility) contributed to the prediction of the frailty score (R ² = 0.25).
Conclusion
The results demonstrate that ADL task performance can be assessed by smartwatch-based measures and further shows task-independent differences between the three levels of frailty. From the pattern of impaired and preserved performance parameters across the tested tasks, we concluded that in persons with frailty ADL performance was more impaired by physiological deficiencies, i.e., physical power and endurance, than by cognitive functioning or sensorimotor control.
... Mardia's test for multivariate normality (p=0.026) was found to have been violated, which introduced a concern with respect to the stability of the results. To address these violations we considered the words of Konietschke et al. [25] who recommend the use of bootstrap as a workaround, specifically they write "for moderate sample sizes, the bootstrap approach provides an improvement to existing methods in particular for situations with nonnormal data and heterogeneous covariance matrices in unbalanced designs". Moreover, per Davison and Hinkley [14] the use of the bootstrapping technique ensures the stability of our results. ...
In this paper, we adapt and validate two constructs, perceived extrinsic warm-glow (PIWG) and perceived intrinsic warm-glow (PIWG), to measure the two dimensions of consumer perceived warm-glow (i.e., extrinsic and intrinsic) for use with the practice of technology adoption model-ing. Taking an experimental approach, participants were exposed to one of four vignettes de-signed to simulate the absence or the presence of warm-glow (specifically, extrinsic warm-glow, intrinsic warm-glow, and concurrently extrinsic and intrinsic warm-glow). The results revealed that both constructs measured their respective forms of warm-glow with two caveats. The first that singularly trying to evoke extrinsic warm-glow led to only a slight increase in consumer perception of extrinsic warm-glow. We attributed this finding to individuals not being attracted to technology products that overtly target and seek to satisfy their vanity, instead preferring technology that does so subtly. The second that singularly trying to evoke intrinsic warm-glow also resulted in the manifestation of extrinsic warm-glow. Thus, warm-glow appears as a blend of extrinsic and intrinsic dimensions. A finding serves to reinforce what has already been reported in the warm-glow literature and the idea of impure altruism.
... The hope of such multivariate analyses is, that the consideration of possible dependencies between the outcomes may lead to procedures with better power (in case of inference) or accuracy (in case of prediction) compared to separate univariate analyses. While the need for the development and use of valid and distributional robust or nonparametric multivariate methods has been recognized and addressed in inferential statistic (Dobler et al., 2020;Friedrich et al., 2019;Konietschke et al., 2015;Smaga, 2017;Vallejo and Ato, 2012;Zimmermann et al., 2020), there do not exist exhausting studies that exploit the potential of multivariate regression methods for prediction. ...
Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether we separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. In this work we compare these methods in extensive simulations to help in answering the primary question when to use multivariate ensemble techniques.
... The conventional wisdom is that negative pairings (larger samples having smaller covariances) lead to inflated type-I error rates, while positive pairings (larger samples having larger covariances) result in conservative inference. 17 In balanced designs, however, the method of moments based finite-sample approximation generally provide satisfactory α-level control. Methods to mitigate the effects of unequal covariance have been investigated in many papers. ...
The paper addresses estimating and testing treatment effects with multivariate outcomes in clinical trials where imperfect diagnostic devices are used to assign subjects to treatment groups. The paper focuses on the pre-post design and proposes two novel methods for estimating and testing treatment effects. In addition, methods for sample size and power calculations are developed. The methods are compared with each other and with a traditional method in a simulation study. The new methods show significant advantages in terms of power, coverage probability, and required sample size. The application of the methods is illustrated with data from electroencephalogram (EEG) recordings of alcoholic and control subjects.
... Estos cálculos incluyen análisis de la varianza usando el paquete MANOVA.RM (Friedrich, Konietschke & Pauly, 2019). Este paquete proporciona una alternativa no paramétrica a la prueba MANOVA, modified ANOVA-type statistic (MATS), que no asume términos de error normales, homogeneidad del tamaño muestral ni homocedasticidad (Bathke et al., 2018;Konietschke et al., 2015). En tanto que esta prueba puede manejar tamaños muestrales heterogéneos, no requiere correcciones de la estadística ni de la muestra. ...
Este estudio casuístico explora las diferencias en la complejidad léxica (LC) de una selección de la prensa de calidad española. Los resultados muestranvariabilidad del léxico entre periódicos yfalta de correlación entre el número de lectores y una alta LC. Se calculan índices LS1 y CVS1 de sofisticación y HD-D, MAAS y MTLDde diversidad para evaluar 2741 artículos editoriales de Abc, El Mundo, El País yEl Periódico publicados en línea durante 2019. Los resultados revelan diferencias significativas tanto en diversidad como en sofisticación, siendo El Mundoel periódico con los con textos más complejos yEl Periódicocon los menos complejos. Adicionalmente, la comparación de HD-D, MAAS y MTLD con variaciones de TTR sugiere una ventaja de los primeros para muestras de tamaño heterogéneo, como las empleadas en el estudio.
In many life science experiments or medical studies, subjects are repeatedly observed and measurements are collected in factorial designs with multivariate data. The analysis of such multivariate data is typically based on multivariate analysis of variance (MANOVA) or mixed models, requiring complete data, and certain assumption on the underlying parametric distribution such as continuity or a specific covariance structure, for example, compound symmetry. However, these methods are usually not applicable when discrete data or even ordered categorical data are present. In such cases, nonparametric rank‐based methods that do not require stringent distributional assumptions are the preferred choice. However, in the multivariate case, most rank‐based approaches have only been developed for complete observations. It is the aim of this work to develop asymptotic correct procedures that are capable of handling missing values, allowing for singular covariance matrices and are applicable for ordinal or ordered categorical data. This is achieved by applying a wild bootstrap procedure in combination with quadratic form‐type test statistics. Beyond proving their asymptotic correctness, extensive simulation studies validate their applicability for small samples. Finally, two real data examples are analyzed.
Functional data analysis is becoming increasingly popular to study data from real-valued random functions. Nevertheless, there is a lack of multiple testing procedures for such data. These are particularly important in factorial designs to compare different groups or to infer factor effects. We propose a new class of testing procedures for arbitrary linear hypotheses in general factorial designs with functional data. Our methods allow global as well as multiple inference of both, univariate and multivariate mean functions without assuming particular error distributions nor homoscedasticity. That is, we allow for different structures of the covariance functions between groups. To this end, we use point-wise quadratic-form-type test functions that take potential heteroscedasticity into account. Taking the supremum over each test function, we define a class of local test statistics. We analyse their (joint) asymptotic behaviour and propose a resampling approach to approximate the limit distributions. The resulting global and multiple testing procedures are asymptotic valid under weak conditions and applicable in general functional MANOVA settings. We evaluate their small-sample performance in extensive simulations and finally illustrate their applicability by analysing a multivariate functional air pollution data set.
Timbre has been identified as a potential component in the communication of affect in music. Although its function as a carrier of perceptually useful information about sound source mechanics has been established, less is understood about whether and how it functions as a carrier of information for communicating affect in music. To investigate these issues, listeners trained in Chinese and Western musical traditions were presented with Phrases, Measures, and Notes of recorded excerpts interpreted with a variety of affective intentions by performers on instruments from the two cultures. Results showed greater accuracy and more extreme responses in Chinese musician listeners and lowest accuracy in nonmusicians suggesting that musical training plays a role in listeners’ decoding of affective intention. Responses were more differentiated and more accurate with more musical information. Excerpts were also analyzed to determine acoustic features that are correlated with timbre characteristics. Temporal, spectral, and spectrotemporal attributes were consistently used in judging affective intent in music, suggesting purposeful use of these properties by listeners. Comparison between listeners’ use of acoustic features reveals a greater number of shared features between Western musicians and nonmusicians compared to Chinese musicians for valence, although the three groups shared more features for arousal. How timbre is utilized in musical communication appears to be different across musical traditions, and valence responses seem to be more culture-specific and arousal responses more similar across cultures.
In this paper, a one-way heteroscedastic ANOVA model is considered with exponentially distributed errors. The likelihood ratio test (LRT) and two multiple comparison tests are developed for testing against ordered alternatives. A parametric bootstrap (PB) approach is proposed for implementation of tests and its asymptotic accuracy is proved. An extensive simulation study shows that all the proposed tests are accurate in terms of achieving the nominal size value, even for small samples. The proposed simultaneous confidence intervals are also seen to maintain the preassigned coverage probability. The powers of these tests are compared with a recently proposed test, which is quite conservative. Finally, the proposed tests are illustrated with the help of three data sets related to medical studies. We have developed an ‘R’ package for implementing our test procedures and shared it on the open platform ‘GitHub.’
An Electrocoagulation with Membrane Bioreactor (EC-MBR) was created in order to treat municipal wastewater and prevent membrane fouling. Experiments were carried out in a few stages to validate the new design. A laboratory-scale (EC-MBR) treating municipal wastewater operates to investigate the structure and distribution of the organic matter removal using the membrane. The study's objectives were to evaluate the organic matter (biological oxygen demand (BOD) and chemical oxygen demand (COD)) removal efficiency of Al-Hawraa wastewater, as well as its connections with statistical indicators. It was decided to sample and analyze effluent from municipal wastewater utilizing biological and chemical treatment techniques using EC-MBR with operating temperature (25 C o), pH (7-8), DO (4-6) mg/L, initial and final concentrations of BOD (184-6 mg/L) and COD (489-20 mg/L). The organic matter removal efficiency might be estimated using MLR model, according to the findings. Furthermore, the results revealed an overall reactor achieved excellent BOD and COD maximum removal efficiencies of (96.7 and 95.9 %) respectively. Finally, based on the coefficient of correlation, the MLR model by its (R 2 of 98.5 %) was efficient.
In this paper, a two-way ANOVA model is considered when interactions between two factors are present and errors are normally distributed with heteroscedastic cell variances. The problem of testing the homogeneity of simple effects against their ordered alternatives has not been studied before in the literature for this model. Here, we develop the likelihood ratio test and two heuristic tests based on multiple contrasts. Two algorithms are proposed for finding solutions of the likelihood equations under the null and full parameter spaces. The existence and uniqueness of solutions and convergence of the algorithms are established. Hence, this paper also finds the maximum likelihood estimators of simple effects when they are order restricted. A parametric bootstrap procedure is used to implement all the tests and the asymptotic accuracy of the parametric bootstrap is proved. An extensive simulation study is carried out to study the size and power performance of the tests. Results show that all the parametric bootstrap-based test procedures achieve nominal sizes for small, moderate, and highly unbalanced sample sizes. Nominal size is controlled even in the case when small samples are combined with large and heterogeneous variances. The robustness of tests is also investigated under departure from normality. The proposed tests are illustrated with the help of three examples. Finally, an “R” package has been developed.
Despite considerable advances in knowledge tracing algorithms, educational technologies that use this technology typically continue to use older algorithms, such as Bayesian Knowledge Tracing. One key reason for this is that contemporary knowledge tracing algorithms primarily infer next-problem correctness in the learning system, but do not attempt to infer the knowledge the student can carry out of the system, information more useful for teachers. The ability of knowledge tracing algorithms to predict problem correctness using data from intelligent tutoring systems has been extensively researched, but data from outcomes other than next-problem correctness have received less attention. In addition, there has been limited use of knowledge tracing algorithms in games, because algorithms that do attempt to infer knowledge from answer correctness are often too simple to capture the more complex evidence of learning within games. In this study, data from a digital learning game, (anonymized), was used to compare ten knowledge tracing algorithms’ ability to predict students’ knowledge carried outside the learning system–measured here by posttest scores–given their game activity. All Opportunities Averaged (AOA), a method proposed by Authors (2020) was used to convert correctness predictions to knowledge estimates, which were also compared to the built-in estimates from algorithms that produced them. Although statistical testing was not feasible for these data, three algorithms tended to perform better than the others: Dynamic Key-Value Memory Networks, Logistic Knowledge Tracing, and a multivariate version of Elo. Algorithms’ built-in estimates of student ability underperformed estimates produced by AOA, suggesting that some algorithms may be better at estimating performance than ability. Theoretical and methodological challenges related to comparing knowledge estimates with hypothesis testing are also discussed.
El objetivo fue analizar el efecto de diferentes tipos de feedback sobre variables psicológicas y de rendimiento en función de la percepción del deportista de la competencia del entrenador. Se realizó un estudio de caso con 33 futbolistas asignados aleatoriamente a tres condiciones experimentales (feedback positivo, negativo y ausencia de feedback). Se midieron velocidad y precisión de lanzamientos a portería, valoración de competencia, competencia percibida, motivación autónoma y vitalidad subjetiva. Se empleó un nivel ? de 0,05 para los análisis. El grupo feedback positivo exhibió niveles más altos de valoración de competencia, competencia percibida, motivación autónoma y bienestar, que los de feedback negativo y ausencia de feedback, en sujetos con alta percepción de competencia del entrenador. Este efecto no se observó en aquellos con baja percepción de competencia del entrenador. La percepción del jugador sobre la competencia del entrenador podría ser un factor en la modulación de las diferencias generadas en cuanto al tipo de feedback.
Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether it is better to separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. These methods are compared in extensive simulations and a real data example to help in answering the primary question when to use multivariate ensemble techniques instead of univariate ones.
In pre-post factorial designs involving clustered units, parametric methods such as generalized linear mixed effects models are used to handle within subject correlations. However, the distributional and parametric model assumptions in these methods are not always satisfied, especially so with modern data sets, and are often difficult to verify in practice. Often times, the assumptions may not even be realistic when data are measured in a non-metric scale as commonly happens, for example, in Quality-of-Life outcomes. In this article, nonparametric effect-size measures for clustered data in factorial designs with pre-post measurements will be introduced. In our setup, the pre and post occasions may also be viewed as two treatment conditions. Arbitrary dependence among observations within a cluster and across treatment groups is allowed. The effect-size estimators along with their asymptotic properties for computing confidence intervals and performing hypotheses tests are investigated. The proposed methods handle within-cluster as well as between-cluster treatment assignments seamlessly. The methods are shown to be effective in finite samples using simulation studies and they have high powers, especially in situations with multiple forms of clustering. Applications are illustrated with data from a three-arm Randomized Trial of Indoor Wood Smoke reduction.
In real life we often deal with independent but not identically distributed observations (i.n.i.d.o), for which the most well-known statistical model is the multiple linear regression model (MLRM) with non-random covariates. While the classical methods are based on the maximum likelihood estimator (MLE), it is well known its lack of robustness to small deviations from the assumed conditions. In this paper, and based on the Rényi’s pseudodistance (RP), we introduce a new family of estimators in case our information about the unknown parameter is given for i.n.i.d.o.. This family of estimators, let us say minimum RP estimators (as they are obtained by minimizing the RP between the assumed distribution and the empirical distribution of the data), contains the MLE as a particular case and can be applied, among others, to the MLRM with non-random covariates. Based on these estimators, we introduce Wald-type tests for testing simple and composite null hypotheses, as an extension of the classical MLE-based Wald test. Influence functions for the estimators and Wald-type tests are also obtained and analysed. Finally, a simulation study is developed in order to asses the performance of the proposed methods and some real-life data are analysed for illustrative purpose.
We introduce a unified approach for testing a variety of rather general null hypotheses that can be formulated in terms of covariance matrices. These include as special cases, for example, testing for equal variances, equal traces, or for elements of the covariance matrix taking certain values. The proposed method only requires very few assumptions and thus promises to be of broad practical use. Different test statistics are defined, and their asymptotic or approximate sampling distributions are derived. In order to particularly improve the small-sample behaviour of the resulting tests, two bootstrap-based methods are developed and theoretically justified. Several simulations shed light on the performance of the proposed tests. The analysis of a real data set illustrates the application of the procedures.
In this paper, we consider the general linear hypothesis testing (GLHT) problem in heteroscedastic one-way MANOVA. The well-known Wald-type test statistic is used. Its null distribution is approximated by a Hotelling T^2 distribution with one parameter estimated from the data, resulting in the so-called approximate Hotelling T^2 (AHT) test. The AHT test is shown to be invariant under affine transformation, different choices of the contrast matrix specifying the same hypothesis, and different labeling schemes of the mean vectors. The AHT test can be simply conducted using the usual F-distribution. Simulation studies and real data applications show that the AHT test substantially outperforms the test of [1] and is comparable to the parametric bootstrap (PB) test of [2] for the multivariate k-sample Behrens-Fisher problem which is a special case of the GLHT problem in heteroscedastic one-way MANOVA.
In this article, we propose and study an approximate Hotelling T2 (AHT) test for heteroscedastic two-way MANOVA. The AHT test is shown to be invariant under affine-transformations, different choices of the contrast matrix used to define the same hypothesis, and different labeling schemes of the cell mean vectors. We demonstrate via intensive simulations that the AHT test generally performs well and outperforms two existing approaches in terms of size and power. An extension of the AHT test for heteroscedastic multi-way MANOVA is briefly described. A dataset from a smoking cessation trial is analyzed to illustrate the methodologies. This article has supplementary material online in a single archive.
We present the FRB package for R, which implements the fast and robust bootstrap. This method constitutes an alternative to ordinary bootstrap or asymptotic inference procedures when using robust estimators such as S-, MM-or GS-estimators. The package considers three multivariate settings: principal components analysis, Hotelling tests and multivariate regression. It provides both the robust point estimates and uncertainty measures based on the fast and robust bootstrap. In this paper we give some background on the method, discuss the implementation and provide various examples.
The recently proposed modified compressive sensing (modified-CS), which utilizes the partially known support as prior knowledge, significantly improves the performance of compressive sensing. In practice, the known part will inevitably involve some errors, which may degrade the gain of modified-CS. Within the stochastic framework, this article discuss the effect of errors in known part on the recoverability of modified-CS. First, based on the probabilistic measure of recoverability, two probability inequalities on recoverability are established, which reflect the changing tendencies of the recoverability of modified-CS with respect to the addition of errors in the known support and sparsity of original sparse vector. A direct corollary reveals further the effect degree of recoverability as adding an error in known support. Second, the maximum number of errors that the modified-CS can bear is also analyzed. We prove a quantitative-bound of errors in known part that relates with the number of samples and the sparsity of original vector. This bound mirrors the fault-tolerance capability of modified-CS. Simulation experiments have been carried out to validate our theoretical results.
In this paper, we investigate tests of linear hypotheses in heteroscedastic one-way MANOVA via proposing a modified Bartlett (MB) test. The MB test is easy to conduct via using the usual χ2-table. It is shown to be invariant under affine transformations, different choices of the contrast matrix used to define the same hypothesis and different labeling schemes of the mean vectors. Simulation studies and real data applications demonstrate that the MB test performs well and is generally comparable to Krishnamoorthy and Lu’s (J Statist Comput Simul 80(8):873–887, 2010) parametric bootstrap test in terms of size controlling and power.
Background
Complex traits may be defined by a range of different criteria. It would result in a loss of information to perform analyses simply on the basis of a final clinical dichotomized affected / unaffected variable.
Results
We assess the performance of four alternative approaches for the analysis of multiple phenotypes in genetic association studies. We describe the four methods in detail and discuss their relative theoretical merits and disadvantages. Using simulation we demonstrate that PCA provides the greatest power when applied to both correlated phenotypes and with large numbers of phenotypes. The multivariate approach had low type I error only with independent phenotypes or small numbers of phenotypes. In this study, our application of the four methods to schizophrenia data provides converging evidence of the relative performance of the methods.
Conclusions
Via power analysis of simulated data and testing of experimental data, we conclude that PCA, creating one variable based on a linear combination of all the traits, performs optimally. We propose that our comparison will provide insight into the properties of the methods and help researchers to choose appropriate strategy in future experimental studies.
This article provides three approximate solutions to the multivariate Behrens–Fisher problem: the F statistic, the Bartlett, as well as the modified Bartlett corrected statistics. Empirical results indicate that the F statistic outperforms the other two and five existing procedures. The modified Bartlett corrected statistic is also very competitive.
We propose robust tests as alternatives to the classical Wilks’ Lambda test in one-way MANOVA. The robust tests use highly robust and efficient multisample multivariate S-estimators or MM-estimators instead of the empirical covariances. The properties of several robust test statistics are compared. Under the null hypothesis, the distribution of the test statistics is proportional to a chi-square distribution. As an alternative to the asymptotic distribution, we develop a fast robust bootstrap method to estimate the distribution under the null hypothesis. We show when it is asymptotically correct to estimate the null distribution in this way and we use simulations to verify the performance of the bootstrap based tests in finite samples. We also investigate the power of the new tests, as well as their robustness against outliers. Finally, we illustrate the use of these robust test statistics on a real data example. Some additional results are provided as supplemental material.
In this paper, we present results for testing main, simple and interaction effects in heteroscedastic two factor MANOVA models.
In particular, we suggest modifications to the MANOVA sum of squares and cross product matrices to account for heteroscedasticity.
Based on these modified matrices, we define some multivariate test statistics and derive their asymptotic distributions under
non-normality for the null as well as non-null cases. Derivation of these results relies on the perturbation method and limit
theorems for independently distributed random matrices. Based on the asymptotic distributions, we devise small sample approximations
for the quantiles of the null distributions. The numerical accuracy of the large sample as well as small sample approximations
are favorable. A real data set from a Smoking Cessation Trial is analyzed to illustrate the application of the methods.
KeywordsMANOVA–Perturbation method–Heteroscedasticity–Non-normality–Local alternatives–Multivariate tests
The question of how to analyze several multivariate normal mean vectors when normality and covariance homogeneity assumptions are violated is considered in this article. For the two-way MANOVA layout, we address this problem adapting results presented by Brunner, Dette, and Munk (BDM; 1997) and Vallejo and Ato (modified Brown-Forsythe [MBF]; 2006) in the context of univariate factorial and split-plot designs and a multivariate version of the linear model (MLM) to accommodate heterogeneous data. Furthermore, we compare these procedures with the Welch-James (WJ) approximate degrees of freedom multivariate statistics based on ordinary least squares via Monte Carlo simulation. Our numerical studies show that of the methods evaluated, only the modified versions of the BDM and MBF procedures were robust to violations of underlying assumptions. The MLM approach was only occasionally liberal, and then by only a small amount, whereas the WJ procedure was often liberal if the interactive effects were involved in the design, particularly when the number of dependent variables increased and total sample size was small. On the other hand, it was also found that the MLM procedure was uniformly more powerful than its most direct competitors. The overall success rate was 22.4% for the BDM, 36.3% for the MBF, and 45.0% for the MLM.
In this paper the behaviour of linear resampling statistics in martingale difference arrays X n,i , i≤k(n) is studied. It is shown that different bootstrap and permutation procedures work if the array (X n,i ) i fulfils the conditions of a general central limit theorem. As an application we obtain amongst others resampling versions of the Kuan-Lee [C.-M. Kuan and W.-M. Lee, Stud. Nonlinear Dyn. Econom. 8, No. 4, Article 1, 24 p. (2004; Zbl 1082.62539)] test for the martingale difference hypothesis.
Resampling methods are frequently used in practice to adjust critical values of nonparametric tests. In the present paper a comprehensive and unified approach for the conditional and unconditional analysis of linear resampling statistics is presented. Under fairly mild assumptions we prove tightness and an asymptotic series representation for their weak accumulation points. From this series it becomes clear which part of the resampling statistic is responsible for asymptotic normality. The results leads to a discussion of the asymptotic correctness of resampling methods as well as their applications in testing hypotheses. They are conditionally correct iff a central limit theorem holds for the original test statistic. We prove unconditional correctness iff the central limit theorem holds or when symmetric random variables are resampled by a scheme of asymptotically random signs. Special cases are the m (n) out of k (n) bootstrap, the weighted bootstrap, the wild bootstrap and all kinds of permutation statistics. The program is carried out for convergent partial sums of rowwise independent infinitesimal triangular arrays in detail. These results are used to compare power functions of conditional resampling tests and their unconditional counterparts. The proof uses the method of random scores for permutation type statistics.
Concentrating mainly on independent and identically distributed (i.i.d.)
real-valued parent sequences, we give an overview of first-order limit theorems available for bootstrapped sample sums for Efron's bootstrap. As a light unifying theme, we expose by elementary means the relationship between corresponding conditional and unconditional bootstrap limit laws. Some open problems are also
posed.
In this paper we provide a provably convergent algorithm for the multivariate Gaussian Maximum Likelihood version of the Behrens--Fisher Problem. Our work builds upon a formulation of the log-likelihood function proposed by Buot and Richards \citeBR. Instead of focusing on the first order optimality conditions, the algorithm aims directly for the maximization of the log-likelihood function itself to achieve a global solution. Convergence proof and complexity estimates are provided for the algorithm. Computational experiments illustrate the applicability of such methods to high-dimensional data. We also discuss how to extend the proposed methodology to a broader class of problems. We establish a systematic algebraic relation between the Wald, Likelihood Ratio and Lagrangian Multiplier Test () in the context of the Behrens--Fisher Problem. Moreover, we use our algorithm to computationally investigate the finite-sample size and power of the Wald, Likelihood Ratio and Lagrange Multiplier Tests, which previously were only available through asymptotic results. The methods developed here are applicable to much higher dimensional settings than the ones available in the literature. This allows us to better capture the role of high dimensionality on the actual size and power of the tests for finite samples.
In this article, we consider the use of permutation tests for comparing multivariate parameters from two populations. First, the underlying properties of permutation tests when comparing parameter vectors from two distributions and are developed. Although an exact level test can be constructed by a permutation test when the fundamental assumption of identical underlying distributions holds, permutation tests have often been misused. Indeed, permutation tests have frequently been applied in cases where the underlying distributions need not be identical under the null hypothesis. In such cases, permutation tests fail to control the Type 1 error, even asymptotically. However, we provide valid procedures in the sense that even when the assumption of identical distributions fails, one can establish the asymptotic validity of permutation tests in general while retaining the exactness property when all the observations are i.i.d. In the multivariate testing problem for testing the global null hypothesis of equality of parameter vectors, a modified Hotelling’s -statistic as well as tests based on the maximum of studentized absolute differences are considered. In the latter case, a bootstrap prepivoting test statistic is constructed, which leads to a bootstrapping after permuting algorithm. Then, these tests are applied as a basis for testing multiple hypotheses simultaneously by invoking the closure method to control the Familywise Error Rate. Lastly, Monte Carlo simulation studies and an empirical example are presented.
Complex multivariate testing problems are frequently encountered in many scientific disciplines, such as engineering, medicine and the social sciences. As a result, modern statistics needs permutation testing for complex data with low sample size and many variables, especially in observational studies. The Authors give a general overview on permutation tests with a focus on recent theoretical advances within univariate and multivariate complex permutation testing problems, this book brings the reader completely up to date with today's current thinking. Key Features: Examines the most up-to-date methodologies of univariate and multivariate permutation testing. Includes extensive software codes in MATLAB, R and SAS, featuring worked examples, and uses real case studies from both experimental and observational studies. Includes a standalone free software NPC Test Release 10 with a graphical interface which allows practitioners from every scientific field to easily implement almost all complex testing procedures included in the book. Presents and discusses solutions to the most important and frequently encountered real problems in multivariate analyses. A supplementary website containing all of the data sets examined in the book along with ready to use software codes. Together with a wide set of application cases, the Authors present a thorough theory of permutation testing both with formal description and proofs, and analysing real case studies. Practitioners and researchers, working in different scientific fields such as engineering, biostatistics, psychology or medicine will benefit from this book.
In this paper, we consider testing the equality of two mean vectors with unequal covariance matrices. In the case of equal covariance matrices, we can use Hotelling’s T 2 statistic, which follows the F distribution under the null hypothesis. Meanwhile, in the case of unequal covariance matrices, the T 2 type test statistic does not follow the F distribution, and it is also difficult to derive the exact distribution. In this paper, we propose an approximate solution to the problem by adjusting the degrees of freedom of the F distribution. Asymptotic expansions up to the term of order N − 2 for the first and second moments of the U statistic are given, where N is the total sample size minus two. A new approximate degrees of freedom and its bias correction are obtained. Finally, numerical comparison is presented by a Monte Carlo simulation.
In this chapter, testing problems for comparing related samples are discussed. Section 4.1 is the introduction to the chapter. In Section 4.2 the two-sample problem and specifically the testing problem for paired data is considered. The observations of the two samples are supposed to be realizations of two non -independent variables. The hypotheses under study concern the significance of a shift in location between the two compared populations. For this problem the Wilcoxon signed rank test and the permutation test for dependent samples are presented. Section 4.3 is dedicated to the generalization of the problem to the multisample case. The typical multisample problem is related to tests for repeated measures, where data come from the observation of the same statistical units on several time occasions. Also in this case we have a location problem and the goal is to test whether the compared populations differ in terms of central tendency. The rank based method proposed by Friedman and the multisample permutation test for repeated measures are the considered procedures.
The present study investigates the operating characteristics of several Box-type and Welch-James (WJ) modifications on factorial designs lacking homogeneity, normality, and orthogonality. For comparison purposes the behaviours of Proc Mixed and Proc GLM, available from the SAS program, were also examined. When the shape of the distribution was symmetric, the Box-type, WJ, and Proc Mixed approaches consistently controlled the rates of error; however, when the distribution was moderately skewed only the Box-type approach limited the number of errors to the nominal value. In distributions with extreme skewness, the procedure was predominantly conservative but showed improved rates of Type-I error control using the Box-Cox method of power transformation. The execution of Proc GLM was considerably influenced by the presence of heterogeneity and scarcely affected by the absence of normality. With regard to test sensitivity, the WJ and Proc Mixed approaches were substantially more powerful than the Box-type approach when variances and cells sizes were negatively paired. However, they were equally powerful when this relationship was positive. When the population variances were homogeneous, the differences in power slightly favoured the Proc GLM approach.
In this article we consider the Two-Way ANOVA model with unequal cell frequencies without the assumption of equal error variances. For the problem of testing no interaction effects and equal main effects, we propose a parametric bootstrap (PB) approach and compare it with existing the generalized F (GF) test. The Type I error rates and powers of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the generalized F-test. The PB test performs very satisfactorily even for small samples while the GF test exhibits poor Type I error properties when the number of factorial combinations or treatments goes up.
In general factorial designs where no homoscedasticity or a particular error distribution is assumed, the well-known Wald-type statistic is a simple asymptotically valid procedure. However, it is well known that it suffers from a poor finite sample approximation since the convergence to its χ2 limit distribution is quite slow. This becomes even worse with an increasing number of factor levels. The aim of the paper is to improve the small sample behaviour of the Wald-type statistic, maintaining its applicability to general settings as crossed or hierarchically nested designs by applying a modified permutation approach. In particular, it is shown that this approach approximates the null distribution of the Wald-type statistic not only under the null hypothesis but also under the alternative yielding an asymptotically valid permutation test which is even finitely exact under exchangeability. Finally, its small sample behaviour is compared with competing procedures in an extensive simulation study.
Some new algebra on pattern and transition matrices is used to determine the degrees of freedom and the parameter matrix, if the distribution of a linear sum of Wishart matrices is approximated by a single Wishart distribution. This approximation is then used to find a solution to the multivariate Behrens-Fisher problem similar to the Welch (1947) solution in the univariate case.
The comparison of the means of two populations on the basis of two independent samples is one of the oldest problems in statistics. Indeed, it has been a testing ground for many methods of inference as well as for a variety of analytic approaches to practical problems. The univariate problem was first studied by Behrens (1929) and his solution was presented by Fisher (1935) in terms of the fiducial theory. Welch studied it in the confidence theory framework and provided an 'approximate degrees of freedom' solution as well as an asymptotic series solution (1936, 1947). Many others have investigated this topic and various methods of approach were also suggested by Jeifreys (1940), Scheff6 (1943), McCullough, Gurland & Rosenberg (1960), Banerjee (1961), and Savage (1961). In the multivariate extension of the Behrens-Fisher problem, Bennett (1951) has extended the Scheff6 solution, and James (1954) the Welch series solution. The present paper studies an extension of the Welch 'approximate degrees of freedom' (APDF) solution provided by Tukey (1959), and discusses the results of a Monte Carlo sampling study on this new APDF solution and its comparison with the James series solution.
Several alternatives to Hotelling's T have been recommended for the case where variance-covariance matrices are unknown and unequal. Type I error rates and power were estimated for Hotelling's T and for seven such solutions, including Bennett's (1951), James' (1954), Yao's (1965), Johansen's (1980), Nel and Van der Merwe's (1986), Hwang and Paulson's RB: and Kim's .Additionally, the power for each solution was calculated after adjusting for Type I error rates. The number of variables, the level of intercorrelation present among the variates, the degree of heteroscedasticity, and the sample sizes associated with each of the two mean vectors were varied in each of 36 factor combinations. For each factor combination, 10,000 repetitions were run. James' procedure almost always had the highest power, but its Type I error rate was almost always greater than the nominal. Kim's and Nel and Van der Merwe's procedures had the highest power among procedures whose Type I error rates were not inflated. Type I error rates for Hotelling's J were almost always inflated, even when sample sizes were equal.
Empirical Type I error and power rates were estimated for (a) the doubly multivariate model, (b) the Welch-James multivariate solution developed by Keselman, Carriere and Lix (1993) using Johansen's results (1980), and for (c) the multivariate version of the modified Brown-Forsythe (1974) procedure. The performance of these procedures was investigated by testing within- blocks sources of variation in a multivariate split-plot design containing unequal covariance matrices. The results indicate that the doubly multivariate model did not provide effective Type I error control while the Welch-James procedure provided robust and powerful tests of the within-subjects main effect, however, this approach provided liberal tests of the interaction effect. The results also indicate that the modified Brown-Forsythe procedure provided robust tests of within-subjects main and interaction effects, especially when the design was balanced or when group sizes and covariance matrices were positively paired.
In this paper, we consider the multivariate case of the so-called nonparametric Behrens–Fisher problem where two samples with independent multivariate observations are given and the equality of the marginal distribution functions under the hypothesis in the two groups is not assumed. Moreover, we do not require the continuity of the marginal distribution functions so that data with ties and, particularly, multivariate-ordered categorical data are covered by this model. A multivariate relative treatment effect is defined which can be estimated by using the mid-ranks of the observations within each component and we derive the asymptotic distribution of this estimator. Moreover, the unknown asymptotic covariance matrix of the centered vector of the estimated relative treatment effects is estimated and its L2-consistency is proved. To test the hypothesis of no treatment effect, we consider the rank version of the Wald-type statistic (as used in Puri and Sen, Nonparametric Methods in Multivariate Analysis, Wiley, New York, 1971) and the rank version of the ANOVA-type statistic which was suggested by Brunner et al. [J. Amer. Statist. Assoc. 92 (1997) 1494–1502] for univariate nonparametric models. Simulations show that the ANOVA-type statistic appears to maintain the pre-assigned level of the test quite accurately (even for rather small sample sizes) while the Wald-type statistic leads to more or less liberal decisions. Regarding the power, none of the two statistics is uniformly superior to the other.
In this article, we consider the problem of comparing several multivariate normal mean vectors when the covariance matrices are unknown and arbitrary positive definite matrices. We propose a parametric bootstrap (PB) approach and develop an approximation to the distribution of the PB pivotal quantity for comparing two mean vectors. This approximate test is shown to be the same as the invariant test given in [Krishnamoorthy and Yu, Modified Nel and Van der Merwe test for the multivariate Behrens–Fisher problem, Stat. Probab. Lett. 66 (2004), pp. 161–169] for the multivariate Behrens–Fisher problem. Furthermore, we compare the PB test with two existing invariant tests via Monte Carlo simulation. Our simulation studies show that the PB test controls Type I error rates very satisfactorily, whereas other tests are liberal especially when the number of means to be compared is moderate and/or sample sizes are small. The tests are illustrated using an example.
Hypothesis-testing methods for multivariate data are needed to make rigorous probability statements about the effects of factors and their interactions in experiments. Analysis of variance is particularly powerful for the analysis of univariate data. The traditional multivariate analogues, however, are too stringent in their assumptions for most ecological multivariate data sets. Non-parametric methods, based on permutation tests, are preferable. This paper describes a new non-parametric method for multivariate analysis of variance, after McArdle and Anderson (in press). It is given here, with several applications in ecology, to provide an alternative and perhaps more intuitive formulation for ANOVA (based on sums of squared distances) to complement the description provided by McArdle and Anderson (in press) for the analysis of any linear model. It is an improvement on previous non-parametric methods because it allows a direct additive partitioning of variation for complex models. It does this while maintaining the flexibility and lack of formal assumptions of other non-parametric methods. The test-statistic is a multivariate analogue to Fisher’s F-ratio and is calculated directly from any symmetric distance or dissimilarity matrix. P-values are then obtained using permutations. Some examples of the method are given for tests involving several factors, including factorial and hierarchical (nested) designs and tests of interactions.
We propose a nonparametric version of Wilks' lambda (the multivariate likelihood ratio test) and investigate its asymptotic properties under the two different scenarios of either large sample size or large number of samples. For unbalanced samples, a weighted and an unweighted variant are introduced. The unweighted variant of the proposed test appears to be novel also in the normal-theory context. The theoretical results are supplemented by a simulation study with parameter settings that are motivated by clinical and agricultural data, considering in particular the performance for small sample sizes, small number of samples, and varying dimensions. Inference methods based on the asymptotic sampling distribution and a small sample approximation are compared to permutation tests and to other parametric and nonparametric procedures. Application of the proposed method is illustrated by examples.
The present paper investigates the asymptotic behaviour of a studentized permutation test for testing equality of (Pearson) correlation coefficients in two populations. It is shown that this test is asymptotically of exact level and has the same power for contiguous alternatives as the corresponding asymptotic test. As a by-product we specify the assumptions needed for the validity of the permutation test suggested in Sakaori (2002). A small simulation study compares the finite sample properties of the considered tests.
We propose different nonparametric tests for multivariate data and derive their asymptotic distribution for unbalanced designs in which the number of factor levels tends to infinity (large a, small ni case). Quasi gratis, some new parametric multivariate tests suitable for the large a asymptotic case are also obtained. Finite sample performances are investigated and compared in a simulation study. The nonparametric tests are based on separate rankings for the different variables. In the presence of outliers, the proposed nonparametric methods have better power than their parametric counterparts. Application of the new tests is demonstrated using data from plant pathology.
A new test to the multivariate Behrens–Fisher problem is obtained by modifying D. G. Nel and C. A. van der Merwe’s [Commun. Stat. Theory Methods 15, 3719–3735 (1986; Zbl 0607.62059)] test. The new test is affine invariant and it simplifies to the Welch’s approximate solution to the univariate case [B. L. Welch, Biometrika 34, 28–35 (1947; Zbl 0029.40802)]. The merits of the new test and two existing invariant tests are evaluated using Monte Carlo methods. Monte Carlo comparison shows that the new test is as powerful as the other two methods while controlling the sizes satisfactorily.
Inference about the difference between two normal mean vectors when the covariance matrices are unknown and arbitrary is considered. Assuming that the incomplete data are of monotone pattern, a pivotal quantity, similar to the Hotelling T2 statistic, is proposed. A satisfactory moment approximation to the distribution of the pivotal quantity is derived. Hypothesis testing and confidence estimation based on the approximate distribution are outlined. The accuracy of the approximation is investigated using Monte Carlo simulation. Monte Carlo studies indicate that the approximate method is very satisfactory even for moderately small samples. The proposed methods are illustrated using an example.
The main purpose of this paper is the study of the multivariate Behrens–Fisher distribution. It is defined as the convolution of two independent multivariate Student t distributions. Some representations of this distribution as the mixture of known distributions are shown. An important result presented in the paper is the elliptical condition of this distribution in the special case of proportional scale matrices of the Student t distributions in the defining convolution. For the bivariate Behrens–Fisher problem, the authors propose a non-informative prior distribution leading to highest posterior density (H.P.D.) regions for the difference of the mean vectors whose coverage probability matches the frequentist coverage probability more accurately than that obtained using the independence-Jeffreys prior distribution, even with small samples.
In recent years permutation testing methods have increased both in number of applications and in solving complex multivariate
problems. When available permutation tests are essentially of an exact nonparametric nature in a conditional context, where
conditioning is on the pooled observed data set which is often a set of sufficient statistics in the null hypothesis. Whereas,
the reference null distribution of most parametric tests is only known asymptotically. Thus, for most sample sizes of practical
interest, the possible lack of efficiency of permutation solutions may be compensated by the lack of approximation of parametric
counterparts. There are many complex multivariate problems, quite common in empirical sciences, which are difficult to solve
outside the conditional framework and in particular outside the method of nonparametric combination (NPC) of dependent permutation
tests. In this paper we review such a method and its main properties along with some new results in experimental and observational
situations (robust testing, multi-sided alternatives and testing for survival functions).
We propose a general bootstrap procedure to approximate the null distribution of non-parametric frequency domain tests about the spectral density matrix of a multivariate time series. Under a set of easy-to-verify conditions, we establish asymptotic validity of the bootstrap procedure proposed. We apply a version of this procedure together with a new statistic to test the hypothesis that the spectral densities of not necessarily independent time series are equal. The test statistic proposed is based on an "L"2-distance between the non-parametrically estimated individual spectral densities and an overall, 'pooled' spectral density, the latter being obtained by using the whole set of "m" time series considered. The effects of the dependence between the time series on the power behaviour of the test are investigated. Some simulations are presented and a real life data example is discussed. Copyright (c) 2009 Royal Statistical Society.
Efron's "bootstrap" method of distribution approximation is shown to be asymptotically valid in a large number of situations, including t-statistics, the empirical and quantile processes, and von Mises functionals. Some counter-examples are also given, to show that the approximation does not always succeed.
Oligonucleotide arrays such as Affymetrix GeneChips use multiple probes, or a probe set, to measure the abundance of mRNA of every gene of interest. Some analysis methods attempt to summarize the multiple observations into one single score before conducting further analysis such as detecting differentially expressed genes (DEG), clustering and classification. However, there is a risk of losing a significant amount of information and consequently reaching inaccurate or even incorrect conclusions during this data reduction.
We developed a novel statistical method called robustified multivariate analysis of variance (MANOVA) based on the traditional MANOVA model and permutation test to detect DEG for both one-way and two-way cases. It can be extended to detect some special patterns of gene expression through profile analysis across k (>or=2) populations. The method utilizes probe-level data and requires no assumptions about the distribution of the dataset. We also propose a method of estimating the null distribution using quantile normalization in contrast to the 'pooling' method (Section 3.1). Monte Carlo simulation and real data analysis are conducted to demonstrate the performance of the proposed method comparing with the 'pooling' method and the usual Analysis of Variance (ANOVA) test based on the summarized scores. It is found that the new method successfully detects DEG under desired false discovery rate and is more powerful than the competing method especially when the number of groups is small.
The package of robustified MANOVA can be downloaded from http://faculty.ucr.edu/~xpcui/software
In the life sciences and other research fields, experiments are often conducted to determine responses of subjects to various treatments. Typically, such data are multivariate, where different variables may be measured on different scales that can be quantitative, ordinal, or mixed. To analyze these data, we present different nonparametric (rank-based) tests for multivariate observations in balanced and unbalanced one-way layouts. Previous work has led to the development of tests based on asymptotic theory, either for large numbers of samples or groups; however, most experiments comprise only small or moderate numbers of experimental units in each individual group or sample. Here, we investigate several tests based on small-sample approximations, and compare their performance in terms of [alpha] levels and power for different simulated situations, with continuous and discrete observations. For positively correlated responses, an approximation based on [Brunner, E., Dette, H., Munk, A., 1997. Box-type approximations in nonparametric factorial designs. J. Amer. Statist. Assoc. 92, 1494-1502] ANOVA-Type statistic performed best; for responses with negative correlations, in general, an approximation based on the Lawley-Hotelling type test performed best. We demonstrate the use of the tests based on the approximations for a plant pathology experiment.
Changes of motor cortical output in patients with spinal cord injury
- R Nardone
- Y Höller
- A Thomschewski
- A C Bathke
- A R Ellis
- A Kunz
- S Golaszewski
- F Brigo
- E Trinka
R. Nardone, Y. Höller, A. Thomschewski, A.C. Bathke, A.R. Ellis, A. Kunz, S. Golaszewski, F. Brigo, E. Trinka, Changes of motor cortical output in patients
with spinal cord injury, 2014 (submitted for publication).
Multivariate multiple comparisons by bootstrap and permutation tests
- E N F Santos
- D F Ferreira
E.N.F. Santos, D.F. Ferreira, Multivariate multiple comparisons by bootstrap and permutation tests, Rev. Bras. Biom. 30 (3) (2012) 381–400.