-
Yuan-Chin Ivan Chang
[show abstract]
[hide abstract]
ABSTRACT: Effectively combining many classification instruments or diagnostic measurements together to improve the classification accuracy of individuals is a common idea in disease diagnosis or classification. These ensemble-type diagnostic methods can be constructed with respect to different kinds of performance criterions. Among them, the receiver operating characteristic (ROC) curve is the most popular criterion, which, together with some indexes derived from it, is commonly used to evaluate and summarize the performance of a classification instrument, such as a biomarker or a classifier. However, the usefulness of ROC curve and its related indexes relies on the existence of a binary label for each individual subject. In many disease diagnosis situations, such a binary variable may not exist, but only the continuous measurement of the true disease status is available. This true disease status is often referred to as the 'gold standard'. The modified area under ROC curve (AUC)-type measure defined by Obuchowski is a method proposed to accommodate such a situation. However, there is still no method for finding the optimal combination of diagnostic measurements, with respect to such an index, to have better diagnostic power than that of each individual measurement. In this paper, we propose an algorithm for finding the optimal combination with respect to such an extended AUC-type measure such that the combined measurement can have more diagnostic power. We illustrate the performance of our algorithm by using some synthesized data and a diabetes data set. Copyright © 2012 John Wiley & Sons, Ltd.
Statistics in Medicine 09/2012; · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In clinical trials, a covariate-adjusted response-adaptive (CARA) design
allows a subject newly entering a trial a better chance of being allocated to a
superior treatment regimen based on cumulative information from previous
subjects, and adjusts the allocation according to individual covariate
information.
Since this design allocates subjects sequentially, it is natural to apply a
sequential method for estimating the treatment effect in order to make the data
analysis more efficient.
In this paper, we study the sequential estimation of treatment effect for a
general CARA design. A stopping criterion is proposed such that the estimates
satisfy a prescribed precision when the sampling is stopped. The properties of
estimates and stopping time} are obtained under the proposed stopping rule. In
addition, we show that the asymptotic properties of the allocation function,
under the proposed stopping rule, are the same as those obtained in the
non-sequential/fixed sample size counterpart.
We then illustrate the performance of the proposed procedure with some
simulation results using logistic models. The properties, such as the coverage
probability of treatment effect, correct allocation proportion and average
sample size, for diverse combinations of initial sample sizes and tuning
parameters in the utility function are discussed.
06/2011;
-
[show abstract]
[hide abstract]
ABSTRACT: Rather than viewing receiver operating characteristic (ROC) curves directly to compare the performances of diagnostic methods, the whole and the partial areas under the ROC curve (area under the ROC curve [AUC] and partial area under the ROC curve [pAUC]) are 2 of the most popularly used summaries of the curve. Moreover, when high specificity is a prerequisite, as in some medical diagnostics, pAUC is preferable. In this paper, we propose a wrapper-type algorithm to select the best linear combination of markers that has high sensitivity within a confined specificity range. The markers selected by the proposed algorithm are different from those selected by AUC-based algorithms and therefore provide different information for further studies. Most notably, for example, within the given range of specificity, the markers selected by the proposed algorithm always have higher individual sensitivities than those selected by other AUC-based methods. This characteristic makes the proposed method a good addition to existing methods. Without assuming the underlying distributions of markers, we prove that the pAUC obtained with the proposed algorithm is a strongly consistent estimate of the true pAUC and then illustrate its performance with numerical studies using synthesized data and 2 real examples. The results are compared with those obtained by its AUC-based counterpart. We found that the classification performance of the final classifier based on the selected markers is very competitive.
Biostatistics 04/2011; 12(2):369-85. · 2.14 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Case-control sampling is popular in epidemiological research because of its cost and time saving. In a logistic regression model, with limited knowledge on the covariance matrix of the point estimator of the regression coefficients a priori, there exists no fixed sample size analysis. In this study, we propose a two-stage sequential analysis, in which the optimal sample fraction and the required sample size to achieve a predetermined volume of a joint confidence set are estimated in an interim analysis. Additionally required observations are collected in the second stage according to the estimated optimal sample fraction. At the end of the experiment, data from these two stages are combined and analyzed for statistical inference. Simulation studies are conducted to justify the proposed two-stage procedure and an example is presented for illustration. It is found that the proposed two-stage procedure performs adequately in the sense that the resultant joint confidence set has a well-controlled volume and achieves the required coverage probability. Furthermore, the optimal sample fractions among all the selected scenarios are close to one. Hence, the proposed procedure can be simplified by always considering a balance design.
Biometrical Journal 02/2011; 53(1):5-18. · 1.25 Impact Factor
-
Computational Statistics & Data Analysis. 01/2010; 54:2203-2213.
-
[show abstract]
[hide abstract]
ABSTRACT: It is well known that the boosting-like algorithms, such as AdaBoost and many of its modifications, may over-fit the training data when the number of boosting iterations becomes large. Therefore, how to stop a boosting algorithm at an appropriate iteration time is a longstanding problem for the past decade (see Meir and Rätsch, 2003). Bühlmann and Yu (2005) applied model selection criteria to estimate the stopping iteration for L2Boosting, but it is still necessary to compute all boosting iterations under consideration for the training data. Thus, the main purpose of this paper is focused on studying the early stopping rule for L2Boosting during the training stage to seek a very substantial computational saving. The proposed method is based on a change point detection method on the values of model selection criteria during the training stage. This method is also extended to two-class classification problems which are very common in medical and bioinformatics applications. A simulation study and a real data example to these approaches are provided for illustrations, and comparisons are made with LogitBoost.
Computational Statistics & Data Analysis 01/2010; 54(10):2203-2213. · 1.03 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We study the best linear combination of markers in terms of the area under the receiver operating characteristic curve, since no single marker is perfect for classification purposes. The sequential fixed-width confidence interval estimate method is applied. We show that the proposed procedure is efficient in terms of the total sample size, with an optimal ratio of cases to controls, and is asymptotically consistent. The performance of our method is illustrated by synthesized data and a real example.
Statistics [?] Probability Letters 01/2009; 79(18):1921-1927. · 0.50 Impact Factor
-
Bioinformatics. 01/2007; 23:2788-2794.
-
Intelligent Data Engineering and Automated Learning - IDEAL 2004, 5th International Conference, Exeter, UK, August 25-27, 2004, Proceedings; 01/2004
-
Yuan-chin Ivan Chang
[show abstract]
[hide abstract]
ABSTRACT: The support vector machine classifier is a linear maximum margin classifier. It performs very well in many classification applications. Although, it could be extended to nonlinear cases by exploiting the idea of kernel, it might still su#er from the heterogeneity in the training examples. Since there are very few theories in the literature to guide us on how to choose kernel functions, the selection of kernel is usually based on a try-and-error manner.
06/2003;
-
Yuan-chin Ivan Chang
[show abstract]
[hide abstract]
ABSTRACT: The adaptive testing is an important testing method in the modern educational/psychological testings. In adaptive mastery testing, the items are selectly adaptively according to the estimated information of the unknown latent ability levels, and given then to the test-takers, sequentially. Hence, the decision (master or non-master; pass or fail) for each test-taker is made sequentially based on each test-taker's responses to a particular sequence of items administered to him/her. Thus, statistically speaking, by the natural character of the adaptive mastery testing, it is a sequential problem with dependent observations. The Wald's (1947) SPRT (sequential probability ratio test) has been applied to this kind of mental testing problem by many researchers in the field of educational/psychological measurement theory; for example, Reckase (1978, 1983), Kingsbury and Weiss (1983) and Spray (1993). Most of their results are empirical studies of the performance of SPRT with di#erent item selection schemes. In statistical literature, the SPRT with iid observations have been intensively studied by many statistician since Wald (1947). In this paper, we concentrate on the properties of stopping time of the SPRT under adaptive mastery testing situation; i.e. the test items (observations) are adaptively selected. Not only there are only few people discuss the properties of SPRT under non-iid setup, but also most of them are just large sample properties, which provide very little information for test-makers to design good mastery tests. Without independence property of the observations, it will require di#erent approaches to analyze its performance. Here we apply some results of linear growth processes and a martingale extension of the Wald's equation to obtain a bound of the expect...
04/2003;
-
Intelligent Data Engineering and Automated Learning - IDEAL 2002, Third International Conference, Manchester, UK, August 12-14, Proceedings; 01/2002
-
[show abstract]
[hide abstract]
ABSTRACT: With the advent of modern computer technology, there have been growing e#orts in recent years to computerize standardized tests, including the popular Graduate Record Examination (GRE), the Graduate Management Admission Test (GMAT) and the Test of English as a Foreign Language (TOEFL). Many of such computer-based tests are known as the computerized adaptive tests, a major feature of which is that, depending on their performance in the course of testing, di#erent examinees may be given with di#erent sets of items (questions). In doing so, items can be e#ciently utilized to yield maximum accuracy for estimation of examinees' ability traits. We consider, in this article, one type of such tests where test lengths vary with examinees to yield approximately same predetermined accuracy for all ability traits. A comprehensive large sample theory is developed for the expected test length and the sequential point and interval estimates of the latent trait. Extensive simulations are conducted with results showing that the large sample approximations are adequate for realistic sample sizes.
08/2001;
-
Yuan-chin Ivan Chang
[show abstract]
[hide abstract]
ABSTRACT: In this paper, by using a last-time random variable, we show the strong consistency for the maximum quasi-likelihood estimate in generalized linear models with adaptive design variables and general link functions. Our approach is based on the Leray-Schauder Theorem and a last-time theorem. The last time that we defined here is based on a sum of martingale differences instead of independent random variables. Under some slightly stronger assumptions on the adaptive design variables, we obtain the almost sure convergence as well as the convergence rate of the estimate.
Statistics [?] Probability Letters 02/1999; 45(3):237-246. · 0.50 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The iterative weighted least squares algorithm is handy for solving generalized estimating equations. In some situations it may be desirable to limit the number of iterations to a fixed finite number, for instance, to keep the breakdown point under control. Such a scheme is called reweighting. Usually reweighting leads to a different large sample theory than full iteration, and the reweighted estimator may inherit deficiencies of the starting value. When might the reweighting scheme work? To answer this question we define a broad class of estimators, namely, approximate GM estimators, and we show that reweighting leads to the same large sample theory as full iteration within this class. As an example, we provide conditions under which one-step Newton-Raphson estimators are approximate GM estimators. We then use the reweighting to construct residual-based graphics for approximate GM estimates, adapting weighted residual plots that have been proposed previously, and developing new plots to provide complementary views of the data.
Journal of Statistical Planning and Inference.
-
[show abstract]
[hide abstract]
ABSTRACT: Let $(\mathbf{X}_i, Y_i)$ be independent, identically distributed observations that satisfy a logistic regression model; that is, for each $i, \log \lbrack P(Y_i = 1 | \mathbf{X}_i)/P(Y_i = 0 |\mathbf{X}_i)\rbrack = \mathbf{X}^T_i \beta_0$, where $Y_i \in \{0, 1\}, \mathbf{X}_i \in \mathbf{R}^p$ and $\beta_0 \in \mathbf{B}^p$ is the unknown parameter vector of the model. The marginal distribution of the covariate vectors $\mathbf{X}_i$ is assumed to be unknown. Sequential procedures for constructing fixed size and fixed proportional accuracy confidence regions for $\beta_0$ are proposed and shown to be asymptotically efficient as the size of the region becomes small.
-
Yuan-chin Ivan Chang
[show abstract]
[hide abstract]
ABSTRACT: Let (Xi, Yi) be independent, identically distributed observations that satisfy a binary regression model; i.e. for each i = 1, 2, …, P (Yi = 1 | Xi) = F (XTiβ0), where F is some continuous distribution function, Yi ∈ {0, 1}, Xi ∈ Rp, and β0 ∈ Rp is the unknown parameter vector of the model. The marginal distribution of Xi is assumed to be unknown. Sequential procedures for constructing fixed size confidence regions for β0 and linear combinations of β0 are proposed and shown to be asymptotically consistent and efficient as the size of the region becomes small. Moreover, a sequential confidence interval for the probability of response at a given factor will also be given.
Journal of Statistical Planning and Inference.
-
[show abstract]
[hide abstract]
ABSTRACT: The boosting as a stochastic approximation algorithm is considered. This new interpretation provides an alternative theoretical framework for investigation. Following the results of stochastic approximation theory a stochastic approximation boosting algorithm, SABoost, is proposed. By adjusting its step sizes, SABoost will have different kinds of properties. Empirically, it is found that SABoost with a small step size will have smaller training and testing errors difference, and when the step size becomes large, it tends to overfit (i.e. bias towards training scenarios). This choice of step size can be viewed as a smooth (early) stopping rule. The performance of AdaBoost is compared and contrasted.
Computational Statistics & Data Analysis.
