No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Modified estimating functions
Abstract
In a parametric model the maximum likelihood estimator of a parameter of interest &psgr; may be viewed as the solution to the equation l′-sub-p(&psgr;) = 0, where l-sub-p denotes the profile <?Pub Caret>loglikelihood function. It is well known that the estimating function l′-sub-p(&psgr;) is not unbiased and that this bias can, in some cases, lead to poor estimates of &psgr;. An alternative approach is to use the modified profile likelihood function, or an approximation to the modified profile likelihood function, which yields an estimating function that is approximately unbiased. In many cases, the maximum likelihood estimating functions are unbiased under more general assumptions than those used to construct the likelihood function, for example under first- or second-moment conditions. Although the likelihood function itself may provide valid estimates under moment conditions alone, the modified profile likelihood requires a full parametric model. In this paper, modifications to l′-sub-p(&psgr;) are presented that yield an approximately unbiased estimating function under more general conditions. Copyright Biometrika Trust 2002, Oxford University Press.
... The theory of profile estimating equations, and of the related quasi-profile likelihood functions, has developed rapidly in recent years; see, among other, Liang and Zeger (1995), Barndorff-Nielsen (1995), Desmond (1997), Heyde (1997), Adimari and Ventura (2002), Severini (2002), Wang and Hanfelt (2003) and JZrgensen and Knudsen (2004). In this paper we address an important issue in parametric inference based on estimating functions in the presence of nuisance parameters, namely modifications of a quasi-profile loglikelihood function, in order to alleviate some of the problems inherent to the presence of nuisance parameters. ...
... As a practical consequence, the estimate of may perform poorly for small or moderate samples, even though it is consistent when n goes to infinity; see e.g. Liang and Zeger (1995) and Severini (2002). ...
... Several methods to modify an estimating function for the parameter of interest and reduce its bias to O(n −1 ) have been proposed. They include Severini (2002), Wang and Hanfelt (2003) and JZrgensen and Knudsen (2004). However, all these authors focus only on the reduction of the bias of (1) but do not consider likelihood-type based procedures associated to the corresponding adjusted profile estimating function. ...
We discuss higher-order adjustments for a quasi-profile likelihood for a scalar parameter of interest, in order to alleviate some of the problems inherent to the presence of nuisance parameters, such as bias and inconsistency. Indeed, quasi-profile score functions for the parameter of interest have bias of order O(1), and such bias can lead to poor inference on the parameter of interest. The higher-order adjustments are obtained so that the adjusted quasi-profile score estimating function is unbiased and its variance is the negative expected derivative matrix of the adjusted profile estimating equation. The modified quasi-profile likelihood is then obtained as the integral of the adjusted profile estimating function. We discuss two methods for the computation of the modified quasi-profile likelihoods: a bootstrap simulation method and a first-order asymptotic expression, which can be simplified under an orthogonality assumption. Examples in the context of generalized linear models and of robust inference are provided, showing that the use of a modified quasi-profile likelihood ratio statistic may lead to coverage probabilities more accurate than those pertaining to first-order Wald-type confidence intervals.
... Since the iterative method is adopted, it is convenient to correct the bias in the concentrated likelihood function. According to the method developed by Severini (2002) and Pace and Salvan (2006), the bias-corrected concentrated likelihood function for NB2 model is (see appendix A.2 for the derivation) ...
... In this case, the concentrated log likelihood is not maximized at the true value of the parameter. According to Severini (2002) and Pace and Salvan (2006), the concentrated likelihood function should be adjusted bŷ ...
Resource exploitation without appropriate property right is often inefficient. Traditional management attempts to limit over-exploitation by directly controlling input. The effective-ness of an input control hinges on individual's response. This paper uses the fishery as a case and examines fishermen's short-run fishing behavior. That is, given fishing capacity, how do fishermen adjust their fishing effort in response to ecological and economic variations? Since a fisherman does not earn wage, a shadow wage is constructed from fishing profit. This shadow wage depends on the fish stock, which is observable to fishermen but latent to econometri-cians. To deal with the latent variable problem in fishing effort models, a three-step estima-tion approach is proposed. First, count data models are used to describe trip frequency, in which stock is lumped into a time effect. Second, a stock index is inferred through a struc-tural harvest-stock relationship. Third, the time effect is regressed on the stock index and other individual-invariant regressors. With three steps, the fishing effort model can be identified without knowing the biological stock. The empirical model is illustrated with an application to the reef-fish fishery in the northeastern Gulf of Mexico. The results indicate that reef-fish fishermen use utility/profit maximizing behavior and the labor supply curve is not backward bending. It is also found that stock, fish price, fuel price, weather, and unemployment rate have significant impacts on fishing effort. However, without accounting for these impacts econometrically, analysts will draw spurious inferences about the effectiveness of traditional fisheries management.
... This is achieved by adjusting a profile quadratic generalized estimating function [12,8] for the bias induced by the fitting of the many stratum-specific effects: we prove that an exact adjustment for plug-in bias is possible under a multiplicative model for the count data. By contrast, standard methods to adjust profile estimating functions generally can reduce the plug-in bias by at most two orders of magnitude without imposing additional modeling assumptions [14,13]. ...
... Our approach for analyzing finely stratified data exactly adjusts the biases of the profile estimating functions, which was achievable owing to the multiplicative form of the marginal mean model (1). For marginal mean models that are not multiplicative, for example logistic regression models for binary responses, the fitted mean responsesμ ijk generally fail to be both unbiased and linear in the responses, and so one can reduce the biases of the resulting profile estimating functions by at most two orders of magnitude without imposing additional modeling assumptions [14,13]. ...
Standard methods for the analysis of cluster-correlated count data fail to yield valid inferences when the study is finely stratified and the interest is in assessing the intracluster correlation structure. We present an approach, based upon exactly adjusting an estimating function for the bias induced by the fitting of stratum-specific effects, that requires modeling only the first two joint moments of the observations and that yields consistent and asymptotically normal estimators of the correlation parameters.
... where v denotes the glm's variance function andβ is a pilot estimator for β. In finite samples both of these objectives have been adapted by adding a perturbation to the estimating equations such that the estimating equation (10) remains unbiased (McCullagh and Tibshirani, 1990;Severini, 2002;Jørgensen and Knudsen, 2004). ...
Generalized linear models are a popular tool in applied statistics, with their maximum likelihood estimators enjoying asymptotic Gaussianity and efficiency. As all models are wrong, it is desirable to understand these estimators' behaviours under model misspecification. We study semiparametric multilevel generalized linear models, where only the conditional mean of the response is taken to follow a specific parametric form. Pre-existing estimators from mixed effects models and generalized estimating equations require specificaiton of a conditional covariance, which when misspecified can result in inefficient estimates of fixed effects parameters. It is nevertheless often computationally attractive to consider a restricted, finite dimensional class of estimators, as these models naturally imply. We introduce sandwich regression, that selects the estimator of minimal variance within a parametric class of estimators over all distributions in the full semiparametric model. We demonstrate numerically on simulated and real data the attractive improvements our sandwich regression approach enjoys over classical mixed effects models and generalized estimating equations.
... For example,Ferguson et al. (1991);DiCiccio et al. (1996);Severini (1998Severini ( , 2000Severini ( , 2002;Sartori (2003);Cox (2006);Bellio and Sartori (2006);Pace and Salvan (2006);Brazzale et al. (2007).4 For more recent general accounts, see e.g.Maronna et al. (2006);Huber and Ronchetti (2009);Heritier et al. (2009). ...
Considering the increasing size of available data, the need for statistical methods that control the finite sample bias is growing. This is mainly due to the frequent settings where the number of variables is large and allowed to increase with the sample size bringing standard inferential procedures to incur significant loss in terms of performance. Moreover, the complexity of statistical models is also increasing thereby entailing important computational challenges in constructing new estimators or in implementing classical ones. A trade-off between numerical complexity and statistical properties is often accepted. However, numerically efficient estimators that are altogether unbiased, consistent and asymptotically normal in high dimensional problems would generally be ideal. In this paper, we set a general framework from which such estimators can easily be derived for wide classes of models. This framework is based on the concepts that underlie simulation-based estimation methods such as indirect inference. The approach allows various extensions compared to previous results as it is adapted to possibly inconsistent estimators and is applicable to discrete models and/or models with a large number of parameters. We consider an algorithm, namely the Iterative Bootstrap (IB), to efficiently compute simulation-based estimators by showing its convergence properties. Within this framework we also prove the properties of simulation-based estimators, more specifically the unbiasedness, consistency and asymptotic normality when the number of parameters is allowed to increase with the sample size. Therefore, an important implication of the proposed approach is that it allows to obtain unbiased estimators in finite samples. Finally, we study this approach when applied to three common models, namely logistic regression, negative binomial regression and lasso regression.
... The theory and use of estimating equations and that of the related quasi-and quasiprofile likelihood functions have received a good deal of attention in recent years; see, among others, Liang and Zeger (1995); Barndorff-Nielsen (1995); Desmond (1997); Heyde (1997); Adimari and Ventura (2002); Severini (2002); Wang and Hanfelt (2003); Jørgensen and Knudsen (2004); Bellio et al. (2008). In addition, Ventura et al. (2010); Lin (2006); Greco et al. (2008) discuss the use of QL functions in the Bayesian setting. ...
Approximate Bayesian Computation (ABC) is a useful class of methods for
Bayesian inference when the likelihood function is computationally intractable.
In practice, the basic ABC algorithm may be inefficient in the presence of
discrepancy between prior and posterior. Therefore, more elaborate methods,
such as ABC with the Markov chain Monte Carlo algorithm (ABC-MCMC), should be
used. However, the elaboration of a proposal density for MCMC is a sensitive
issue and very difficult in the ABC setting, where the likelihood is
intractable. We discuss an automatic proposal distribution useful for ABC-MCMC
algorithms. This proposal is inspired by the theory of quasi-likelihood (QL)
functions and is obtained by modelling the distribution of the summary
statistics as a function of the parameters. Essentially, given a real-valued
vector of summary statistics, we reparametrize the model by means of a
regression function of the statistics on parameters, obtained by sampling from
the original model in a pilot-run simulation study. The QL theory is well
established for a scalar parameter, and it is shown that when the conditional
variance of the summary statistic is assumed constant, the QL has a closed-form
normal density. This idea of constructing proposal distributions is extended to
non constant variance and to real-valued parameter vectors. The method is
illustrated by several examples and by an application to a real problem in
population genetics.
... Mykland [25] showed that this result is adequate to establish that adjustments of affine form r † ψ = {r ψ − E(r ψ )}/ SD(r ψ ) are standard normal to second order, and in usual settings these are second-order equivalent to r * ψ . Moreover, it follows from Severini [34] that for likelihood-like objects that are not true likelihoods, the first two Bartlett identities are enough to validate the NP adjustment referred to above. ...
To go beyond standard first-order asymptotics for Cox regression, we develop
parametric bootstrap and second-order methods. In general, computation of
P-values beyond first order requires more model specification than is
required for the likelihood function. It is problematic to specify a censoring
mechanism to be taken very seriously in detail, and it appears that
conditioning on censoring is not a viable alternative to that. We circumvent
this matter by employing a reference censoring model, matching the extent and
timing of observed censoring. Our primary proposal is a parametric bootstrap
method utilizing this reference censoring model to simulate inferential
repetitions of the experiment. It is shown that the most important part of
improvement on first-order methods - that pertaining to fitting nuisance
parameters - is insensitive to the assumed censoring model. This is supported
by numerical comparisons of our proposal to parametric bootstrap methods based
on usual random censoring models, which are far more unattractive to implement.
As an alternative to our primary proposal, we provide a second-order method
requiring less computing effort while providing more insight into the nature of
improvement on first-order methods. However, the parametric bootstrap method is
more transparent, and hence is our primary proposal. Indications are that
first-order partial likelihood methods are usually adequate in practice, so we
are not advocating routine use of the proposed methods. It is however useful to
see how best to check on first-order approximations, or improve on them, when
this is expressly desired.
... The conditions required by Severini (2002) ...
We consider methods for reducing the effect of fitting nuisance parameters on a general estimating function, when the estimating function depends on not only a vector of parameters of interest, θ, but also on a vector of nuisance parameters, λ. We propose a class of modified profile estimating functions with plug-in bias reduced by two orders. A robust version of the adjustment term does not require any information about the probability mechanism beyond that required by the original estimating function. An important application of this method is bias correction for the generalized estimating equation in analyzing stratified longitudinal data, where the stratum-specific intercepts are considered as fixed nuisance parameters, the dependence of the expected outcome on the covariates is of interest, and the intracluster correlation structure is unknown. Furthermore, when the quasi-scores for θ and λ are available, we propose an additional multiplicative adjustment term such that the modified profile estimating function is approximately information unbiased. This multiplicative adjustment term can serve as an optimal weight in the analysis of stratified studies. A brief simulation study shows that the proposed method considerably reduces the impact of the nuisance parameters.
... An estimator of the bias of the form of (36) that uses the observed Hessian but an expectation-based estimate of the outer product term Υ i (θ) is closely related to Severini (1998)'s approximation to the modified profile likelihood. Severini (2002) extends his earlier results to pseudo ML estimation problems , and Sartori (2003) considers double asymptotic properties of modified concentrated likelihoods in the context of independent panel or stratified data with fixed effects. ...
The purpose of this paper is to review recently developed methods of estimation of nonlinear fixed effects panel data models with reduced bias properties. We begin by describing fixed effects estimators and the incidental parameters problem. Next we explain how to construct analytical bias correction of estimators, followed by bias correction of the moment equation, and bias corrections for the concentrated likelihood. We then turn to discuss other approaches leading to bias correction based on orthogonalization and their extensions. The remaining sections consider quasi maximum likelihood estimation for dynamic models, the estimation of marginal effects, and automatic methods based on simulation.
... These methods are based on the idea of estimating a onedimensional subproblem of the original problem so that the obtained estimator is leastfavorable in the sense of Stein (1956). Severini (1998, 1999, 2002) constructed some modified profile likelihood functions, or some approximations to the modified profile likelihood functions through known distribution of data, which yield some estimating functions for the parameter of interest satisfying approximate unbiasedness. Small and McLeish (1994) in Chapter 5 of their book summed up some Hilbert space methods to obtain the estimating function for the parameter of interest, which are based on the version of parameter orthogonality and use the projection of an estimating function for full parameters onto the E-ancillary subspace of estimating functions to make the *The first author is supported by NNSF projects (10371059 and 10171051) of China. ...
This paper introduces a profile empirical likelihood and a profile conditionally empirical likelihood to estimate the parameter
of interest in the presence of nuisance parameters respectively for the parametric and semiparametric models. It is proven
that these methods propose some efficient estimators of parameters of interest in the sense of least-favorable efficiency.
Particularly, for the decomposable semiparametric models, an explicit representation for the estimator of parameter of interest
is derived from the proposed nonparametric method. These new estimations are different from and more efficient than the existing
estimations. Some examples and simulation studies are given to illustrate the theoretical results.
... An estimator of the bias of the form of (36) that uses the observed Hessian but an expectation-based estimate of the outer product term Υ i (θ) is closely related to Severini (1998)'s approximation to the modified profile likelihood. Severini (2002) extends his earlier results to pseudo ML estimation problems , and Sartori (2003) considers double asymptotic properties of modified concentrated likelihoods in the context of independent panel or stratified data with fixed effects. ...
The purpose of this paper is to review recently development methods of estimation of nonlinear fixed effects panel data models with reduced bias properties. We begin by describing fixed effects estimators and the incidental parameters problem. Next the explain how to construct analytical bias correction of estimators, followed by bias correction of estimators, followed by bias correction of the moment equation, and bias corrections for the concentrated likelihood. We then turn to discuss other approaches leading to bias correction based on orthogonalization and their extensions. The remaining sections consider quasi maximum likelihood estimation for dynamic models, the estimation of marginal effects, and automatic methods based on simulation.
We show how modified profile likelihood methods, developed in the statistical literature, may be effectively applied to estimate the structural parameters of econometric models for panel data, with a remarkable reduction of bias with respect to ordinary likelihood methods. Initially, the implementation of these methods is illustrated for general models for panel data including individual-specific fixed effects and then, in more detail, for the truncated linear regression model and dynamic regression models for binary data formulated along with different specifications. Simulation studies show the good behavior of the inference based on the modified profile likelihood, even when compared to an ideal, although infeasible, procedure (in which the fixed effects are known) and also to alternative estimators existing in the econometric literature. The proposed estimation methods are implemented in an R package that we make available to the reader.
In a parametric model, parameters are often partitioned into parameters of interest and nuisance parameters. However, as the data structure becomes more complex, inference based on the full likelihood may be computationally intractable or sensitive to potential model misspecification. Alternative likelihood-based methods proposed in these settings include pseudo-likelihood and composite likelihood. We propose a simple adjustment to these likelihood functions to reduce the impact of nuisance parameters. The advantages of the modification are illustrated through examples and reinforced through simulations. The adjustment is still novel even if attention is restricted to the profile likelihood. The Canadian Journal of Statistics xx: 1–19; 2014 © 2014 Statistical Society of CanadaRésuméLes paramètres d'un modèle sont souvent catégorisés comme nuisibles ou d'intérêt. À mesure que la structure des données devient plus complexe, la vraisemblance peut devenir incalculable ou sensible à des erreurs de spécification. La pseudo-vraisemblance et la vraisemblance composite ont été présentées comme des solutions dans ces situations. Les auteurs proposent un ajustement simple de ces fonctions de vraisemblance afin d'atténuer l'effet des paramètres nuisibles. Les avantages offerts par cette modification sont illustrés par des exemples et appuyés par des simulations. Cet ajustement est inédit même si les auteurs restreignent leur attention aux profils de vraisemblance. La revue canadienne de statistique xx: 1–19; 2014 © 2014 Société statistique du Canada
When the data are sparse but not exceedingly so, we face a trade-off between bias and precision that makes the usual choice
between conducting either a fully unconditional inference or a fully conditional inference unduly restrictive. We propose
a method to relax the conditional inference that relies upon commonly available computer outputs. In the rectangular array
asymptotic setting, the relaxed conditional maximum likelihood estimator has smaller bias than the unconditional estimator
and smaller mean square error than the conditional estimator.
Complex models typically involve intractable likelihood functions which, from a Bayesian perspective, lead to intractable posterior distributions. In this context, Approximate Bayesian computation (ABC) methods can be used in order to obtain a valid posterior approximation. However, when simulation from the model is computationally demanding, then the ABC approach may be cumbersome. We discuss an alternative method, where the intractable likelihood is approximated by a quasi-likelihood calculated through an algorithm that is reminiscent of the ABC. The proposed approximation method requires less computational effort than ABC. An extension to multiparameter models is also considered and the method is illustrated by several examples.
We show how modified profile likelihood methods, developed in the statistical literature, may be effectively applied to estimate the structural parameters of econometric models for panel data, with a remarkable reduction of bias with respect to ordinary likelihood methods. Initially, the implementation of these methods is illustrated for general models for panel data including individual-specific fixed effects and then, in more detail, for the truncated linear regression model and dynamic regression models for binary data formulated along with different specifications. Simulation studies show the good behavior of the inference based on the modified profile likelihood, even when compared to an ideal, although infeasible, procedure (in which the fixed effects are known) and also to alternative estimators existing in the econometric literature. The proposed estimation methods are implemented in an R package that we make available to the reader.
We propose an orthogonal locally ancillary estimating function that provides first-order bias correction of inferences. It requires the specification of merely the first two moments of the observations when applying to analysis of stratified clustered (continuous or binary) data with the parameters of interest in both the first and second joint moments of dependent data. Simulation results confirm that the estimators obtained using the proposed method are substantially improved over those using regular profile estimating functions.
In parametric likelihood models, the first order bias in the posterior mode and the posterior mean can be removed using objective Bayesian priors. These bias-reducing priors are defined as the solution to a set of differential equations. We provide a simple and tractable data dependent prior that solves the differential equations asymptotically and removes the first order bias. When we consider the posterior mode, this approach can be interpreted as penalized maximum likelihood in a frequentist setting. We illustrate the construction and use of the bias-reducing priors in a brief simulation study and an empirical application to foreign exchange rates.
In some problems of practical interest, a standard Bayesian analysis can be difficult to perform. This is true, for example, when the class of sampling parametric models is unknown or if robustness with respect to data or to model misspecifications is required. These situations can be usefully handled by using a posterior distribution for the parameter of interest which is based on a pseudo-likelihood function derived from estimating equations, i.e. on a quasi-likelihood, and on a suitable prior distribution. The aim of this paper is to propose and discuss the construction of a default prior distribution for a scalar parameter of interest to be used together with a quasi-likelihood function. We show that the proposed default prior can be interpreted as a Jeffreys-type prior, since it is proportional to the square-root of the expected information derived from the quasi-likelihood. The frequentist coverage of the credible regions, based on the proposed procedure, is studied through Monte Carlo simulations in the context of robustness theory and of generalized linear models with overdispersion.
This paper constructs a two-country (Home and Foreign) general equilibrium model of Schumpeterian growth without scale effects. The scale effects property is removed by introducing two distinct specifications in the knowledge production function: the permanent effect on growth (PEG) specification, which allows policy effects on long-run growth; and the temporary effects on growth (TEG) specification, which generates semi-endogenous long-run economic growth. In the present model, the direction of the effect of the size of innovations on the pattern of trade and Home’s relative wage depends on the way in which the scale effects property is removed. Under the PEG specification, changes in the size of innovations increase Home’s comparative advantage and its relative wage, while under the TEG specification, an increase in the size of innovations increases Home’s relative wage but with an ambiguous effect on its comparative advantage.
This paper presents a robust method to conduct inference in finely stratified familial studies under proband-based sampling.
We assume that the interest is in both the marginal effects of subject-specific covariates on a binary response and the familial
aggregation of the response, as quantified by intrafamilial pairwise odds ratios. We adopt an estimating function for proband-based
family studies originally developed by Zhao and others (1998) in the context of an unstratified design and treat the stratification effects as fixed nuisance parameters. Our method requires
modeling only the first 2 joint moments of the observations and reduces by 2 orders of magnitude the bias induced by fitting
the stratum-specific nuisance parameters. An analytical standard error estimator for the proposed estimator is also provided.
The proposed approach is applied to a matched case–control familial study of sleep apnea. A simulation study confirms the
usefulness of the approach.
In settings where the full probability model is not specified, consider a general estimating function g(&thgr;, &lgr;; y) that involves not only the parameters of interest, &thgr;, but also some nuisance parameters, &lgr;. We consider methods for reducing the effects on g of fitting nuisance parameters. We propose Cox--Reid-type adjustment to the profile estimating function, g(&thgr;, &lgr;ˆ-sub-&thgr;; y), that reduces its bias by two orders. Typically, only the first two moments of the response variable are needed to form the adjustment. Important applications of this method include the estimation of the pairwise association and main effects in stratified, clustered data and estimation of the main effects in a matched pair study. A brief simulation study shows that the proposed method considerably reduces the impact of the nuisance parameters. Copyright Biometrika Trust 2003, Oxford University Press.
It is well known, at least through many examples, that when there are many nuisance parameters modified profile likelihoods often perform much better than the profile likelihood. Ordinary asymptotics almost totally fail to deal with this issue. For this reason, we study asymptotic properties of the profile and modified profile likelihoods in models for stratified data in a two-index asymptotics setting. This means that both the sample size of the strata, m, and the dimension of the nuisance parameter, q, may increase to infinity. It is shown that in this asymptotic setting modified profile likelihoods give improvements, with respect to the profile likelihood, in terms of consistency of estimators and of asymptotic distributional properties. In particular, the modified profile likelihood based statistics have the usual asymptotic distribution, provided that 1/m = o(q-super- - 1/3), while the analogous condition for the profile likelihood is 1/m = o(q-super- - 1). Copyright Biometrika Trust 2003, Oxford University Press.
A conditional method is presented that renders an estimating function insensitive to nuisance parameters. The approach is a generalisation of the conditional score method to a general estimating function context and does not require complete specification of the probability model. We exploit the informal relationship between general estimating functions and score functions to derive simple generalisations of sufficient and partially ancillary statistics, referred to as G-sufficient and G-ancillary statistics, respectively. These two types of statistic are defined in a manner that does not require complete knowledge of the probability model and thus are more suitable for use with estimating functions. If we condition on a G-sufficient statistic for the nuisance parameters, the resulting conditional estimating function is insensitive to nuisance parameters and in particular achieves the plug-in unbiasedness property. Furthermore, if the conditioning argument is also G-ancillary for the parameters of interest, then the conditional estimating function possesses an attractive optimality property. Copyright Biometrika Trust 2003, Oxford University Press.
This paper presents a new quasi-profile loglikelihood with the standard kind of distributional limit behaviour, for inference about an arbitrary one-dimensional parameter of interest, based on unbiased estimating functions. The new function is obtained by requiring the corresponding quasi-profile score function to have bias and information bias of order O(1). We illustrate the use of the proposed pseudo-likelihood with an application to robust inference in linear models.
SUMMARY When the number of nuisance parameters increases in proportion to the sample size, the Cramer-Rao bound does not necessarily
give an attainable lower bound for the asymptotic variance of an estimator of the structural parameter. The present paper
presents a new lower bound under a criterion called information uniformity. The bound is expressed as the inverse of the sum
of the partial information and a certain nonnegative term, which is derived by differential-geometrical considerations. The
optimal estimating function meeting this lower bound, when it exists, is also obtained in a decomposed form. The first term
is the modified score function, and the second term is, roughly speaking, given by the normal component of the mixture covariant
derivative of some random variable. Furthermore, special versions of these results are given in concise form, and these are
then applied to elucidate the efficiency of some examples.
An approximation is derived for tests of one-dimensional hypotheses in a general regular parametric model, possibly with nuisance parameters. The test statistic is most conveniently represented as a modified log-likelihood ratio statistic, just as the R*-statistic from Barndorff-Nielsen (1986). In fact, the statistic is identical to a version of R*, except that a certain approximation is used for the sample space derivatives required for the calculation of R*. With this approximation the relative error for large-deviation tail probabilities still tends uniformly to zero for curved exponential models. The rate may, however, be O(n-1/2) rather than O(n-1) as for R*. For general regular models asymptotic properties are less clear but still good compared to other general methods. The expression for the statistic is quite explicit, involving only likelihood quantities of a complexity comparable to an information matrix. A numerical example confirms the highly accurate tail probabilities. A sketch of the proof is given. This includes large parts which, despite technical differences, may be considered an overview of Barndorff-Nielsen's derivation of the formulae for p* and R*.
Patterson & Thompson (1971) proposed estimating the variance components of a mixed analysis of variance model by maximizing
the likelihood of a set of error contrasts. In the present paper, a convenient representation is obtained for that likelihood.
It is shown that, from a Bayesian viewpoint, using only error contrasts to make inferences on variance components is equivalent
to ignoring any prior information on the fixed effects and using all the data.
SUMMARY The formation of a combination of the observed random variables and a parameter of interest ψ having a distribution depending
only on ψ provides one approach to the analysis of problems with many nuisance parameters. The condition for the resulting
formal maximum likelihood estimating equation for ψ to be unbiased is derived. Applications include a regression model for
exponentially distributed random variables with errors in the explanatory variables.
Consider a model parameterised by a scalar parameter of interest ψ and a nuisance parameter λ. Inference about ψ may be based on the signed square root of the likelihood ratio statistic, R. The statistic R is asymptotically distributed according to a standard normal distribution, with error O(n -1/2 ). To reduce the error of this normal approximation, several modifications to R have been proposed such as O.E. Barndorff-Nielsen’s [ibid 73, 307-322 (1986; Zbl 0605.62020); ibid. 78, No. 3, 557-563 (1991)] modified directed likelihood statistic, R * . In this paper, an approximation to R * is proposed that can be calculated numerically for a wide range of models. This approximation is shown to agree with R * with error of order O p (n -1 ). The results are illustrated on several examples.
Godambe & Thompson (1974) characterized optimum estimating equations for some situations when the underlying frequency function contained a nuisance parameter. In the present paper the nuisance parameter case is related to concepts of conditional likelihood and complete sufficiency. Under suitable conditions it is shown that the maximum conditional likelihood equation provides the optimum estimating equation, the criterion of optimality being independent of conditioning. In the light of this, some definitions of ancillarity and sufficiency in the presence of nuisance parameters are discussed.
An approximation to the modified profile likelihood function is proposed. This approximation is invariant under interest-respecting reparameterisations, it agrees with the modified profile likelihood function to order O(n -1 ) in the moderate-deviation sense and to order O(n -1/2 ) in the large-deviation sense, and it is stable in the sense of conditional inference. For the case of a cured exponential family model this approximation agrees with the one proposed by Barndorff-Nielsen (1995). A general expression for the approximate modified profile likelihood function is given for nonlinear regression models and for generalised linear models with known dispersion parameter.
Point estimators are usually judged in terms of their centering and spread properties, the most common measures being expectation and mean squared error. These measures, however, do not agree with the requirement of equivariance under reparameterization. In exponential families, when the parameter of interest is a scalar linear function of the natural parameter, an optimal equivariant estimator exists and is given by the zero-level optimal confidence interval. This optimal procedure is obtained by conditioning on the sufficient statistic for the nuisance parameter. However, the required conditional distribution is usually difficult to compute exactly. Quite accurate approximations are provided by recently developed higher-order asymptotic methods, in particular, by the modified directed likelihood introduced by Barndorff-Nielsen (1986). In this paper, an approximation for the optimal conditional estimator is obtained, using the modified directed likelihood as an estimating function. A simple explicit version of the approximate optimal conditional estimator is also derived. Simulation results with many nuisance parameters confirm that the approximate conditional estimators improve remarkably on the maximum likelihood estimator.
The author describes the relationship between the extended generalized estimating equations (EGEEs) of Hall & Severini (1998) and various similar methods. He proposes a true extended quasi-likelihood approach for the clustered data case and explores restricted maximum likelihood-like versions of the EGEE and extended quasi-likelihood estimating equations. He also presents simulation results comparing the various estimators in terms of mean squared error of estimation based on three moderate sample size, discrete data situations.L'auteur décrit la relation entre les équations d'estimation généralisées étendues (EEGE) de Hall & Severini (1998) et plusieurs méthodes similaires. II propose une véritable approche de quasivraisemblance étendue adaptée au cas de données regroupées et étudie des versions de type maximum de vraisemblance restreint des EEGE et des équations d'estimation de quasi-vraisemblance. II présente de plus des résultats de simulation comparant les différems estimateurs en terme d'erreur quadratique moyenne pour des échantillons de données discrètes de taille moyenne.
In many studies, the scientific objective can be formulated in terms of a statistical model indexed by parameters, only some of which are of scientific interest. The other "nuisance parameters" are required to complete the specification of the probability mechanism but are not of intrinsic value in themselves. It is well known that nuisance parameters can have a profound impact on inference. Many approaches have been proposed to eliminate or reduce their impact. In this paper, we consider two situations: where the likelihood is completely specified; and where only a part of the random mechanism can be reasonably assumed. In either case, we examine methods for dealing with nuisance parameters from the vantage point of parameter estimating functions. To establish a context, we begin with a review of the basic concepts and limitations of optimal estimating functions. We introduce a hierarchy of orthogonality conditions for estimating functions that helps to characterize the sensitivity of inferences to nuisance parameters. It applies to both the fully and partly parametric cases. Throughout the paper, we rely on examples to illustrate the main ideas.
The approximate conditional likelihood method proposed by Cox & Reid (1987) is applied to the estimation of a scalar parameter Θ , in the presence of nuisance parameters. The estimating function of Θ based on the approximate conditional likelihood is shown to be preferable to that based on the profile likelihood. A sufficient condition for both approaches to be equivalent is given. The role of parameter orthogonality is emphasized. Several examples including bivariate normal means with known coefficient of variation are presented.
Cox & Reid (1987) proposed the technique of orthogonalizing parameters, to deal with the general problem of nuisance parameters, within fully parametric models. They obtained a large-sample approximation to the conditional likelihood. Along the same lines Davison (1988) studied generalized linear models. In the present paper we deal with the problem of nuisance parameters, within a semiparametric setup which includes the class of distributions associated with generalized linear models. The technique used is that of optimum orthogonal estimating functions (Godambe & Thompson, 1989). The results are related to those of Cox & Reid (1987).
Methods are proposed for deriving unbiased estimating equations for the parameters of interest in a statistical model which
includes further incidental or nuisance parameters. We start with pivot-like quantities which eliminate the dependence on
the incidental parameters, and derive functions of these which give estimating equations which are most efficient in a certain
sense. These equations involve the incidental parameters, which are then replaced by estimators. The methods overcome some
of the difficulties encountered in likelihood and least squares estimation when there are many incidental parameters, and
our theory reduces to likelihood and least squares estimation when there are none. Thus the optimality results apply to these
methods too. Some examples are discussed.
Typically, the analysis of data consisting of multiple observations on a cluster is complicated by within-cluster correlation. Estimating equations for generalized linear modelling of clustered data have recently received much attention. This paper proposes an extension to the generalized estimating equation method proposed by Liang and Zeger (1986). Liang and Zeger's approach was to treat within-cluster correlations as nuisance parameters. This paper, using ideas from extended quasilikelihood, provides estimating equations for regression and association parameters simultaneously. The resulting estimators are proven to be asymptotically normal and consistent under certain conditions. The consistency of regression estimators allows incorrect modelling of the correlation among repeated responses. The method is illustrated with an analysis of data from a developmental toxicity study. KEY WORDS: Correlation, Extended quasi-likelihood, Generalized linear models, Longitudinal data, Marginal ...