George Judge

• University of California, Berkeley

In this paper, we introduce a flexible and widely applicable nonparametric entropy-based testing procedure that can be used to assess the validity of simple hypotheses about a specific parametric population distribution. The testing methodology relies on the characteristic function of the population probability distribution being tested and is attr...

In this paper, we introduce a flexible and widely applicable nonparametric entropy-based testing procedure that can be used to assess the validity of simple hypotheses about a specific parametric population distribution. The testing methodology relies on the characteristic function of the population probability distribution being tested and is attr...

The focus of this paper is an information theoretic-symbolic logic approach to extract information from complex economic systems and unlock its dynamic content. Permutation Entropy (PE) is used to capture the permutation patterns-ordinal relations among the individual values of a given time series; to obtain a probability distribution of the access...

We use information theoretic information recovery methods, on a 2005 sample of household income data from the Chinese InterCensus, to estimate the income distribution for China and each of its 31 provinces and to obtain corresponding measures of income inequality. Using entropy divergence methods, we seek a probability density function solution tha...

In this paper, we borrow some of the key concepts of nonequilibrium statistical systems, to develop a framework for analyzing a self-organizing-optimizing system of independent interacting agents, with nonlinear dynamics at the macro level that is based on stochastic individual behavior at the micro level. We demonstrate the use of entropy-divergen...

In this paper, instead of likelihood based methods that are fragile under model uncertainty, we use entropy based methods on time-ordered household income data to recover income distribution information on European countries and obtain an inequality income measure. For information recovery, we use a family of information theoretic entropy divergenc...

Since Benford’s law is an empirical phenomenon that occurs in a range of data sets, this raises the question as to whether or not the same thing might be true in terms of the Chinese income distribution data. We focus on the first significant digit (FSD) distribution of Chinese micro income data from the 2005 Inter-Census sample, which corresponds...

This paper offers a general and comprehensive definition of the day-of-the-week effect. Using symbolic dynamics, we develop a unique test based on ordinal patterns in order to detect it. This test uncovers the fact that the so-called "day-of-the-week" effect is partly an artifact of the hidden correlation structure of the data. We present simulatio...

This paper offers a general and comprehensive definition of the day-of-the-week effect. Using symbolic dynamics, we develop a unique test based on ordinal patterns in order to detect it. This test uncovers the fact that the so-called "day-of-the-week" effect is partly an artifact of the hidden correlation structure of the data. We present simulatio...

In this paper, we focus on the first significant digit (FSD) distribution of European micro income data and use information theoretic-entropy based methods to investigate the degree to which Benford's FSD law is consistent with the nature of these economic behavioral systems. We demonstrate that Benford's law is not an empirical phenomenon that occ...

Regarding the current econometric scene, in this review I argue that (a) traditional econometric modeling approaches do not provide a reliable basis for making inferences about the causal effect of a supposed treatment of data in observational and quasi-experimental settings; and (b) the focus on conventional reductionist models and information rec...

In this paper, we suggest an approach to recovering behavior-related, preference-choice network information from observational data. We model the process as a self-organized behavior based random exponential network-graph system. To address the unknown nature of the sampling model in recovering behavior related network information, we use the Cress...

In this paper, we consider the question and present evidence as to whether or not Benford’s exponential first significant digit (FSD) law reflects a fundamental principle behind the complex and nondeterministic nature of large-scale physical and behavioral systems. As a behavioral example, we focus on the FSD distribution of Australian micro income...

This chapter examines and searches for evidence of fraud in two clinical data sets from a highly publicized case of scientific misconduct. In this case, data were falsified by Eric Poehlman, a faculty member at the University of Vermont, who pleaded guilty to fabricating more than a decade of data, some connected to federal grants from the National...

In the context of agent based modeling and network theory, we focus on the problem of recovering behavior-related choice information from origin-destination type data, a topic also known under the name of network tomography. As a basis for predicting agents' choices we emphasize the connection between adaptive intelligent behavior, causal entropy m...

As a basis for information recovery in open dynamic microeconomic systems, we emphasize the connection between adaptive intelligent behavior, causal entropy maximization and self-organized equilibrium seeking behavior. This entropy-based causal adaptive behavior framework permits the use of information-theoretic methods as a solution basis for the...

Although economic processes and systems are in general simple in nature, the underlying dynamics are complicated and seldom understood. Recognizing this, in this paper we use a nonstationary-conditional Markov process model of observed aggregate data to learn about and recover causal influence information associated with the underlying dynamic micr...

In this article, we formulate an information theoretic approach to information recovery for a network flow transportation problem as an ill-posed inverse problem and use nonparametric information theoretic methods to recover the unknown adaptive-intelligent behaviour traffic flows. We indicate how, in general, information theoretic methods may prov...

In this paper we assess two economic-econometric information based approaches-paths to recovering behavior related choice information and making inferences from observational data. The traditional path, starts by assuming a stochastic model based on economic, econometric and inferential statistics foundations. The unknown and unobservable parameter...

We focus discussion on extracting probability distribution functions (PDFs) from semi-chaotic time series (TS). We wish to ascertain what is the best extraction approach and to such an end we use an extremely well known semiclassical system in its classical limit [1, 2]. Since this systems possesses a very rich dynamics, it can safely be regarded a...

In this work, a method based on information theory is developed to make predictions from a sample of nonlinear time series data. Numerical examples are given to illustrate the effectiveness of the proposed method.

The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant “additivity” properties: (i) existence of a monotonic transformation which makes the functional additive with respect to the joining of independent systems and (...

In this paper we emphasize the connection between adaptive intelligent behavior, causal entropy maximization and self-organized equilibrium seeking behavior in an open dynamic micro economic systems. This entropy based causal adaptive behavior framework permits the use of information theoretic methods, as a solution basis for the resulting pure and...

The purpose of this paper is to initiate a discussion on the incorrect nature of our economic–econometric models and methods, and to make a plea for information theoretic recovery methods consistent with the data that we must use and with the questions that we need to ask.

Presentation at the Conference "Modern Problems of Mathematics, Informatics and Bioinformatics", devoted to the 100th anniversary of professor Alexei A. Lyapunov, Novosibirsk, Russia, 2011, October 11-14 Entropy was born in the 19th century as a daughter of energy. Clausius, Boltzmann and Gibbs (and others) developed the physical notion of entropy...

We discuss the connection between causal adaptive economic behavior and entropy maximization and suggest estimation and inference methods consistent with information recovery in dynamic economic systems. In particular we emphasize the connection between adaptive intelligent behavior, causal entropy maximization and self-organized equilibrium seekin...

In the context of competing theoretical economic–econometric models and corresponding estimators, we demonstrate a semiparametric combining estimator that, under quadratic loss, has superior risk performance. The method eliminates the need for pretesting to decide between members of the relevant family of econometric models and demonstrates, under...

Time-series (TS) are employed in a variety of academic disciplines. In this paper we focus on extracting probability density functions (PDFs) from TS to gain an insight into the underlying dynamic processes. On discussing this “extraction" problem, we consider two popular approaches that we identify as histograms and Bandt–Pompe. We use an informat...

To address the unknown nature of probability-sampling models, in this paper we use information theoretic concepts and the Cressie-Read (CR) family of information divergence measures to produce a flexible family of probability distributions, likelihood functions, estimators, and inference procedures. The usual case in statistical modeling is that th...

Although economic processes and systems are in general simple in nature, the underlying dynamics are complicated and seldom understood. Recognizing this, in this paper we use a nonstationary-conditional Markov process model of observed aggregate data to learn about and recover causal influence like information associated with the underlying dynamic...

By the Bayesian Law of Large Numbers for iid data, the post-data measure concentrates on the distribution(s) that minimize the L-divergence (aka the reverse I-divergence). Consequently, methods based on minimization of other divergences are not consistent under misspecification, and in this sense ruled out. This point, which has already been establ...

To address the unknown nature of probability-­sampling models, in this paper we use information theoretic concepts and the Cressie-Read (CR) family of information divergence measures to produce a flexible family of probability distributions, likelihood function relationships, estimators, and inference procedures. The usual case in statistical model...

The focus of this paper is on starting a critical discussion on the state of econometrics. The problem of information recovery in economics is discussed, and information theoretic methods are suggested as an estimation and inference framework for analyzing questions of a causal nature and learning about hidden dynamic micro and macro processes and...

Suich and Derringer's reply (1980) to Hill, Judge and Fomby (1978) is examined and their decision rule characterized.

Despite the productive efforts of economists, the disequilibrium nature of the economic system and imprecise predictions persist. One reason for this outcome is that traditional econometric models and estimation and inference methods cannot provide the necessary quantitative information for the causal influence-dynamic micro and macro questions we...

The minimum empirical divergence methods operate within the empirical estimating equations (E 3 ) approach to estimation and inference. Consequently, the existence of E 3 estimators and tests is data dependent and subject to the Empty Set Problem (ESP). As one option to avoid ESP we suggest the Revised Empirical Likelihood (ReEL) method, which oper...

The minimum discrimination information principle is used to identify an appropriate parametric family of probability distributions and the corresponding maximum likelihood estimators for binary response models. Estimators in the family subsume the conventional logit model and form the basis for a set of parametric estimation alternatives with the u...

This book is intended to provide the reader with a firm conceptual and empirical understanding of basic information-theoretic econometric models and methods. Because most data are observational, practitioners work with indirect noisy observations and ill-posed econometric models in the form of stochastic inverse problems. Consequently, traditional...

The estimation of K (K ≥ 3) location parameters is considered under quadratic loss when the coordinates of the best invariant estimators are spherically symmetrically distributed. Under these stochastic mechanisms traditional Stein estimators are evaluated for finite samples and shown to have a risk performance superior to some conventional rules.

With an eye to providing a methodology for tracking the dynamic integrity of prices for important market indicators, in this paper we use Benford second digit reference distribution to track the daily London Interbank Offered Rate (Libor) over the period 2005-2008. This reference, known as Benford’s law, is present in many naturally occurring num...

The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant “additivity” properties: (i) existence of a monotonic transformation which makes the functional additive with respect to the joining of independent systems and (...

Benford's Law can be seen as one of the many first significant digit (FSD) distributions in a family of monotonically decreasing distributions. We examine the interrelationship between Benford and other monotonically decreasing distributions such as those arising from Stigler, Zipf, and the power laws. We examine the theoretical basis of the Stigle...

In an influential work, Qin and Lawless (1994) proposed a general estimating equations (GEE) formulation for maximum empirical likelihood (MEL) estimation and inference. The formulation replaces a model specified by GEE with a set of data-supported probability mass functions that satisfy empirical estimating equations (E3). In this paper we use sev...

Empirical Likelihood (EL) and other methods that operate within the Empirical Estimating Equations (E3) approach to estimation and inference are challenged by the Empty Set Problem (ESP). ESP concerns the possibility that a model set, which is data-dependent, may be empty for some data sets. To avoid ESP we return from E3 back to the Estimating Equ...

Methods, like Maximum Empirical Likelihood (MEL), that operate within the Empirical Estimating Equations (E3) approach to estimation and inference are challenged by the Empty Set Problem (ESP). We propose to return from E3 back to the Estimating Equations, and to use the Maximum Likelihood method. In the discrete case the Maximum Likelihood with Es...

In this paper we are interested in empirical likelihood (EL) as a method of estimation, and we address the following two problems: (1) selecting among various empirical discrepancies in an EL framework and (2) demonstrating that EL has a well-defined probabilistic interpretation that would justify its use in a Bayesian context. Using the large devi...

In this paper a range of information theoretic distance measures, based on Cressie-Read divergence, are combined with mean-zero estimating equations to provide an efficient basis for semi parametric estimation and testing. Asymptotic properties of the resulting semi parametric estimators are demonstrated and issues of implementation are considered.

In an influential work, Qin and Lawless (1994) proposed a general estimating equations (GEE) formulation for maximum empirical likelihood (MEL) estimation and inference. The formulation replaces a model specified by GEE with a set of data-supported probability mass functions that satisfy empirical estimating equations (E3). In this paper we use sev...

In the Empirical Estimating Equations (E^3) approach to estimation and inference estimating equations are replaced by their data-dependent empirical counterparts. It is odd but with E^3 there are models where the E^3-based estimator does not exist for some data set, and does exist for others. This depends on whether or not a set of data-supported p...

This paper uses information theoretic methods to introduce a new class of probability distributions and estimators for competing explanations of the data in the binary choice model. No explicit parameterization of the function connecting the data to the Bernoulli probabilities is stated in the specification of the statistical model. A large class o...

In this article, we consider the problem of criterion choice in information recovery and inference in a large-deviations (LD) context. Kitamura and Stutzer recognize that the Maximum Entropy Empirical Likelihood estimator can be given a LD justification (Kitamura and Stutzer, 2002). We demonstrate there exists a similar LD justification for Owen's...

The maximum likelihood estimation principle, unbiasedness and hypothesis testing serve as foundation stones for much that goes on in the lives of theoretical and applied econometricians. In this context, the purpose of these words and other symbols is to review the statistical implications of pursuing these estimation and inference goals and to sug...

Clinical data serve as a necessary basis for medical decisions. Consequently, the importance of methods that help officials quickly identify human tampering of data cannot be underestimated. In this paper, we suggest Benford’s Law as a basis for objectively identifying the presence of experimenter distortions in the outcome of clinical research d...

Using a large deviations approach, Maximum A-Posteriori Probability (MAP) and Empirical Likelihood (EL) are shown to possess, under misspecification, an exclusive property of Bayesian consistency. Under conditions of consistency, regardless of prior the MAP estimator asymptotically coincides with EL. The consistency property is also studied for sam...

"It is 15:00 in Nairobi. Do you know where your enumerators are??" Good quality data is paramount for applied economic research. If the data are distorted, corresponding conclusions may be incorrect. We demonstrate how Benford's law, the distribution that first digits of numbers in certain data sets should follow, can be used to test for data abnor...

When there is uncertainty concerning the appropriate statistical model and corresponding estimators and inference methods, we use the Cressie–Read measure of divergence to define a semiparametric estimator, , that combines plausible estimation problems. This estimation procedure identifies, conditional on the data, an optimal combination of competi...

In a bivariate context, we consider ill-posed inverse problems with incomplete theoretical and data information. We demonstrate the use of information theoretic methods for information recovery for a range of under-identified choice problems with more unknowns than data points.

In this paper we demonstrate, in a parametric Estimating Equations setting, that the Empirical Likelihood (EL) method is an asymptotic instance of the Bayesian non-parametric Maximum-A-Posteriori approach. The resulting probabilistic interpretation and justifcation of EL rests on Bayesian non-parametric consistency in L-divergence.

The Bayesian Sanov Theorem (BST) identifies, under both correct and incorrect specification of infinite dimensional model, the points of concentration of the posterior measure. Utilizing this insight in the context of Polya urn sampling, Bayesian nonparametric consistency is established. Polya BST is also used to provide an extension of Maximum Non...

A mathematical expression known as Benford's law provides an example of an unexpected relationship among randomly selected sequences of first significant digits (FSDs). Newcomb [Note on the frequency of use of the different digits in natural numbers, Am. J. Math. 4 (1881) 39–40], and later Benford [The law of anomalous numbers, Proc. Am. Philos. So...

A mathematical expression known as Benford's law provides an example of an unexpected relationship among randomly selected sequences of first significant digits (FSDs). Newcomb [Note on the frequency of use of the different digits in natural numbers, Am. J. Math. 4 (1881) 39–40], and later Benford [The law of anomalous numbers, Proc. Am. Philos. So...

Criterion choice is such a hard problem in information recovery and in estimation and inference. In the case of inverse problems with noise, can probabilistic laws provide a basis for empirical estimator choice? That is the problem we investigate in this paper. Large Deviations Theory is used to evaluate the choice of estimator in the case of two f...

Asymptotically, semi parametric estimators of the parameters in linear structural models have the same sampling properties. In finite samples the sampling properties of these estimators vary and large biases may result for sample sizes often found in practice. With a goal of improving asymptotic risk performance and finite sample efficiency propert...

This paper introduces a new class of estimators based on minimization of the Cressie-Read (CR)power divergence measure for binary choice models, where neither a parameterized distribution nor a parameterization of the mean is specified explicitly in the statistical model. By incorporating sample information in the form of conditional moment conditi...

The purpose of this chapter is to formulate and demonstrate information theoretic, moment-based approaches to processing and recovering information from aggregate voter data. In the context of the ecological inference problem, we focus on the recovery of unknown conditional vote counts for a precinct or district, given the observed number of votes...

This paper considers estimation and inference for the binary response model in the case where endogenous variables are included as arguments of the unknown link function. Semiparametric estimators are proposed that avoid the parametric assumptions underlying the likelihood approach as well as the loss of precision when using nonparametric estimatio...

When there is uncertainty concerning the appropriate statistical model-estimator to use in representing the data sampling process, we consider a basis for optimally combining estimation problems. The objective is to produce natural adaptive estimators that are free of subjective choices and tuning parameters. In the context of two competing multiva...

This paper considers estimation and inference for the multinomial response model in the case where endogenous variables are arguments of the unknown link function. Semiparametric estimators are proposed that avoid the parametric assumptions underlying the likelihood approach as well as the loss of precision when using nonparametric estimation. A da...

This paper considers estimation and inference for the multinomial response model in the case where endogenous variables are included as arguments of the unknown link function. Semiparametric estimators are proposed that avoid the parametric assumptions underlying the likelihood approach as well as the loss of precision when using nonparametric esti...

This paper presents empirical evidence concerning the finite sample performance of conventional and generalized empirical likelihood-type estimators that utilize instruments in the context of linear structural models characterized by endogenous explanatory variables. There are suggestions in the literature that traditional and non-traditional asymp...

We seek to identify the impact of data measurement error problems in the context of ecological inference applications. We explore the statistical and substantive implications of using inaccurate proxy variables in the estimation and inference process. The focus of our analysis is on applications of ecological inference in cases involving the Voting...

When there is uncertainty concerning the appropriate statistical model to use in representing the data sampling process and corresponding estimators, we consider a basis for optimally combining estimation problems. In the context of the multivariate linear statistical model, we consider a semi-parametric Stein-like (SPSL) estimator, B( ), that shri...

In the context of a semiparametric regression model with underlying probability distribution unspecified, an extremum estimator formulation is proposed that makes use of empirical likelihood and information theoretic estimation and inference concepts to mitigate the problem of an ill-conditioned design matrix. A squared error loss measure is used t...

Information theoretic estimators are specified for a system of linear simultaneous equations, including maximum empirical likelihood, maximum empirical exponential likelihood, and maximum log Euclidean likelihood. Monte Carlo experiments are used to compare finite sample performance of these estimators to traditional generalized method of moments.

We extend the empirical likelihood method of estimation and inference proposed by Owen and others and demonstrate how it may be used in a general linear model context and to mitigate the impact of an ill-conditioned design matrix. A dual loss information theoretic estimating function is used along with extended moment conditions to yield a data bas...

This integrated textbook and CD-ROM develop step by step a modern approach to econometric problems. Aimed at upper-level undergraduates, graduate students, and professionals, they describe the principles and procedures for processing and recovering information from samples of economic data. In the real world such data are usually limited or incompl...

Abstract not available

We propose a new formulation of the statistical model and the use of the maximum entropy principle for recovering information when the dependent variable is censored or ordered. This approach makes use of weak sampling assumptions and performs well over a range of non Gaussian error distributions and ill-posed and well-posed problems. Analytical an...

Abstract not available

The classical maximum entropy (ME) approach to estimating the unknown parameters of a multinomial discrete choice problem, which is equivalent to the maximum likelihood multinomial logit (ML) estimator, is generalized. The generalized maximum entropy (GME) model includes noise terms in the multinomial information constraints. Each noise term is mod...

The classical maximum entropy (ME) approach to estimating the unknown parameters of a multinomial discrete choice problem, which is equivalent to the maximum likelihood multinomial logit (ML) estimator, is generalized. The generalized maximum entropy (GME) model includes noise terms in the multinomial information constraints. Each noise term is mod...

In this paper we consider estimation problems based on dynamic discrete time models. The first problem involves noisy state observations, where the state equation and the observation equation are nonlinear. The objective is to estimate the unknown parameters of the state and observation equations and the unknown values of the state variable. Next w...

Using a maximum entropy technique, the authors estimate the market shares of each firm in an industry using the available government summary statistics such as the four-firm concentration ratio and the Herfindahl-Hirschmann Index. They show that their technique is very effective in estimating the distribution of market shares in twenty industries....

A framework is developed to recover parameters in the case of incomplete data and underdetermined economic models. Within this context, the maximum entropy formalism is used as the criterion for recovering and making inferences relative to the unknown parameters. Examples are given to suggest the general nature of the problem specification and the...