Fumiko Samejima’s research while affiliated with The University of Tennessee Medical Center at Knoxville and other places

Samejima (Psychometrika 65:319–335, 2000) proposed the logistic positive exponent family of models (LPEF) for dichotomous responses in the unidimensional latent space. The objective of the present paper is to propose and discuss a graded response model that is expanded from the LPEF, in the context of item response theory (IRT). This specific graded response model belongs to the general framework of graded response model (Samejima, Psychometrika Monograph, No. 17, 1969 and No. 18, 1972; Handbook of modern item response theory, Springer, New York, 1997; Encyclopedia of Social Measurement, Academic Press, San Diego, 2004), and, in particular to the heterogeneous case (Samejima, Psychometrika Monograph, No. 18, 1972). Thus, the model can deal with any number of ordered polytomous responses, such as letter grades (e.g., A, B, C, D, F), etc. For brevity, hereafter, the model will be called the LPEF graded response model, or LPEFG. This model reflects the opposing two principles contained in the LPEF for dichotomous responses, with the logistic model (Birnbaum, Statistical theories of mental test scores, Addison Wesley, Reading, 1968) as their transition, which provide a reasonable rationale for partial credits in LPEFG, among others.

Abstract In estimating the operating characteristic (OC) of an item, in contrast to parametric estimation, nonparametric estimation directly approaches the entire conditional probabilty curve, without assuming any mathematical form. Samejima,proposed several nonparametric methods,in the 1970s and 1980s, under several multi-year research contracts with the Office of Naval Research. Later, one of them, the conditional p.d.f. approach, was adapted to the environment of computerized adaptive testing, utilizing its strengths. In the present research, the truncated logistic model, which leads to higher accuracy in estimating the item characteristic functions (ICFs) of dichotomous items, was used. The results of simulations showed that this method faithfully depicted even complicated changes in nonmonotonic ICFs using as few as 1,202 hypothetical examinees.

The paper addresses and discusses whether the tradition of accepting point-symmetric item characteristic curves is justified by uncovering the inconsistent relationship between the difficulties of items and the order of maximum likelihood estimates of ability. This inconsistency is intrinsic in models that provide point-symmetric item characteristic curves, and in this paper focus is put on the normal ogive model for observation. It is also questioned if in the logistic model the sufficient statistic has forfeited the rationale that is appropriate to the psychological reality. It is observed that the logistic model can be interpreted as the case in which the inconsistency in ordering the maximum likelihood estimates is degenerated. The paper proposes a family of models, called the logistic positive exponent family, which provides asymmetric item chacteristic curves. A model in this family has a consistent principle in ordering the maximum likelihood estimates of ability. The family is divided into two subsets each of which has its own principle, and includes the logistic model as a transition from one principle to the other. Rationale and some illustrative examples are given.

Rationale and the actual procedures of two nonparametric approaches, called Bivariate P.D.F. Approach and Conditional P.D.F. Approach, for estimating the operating characteristic of a discrete item response, or the conditional probability, given latent trait, that the examinee's response be that specific response, are introduced and discussed. These methods are featured by the facts that: (a) estimation is made without assuming any mathematical forms, and (b) it is based upon a relatively small sample of several hundred to a few thousand examinees. Some examples of the results obtained by the Simple Sum Procedure and the Differential Weight Procedure of the Conditional P.D.F. Approach are given, using simulated data. The usefulness of these nonparametric methods is also discussed.

Normal assumptions have been used in many psychometric methods, to the extent that most researchers do not even question their adequacy. With the rapid advancement of computer technologies in recent years, psychometrics has extended its territory to include intensive cognitive diagnosis, etcetera, and substantive mathematical modeling ha become essential. As a natural consequence, it is time to consider departure from normal assumptions seriously. As examples of models which are not based on normality or its approximation, the logistic positive exponent family of models is discussed. These models include the item task complexity as the third parameter, which determines the single principle of ordering individuals on the ability scale.

The graded response model represents a family of mathematical models that deals with ordered polytomous categories. These ordered categories include rating such as letter grading, A, B, C, D, and F, used in the evaluation of students’ performance; strongly disagree, disagree, agree, and strongly agree, used in attitude surveys; or partial credit given in accordance with an examinee’s degree of attainment in solving a problem.

In this paper, mathematical modeling is treated as distinct from curve fitting. Considerations of psychological reality behind our data are emphasized, and criteria such as additivity in a model, its natural generalization to a continuous response mode], satisfaction of the unique maximum condition and orderliness of the modal points of the operating characteristics of the ordered polychotomous responses are proposed. Strengths and weaknesses of mathematical models for ordered polychotomous responses that include the normal ogive model, the logistic model, the acceleration model and the family of ordered polychotomous models developed from Bock's nominal model are observed and discussed in terms of such criteria. It was concluded that it will be better to leave Bock's model as a nominal model as he intended it to be, without expanding it to ordered polychotomous models.

A new model, called acceleration model, is proposed in the framework of the heterogenous case of the graded response model, based on processing functions defined for a finite or enumerable number of steps. The model is expected to be useful in cognitive assessment, as well as in more traditional areas of application of latent trait models. Criteria for evaluating models are proposed, and soundness and robustness of the acceleration model are discussed. Graded response models based on individual choice behavior are also discussed, and criticisms on model selection in terms of fitnesses of models to the data are also given.

The Level 11 Vocabulary Subtest of the Iowa Tests of Basic Skills was analyzed using a two-stage latent trait approach and an empirical dataset of 2,356 examinees. First, each of the 43 multiple-choice test items was scored dichotomously; then, assuming the (two-parameter) normal ogive model the item parameters were estimated. The operating characteristics of the correct answer and of the three distractors were estimated using a nonparametric approach called the simple sum procedure of the conditional probability density function approach combined with the normal approach method. Differential information was provided by the distractors, and these operating characteristics were named the plausibility functions of the distractors. The operating characteristic of the correct answer of each item estimated by assuming the normal ogive model was compared with the nonparagnetrically estimated operating characteristic for model validation. It was concluded that the nonparametric approach leads to efficient estimation of the latent trait. Index terms: distractors, item response theory, latent trait models, multiple-choice test items, nonparametric estimation, plausibility functions of distractors.

The test information function serves important roles in latent trait models and in their applications. Among others, it has been used as the measure of accuracy in ability estimation. A question arises, however, if the test information function is accurate enough for all meaningful levels of ability relative to the test, especially when the number of test items is relatively small (e.g., less than 50). In the present paper, using the constant information model and constant amounts of test information for a finite interval of ability, simulated data were produced for eight different levels of ability and for twenty different numbers of test items ranging between 10 and 200. Analyses of these data suggest that it is desirable to consider some modification of the test information function when it is used as the measure of accuracy in ability estimation.