Distinctions between Item Format and Objectivity in Scoring

Abstract

This paper clarifies important distinctions in item writing and item scoring and considers the implications of these distinctions for developing guidelines related to test construction for training teachers. The terminology used to describe and classify paper and pencil test questions frequently confuses two distinct features of questions: response format, and objectivity in scoring. There are three major response formats: (1) choice response; (2) free response; and (3) any combination of choice response and free response. Objectivity may be operationally defined by asking subject matter experts to indicate which responses should be accepted as correct. The consensus among the experts provides an empirical definition of the objectivity of the item. Results of a review of 10 commonly used textbooks on classroom measurement are discussed, which reveal inconsistency and confusion in the treatment of issues related to objectivity. Textbooks often imply that objectivity is guaranteed by specific response formats; illustrations are provided to show this is not true. Two figures and one table illustrate the discussion. A bibliography of the 10 textbooks on classroom measurement that were reviewed is appended. (SLD)