Richard D Roberts

Research and Assessment Design: Science Solution

A focus on implementing social and emotional (SE) learning into curricula continues to gain popularity in K-12 educational contexts at the policy and practitioner levels. As it continues to be elevated in educational discourse, it becomes increasingly clear that it is important to have reliable, validated measures of students’ SE skills. Here we ar...

This article describes (a) the origins of, (b) rationale underlying, and (c) preliminary validity evidence for, a new ability measure of emotional understanding (EU), a major component of emotional intelligence. A novel conceptual approach-the empathic agent paradigm (EAP)-provided the theoretical foundation. The EAP results in a veridical, logical...

Forced choice (FC) measures may be a desirable alternative to single stimulus (SS) Likert items, which are easier to fake and can have associated response biases. However, classical methods of scoring FC measures lead to ipsative data, which have a number of psychometric problems. A Thurstonian item response theory (TIRT) model has been introduced...

This paper investigates a quadrant-based typology of circadian preference including morning (M) types (high morningness, low eveningness), evening (E) types (low morningness, high eveningness), low M-E types (low morningness and low eveningness), and high M-E types (high morningness and eveningness). In Study 1, a latent class analysis of circadian...

Situational Judgment Tests (SJTs) are criterion valid low fidelity measures that have gained much popularity as predictors of job performance. A broad variety of SJTs have been studied, but SJTs measuring personality are still rare. Personality traits such as Conscientiousness are valid predictors of many educational, work and life-related outcomes...

The theory of planned behavior (TPB) offers a theoretically meaningful framework for examining students' beliefs and attitudes toward mathematics at school. However, longitudinal investigations of mathematics beliefs and attitudes using the TPB are scarce at best. To redress this imbalance, we examined the predictive validity of mathematics beliefs...

Data from self-report tools cannot be readily compared between cultures due to culturally specific ways of using a response scale. As such, anchoring vignettes have been proposed as a suitable methodology for correcting against this difference. We developed anchoring vignettes for the Big Five Inventory-44 (BFI-44) to supplement its Likert-type res...

Test behavior refers to any type of mental or physical response that an individual or group exhibits in anticipation of, during, and following the administration of a psychological or educational test—an instrument or method designed to measure a specific characteristic or a set of characteristics of an individual or social group. To ensure that a...

The current study investigated whether manipulating participants' pre-exposure to reward and punishment affects the extent to which sensation seeking and values predict risk-taking behavior. Participants (n = 195) were randomly allocated to one of two conditions, defined by the order at which they were rewarded or punished for risk-taking behavior....

Although educational catchphrases like “reading, writing, and arithmetic” seem dated and quaint, modern assessment systems still place inordinate emphasis on these narrow domains. Focusing on academic content knowledge to the exclusion of noncognitive skills leaves a lot of unexplained variance on the table. Broadening our definition of readiness t...

Emotional intelligence (EI) refers to a set of competencies for understanding and managing emotions and emotional encounters. It is an elusive construct that has inspired a variety of conceptualizations and measurement strategies. Current perspectives divide into those that define EI as an ability akin to conventional intelligence, and those that a...

This chapter introduces the book by asking the question of whether “building better students” to improve the state of workforce readiness, which is currently experiencing a “skills gap,” is really a necessity. The chapter concludes that building better students is indeed necessary and introduces one theory for how society reached this point. This t...

The face of the workforce is rapidly changing. Technological advances mean that jobs previously serving as major drivers of the world’s economy are now fully automated. Furthermore, the automatization of many common work activities means that those currently entering the workforce require a different set of skills than those entering the workforce...

In this final concluding chapter, we synthesize the running themes across previous chapters in the book to arrive at a broad taxonomy and working definition of workforce readiness. Next, we look to the future of workforce readiness research by examining the global trends that will influence readiness over time, examining the impact both on workforc...

A relatively new item response theory (IRT) approach (Böckenholt, 2012) and its multidimensional extension (Khorramdel & von Davier, 2014; von Davier & Khorramdel, 2013) to test and correct for response styles was applied to international large-scale assessment data – the Programme for International Student Assessment 2012 field trial – for the fir...

This study explores relations between measures of individuals' circadian preferences and the Big Five. To this end, we compared a model of circadian preferences that acknowledges morningness (M) and eveningness (E) as separate dimensions to that of a model that places M and E on a single continuum (M-E). Analyses of 620 correlations from 44 indepen...

Working effectively with people from different cultures is critically important for U.S. Soldiers. However, there is evidence to suggest that a majority of Soldiers fall short in this ability (Baker and Hamilton in The Iraq study group report: The way forward - A new approach. Vintage, New York, 2006). Recently, this ability has been deemed cross-c...

The Situational Judgment Test of Emotional Management (STEM) represents one of the few open-source instruments for the assessment of the fourth branch of emotional intelligence. In this presentation, we will discuss a range of studies designed to provide a clearer understanding of the STEM’s nomological network, and subsequent construct validity ev...

Emotional intelligence (EI) stands at the nexus between intelligence and emotion disciplines, and we outline how EI research might be better integrated within both theoretical frameworks. From the former discipline, empirical research focused upon whether EI is an intelligence and what type of intelligence it constitutes. It is clear that ability-b...

Emotional intelligence (EI) stands at the nexus between intelligence and emotion disciplines, and we outline how EI research might be better integrated within both theoretical frameworks. From the former discipline, empirical research focused upon whether EI is an intelligence and what type of intelligence it constitutes. It is clear that ability-b...

In this response to the critiques of Fontaine (2016) and Hughes and Evans (2016), we touch on main points of consensus and contention, and offer some suggestions for future programs of research.

Meta-analyses suggest that morning-oriented students obtain better school grades than evening-oriented students. This finding has generally been found for students in high school using self-report data for the assessment of circadian preference. Two studies (N = 2718/192) investigated whether these findings generalize across samples (i.e. elementar...

Accurate appraisal of others’ behavior is critical for the production of skilled interpersonal behavior. We used an ecologically valid methodology, a video-based situational judgment test with true-false items, to assess the accuracy with which students (N = 947) perceive the interpersonal behavior of actors involved in workplace situations. Studen...

This article reports the results of a quasi-experimental study conducted to examine the efficacy of a new time management intervention designed for high school students. Participants were 149 students from a highly selective private high school in the northeastern United States who were in the ninth grade. Half of the students participated in a 5-w...

The Mission Skills Assessment (MSA) is an online assessment that targets 6 noncognitive constructs: creativity, curiosity, ethics, resilience, teamwork, and time management. Each construct is measured by means of a student self-report scale, a student alternative scale (e.g., situational judgment test), and a teacher report scale. Use of the MSA pr...

In this concluding commentary, we summarize the main themes discussed by our contributors and offer ideas for future research and policy implications. We begin by describing psychosocial skills both as predictors of academic outcomes and indicators of a broader definition of academic success, raising issues that we feel may need attention in future...

This study examined relationships between conscientiousness facets and both broad factors of cognitive ability and collegiate GPA. Students responded to 117 Conscientiousness items and 15 cognitive tests demarcating fluid intelligence, crystallized intelligence, quantitative reasoning, visual processing, and broad retrieval ability. Confirmatory fa...

This book provides a comprehensive overview and in-depth analysis of research on psychosocial skills, examining both theory and areas of application. It discusses students’ psychosocial skills both as components of academic success and desired educational outcomes in grades K through 12. The book describes an organizing framework for psychosocial s...

Personality traits are associated with stable affective dispositions, including vulnerability to stress. Coping is conceptualized as a process that mediates interactions between people and sources of environmental demand. Thus, research on coping may help to explain associations between personality and affective response. Various traits correlate w...

The aim of this study was to investigate the predictive and moderating effects of HEXACO personality factors, in addition to theory of planned behavior (TPB) variables, on fruit and vegetable consumption. American college students (N = 1036) from 24 institutions were administered the TPB, HEXACO and a self-reported fruit and vegetable consumption m...

This paper provides a theoretical overview and empirical demonstration of Partial Least Squares Path Modeling (PLS) comparing it to Structural Equation Modeling (SEM). The two data analysis techniques share historical origins but differ in terms of underlying algorithms, which gives rise to other key differences: (a) the treatment of unexplained in...

The Reading the Mind in the Eyes Test is a popular measure of individual differences in Theory of Mind that is often applied in the assessment of particular clinical populations (primarily, individuals on the autism spectrum). However, little is known about the test's psychometric properties, including factor structure, internal consistency, and co...

The numerous advantages of structural equation modeling (SEM) for the analysis of multitrait–multimethod (MTMM) data are well known. MTMM-SEMs allow researchers to explicitly model the measurement error, to examine the true convergent and discriminant validity of the given measures, and to relate external variables to the latent trait as well as th...

Previous research has established the association between Extraversion and pleasure orientation. This paper fo- cuses on how Extraversion predicts problem-solving ability. The present study examined the role of Extraversion and pleasure-orientation values (i.e., Hedonism & Stimulation) and welfare-orientation values (i.e., Universalism & Benevolenc...

People process emotional information using visual, vocal, and verbal cues. However, emotion management is typically assessed with text-based rather than multimedia stimuli. This study (N = 427) presents the new Multimedia Emotion Management Assessment (MEMA) and compares it to the text-based assessment of emotion management used in the MSCEIT. The...

Meta-analyses have demonstrated that other-ratings of Conscientiousness are stronger predictors of academic achievement than are self-ratings. The current study (N = 410 high school students) examined whether this effect applies for all facets of Conscientiousness. Compared to self-reports, parent-reports showed stronger prediction of GPA and of ot...

The Situational Judgment Test of Emotional Management (STEM) represents one of the few open-source instruments for the assessment of the fourth branch of emotional intelligence. In this presentation, we will discuss a range of studies designed to provide a clearer understanding of the STEM’s nomological network, and subsequent construct validity ev...

Both creativity and ethical decision making have been identified as 21st century skills that need to be facilitated in modern educational policy and practice. Prior research on the “dark side of creativity” suggests that creativity impacts ethical decision making adversely. This study is the first to study the reciprocity of students’ creativity an...

This study uses a 3-Parameter Logistic item response theory (IRT) model to develop an 18-item short-form of the Situational Test of Emotion Management (STEM). In a sample of 900 people, the short form showed acceptable reliability (reliability index = .87; Cronbach’s alpha = .84) and a meaningful correlation with another indicator of emotional inte...

Objectives: To explore whether HEXACO personality domains could incrementally increase the prediction of exercise behavior above Theory of Planned Behavior (TPB) variables, and whether personality moderated the intention-behavior gap. Methods: US College students (N = 1017) completed demographic, TPB, HEXACO domain and exercise behavior measures...

This study tested whether trait emotional intelligence (TEI) measures of narrow bandwidth predict perception of facial emotion, using two tasks: identification of microexpressions of emotion and controlled visual search for target emotions. A total of 129 undergraduates completed multiple scales for TEI, as well as cognitive ability, personality, a...

This paper provides a theoretical overview and empirical demonstration of Partial Least Squares Path Modeling (PLS) comparing it to Structural Equation Modeling (SEM). The two data analysis techniques share historical origins but differ in terms of underlying algorithms, which gives rise to other key differences: (a) the treatment of unexplained in...

With globalization, cross-cultural competence is increasingly important to effective policies in international relations, business, and even in our schools and communities. Can we assess the skills and attributes relevant to gaining proficiency in other cultures? What kinds of training can help people toward this goal? Evidence on the assessment qu...

There is a growing interest in the application of item response theory (IRT) models to emotional intelligence (EI) as a means of instrument development. The goal of this study was to create a short form of the Situational Test of Emotional Understanding (STEU) based on analysis from a 3-PL IRT model. Analysis of 42 STEU item responses from 822 part...

Parents consult with schools on how to help their children succeed, but schools rarely consult with parents, even though most parents have considerable expertise concerning their children's thoughts, feelings, and abilities. This study compares the prediction of academic achievement from self- and parent-ratings of feelings towards school (both pos...

This article concerns how noncognitive constructs—personality and motivation—can be assessed and developed to increase students' readiness for college. We propose a general framework to account for personality and motivational differences between students. We review numerous studies showing that personality and motivational factors are related to e...

Emotional intelligence (EI) may promote more effective decision-making under stress. In the present study, 167 participants completed a situation judgment test for EI, and performed a decision-making task based on an Antarctic rescue scenario. Participants were assigned to either a negative or neutral feedback group. Negative feedback significantly...

Emotional intelligence (EI) may promote more effective decision-making under stress. In the present study, 167 participants completed a situation judgment test for EI, and performed a decision-making task based on an Antarctic rescue scenario. Participants were assigned to either a negative or neutral feedback group. Negative feedback significantly...

Self-report ratings of emotional intelligence (EI) can be faked in high-stakes situations. Although forced-choice administration can prevent response distortion, it produces ipsative scores when scored conventionally. This study (n = 486) develops an 18-item EI rating scale assessing emotion perception, understanding, and management. We compare val...

This article examines the status of emotional intelligence (EI) within the structure of human cognitive abilities. To evaluate whether EI is a 2nd-stratum factor of intelligence, data were fit to a series of structural models involving 3 indicators each for fluid intelligence, crystallized intelligence, quantitative reasoning, visual processing, an...

The current paper reports the results of 2 quasiexperimental studies conducted to examine the efficacy of a new time management intervention designed for high school students. In both studies, there was no difference between the treatment and control groups in improvement in self-reported time management skills as a result of the intervention. Howe...

This study investigated the protective effects of emotional intelligence (EI) during decision-making under stress. The researchers assigned 172 participants to either a negative or neutral feedback group and assessed EI, distress, information search prior to choice, and decision-making performance. We predicted EI would be associated with superior...

Oprah Winfrey’s lifetime accomplishments are impressive by any measure. Named one of Time magazine’s 100 most influential people of the twentieth century, she hosted one of the world’s most watched television shows for 24 years (Academy of Achievement, 2011). She is also a film producer, runs her own magazine and cable television station, and was n...

Despite near universal acceptance in the value of higher education for individuals and society, college persistence rates in 4-year and community colleges are low. Only 57% of students who began college at a 4–year institution in 2001 had completed a bachelor's degree by 2007, and only 28% of community college students who started school in 2005 ha...

A situational judgment test of emotion management was administered to 382 eighth-grade students with typical-performance instructions (i.e., “What would you do in this situation?”). The emotion management test was also administered to a parent of each student in an observer-judgment format (i.e., “What would your child do in this situation?”). Comp...

Abstract This study examines the role of some personal and professional factors in compassion fatigue among health-care professionals. Research participants included 182 (89 mental and 93 medical) health-care professionals who completed an assessment battery measuring compassion fatigue, emotion management, trait emotional intelligence, situation-s...

Background. Chronotype refers to individuals’ preference for morning or evening activities. Its two dimensions (morningness and eveningness) are related to a number of academic outcomes. Aims. The main goal of the study was to investigate the incremental validity of chronotype as a predictor of academic achievement after controlling for a number of...

This paper examines relationships between the Big Five personality factors, time management, and grade-point-average in 556 community colleges students. A path model controlling for vocabulary, gender, and demographic covariates demonstrated that time management mediates the relationship between conscientiousness and students' academic achievement...

This article describes an exploration of the distinction between typological and factorial latent variables in the domain of personality theory. Traditionally, many personality variables have been considered to be factorial in nature, even though there are examples of typological constructs dating back to Hippocrates. Recently, some reconceptualiza...

This comment responds to Maul's (2012) article evaluating the validity evidence and argument for the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) as a measure of emotional intelligence (EI). We suggest that Maul's standards for establishing validity evidence are unrealistically high, and may not be met by other established psychometric...

The current study investigated self-reported positive affect (PA) and negative affect (NA) in high school students (N = 451) within three academic contexts: homework, classwork/tests, and after-school activities. We examined whether context-specific emotions predicted grades, life satisfaction, and discipline records. Our findings revealed that con...

This research was conducted to examine whether people high in emotional intelligence (EI) have greater well-being than people low in EI. The Situational Test of Emotion Management, Scales of Psychological Well-being, and Day Reconstruction Method were completed by 131 college students. Responses to the Situational Test of Emotion Management were st...