Boris Forthmann

University of Münster | WWU · Institute of Psychology in Education

Creativity research commonly involves recruiting human raters to judge the originality of responses to divergent thinking tasks, such as the alternate uses task (AUT). These manual scoring practices have benefited the field, but they also have limitations, including labor-intensiveness and subjectivity, which can adversely impact the reliability an...

Human ratings are ubiquitous in creativity research. Yet the process of rating responses to creativity tasks—typically several hundred or thousands of responses, per rater—is often time consuming and expensive. Planned missing data designs, where raters only rate a subset of the total number of responses, have been recently proposed as one possible...

A situational judgment test (SJT) is a psychological instrument typically used to assess the suitability of applicants in personnel selection or development. Interest in SJTs has increased over the past decades as research has shown considerable validity of SJTs and various other benefits. Researchers often provide information about internal consis...

Our study examines individual differences in vacation‐related well‐being gains by investigating general work engagement and general well‐being as moderators. We examined the effect of vacation on employees' affective well‐being (negative activation and vigor) concerning three different vacation effects (change in affective well‐being over time): “v...

In response to pandemic-related learning gaps, educational policies have been put in place to help close those gaps. In this Think Piece, we complement existing analyses of the compensatory programs with a discussion of the role that student assessment plays both at the level of system monitoring and in guiding instructional decisions in the respec...

Even though a relationship between psychopathology and creativity has been postulated since the time of ancient Greece, systematic meta-analyses on this topic are still scarce. Thus, the metaanalysis described here can be considered the first to date that specifically focuses on the relationship between creative potential, as measured by divergent...

Chance models of scientific creative productivity allow estimation of researcher capacity. One prominent such model is the Q model in which the impact of a scholarly work is modeled as a multiplicative function of researcher capacity and a potential impact (i.e., luck) parameter. Previous work estimated researcher capacity based on an approximation...

The present study aimed to integrate evidence on the relationship among broad retrieval ability (Gr), processing speed (Gs), and divergent thinking (DT) with a three-level meta-analytic approach. The analysis was conducted on 560 effect sizes obtained from 47 studies with an overall sample of 10,391 participants. Results indicated moderate mean cor...

Creativity research often relies on human raters to judge the novelty of participants’ responses on open- ended tasks, such as the Alternate Uses Task (AUT). Albeit useful, manual ratings are subjective and labor intensive. To address these limitations, researchers increasingly use automatic scoring methods based on a natural language processing te...

Scoring divergent thinking tasks opens multiple avenues and possibilities – decisions researchers have to make. While some scholars postulate that scoring should focus on the best ideas provided, the measurement of the best responses (e.g., “top scoring”) comes along with challenges. More specifically, compared to the average quality across all res...

In education, among the most anticipated consequences of the COVID-19 pandemic are that student performance will stagnate or decline and that existing inequities will increase. Although some studies suggest a decline in student performance and widening learning gaps, the picture is less clear than expected. In this study, we add to the existing lit...

The present study aimed to integrate evidence on the relationship among divergent thinking (DT), broad retrieval ability (Gr), and processing speed (Gs) with a three-level meta-analytic approach. The analysis was conducted on 536 effect sizes obtained from 41 studies with an overall sample of 9055 participants. Results indicated moderate mean corre...

Human ratings are ubiquitous in creativity research. Yet the process of rating responses to creativity tasks—typically several hundred or thousands of responses, per rater—is often time consuming and expensive. Planned missing data designs, where raters only rate a subset of the total number of responses, have been recently proposed as one possible...

The equal odds baseline model of creative scientific productivity proposes that the number of high-quality works depends linearly on the number of total works. In addition, the equal odds baseline implies that the percentage of high-quality works and total works are uncorrelated. The tilted funnel hypothesis proposes that the linear regression impl...

To put creative ideas and insights into action, people need to overcome obstacles, monitor their processes, and effectively evaluate the steps they take. Across two studies (N = 832 and N = 843), we explored the structure, correlates, and cross-domain similarity and specificity of creative self-regulation. Both studies supported a seven-factor mode...

Reliable learning progress information is crucial for teachers’ interpretation and data-based decision making in everyday classrooms. Slope estimates obtained from simple regression modeling or more complex latent growth models are typically used in this context as indicators of learning progress. Research on progress monitoring has used mainly two...

The goal of the current study was to gain insight into what elements encompass business-as-usual (BAU) reading instruction and to what extent BAU reading instruction includes elements that have been found to positively impact reading competence. In addition, we examined whether and how these evidence-based elements are incorporated and how they clu...

Semantic distance scoring provides an attractive alternative to other scoring approaches for responses in creative thinking tasks. In addition, evidence in support of semantic distance scoring has increased over the last years. In one recent approach, it has been proposed to combine multiple semantic spaces to better balance the idiosyncratic influ...

This study explores long-term stability of creative self-concept variables, which have gained attention in the past decade, but lacked specific longitudinal investigation and strong analytical decisions. We conducted two higher-order confirmatory factor analyses based on latent state-trait theory to demonstrate the underlying latent structure of tw...

Automated scoring of divergent thinking tasks is a current hot topic in creativity research. Most of the debated approaches are unsupervised machine learning approaches and researchers seemingly just started to evaluate supervised approaches. Hence, rediscovering the seminal work of Paulus et al. (1970) came as a big surprise to us. More than fifty...

Traditionally, researchers employ human raters for scoring responses to creative thinking tasks. Apart from the associated costs this approach entails two potential risks. First, human raters can be subjective in their scoring behavior (inter-rater-variance). Second, individual raters are prone to inconsistent scoring patterns (intra-rater-variance...

Automated scoring of divergent thinking tasks is a current hot topic in creativity research. Most of the debated approaches are unsupervised machine learning approaches and researchers seemingly just started to evaluate supervised approaches. Hence, rediscovering the seminal work of Paulus et al. (1970) came as a big surprise to us. More than fifty...

Creativity research often relies on human raters to judge the novelty of participants’ responses on open-ended tasks, such as the Alternate Uses Task (AUT). Albeit useful, manual ratings are subjective and labor intensive. To address these limitations, researchers increasingly use automatic scoring methods based on a natural language processing tec...

The idea of data-based decision-making (DBDM) at the classroom level is that teachers use assessment data to adapt their instruction to students’ individual needs and thus improve students’ learning progress. In this study, we first investigate this theoretically assumed DBDM process, and second, we evaluate the effectiveness of teacher support on...

Although the behaviors displayed by assessees are the currency of assessment centers (ACs), they have remained largely unexplored. This is surprising because a better understanding of assessees’ behaviors may provide the missing link between research on the determinants of assessee performance and research on the validity of performance ratings. Th...

Aesthetics is essential to the design of products. Nevertheless, aesthetic quality is often assessed with inaccurate, ad-hoc scales. Therefore, we have developed the Product Aesthetics Inventory (PAI) and its short version the PAI-S. A pre-study using face-to-face interviews (N = 6 design experts, N = 4 product users) served as basis for the develo...

Star inventors generate superior innovation outcomes. Their capacity to invent high-quality patents might be decisive beyond mere productivity. However, the relationship between quantitative and qualitative dimensions has not been exhaustively investigated. The equal odds baseline (EOB) framework can explicitly model this relationship. This work co...

Although the behaviors displayed by assessees are considered to be the currency of assessment centers (ACs), they have remained largely unexplored. This is surprising because a better understanding of assessees’ behaviors may provide the missing link between research on the determinants of assessee performance and research on the validity of perfor...

Monitoring the progress of student learning is an important part of teachers’ data-based decision making. One such tool that can equip teachers with information about students’ learning progress throughout the school year and thus facilitate monitoring and instructional decision making is learning progress assessments. In practical contexts and res...

Learning progress assessments (LPA) are increasingly used by teachers to inform instructional decisions. This study presents evidence for the reliability, validity, and measurement invariance of a newly developed LPA for reading in Grade 2 (quop-L2 – quop Lesetest für zweite Klassen) that assesses the development of reading comprehension in German...

Background We examine the role of learning-family conflicts for the relation between commuting strain and health in a sample of medical university students. The first goal of the study was to investigate the mediating role of learning-family conflicts. The second goal was to extend the temporal view on relations between study variables. Therefore,...

Background. When students generate ideas, important inter-individual variance exists both in the quantity and the quality of ideas they are able to produce (e.g., perfectionists who have few highly creative ideas or mass producers who produce a lot of uncreative ideas). In educational psychology research on creativity, the relation between the quan...

Correlations are ubiquitous in scientometric research. The present work illustrates a formula to quantify the predicted correlation between a composite indicator and a primary indicator (i.e., the composite indicator can be expressed as a weighted sum of the primary indicator), for example. Total citations received and number of self-citations or t...

Semantic distance is increasingly used for automated scoring of originality on divergent thinking tasks, such as the Alternate Uses Task (AUT). Despite some psychometric support for semantic distance—including positive correlations with human creativity ratings—additional work is needed to optimize its reliability and validity, including identifyin...

Both researchers and practitioners agree that having highly engaged employees results in individuals and organizations reaping various positive consequences related to performance and absenteeism. However, available research syntheses date from the early years of this line of research, thus cover only a small fraction (under 10%) of the available s...

Social skills (e.g., persuading others, showing compassion, staying calm) are of key importance in work and education settings. Accordingly, the goal of many selection processes is to identify candidates who excel in desired skills. For this, high-fidelity simulations such as assessment centers (ACs) are regarded as ideal procedures because they ca...

This paper presents a meta-analysis of the links between intelligence test scores and creative achievement. A three-level meta-analysis of 117 correlation coefficients from 30 studies found a correlation of r = .16 (95% CI: .12, .19), closely mirroring previous meta-analytic findings. The estimated effects were stronger for overall creative achieve...

Creativity—as any other object of scientific endeavor—requires a sound measurement that adheres to quality criteria. For decades, creativity science has been criticized as falling short in developing valid and reliable measures of creative potential, activity, and achievement. Recent years have witnessed growth of theoretical and empirical works th...

Even though a relationship between psychopathology and creativity has been postulated since the time of ancient Greece, systematic meta-analyses on this topic are still scarce. Thus, the meta-analysis described here can be considered the first to date that specifically focuses on the relationship between creative potential, as measured by divergent...

In most general education classrooms in Germany, students with and without special educational needs are taught together. To support teachers in adapting instruction to these heterogeneous classrooms, we have developed learning progress assessment (LPA) and reading instructional materials, the Reading Sportsman (RS) in line with the theoretical fra...

Theoretischer Hintergrund Zu den von Politiker*innen, Lehrkräften und Forschenden am meisten befürchteten Folgen der COVID-19 Pandemie im Bildungsbereich gehört, dass die Lernleistungen stagnieren oder sinken und dass bestehende Ungleichheiten zunehmen (Forsa, 2020; Leopoldina, 2020). Diese Sorgen sind aus mehreren Gründen berechtigt, denn a) Sch...

This paper presents a meta-analysis of the links between intelligence test scores and creative achievement. A three-level meta-analysis of 117 correlation coefficients from 30 studies has found a correlation of r = .16 (95% CI: .12, .19), closely mirroring previous meta-analytic findings. The estimated effects were stronger for overall creative ach...

This paper provides a meta-analytic update on the relationship between intelligence and divergent thinking (DT), as research on this topic has increased, and methods have diversified since Kim’s meta-analysis in 2005. A three-level meta-analysis was used to analyze 875 correlation coefficients from 112 studies with an overall N = 33,897. The overal...

Social skills (e.g., persuading others, showing compassion, staying calm) are of key importance in work and education settings. Accordingly, the goal of many selection processes is to identify candidates who excel in desired skills. For this, high-fidelity simulations such as assessment centers (ACs) are regarded as ideal procedures because they ca...

Item-response models from the psychometric literature have been proposed for the estimation of researcher capacity. Canonical items that can be incorporated in such models to reflect researcher performance are count data (e.g., number of publications, number of citations). Count data can be modeled by Rasch's Poisson counts model that assumes equid...

Up to now, support for the idea that a controlled component exists in creative thought has mainly been supported by correlational studies; to further shed light on this issue, we employed an experimental approach. We used four alternate uses tasks that differed in instruction type (“be fluent” vs. “be creative”) and concurrent secondary workload (l...

Creativity—as any other object of scientific endeavor—requires a sound measurement that adheres to quality criteria. For decades, creativity science has been criticized as falling short in developing valid and reliable measures of creative potential, activity, and achievement. Recent years have witnessed growth of theoretical and empirical works th...

A thorough understanding of the relationship between quality and quantity of creative productions is critically important for creativity researchers and practitioners. The current study examines the equal odds baseline as a simple model to describe the quality-quantity relationship. Among other predictions, the equal odds baseline posits the presen...

Die zunehmende Heterogenität der SchülerInnen geht mit der Herausforderung für Lehrkräfte einher, die unterschiedlichen Lernausgangslagen zu diagnostizieren und die schulische Förderung entsprechend differenziert zu gestalten. Der vorliegende Beitrag stellt mit der Lernverlaufsdiagnostik quop sowie dem Förderprogramm 'Der Lese-Sportler' ein Konzept...

Quantifying the creative quality of scholarly work is a difficult challenge, and, unsurprisingly, empirical research in this area is scarce. This investigation builds on the theoretical distinction between impact (e.g., citation counts) and creative quality (e.g., originality) and extends recent work on using objective measures to assess the origin...

Among scientists who study scientific production, the relationship between the quantity of a scientist’s production and the quality of their work has long been a topic of empirical research and theoretical debate. One principal theoretical perspective on the quantity-quality relationship has been the equal odds baseline, which posits that a scienti...

The equal odds baseline is a parsimonious model that describes the relationship between quantity and quality of output in scientific creativity. Specifically, it is posited that quality is a linear function of quantity, and therefore strong positive correlations between these two variables are expected. Strong positive correlations also play a cruc...

Distractors might display discriminatory power with respect to the construct of interest (e.g., intelligence), which was shown in recent applications of nested logit models to the short-form of Raven's progressive matrices and other reasoning tests. In this vein, a simulation study was carried out to examine two effect size measures (i.e., a varian...

As measures of general language proficiency, C-tests are ubiquitous in language testing. Speeded C-tests are quite recent developments in the field and are deemed to be more discriminatory and provide more accurate diagnostic information than power C-tests especially with highability participants. Item response theory modeling of speeded C-tests ha...

The present study contributes to the further development of mandatory employment counseling practice by creating a new measure to describe differences in counselors’ use of discretionary power. The measure categorizes counselors’ acting and thinking into concrete counseling behavior as well as prioritizations of goals and topics for the counseling...

Simonton’s equal odds baseline assumes that the number of creative hits is a positive linear function of the number of attempts (i.e., products). It has importance for productivity of innovators and scientists, small-group brainstorming, and divergent thinking research. It has been proposed within a stochastic model for productions in the field of...

Background: One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that...

Prior research has reported less favorable attitudes toward and more violent crimes against ethnic out-group members in East (vs. West) Germany. We conducted two pre-registered lost letter studies in West versus East German cities (Study 1, N = 400) and in West versus East German rural areas (Study 2, N = 400). To investigate supportive behavior re...

Prior research has reported less favorable attitudes toward and more violent crimes against ethnic out-group members in East (vs. West) Germany. We conducted two pre-registered lost letter studies in West versus East German cities (Study 1, N = 400) and in West versus East German rural areas (Study 2, N = 400). To investigate supportive behavior re...

Background. The originality of divergent thinking production is one of the most critical indicators of creative potential. It is commonly scored using the statistical infrequency of responses relative to all responses provided in a given sample. Aims. Response frequency estimates vary in terms of measurement precision. This issue has been widely ov...

One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that do not depen...

Um kreatives Denken optimal zu fördern, müssen Produkte kreativer Denkprozesse bewertbar sein, damit man möglichst konkrete Hilfestellung geben kann. Dass eine solche Bewertung nicht möglich sei, ist ein bekanntes Klischee, welches durch mystifizierende Vorstellungen zu Kreativität im Allgemeinen genährt wird. Wo Kreativität entsteht bzw. gedacht w...

Count data naturally arise in several areas of cognitive ability testing, e.g., processing speed, memory, verbal fluency, and divergent thinking. Contemporary count data item response theory models, however, are not flexible enough, especially to account for overand underdispersion at the same time. For example, the Rasch Poisson counts model assum...

Divergent thinking (DT) ability (i.e., the ability to come up with creative ideas) is a complex cognitive construct that has been associated with several specific components of the Cattel-Horn-Carroll (CHC) model. In this study, we employed a nested latent variable approach to examine the specific role of mental speed (Gs) and general retrieval abi...

In the presented work, a shift of perspective with respect to the dimensionality of divergent thinking (DT) tasks is introduced moving from the question of multidimensionality across DT scores (i.e., fluency, flexibility, or originality) to the question of multidimensionality within one holistic score of DT performance (i.e., snapshot ratings of cr...

Divergent thinking tests are often used in creativity research as measures of creative potential. However, measurement approaches across studies vary to a great extent. One facet of divergent thinking measurement that contributes strongly to differences across studies is the scoring of participants’ responses. Most commonly, responses are scored fo...

In the presented work, a shift of perspective with respect to the dimensionality of divergent thinking tasks is introduced moving from the question of multidimensionality across divergent thinking scores to the question of multidimensionality across the scale of divergent thinking scores. We apply IRTree models to test if the same latent trait can...

The Remote Associates Test (RAT; Mednick, 1962; Mednick & Mednick, 1967) is a commonly employed test of creative convergent thinking. The RAT is scored with a dichotomous scoring, scoring correct answers as 1 and all other answers as 0. Based on recent research into the information processing underlying RAT performance, we argued that the dichotomo...