Article
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

The purpose of this meta-analysis is to examine the nature and magnitude of the relation between complex problem-solving skills (CPS) and intelligence, a topic that has been widely discussed and that has instigated a vast array of partially contradicting findings in the past. Theoretically, researchers have hypothesized the two constructs to be everything from completely separate to identical. Over the course of almost four decades, empirical studies yielded results in support of both arguments. Our meta-analysis of 47 studies containing 60 independent samples and a total sample size of 13,740 participants revealed a substantial correlation of CPS and intelligence with an average effect size of M(g) = .433. In addition, we investigated whether the operationalization of CPS and intelligence moderated this correlation. Whereas there were no significant correlation differences considering the operationalization of intelligence, the approach used to measure CPS moderated the correlation of CPS and intelligence. Especially the most recent approach towards the assessment of CPS yielded the strongest associations between the two constructs. Implications for existing theories and future research are discussed.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The two constructs intelligence and complex problem solving (CPS) are conceptually and empirically connected (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). Taking a theoretical stance, a high level of intelligence facilitates successful problem-solving behavior (Kröner, Plass, & Leutner, 2005;. ...
... However, research has shown that when intelligence and CPS are both reliably and validly measured, they correlate substantially. In their meta-analysis, Stadler et al. (2015) analyzed the intelligence-CPS-correlations extracted from 47 studies and found a weighted average correlation of r = .43, indicating that intelligence and CPS are closely related but not identical (Stadler et al., 2015). ...
... In their meta-analysis, Stadler et al. (2015) analyzed the intelligence-CPS-correlations extracted from 47 studies and found a weighted average correlation of r = .43, indicating that intelligence and CPS are closely related but not identical (Stadler et al., 2015). Descriptively reviewing correlations that were reported in studies that employed more recent CPS measures (that resolved psychometric weaknesses of earlier measures; Wüstenberg et al., 2012) and broader measures of intelligence in the sense of g instead of narrower measures such as (figural) reasoning, we observed that the coefficients tended to be even higher and ranged in the area of .69 ...
Article
Intelligence and complex problem solving (CPS) correlate closely, but little is known about the mechanism that translates intelligence into successful CPS. Therefore, this study considered the strategic exploration behaviors VOTAT (vary-one-thing-at-a-time) and NOTAT (vary no-thing-at-a-time) as possible mediators. A sample of 495 high-school students worked on nine CPS tasks, six of which with solely direct effects and three with direct and eigendynamic effects. We expected substantial mediation effects if the applied strategic behaviors were optimal to identify the particular underlying effect types (i.e., direct effects: VOTAT; direct and eigendynamic effects: VOTAT and NOTAT). The model for tasks with only direct effects revealed VOTAT and NOTAT to be substantial mediators: Whereas VOTAT showed substantial positive relations to intelligence and CPS performance, NOTAT unexpectedly showed substantial negative relations. Both VOTAT and NOTAT resulted in significant indirect mediation effects. The model for tasks with direct and eigendynamic effects showed substantial positive relations of VOTAT and NOTAT to intelligence and CPS-performance and resulted in significant and positive indirect mediation effects. Moreover, the indirect effects differed between VOTAT and NOTAT and across the two facets of CPS performance. Overall, strategic exploration behaviors are relevant for explaining the g-CPS-relation.
... However, strongest evidence for this assumption does not result from CPS correlations with other constructs beyond reasoning. Rather, it is primarily based on evidence for a less than perfect meta-analytically estimated correlation of reasoning and CPS (Stadler et al., 2015). Beyond that, numerous studies aimed at providing evidence for incremental value of CPS beyond established measures of cognitive abilities regarding while explaining other criteria including school grades or success in tertiary education (Kretzschmar, Neubert, & Greiff, 2014;Kretzschmar, Neubert, Wüstenberg, & Greiff, 2016;Lotz, Sparfeldt, & Greiff, 2016;Molnár, Greiff, & Csapó, 2013;Schweizer, Wüstenberg, & Greiff, 2013). ...
... During the past three decades, computer-based scenarios have been applied in numerous studies for CPS assessment (Dörner, Kreuzig, Reither, & Stäudel, 1983;Funke, 1993;Rigas & Brehmer, 1999;Stadler et al., 2015). Compared to classical tests, in CPS simulations data resulting from both exploration and control of dynamic systems may be analysed (Brehmer & Dörner, 1993;Funke, 2010), which enables the observation of "intelligence in action" (Lotz, Scherer, Greiff, & Sparfeldt, 2017). ...
... As stated above, among the main arguments was the less than perfect correlation between reasoning and CPS and incremental effects of CPS dimensions on GPA beyond reasoning. Stadler et al.'s (2015) claim of a non-equivalence of intelligence and CPS was based on a meta-analysis of 47 studies investigated measures of general intelligence and reasoning as correlates of CPS. This had resulted in a mean weighted Hedges' correlation M(g) between CPS and measures of intelligence of M(g) CPS -Intelligence = .43 ...
Article
Complex Problem Solving (CPS) and reasoning are considered as correlated, but not equivalent (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). However, empirical evidence for incremental validity of constructs beyond reasoning regarding CPS is scarce. This includes evidence for CPS self-concept (CPS SC; cf. Meißner, Greiff, Frischkorn, & Steinmayr, 2016). Strongest evidence would result from applying CPS scenarios as criteria that have been constructed to maximize correlations with conventional reasoning tests (e.g. MultiFlux, Kröner, 2001; Kröner, Plass, & Leutner, 2005; Zech, Bühner, Kröner, Heene, & Hilbert, 2017). Thus, we applied MultiFlux as a criterion to assess CPS performance of n = 240 university students and investigated incremental validity of CPS SC above and beyond reasoning according to a figural matrices test (DESIGMA, Becker et al., 2016; ω > .75 for all constructs). Results showed substantial unique effects of CPS SC on CPS above and beyond reasoning (β = .36; 16% incremental variance), both for a latent CPS factor and for all four CPS subdimensions. Despite a substantial amount of shared variance, reasoning and CPS proved to be distinct constructs and CPS SC showed a substantial unique effect beyond reasoning in explaining CPS. Avenues for exploring relevance of further constructs for explaining CPS variance are discussed.
... The question of how complex problem solving (CPS) skills are to be integrated into the nomological network of intellectual abilities was and still is one of the most examined questions in CPS research (e.g., Dörner and Funke 2017;Dörner et al. 1983;Funke and Frensch 2007;Kretzschmar et al. 2016;Süß 1996;Süß and Kretzschmar 2018). While most studies have investigated the association between (subconstructs of) intelligence and CPS (for an overview, see Stadler et al. 2015), there are only a few studies that have additionally considered working memory as a relevant cognitive ability to solve complex problems (for an overview, see Zech et al. 2017). ...
... Whereas there is a consensus regarding the high correlation between intelligence and CPS (see, e.g., Kretzschmar et al. 2016;Stadler et al. 2015), the results regarding the effect of working memory on CPS are inconsistent. For example, in the studies of Wittmann and Süß (1999) and Greiff et al. (2016), both working memory and (subconstructs of) intelligence significantly explained variance in CPS. ...
... The relation of working memory and intelligence (e.g., Ackerman et al. 2005;Oberauer et al. 2005) and of intelligence and CPS (e.g., Kretzschmar et al. 2016;Stadler et al. 2015) within the nomological network of cognitive abilities have stimulated a great amount of research. However, only a few studies have simultaneously considered all three constructs with inconsistent results. ...
Article
Full-text available
In order to investigate the nature of complex problem solving (CPS) within the nomological network of cognitive abilities, few studies have simultantiously considered working memory and intelligence, and results are inconsistent. The Brunswik symmetry principle was recently discussed as a possible explanation for the inconsistent findings because the operationalizations differed greatly between the studies. Following this assumption, 16 different combinations of operationalizations of working memory and fluid reasoning were examined in the present study (N=152). Based on structural equation modeling with single-indicator latent variables (i.e., corrected for measurement error), it was found that working memory incrementally explained CPS variance above and beyond fluid reasoning in only 2 of 16 conditions. However, according to the Brunswik symmetry principle, both conditions can be interpreted as an asymmetrical (unfair) comparison, in which working memory was artificially favored over fluid reasoning. We conclude that there is little evidence that working memory plays a unique role in solving complex problems independent of fluid reasoning. Furthermore, the impact of the Brunswik symmetry principle was clearly demonstrated as the explained variance in CPS varied between 4 and 31%, depending on which operationalizations of working memory and fluid reasoning were considered. We argue that future studies investigating the interplay of cognitive abilities will benefit if the Brunswik principle is taken into account.
... If the causes are interdependent the solution may be unique; if the causes are independent, the formulation of a solution for each cause may have to be considered. The solution proposal is typically a faster exercise than the diagnosis of the problem, which when done with quality, allows the preparation of the solution to be a more administrative act [23]. ...
... The formulation of the solution focuses substantively on the analysis of causes [23]. With the causes rigorously identified, the solution is easier to work with. ...
... Only after a clear understanding of the problem should the proposition of solutions begin. Furthermore, there are different ways to identify the possible causes of a problem [23,27,28]; however, they need to be validated with facts afterwards. ...
Article
Full-text available
Problem solving skills are increasingly important to be able to tackle the complex problems encountered in the business world. Nowadays is increasingly important to achieve sustainable development, focusing not only on economic profit but also on creating social value. It is widely agreed that the principles of scientific management can lead to more effective solutions for complex problems. Problems have to be looked at objectively, with methodology and intellectual integrity and modesty. Several techniques have been developed to help analyze the causes of the problem or formulate solutions. Although these business research techniques are important tools, they are presented as isolated measures. Pereira Problem Solving methodology presented provides guide to address business and management problems. It is an integrative and easy-to-use instrument that helps organizations adopt scientific management practices and will enhance the efficiency of the solutions encountered.
... The so-called microworlds deal with, for example, the job of a small-town mayor (Lohhausen; or a business manager of a factory (Tailorshop; Putz-Osterloh, 1981). Such microworlds featuring a high degree of complexity and interconnectivity can claim a high degree of face validity (e.g., Stadler et al., 2015). Nevertheless, these early assessment procedures faced severe difficulties in meeting the standards of satisfactory psychometric measurement, as has been mentioned before by different authors (e.g., Greiff & Funke, 2010;Kröner, 2001). ...
... The theoretically derived CPS dimensions knowledge acquisition and knowledge application were supported in confirmatory factor analyses (Greiff & Neubert, 2014). Additionally and regarding the relations with other important variables, evidence for convergent construct validity of MCS with intelligence (see Stadler et al., 2015) as well as incremental predictive validity of MCS in explaining school grades beyond, for example, figural reasoning (as a narrow indicator for intelligence) was found (e.g., . To conclude, MCS scenarios allow psychometrically convincing assessments of CPS. ...
... Whereas intelligence is usually assessed with static, well defined, and transparent problems, CPS is assessed with computer based simulated scenarios that lack several of these static elements and include instead dynamic components (see e.g., Süß & Kretzschmar, 2018). The debate, whether CPS constitutes a construct distinct from intelligence or whether CPS is conceptualized as part of or can be integrated in an extended construct of intelligence has not yet been resolved (e.g., Kretzschmar et al., 2016;Stadler et al., 2015;Stadler, Niepel, & Greiff, 2019). On a conceptual level, intelligence and CPS overlap substantially regarding, for example, the ability to solve problems, as mentioned (see Gottfredson, 1997). ...
Article
Although the relation between intelligence and complex problem solving (CPS) has been investigated repeatedly, the moderating effect of prior knowledge on this relation still remains an open question. The Elshout-Raaheim hypothesis (see Leutner, 2002) predicts a higher correlation between intelligence and CPS at a medium level of problem specific prior knowledge and lower correlations at higher as well as lower levels of prior knowledge, thus a curvilinear moderation. We investigated this hypothesis in a sample of N = 495 high school students by using general intelligence (g) and a minimal complex systems approach (MCS) of CPS. Strategic prior knowledge in the sense of the relative frequency of the vary-one-thing-at-a-time strategy (VOTAT; Tschirgi, 1980) increased across MCS tasks in the knowledge acquisition phase of CPS. With increasing prior knowledge, correlations followed the predicted inverted U-shaped pattern in the knowledge acquisition phase and the knowledge application phase of CPS, thus supporting the Elshout-Raaheim hypothesis. The moderating effect of strategic prior knowledge for the intelligence–CPS relation and its relevance are discussed.
... Alternatively, strategy games or complex simulations seem to be more suitable to assess general mental ability (GMA), reasoning, critical thinking, and decision-making. In line with this, scores from games that were designed to simulate complex problem solving usually correlate substantially with measures of GMA or reasoning (Stadler et al., 2015). ...
... This seems to be in contrast to more recent studies that found positive correlations between performance in complex problem solving simulations and cognitive ability. In a meta-analysis of research that used computer simulations to investigate complex problem solving, Stadler et al. (2015) found an (uncorrected) mean correlation of 0.43 between performance in these simulations and cognitive ability measures. However, for the older simulations from this domain, which are also more similar to the simulation game used in the present study, the mean correlation was only 0.34, whereas the mean correlation was even 0.59 for newer simulations, which were often also more reliable. ...
... The reason for this is that this specific GBA targeted complex problem solving, which can be defined as "the successful interaction with task environments that are dynamic (i.e., change as a function of the user's interventions and/or as a function of time) and in which some, if not all, of the environment's regularities, can only be revealed by successful exploration and integration of the information gained in that process" (Frensch & Funke, 1995, p. 14). Based on this definition, it has been argued that GMA-and also reasoning as a more specific ability-should contribute to performance in these simulation games (see Stadler et al., 2015). Even though previous research suggests that males and females do not differ concerning GMA (Ployhart & Holtz, 2008), recent meta-analytic research revealed a small difference favoring men over women for reasoning (Waschl & Burns, 2020). ...
Article
Gamified and game-based assessments (GBAs) are increasingly used for personnel selection but there are concerns that males and younger applicants have an advantage in these assessments. However, hardly any research has addressed whether sex and age are related to GBA performance. Similarly, the criterion-related validity of GBAs is also not sufficiently confirmed. Therefore, we analyzed archival data from a high-stakes setting in which applicants completed a computer-based simulation game targeting complex problem solving. The analyses confirmed expectations for the present simulation game of better performance for males than for females and for younger than for older applicants. However, the effect sizes were small. Furthermore, performance in the current simulation game correlated with job-related performance as measured in an assessment center.
... A natural question is what is the relationship between Gf and CPS? This question has been the subject of debate over the past several years [11][12][13][14][15][16][17]. In this paper, we demonstrate that there is a strong relationship between fluid ability (Gf) and complex problem solving (CPS) skill, both conceptually and empirically. ...
... Empirically, we examine the correlations between ratings of Gf and CPS importance for several hundred occupations in the U.S. Department of Labor's O*NET database. We believe that this analysis complements other approaches to the question of the relationship between CPS and Gf, which have examined the correlations between performance on CPS tests and performance on Gf tests (e.g., [15]). ...
... Beyond the German CPS tradition, there are categories of tasks referred to as microworlds, which date back to Pappert and the Logo language [63], game-based assessments [64][65][66], such as Space Fortress [67], complex dynamic control tasks [68]; even intelligent tutoring systems may be thought to fit into this category [69]. Dynamic Decision Making [70] and Systems Thinking have been described as constructs that "overlap greatly with CPS in their respective definitions" ( [15], p. 37). ...
Article
Full-text available
Complex problem solving (CPS) has emerged over the past several decades as an important construct in education and in the workforce. We examine the relationship between CPS and general fluid ability (Gf) both conceptually and empirically. A review of definitions of the two factors, prototypical tasks, and the information processing analyses of performance on those tasks suggest considerable conceptual overlap. We review three definitions of CPS: a general definition emerging from the human problem solving literature; a more specialized definition from the "German School" emphasizing performance in many-variable microworlds, with high domain-knowledge requirements; and a third definition based on performance in Minimal Complex Systems (MCS), with fewer variables and reduced knowledge requirements. We find a correlation of 0.86 between expert ratings of the importance of CPS and Gf across 691 occupations in the O*NET database. We find evidence that employers value both Gf and CPS skills, but CPS skills more highly, even after controlling for the importance of domain knowledge. We suggest that this may be due to CPS requiring not just cognitive ability but additionally skill in applying that ability in domains. We suggest that a fruitful future direction is to explore the importance of domain knowledge in CPS.
... In the 2000s, there was a belief that systems thinking can be an answer to complex systems problems [11][12][13], and there is convergence around their definitions [14,15], This belief was translated later into action, where some studies appeared to show the significance of systems thinking in the domain of complex systems and recruiting employees [16]. However, what remained unanswered is the relationship between an individual's systems thinking (ST) and his/her general perception of different stages in the CPS process-that is a current gap in the literature. ...
... Numerous studies have linked complex systems and issues to systems thinking (ST) (e.g., [1,16,[62][63][64][65]). Several researchers [14,15] stated that the definitions of CPS and ST have some overlap. Funke [14] stated that five attributes distinguish complex problems from simple problems, which include (1) the complexity of the problem situation, (2) the relationships between the variables involved, (3) the dynamics of the situation and developments within the system, and the role of time, (4) partial or complete lack of transparency, and (5) polytely (a Greek term for "many goals") and the possibility of conflict in the existence of several goals. ...
Article
Full-text available
The ability to solve modern complex systems becomes a necessity of the 21st century. The purpose of this study is the development of an instrument that measures an individual’s perception toward solving complex problems. Based on literature and definitions, an instrument with four stages named perceived complex problem-solving (PCPS) was designed through exploratory and confirmatory stages. The instrument is validated and scaled through different models, and the final model is discussed. After completing validation and scale development of the PCPS instrument, the final model of the PCPS instrument was introduced to resolve the gap in the literature. The final model of the PCPS instrument is able to find and quantify the degree of perception an individual holds in dealing with complex problems and can be utilized in different settings and environments. Further research about the relationship between Systems Thinking and CPS revealed individuals with a high level of systems thinking have a better understanding of the characteristics of complex problems and so better perception of CPS.
... A recent meta-analysis [16], which included 60 studies from 1982 to 2014, showed that the correlations between successful complex problem-solving and intelligence varied from 0.339 to 0.585, depending on the type of the complex problem. The weakest correlation (0.339) was obtained for "classical" complex problem tasks (LOHHAUSEN, MORO, TAILORSHOP, etc.,) [6,13], while the strongest (0.585)-for minimal complex systems tasks (MicroDYN, MicroFIN, etc.,) [16]. ...
... A recent meta-analysis [16], which included 60 studies from 1982 to 2014, showed that the correlations between successful complex problem-solving and intelligence varied from 0.339 to 0.585, depending on the type of the complex problem. The weakest correlation (0.339) was obtained for "classical" complex problem tasks (LOHHAUSEN, MORO, TAILORSHOP, etc.,) [6,13], while the strongest (0.585)-for minimal complex systems tasks (MicroDYN, MicroFIN, etc.,) [16]. ...
Article
Full-text available
Current studies of complex problem-solving do not commonly evaluate the regulatory role of such personality-based variables as tolerance for uncertainty, risk-readiness, and patterns for coping with decisional conflict. This research aims to establish the contribution of those traits into individual parameters of complex problem-solving strategies. The study was conducted on 53 healthy individuals 17 to 29 years old (M = 20.42; SD = 2.34). Our own computerized complex problem task “The Anthill” was developed for this research. We identified five measurable parameters of the participants’ problem-solving strategies: preferred orientational level (POL); orientational level variability (OLV); class quotas‘ range (R); mean and median quotas shift (MS and MeS); and abrupt changes of strategy (AC). Psychodiagnostic methods included: new questionnaire of tolerance/intolerance for uncertainty; personal decision-making factors questionnaire; Melbourne Decision Making Questionnaire; Subjective Risk Intelligence Scale; Eysencks’ Impulsiveness Scale. The study showed the role of tolerance for uncertainty, risk-readiness, negative attitude toward uncertainty, and decision-making styles in the regulation of complex problem-solving strategies. Specifically, procrastination, tolerance for uncertainty, and risk-readiness were significant predictors of individual strategy indicators, such as POL, OLV, and MeS. Thus, personality traits were shown to regulate resource allocation strategies and the required level of orientation in a complex problem.
... Cognitive developmental priorities of successive developmental cycles constrain the situations presenting problems to be solved and the type of solutions that may be attempted (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). Mastering interaction with objects is a major source of problems in infancy. ...
... (Frensch & Funke, 1995, p. 14). So defined, problem solving is moderately related with the general intelligence (.43; Stadler et al., 2015). At this phase of development, problem solving requires of critical thinking: ...
Preprint
Full-text available
This paper summarizes a theory of cognitive development and discusses its educational implications. The paper first outlines a set of principles that might allow tuning developmental priorities with educational priorities. It postulates, in contrast to several classic developmental theories, that developmental priorities change with development. It outlines the cognitive profile of four successive developmental cycles and presents evidence showing that developmental priorities change from interaction control in infancy to representational control in preschool to inferential control in primary school to logical truth control in adolescence. Studies are then summarized showing that the cognitive priorities of each cycle are the best predictors of school achievement in this or later cycles. Finally, we outline developmental changes in general problem-solving skills and show that learning in different domains, such as language and mathematics, depends on an interaction between the general cognitive processes dominating in each cycle and the state of the symbol systems associated with this domain. If command of any of these systems is deficient, specific learning deficiencies may emerge, as in dyslexia and dyscalculia. Principles for ameliorating these conditions are outlined.
... Cognitive developmental priorities of successive developmental cycles constrain the situations presenting problems that may be solved and the solutions possible (Stadler et al., 2015). The respective column in Table 1 summarizes changes in problemsolving possibilities with development. ...
... Adolescents must understand that problem solving is a dynamic interaction with task situations which change as a function of the own interventions aiming to reveal their regularities emerging from ongoing explorations and their integration into rules and solutions (Frensch and Funke, 1995). Expectedly, problem solving is moderately related with general intelligence (Stadler et al., 2015) Frontiers in Psychology 11 frontiersin.org and critical thinking. ...
Article
Full-text available
This paper summarizes a theory of cognitive development and elaborates on its educational implications. The theory postulates that development occurs in cycles along multiple fronts. Cognitive competence in each cycle comprises a different profile of executive, inferential, and awareness processes, reflecting changes in developmental priorities in each cycle. Changes reflect varying needs in representing, understanding, and interacting with the world. Interaction control dominates episodic representation in infancy; attention control and perceptual awareness dominate in realistic representations in preschool; inferential control and awareness dominate rule-based representation in primary school; truth and validity control and precise self-evaluation dominate in principle-based thought in adolescence. We demonstrate that the best predictors of school learning in each cycle are the cycle’s cognitive priorities. Also learning in different domains, e.g., language and mathematics, depends on an interaction between the general cognitive processes dominating in each cycle and the state of the representational systems associated with each domain. When a representational system is deficient, specific learning difficulties may emerge, e.g., dyslexia and dyscalculia. We also discuss the educational implications for evaluation and learning at school.
... Three-Level Meta-Analysis The second example was based on the data set from Stadler, Becker, Gödker, Leutner, and Greiff (2015), Table 1). These authors investigated the correlation 601 122 114 113 93 85 84 83 82 81 80 79 78 71 59 56 54 52 44 38 28 25 24 19 17 14 11 10 6 between complex problem solving and intelligence. ...
... In their Table 1, however, they did not provide explicit information on how these independent samples were nested. Stadler et al. (2015) conducted their meta-analysis without taking the non-independence of the effect sizes into account. Based on the information on "Authors" and "Year," we could only identify 44 clusters. ...
Article
Full-text available
Conventional meta-analytic procedures assume that effect sizes are independent. When effect sizes are not independent, conclusions based on these conventional procedures can be misleading or even wrong. Traditional approaches, such as averaging the effect sizes and selecting one effect size per study, are usually used to avoid the dependence of the effect sizes. These ad-hoc approaches, however, may lead to missed opportunities to utilize all available data to address the relevant research questions. Both multivariate meta-analysis and three-level meta-analysis have been proposed to handle non-independent effect sizes. This paper gives a brief introduction to these new techniques for applied researchers. The first objective is to highlight the benefits of using these methods to address non-independent effect sizes. The second objective is to illustrate how to apply these techniques with real data in R and Mplus. Researchers may modify the sample R and Mplus code to fit their data.
... In most conventional measures of cognitive ability, complex problem solving is one of the skills being tested. It is thus logical that (complex) problem solving is associated with cognitive ability (Greiff & Neubert, 2014;Lotz et al., 2016;Stadler et al., 2015). ...
... For example, a recent study showed that problem solving is not correlated with academic achievement once cognitive ability is accounted for, except in mathematics (Lotz et al., 2016). However, the extent to which problem solving is associated with academic performance after accounting for cognitive ability depends on the definition and measurement of cognitive ability and problem solving (Greiff & Neubert, 2014;Stadler et al., 2015). ...
... Therefore, it can be concluded that learners with a good level of Typical Intellectual Engagement are good Reflectors as well. Stadler, Becker, Gödker, Leutner, and Greiff (2015) hold ...
... Results indicated the relationship between Pragmatist and Typical Intellectual Engagement which was moderately significant (r=.30, p<.05). Such a positive relationship gained in this study was in agreement with Stadler et al. (2015) who came to the conclusion that intelligence is a strong predictor of complex problem solving. So, it can be inferred that learners with a total typical Intellectual Engagement tend to be more Pragmatists and interested in problem solving tasks. ...
Article
Full-text available
One of the most significant issues in the process of teaching and learning is to consider students' individual differences. The aim of this study was to investigate the interrelationships among Iranian EFL Learners' Typical Intellectual Engagement, Learning Style and Preference for Assessment Methods. The sample included 200 English learners majoring in TEFL (Teaching English as a Foreign Language) from 3 different universities in Mashhad, Iran. To this end, three questionnaires; Typical Intellectual Engagement questionnaire, learning style inventory, and preference for assessment methods questionnaire were administered. The interrelationships among all variables were examined using path analysis. The model was fitted after some modifications. Findings revealed that Typical Intellectual Engagement was a positive and significant predictor of all four sub-factors of EFL learners' learning styles except Activist. Also, all four sub-factors of EFL learners' learning styles except pragmatist were significant predictors of preference for assessment methods. Finally, the fit model showed that Typical Intellectual Engagement was a positive significant predictor of preferred assessment methods (r = .24, .22, p < .05). This study has several implications for teachers, learners, and syllabus designers.
... In search of reasons for this heterogeneity, and therefore in search of the real relationship between measures of intelligence and measures of CPS, many hypotheses were formulated and tested. Extensive research has generally indicated a positive relationship between intelligence and performance in dealing with complex problems (e.g., Beckmann, 1994;Kersting, 1999;Schoppek, 1996;Stadler, Becker, Gödker, Leutner, & Greiff, 2015;Süß, 1996Süß, , 1999Süß, , 2001, while different interpretations of the relationship would seem plausible. Particularly, it could be assumed that reasoning has an essential role for the processes of gathering information about variables and thereby understanding complex problem situations. ...
Article
Full-text available
Complex Problem Solving (CPS) can be defined as those psychological processes that enable a person to achieve goals under complex conditions, which are characterized by their complexity, connectivity, dynamics, lack of transparency, and polytely. Although many hypothesized influences have previously been tested concerning their relevance for the process of solving complex problems (e.g., general intelligence), results were often found to be rather heterogeneous. As this was found to be partially caused by fundamental differences between measurements of CPS, a new operationalization was used in the present study: Following the Microworld approach, CPS was assessed in the simulation game Cities: Skylines, as its aptitude as a Microworld could be justified from a theoretical perspective. A parameter for CPS performance was defined to investigate the following hypotheses: Both reasoning and achievement motivation were expected to be positively correlated to CPS performance. Furthermore, a gender difference favoring male participants was expected. Participants in the present study (N = 27) first provided demographic information, then subsequently completed a short test of reasoning and an objective personality test of achievement motivation, and finally were given a mission in a complex scenario implemented in Cities: Skylines. The results supported all three hypotheses, indicating significant small to moderate positive relationships of both reasoning and achievement motivation with CPS performance, and a significant gender difference favoring male participants in CPS performance. Furthermore, significant gender differences favoring males were also found for reasoning and achievement motivation. Results are discussed and the Microworld operationalization is evaluated.
... Intelligence has repeatedly been shown to impact almost all aspects of daily life and was able to predict a broad array of successful human behaviors, performances, and outcomes, including increased creativity, elevated potential, improved health, better job performance, higher income, and longer employment (e.g., Fergusson, Horwood, & Ridder, 2005;Kuncel, Hezlett, & Ones, 2004;Strenze, 2007). Intelligence is further linked to an ability to solve complex problems, and is especially valuable regarding complex tasks, complex information processing, or complex situations (e.g., Gottfredson, 1997;Kuncel et al., 2004;Roth et al., 2015;Stadler, Becker, Gödker, Leutner, & Greiff, 2015). Furthermore, many studies found intelligence to be strongly and positively correlated to long-term learning and academic achievement, and it is often cited as one of the strongest predictors of long-term learning and academic achievement. ...
Article
Intelligence is an important predictor of long-term learning and academic achievement. In two studies we focused on the relation among intelligence, desirable difficulties–active generation/production of information and taking tests–, and long-term learning. We hypothesized that intelligence is positively correlated to long-term learning and that difficult learning situations, as opposed to easier reading, increase later long-term learning. We further assumed that the beneficial effects of difficult learning would be moderated by intelligence, thus, we supposed the positive effects to be stronger for learners with higher intelligence and weaker for learners with lower intelligence. We in turn conducted two experiments (N1 = 149, N2 = 176, respectively), measured participants' intelligence, applied desirable difficulties–generation/testing–in contrast to control tasks, and later assessed long-term learning indicated by delayed final test performance. Both studies showed positive correlations between intelligence and later long-term learning. Study 2 further found the expected beneficial effect of difficult learning, which was also moderated by intelligence. There was no difference between difficult tasks and control tasks for participants with relatively low intelligence. Retrieving answers in learning tests was, however, beneficial for participants with average intelligence and even more beneficial for participants with higher intelligence. In general, our two experiments highlight the importance of intelligence for complex and challenging learning tasks that are supposed to stimulate deeper encoding and more cognitive processing. Thus, specifically learners with higher, or at least average, intelligence should be confronted with difficulties to increase long-term learning and test performance.
... One final point is worth making about potential contributions to a science of practical wisdom, which is that the concept clearly overlaps with other more traditional foci of psychological research such as judgment and problem-solving, and it may be valuable to mine these literatures to enhance the understanding of phronesis. For example, decision-making competence (Fischhoff, 2010) and complex problem-solving skills (Stadler et al., 2015) have both been found to correlate about 0.50 with measures of cognitive ability, a substantial relationship. At the same time, Fischhoff reported competence was also associated with higher socioeconomic status, absence of paternal substance use, and a more positive peer environment even after controlling for cognitive variables, suggesting better environmental circumstances can contribute to better decisionmaking skills (also see Odom, 1967). ...
Article
Full-text available
The VIA Classification of Character Strengths and Virtue has received substantial attention since its inception as a model of 24 dimensions of positive human functioning, but less so as a potential contributor to a psychological science on the nature of virtue. The current paper presents an overview of how this classification could serve to advance the science of virtue. Specifically, we summarize previous research on the dimensional versus categorical characterization of virtue, and on the identification of cardinal virtues. We give particular attention to the three-dimensional model of cardinal virtues that includes moral, self-regulatory, and intellectual domains. We also discuss the possibility that these three clusters be treated as fundamental elements of a virtue model, meaning that they clearly and directly contribute to both individual and communal flourishing across various cultures. This discussion includes a summary of previous speculations about the evolution of adaptations underlying the human capacity for using behavioral repertoires associated with the three virtues, as well as discussing ways in which they simultaneously enhance community and individual, in the last case focusing particularly on evidence concerning mating potential. We then discuss the relationship between the evolutionary perspective on virtues and Aristotle’s concept of the reciprocity of the virtues. Finally, we provide speculations about the nature of practical wisdom. While accepting the potential value of future revisions to the VIA model, that model even under its current conditions has the potential to generate a number of intriguing and testable hypotheses about the nature of virtue.
... La prima posizione è nota come redundancy perspective -complex problem solving as intelligence (Kröner et al., 2005;Süß, 1996), mentre la seconda è nota come distinctiveness perspective -complex problem solving as a separate construct (Greiff note e punti di vista Scuolademocratica 1/2019 Wüstenberg et al., 2012;Spering et al., 2005;Barth e Funke, 2010;Hermes e Stelling, 2016). Naturalmente anche se dirimere la questione non può rientrare tra gli obiettivi del progetto TECO, resta il fatto che gli studi che l'hanno affrontata mostrano una correlazione significativa tra la capacità di CPS e l'intelligenza, che tuttavia sembra dipendere dalle modalità adottate per operazionalizzare il CPS e che non necessariamente implica che siano riferibili a un unico costrutto (Stadler et al., 2015). Inoltre, i risultati della somministrazione di problemi complessi tendono ad avere capacità predittiva come minimo sovrapponibile (redundancy perspective), e forse un po' migliore (distinctiveness perspective), rispetto ai risultati dei test di intelligenza classici (Greiff et al., 2013;Wüstenberg et al., 2012). ...
... Finally, as hypothesized, controlling GPA for general intelligence reduced the relation between the log data and GPA substantially. Since previous research consistently reported substantial overlap between CPS ability and general intelligence, this provides additional support for the validity of the assumption that log data actually reflects CPS ability (Kretzschmar et al., 2016;Stadler et al., 2015). The relation between general intelligence and GPA (r = .57) ...
Article
Analyzing test-taking behavior allows researchers to investigate the steps and actions resulting in the specific test outcome. The underlying assumption is that test-taking behavior is a valid indicator of the tested ability. The aim of this paper is to scrutinize this assumption in the context of complex problem solving (CPS) by analyzing individual differences in test-taking behavior and their relation to individual differences in established correlates of CPS ability. We investigated a sample of Finnish students who achieved the maximum score on five CPS tasks and worked on an additional intelligence measure. We logged the number of interactions with the CPS tasks and time-on-task. Students showed significant variance in both time-on-task (s² = 260.09, p = .005) and the number of interactions (s² = 0.381, p < .001) despite having no variance in their CPS test scores. Using structural equation modeling, both time-on-task (β = 0.17, p = .015) and the number of interactions (β = −0.17, p < .001) significantly predicted students' GPA, which has frequently shown to be associated with CPS ability. When adding intelligence – another established correlate of CPS – as predictor of GPA (β = 0.55, p < .001), the relation both between time-on-task (β = −0.09, p = .053) and the number of interactions (β = 0.09, p = .100) and students’ GPA was reduced to negligible effects. Taken together, our study supports the assumption of test-taking behavior actually representing differences in CPS ability – over and above test scores.
... On an empirical level, research over the last decades has shown CPS to be related to other cognitive performance measures, including reasoning, working memory, and school achievement (e.g., Kretzschmar et al. 2016;Stadler et al. 2015). Some personality aspects also exhibit relations with CPS, albeit weakly (for instance, with the Big 5; Greiff and Neubert 2014). ...
Article
Complex problem solving (CPS) is an up-and-coming twenty-first century skill that requires test-takers to solve dynamically changing problems, often assessed using computer-based tests. The log data that users produce when interacting with a computer-based test provide valuable information about each individual behavioral action they undertake, but such data are rather difficult to handle from a statistical point of view. This paper addresses this issue by building upon recent research focused on decoding log data and aims to identify homogeneous student profiles with regard to their ability to solve CPS tasks. Therefore, we estimated a discrete two-tier item response theory model, which allowed us to profile units (i.e., students) while taking into account the multidimensionality of the data and the explanatory effect of individual characteristics. The results indicate that: (1) CPS can be thought of as a three-dimensional latent variable; (2) there are ten latent classes of students with homogenous profiles regarding the CPS dimensions; (3) students in the higher latent classes generally demonstrate higher cognitive and non-cognitive performances; (4) some of the latent classes seem to profit from learning-by-doing within tasks, whereas others seem to exhibit the reverse behavior; (5) cognitive and non-cognitive skills, as well as gender and to some extent age, contribute to distinguishing among the latent classes.
... It involves aspects of complex cognition and is usually decomposed into two dimensions: knowledge acquisition and knowledge application (Novick & Bassok, 2005). As a broad and general cognitive ability, CPS is related to and yet distinct from reasoning and other measures of intelligence (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). It is an important predictor of external criteria such as academic achievement (Greiff et al., 2013) or supervisory ratings (Danner, Hagemann, Schankin, Hager, & Funke, 2011). ...
Article
Full-text available
We explored the behavioral mechanisms underlying the link between extraversion and cognitive ability. We focused on reasoning and complex problem solving as cognitive abilities, and response latencies and action orientation during test taking as behavioral mechanisms indicating working style. Results obtained in a sample of 326 adults generally indicated that specific working styles such as slower response latencies and higher action orientation mediated the link between extraver-sion and complex problem solving but not reasoning. We discuss the results in light of Eysenck's cortical arousal theory as well as more recent findings and suggest directions for future research.
... In studies comparing violent and non-violent offenders, violent offenders were found to have significantly lower 38,39 IQ scores than non-violent offenders . Some of the reasons why intelligence is thought to be negatively associated with violence include the fact that those with low intelligence may have weaker 40 skills in dealing with complex problem solving , which makes it difficult to choose non-violent options in a complex encounter. Other studies also reported that individuals with low intelligence are less able to understand the inner workings of other 41,42 persons which is associated with empathy hence 42 their violent acts . ...
... The Future of Jobs -Employment, Skills and Workforce Strategy for the Fourth Industrial Revolution (problem solving, which is highly correlated with general mental ability (Stadler, Becker, Gödker, Leutner, & Greiff, 2015); The Future of Skills: employment in 2030 (problem solving); Key Skills for the 21st Century: an evidence-based review (problem solving). ...
Article
Full-text available
An evidence-based research to find out which skills and abilities will allow knowledge workers to thrive in the age of automation.
... So konnte etwa gezeigt werden, dass die Kompetenz zur Lösung komplexer Probleme -obwohl deutlich mit Intelligenz korreliert (vgl. die Metanalyse von Stadler et al. 2015) -dennoch aber inkrementelle Validität bei der Vorhersage schulischer Leistungen besitzt, wenn Intelligenz als schlussfolgerndes Denken operationalisiert wird ( Kretzschmar et al. 2016). Dabei wird von Dörner und Funke (2017) allerdings kritisch hinterfragt, inwieweit die in solchen Studien verwendeten psychometrischen Tests (bestehend aus Items, die nur minimal komplexe Systeme repräsentieren; Funke und Greiff 2017) tatsächlich noch das erfassen, was ursprünglich mit komplexem Problemlösen gemeint war. ...
Article
Focusing on Germany, this article presents results from the international comparison of fifteen-year-olds in collaborative problem solving and a cross validation of the scaling in the Programme for International Student Assessment (PISA) 2015. A new computer-based test was used requesting students to solve a problem jointly with simulated group members. Data from collaborative problem solving of fifteen-year-olds (n = 124,994) in 51 countries were assessed. The German mean competence level (525 points) is a quarter standard deviation above the OECD average (500 points) and a quarter standard deviation below the OECD’s top performing country Japan (552 points). In all participating countries, girls outperform boys. While the percentage of top-performing students in Germany is comparable to proportions in the best-performing OECD countries, 21% of the students in Germany only reach competence level I or below, twice as many as in Japan. National results are presented as well as empirical evidence on the quality of the test, which is critically discussed.
... Teachers in the 21st century are required not only to be able to teach and manage classroom activities effectively but also to be creative in making learning more interesting, developing variations in learning by elaborating learning using technology to support improving the quality of teaching, and reflecting and evaluating their learning practices continuously [8]. Increasing students' CPS abilities can be obtained by applying a learning model that guides in solving problems [9]. The CPS of students can be skilled through computer-based training and evaluation, which requires completion steps quickly and precisely [10]. ...
Article
Full-text available
Case-Based Games Learning (CBGL) is a strategy of active learning in which the students implement knowledge and analytical skill related to the real complex situation (contextual) and relevant with subject lessons. Give case-based task or question routinely can help students to solve case-based mathematics problems. This study aims to analyze the effectiveness of CBGL using Quizizz dan Case Base Learning (CBL) without Quizizz based on students’ conceptual understanding in mathematics by finding out the percentage of the correct answer. Students were grouping by using a purposive sampling method based on the conceptual understanding indicator. The result of this study is the average conceptual understanding CBGL using Quizizz application strategy is more effective than CBL without Quizizz application in evaluating student work results quickly, precisely, and accurately. Therefore the use of Quizzes application with CBGL was more effective than CBL in assessing the work result quickly, right, and accurately.
... For instance, when deciding whether to purchase life insurance, an individual may consider the price of the insurance, the subjective probability of getting a disease, objective chances of disease occurrence, recommendations from other people, subjective risk preferences, etc. Intuitively, superior decisions would result from logical reasoning about decision problems (e.g., their outcomes and probabilities). Indeed, numerous studies have demonstrated that intelligence is an important predictor of performance in various problem-solving tasks (Goldstein, Princiotta, & Naglieri, 2015;Stadler, Becker, Gödker, Leutner, & Greiff, 2015). However, recent research has also suggested that in a class of problems in which outcomes are uncertain and probabilistic, numeracy (i.e., the ability to understand statistical and probability information and to use it in everyday contexts) can outperform other cognitive abilities in predicting decision making. ...
Article
Full-text available
The goal of the present study was to compare the relative contribution of different cognitive abilities and preferences to superior decision making. Additionally, we aimed to test whether skilled decision makers have better and more sophisticated long-term memory representations of personally meaningful risky situations. A large sample from the general population completed a series of tasks and questionnaires measuring cognitive abilities and preferences (fluid intelligence, cognitive reflection, and multiple numeric competencies: statistical numeracy, subjective numeracy, approximate numeracy) and decision making outcomes (a set of monetary lotteries and a self-report inventory measuring success in avoiding negative decision outcomes in real-life). We also designed a memory task in which participants were instructed to discriminate between decision outcomes presented in the first stage of the study and distractors. We found that multiple numeric competencies predicted decision making beyond fluid intelligence and cognitive reflection. Especially, the acuity of symbolic-number mapping (a measure of approximate numeracy) was the most robust single predictor of superior decision making. Moreover, a combination of different cognitive abilities contributed to a better understanding of decision outcomes. For example, superior decision making in monetary lotteries was best predicted by approximate numeracy, statistical numeracy, and fluid intelligence, while avoiding negative decision outcomes in real-life was best predicted by approximate and subjective numeracy. Finally, we demonstrated that people with high approximate numeracy had better memory for decision outcomes and produced more vivid mental representations, suggesting that memory processes can be crucial to superior decision making.
... Teachers in the 21st century are required not only to be able to teach and manage classroom activities effectively but also to be creative in making learning more interesting, developing variations in learning by elaborating learning using technology to support improving the quality of teaching, and reflecting and evaluating their learning practices continuously [8]. Increasing students' CPS abilities can be obtained by applying a learning model that guides in solving problems [9]. The CPS of students can be skilled through computer-based training and evaluation, which requires completion steps quickly and precisely [10]. ...
Article
Full-text available
One of the competencies needed currently is non-routine competency in solving a complex problem. Students with complex problem solving will be able to identify the complex problem, understand to review related information, and create solutions for the problems. However, the improvement of complex problem-solving abilities lacks attention. This phenomenon occurs when students are given problem-solving, students have not been able to understand the problem, plan solutions, and provide solutions to problems. To overcome these problems, the LAPS-Talk-Ball learning model integrated with interactive games that guide the students in solving problems was implemented by using interactive games and speech balloons to stimulate each student to express their ideas. This research is quantitative experimental research to determine the success of the LAPS-Talk-Ball learning model integrated with android-based interactive games in training complex problem-solving students using saturated sampling and non-equivalent control group research designs. From the results of gain test results high improvement category, which means that the LAPS-Talk-Ball learning model integrated with android-based interactive games can train the complex problem-solving abilities of students.
... It is an individual skill that has gained importance in educational and psychological research, since it requires skills beyond routine reasoning (Scherer, Greiff, & Hautamäki, 2015). Further, complex problem solving tasks demand the performance of more complex mental processes than intelligence tests do, such as the active interaction with the problem to acquire knowledge about the problem environment (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). Overall, complex problem solving takes place for reducing the barrier between a given start state and an intended goal state with the help of cognitive activities and behavior (Funke, 2012). ...
Article
Full-text available
The present study aimed to determine the effect of vodcast utilization in the enhancement the skills of Canadian smocking and its role in complex problem solving. The sample included (120) learners who were in their fourth academic level in the department of home economics. They were randomly divided into experimental and control group, with 60 learners in each group. A product assessment card and 7 Micro-DYN test battery tasks were used as main study instruments. Results proved showed that performance of students who learned through video lectures of handmade embroidery or vodcasts was better than the performance of colleagues in the control group with regard to the development of Canadian smocking skills and complex problem solving and mainly knowledge acquisition and application. The results provide significant proof of the vodcast advantages in educational technology studies and development.
... A number of explanations exist for the inconsistency of the findings on the relation between CPS and intelligence. Stadler et al. (2015) concluded that the zero-correlation found in early studies could be explained by using broader measures of intelligence that include a variety of different cognitive tasks (e.g., factual knowledge). Since more recent studies use more specific latent factors of intelligence that are conceptually closer to the construct of CPS (e.g., fluid intelligence), this relationship was found to be stronger. ...
Article
Full-text available
The main aim of this study was to (a) test the construct validity of complex problem solving (CPS); (b) examine the ability to acquire knowledge as a mediator of the relationship between intelligence and CPS performance; and (c) investigate the personal need for structure as a moderator of the relationship between intelligence and knowledge acquisition. A total of 128 participants completed the self-report Personal Need for Structure scale; the Vienna Matrix Test to assess intelligence; and a new multiple complex systems approach method to assess CPS skills. When analyzing the internal structure of CPS, we found that a two-dimensional model consisting of knowledge acquisition and knowledge application best fitted the data. We also found that the relationship between intelligence and CPS performance was partially mediated by the ability to acquire knowledge. Finally, personal need for structure did not moderate the relationship between intelligence and the ability to acquire knowledge. Our results indicate a need to further investigate other cognitive abilities in interaction with contextual situational factors that could additionally explain variance in CPS performance. Moreover, we also highlight the importance of deeper observation of the knowledge application phase of CPS process.
... CPS tasks having conceptual links with intelligence and decades of successful application in training and education (Wood et al. 2009). However, they are often discounted as intelligence measures because of the challenge in extracting psychometrically reliable and valid performance indicators that correlate sufficiently with static tests of intelligence (Beckmann and Guthke 1995;Greiff et al. 2015;Stadler et al. 2015). Consistent with others , we argue that emphasis on classic psychometric qualities has led to an advocacy for MCS-like tasks, a reduction in multi-phasic task complexity, and questionable validity as tests of "true" CPS ability . ...
Article
Full-text available
Despite substantial evidence for the link between an individual’s intelligence and successful life outcomes, questions about what defines intelligence have remained the focus of heated dispute. The most common approach to understanding intelligence has been to investigate what performance on tests of intellect is and is not associated with. This psychometric approach, based on correlations and factor analysis is deficient. In this review, we aim to substantiate why classic psychometrics which focus on between-person accounts will necessarily provide a limited account of intelligence until theoretical considerations of within-person accounts are incorporated. First, we consider the impact of entrenched psychometric presumptions that support the status quo and impede alternative views. Second, we review the importance of process-theories, which are critical for any serious attempt to build a within-person account of intelligence. Third, features of dynamic tasks are reviewed, and we outline how static tasks can be modified to target within-person processes. Finally, we explain how multilevel models are conceptually and psychometrically well-suited to building and testing within-individual notions of intelligence, which at its core, we argue is cognitive flexibility. We conclude by describing an application of these ideas in the context of microworlds as a case study.
... Komplexes Problemlösen korreliert jedoch relativ eng mit Intelligenz: In einer Metaanalyse resultierte ein mittlerer Zusammenhang von M(g) = .43 (Stadler et al., 2015); bei breiter Operationalisierung der Intelligenz und psychometrisch überzeugender Erfassung komplexen Problemlösens liegen die latenten Korrelationen teils noch höher (r = .85 bei Kretzschmar, Neubert, Wüstenberg & Greiff, 2016; r ≈ .70 ...
Article
Full-text available
Zusammenfassung. Die vorliegende Standortbestimmung zeigt die hohe wissenschaftliche Qualität der Intelligenzforschung und von Intelligenztests. Es werden aber auch mögliche Missverständnisse und Einseitigkeiten der Ergebnisrezeption und -interpretation thematisiert. Im Einzelnen werden (1) die hohe prognostische und kriterienbezogene Validität bei gleichzeitigen Vorbehalten wie teils niedriger Akzeptanz bzw. Augenscheinvalidität, (2) die Darstellung empirischer Befunde aus der Perspektive ausgewählter Theorien sowie (3) die Bedeutung von Umwelteinflüssen und hohen Erblichkeitskoeffizienten eingehender betrachtet. Für jeden dieser Bereiche wird verdeutlicht, dass vor allem Präzision bei der Rezeption und Darstellung von Forschungsergebnissen notwendig ist, um Einseitigkeiten, Missverständnisse und Instrumentalisierungen zu vermeiden. Der vorliegende Beitrag zeigt, dass einiges, was als Problem der Intelligenzforschung und von Intelligenztests kritisiert wird, letztendlich auf die dargestellten Missverständnisse zurückzuführen ist. Vor diesem Hintergrund wird der Unterschied zwischen der qualitativ hochwertigen Intelligenzforschung und Intelligenztestung einerseits sowie den Missverständnissen und Einseitigkeiten bei der Rezeption andererseits herausgearbeitet. Weiterhin werden berechtigte Kritikpunkte an der Intelligenzforschung und an Intelligenztests sowie Forschungsdesiderata benannt.
... In a meta-analysis, an unadjusted mean relationship between the two constructs of M(g) = .43 was reported (Stadler et al., 2015). In further studies with a broad operationalization of intelligence and a psychometrically strong assessment of CPS, substantially higher correlation coefficients were reported (Lotz et al., 2016: r = .69 ...
Article
Human beings are frequently confronted with complex systems that change dynamically over time. Correspondingly, so called eigendynamic effects are an important feature in complex problem solving (CPS) tasks. Eigendynamic effects describe increases or decreases in outcome variables over time without the individual's intervention. Identifying the eigendynamic effects from the start while exploring the system is reasonably an effective strategic behavior when systems are comprised of different effect types (i.e., eigendynamic effects alongside other effects). In this study, we hypothesized that the strategic behavior to identify eigendynamic effects early when exploring the system was related to CPS performance and mediated the relationship between intelligence and CPS performance. In a sample of N = 262 German high school students, the investigated strategic behavior predicted CPS performance when exploring the system (knowledge acquisition phase) and when controlling the system (knowledge application phase) in single CPS tasks and in a set of CPS tasks comprising of different effect types. In mediation models the strategic behavior to identify eigendynamic effects early mediated the intelligence-CPS performance relation in the knowledge acquisition phase, but not in the knowledge application phase. We discuss these findings in the light of how students interact with complex and dynamic systems, pointing to the importance of this strategic behavior for the relationship between intelligence and CPS.
... In the Guerin et al (2021) study, both improvements on IQ tests and academic achievement measures are counted as far transfer effects in this respect, assuming that both tap the same underlying reasoning skills that have improved with training. On this conceptualisation of far transfer, in addition to traditional psychometric measures of intelligence such as fluid intelligence or working memory, there is a pressing need for the incorporation of ecologically valid tests that measure performance in meaningful real world contexts, such as performance in everyday functioning, in education, at work or in sports (see also Burgess et al., 2006;Stadler et al., 2015). This theory builds on the assumption that skilled behaviours occur within a cultural, public context, with norms about appropriate or expected activities and performance-levels. ...
Preprint
Full-text available
This article addresses this concern by outlining a theoretical framework for constraints of effective transfer from cognitive training: the skill-based latent variable intervention (LVI) framework. This framework builds on traditional psychometric factor analytic accounts of intelligence. Key concepts in this psychometric approach will now be briefly reviewed.
... However, there are also concerns that essential skills for solving problems in our current society are not fully reflected in narrow definitions of intelligence and, thus, are not part of the standardized intelligence tests (Halpern and Dunn 2021;Stanovich 2009;and Stanovich 2014). For instance, although problem-solving skills and intelligence are both theoretically and empirically related (e.g., Greiff et al. 2014 andStadler et al. 2015), there are certain problem-solving skills, which are essential for solving consequential world problems but not reflected in traditional intelligence tests (for instance, detecting and controlling autonomous changes in problems; see Stadler et al. 2019). In the current article, we review the role of higher-order thinking (HOT) skills as one promising candidate reflecting integral, established problem-solving skills. ...
Article
Full-text available
The development of a vaccine marks a breakthrough in the fight against infectious diseases. However, to eradicate highly infectious diseases globally, the immunization of large parts of the population is needed. Otherwise, diseases, such as polio, measles, or more recently COVID-19, will repeatedly flare-up, with devastating effects on individuals and, in the worst case, on significant shares of the world population. For example, polio has been almost eradicated over the past three decades through an unprecedented global effort, but complete immunization has not yet been achieved. In this article, we use polio as an example to show how the global effort of developing and administering a vaccine can be understood as solving a complex problem since it involves cultural, political, and geographical barriers that demand solutions in dynamically changing and highly versatile environments. Referring to the literature on problem-solving, higher-order thinking, and complex reasoning, we demonstrate how the ability to deal with real-world environments that are complex and dynamically changing, adapting initial solutions to new circumstances and collaborate efficiently with others, has been essential for this endeavor. We argue that problem-solving abilities form one basis for solving consequential world problems.
... challenging problem scenarios (Ifenthaler, 2012;Jonassen, 2011;Milbourne & Wiebe, 2017;Stadler et al., 2015), and high achievement is strongly linked to aspects of self-regulation (Tortop, 2015). In other studies, Klassen (2010) reported that young adolescent learners with LD rated their self-regulatory skills at a lower level than students without learning difficulties, and low self-regulation has been shown to predict low achievement levels (Zimmerman & Kitsantas, 2014). ...
Article
Problem-based learning (PBL) is increasingly referenced in secondary education as a teaching strategy. This quasi-experimental, field-based study is unique in that it examined whether PBL was effective for all students, including identified groups of exceptional learners within 10 mainstream, Grade 8, Australian classrooms in two schools. Each class completed the same three-week PBL unit. Pre- and post-unit indicators of key capacities - topic knowledge, understanding of the problem-solving process, and self-regulatory skills essential to achieving PBL goals - were measured. Significant differences both in initial capacity and in pre-post-assessment changes across the cohort and between identified groups of learners were found. The implications of these findings are discussed.
... The exercise should start with a profound and meticulous appreciation of the problem in order to ensure that it is fully understood and correctly defined (Pereira et al., 2021a). In line with this approach, problem-solving literature recommends dedicating more time in the analysis of the problem, instead of in the analysis of the solution (Bhardwaj et al., 2018;Stadler et al., 2015;Awudu et al., 2020). The first step in the Pereira problem solving model is the problem definition step, followed by the identification of its main business impacts (e.g., in costs, revenue, efficiency) and by a trend analysis of these impacts. ...
... Suboptimal performance on a DDM task may result from limitations in cognitive capacities such as fluid intelligence and abstract thinking skills, as well as limited task-specific context knowledge (Kerstholt and Raaijmakers, 1997;Goode and Beckmann, 2010;Gonzalez et al., 2017). Working memory capacity , fluid intelligence (Stadler et al., 2015;Kyllonen et al., 2017), along with external factors, such as cognitive workload (Nicholson & O'Hare, 2014), time pressure (Gonzalez, 2004(Gonzalez, , 2005 and task complexity (Fu et al., 2019) have also been found to influence DDM performance. Cognitive workload has been shown to affect DDM regardless of prior task experience (Nicholson & O'Hare, 2014). ...
Article
The aim of this research was to examine how broadband noise which is present in many workplaces affects dynamic decision-making. The effect of potential moderating factors, cognitive workload and sex, were also examined. Forty-eight participants (24 females) with an average age of 27.38 years (SD = 12.34) were asked to complete a dynamic decision-making task over three consecutive-days. Independent variables were Noise (Broadband - 0dBA vs. 75dBA above background) and Cognitive Workload (Low vs. High, manipulated via presence of a secondary task). Among females, broadband noise significantly impaired performance in low workload, but significantly improved performance in high workload. In contrast, among males broadband noise had no significant effect on overall performance. From an applied perspective, understanding the interaction between noise, cognitive workload and sex allows for the design of a training environment to ensure maximum performance by all staff.
... Intelligence has often been shown to be one of the strongest predictors for long-term learning, information retrieval, or academic achievement, and it is also argued to be especially valuable and predictive for difficult and stimulating learning environments and complex materials (e.g., Gottfredson, 1997;Kuncel et al., 2004;Fergusson et al., 2005;Bornstein et al., 2013;Roth et al., 2015;Stadler et al., 2015;Stern, 2015Stern, , 2017Strenze, 2015). Moreover, intelligence is even defined as the ability to learn, to reason, and to solve problems and has also often been found to be associated with successful information processing, successful retrieval from long-term memory, and higher working memory capacities (see, e.g., Gottfredson, 1997;Sternberg, 1997;Oberauer et al., 2005;Bornstein et al., 2013;Stern, 2015Stern, , 2017Wang et al., 2017). ...
Article
Full-text available
It has often been shown that tests as intentionally hindered and difficult learning tasks increase long-term learning compared to easier tasks. Previous work additionally indicated that higher intelligence might serve as a prerequisite for such beneficial effects of tests. Nevertheless, despite their long-term learning effects, tests were also found to be evaluated as more negative and to lead to more stress and anxiety compared to easier control tasks. Stress and anxiety, in turn, often yield detrimental effects on learning outcomes. Hence, we hypothesized that tests increase later learning outcomes but simultaneously also lead to more stress perceptions. Such increased stress was, in turn, hypothesized to reduce later learning outcomes (thus, stress might serve as a mediator of the beneficial effects of tests on learning). All these assumed effects should further be moderated by intelligence, insofar as that higher intelligence should increase beneficial effects of tests on learning, should decrease stress perceptions caused by tests, and should reduce detrimental effects of stress on learning outcomes. Higher intelligence was also assumed to be generally associated with higher learning. We conducted a laboratory study ( N =89) to test these hypotheses: Participants underwent an intelligence screening, then worked on either a test or a re-reading control task, and reported their immediate stress perceptions. Later learning outcomes were assessed after 1week. The results supported all assumed main effects but none of the assumed interactions. Thus, participants using tests had higher long-term learning outcomes compared to participants using re-reading tasks. However, participants using tests also perceived more immediate stress compared to participants that only re-read the materials. These stress perceptions in turn diminished the beneficial effects of tests. Stress was also generally related to lower learning, whereas higher intelligence was linked to higher learning and also to lower stress. Hence, our findings again support the often assumed benefits of tests—even when simultaneously considering learners’ intelligence and and when considering the by tests caused stress perceptions. Notably, controlling for stress further increases these long-term learning benefits. We then discuss some limitations and boundaries of our work as well as ideas for future studies.
... 95% CI [.26, .48]), which is also in line with previous literature (e.g., Stadler et al., 2015). More importantly, the short measure based on Model 43 is related with the respective construct to a similar extent (WMC, r = .23, ...
Article
Full-text available
Rationale: Assessing complex problem-solving skills (CPS) is of great interest to many researchers. However, existing assessments require long testing times making them difficult to include in many studies and experiments. Here, we propose a specific composition of microworlds based on the MicroDYN approach, which allows for valid estimation of CPS in a substantially reduced amount of time (<20 min). Methods: Based on the reanalysis of a sample of N = 232 university students who worked on 11 microworlds of increasing difficulty, we conducted multiple confirmatory factor analyses to test all possible combinations of microworlds, which were theoretically justified in advance. Results/ Discussion: We demonstrate one best fitting set with five microworlds, which shows excellent factorial validity and relates to both conventional measures of intelligence and to school grades. We hope that this will allow other researchers to include CPS into their study designs even when testing time is limited.
... Also, complex problem solving and intelligence showed significant and substantial correlations. Brain skills play critical parts in doing problemsolving (19) . ...
... Overall, individuals with higher mental abilities are generally better at complex analytical problem-solving tasks such as optimisation (e.g. Burgoyne, Sala, Gobet, et al., 2016;Stadler et al., 2015). In other words, they are relatively more likely to find a good solution when searching for the best bundles of slack resources to employ. ...
Article
Full-text available
The management literature has been investigating teams’ human capital resources as a predictor of their task performance. However, our knowledge regarding the precise structure of the human capital-performance relationship, as well as the resource orchestrator role managers play in this relationship, remains limited. In this study, we relax the assumption that human capital resources are used effectively, and conceptually extend the human capital resources construct by distinguishing between gross and active human capital resources. Doing so both helps to better understand the human capital-performance link and clarify the exact role that managers play in this link. Using 98 teams’ data over 2 years (5492 sets of player-level data aggregated to 196 sets of team-level data) from European Big Five football (soccer) leagues, we test our predictions. Our study has implications for the human capital literature as well as for the resource-based view literature on organisational slack.
Article
Full-text available
David Sherry's pioneering work on the neuroecology of spatial memory has three characteristics that could inspire studies on other cognitive processes: it was grounded in a robust prior literature in psychology and neuroscience; it identified several natural history contexts in which repeated independent evolution of spatial memory differences had occurred in different clades; it involved a precise cognitive ability with a precise neural substrate. We discuss the application of these three principles to a more domain-general trait—innovation. We argue that targeting the caudolateral nidopallium and its connected areas, favoring problem-solving over reversal learning as an experimental assay, and focusing on situations that involve environmental change, such as urbanization and invasion, can help the study of innovation progress, like the field of spatial memory has since 1989.
Article
The current study assessed the role of differential offense category involvement in the link between cognitive ability and arrest. Using structural equation modeling, the current study relied on data from the Pathways ( n = 1,354) and Add Health ( n = 3,605) to address three research questions on the differential etiology of offending category (aggressive and property), the mediating effects of disaggregated offending on the relationship between IQ and arrest, and how those effects vary across samples. The results suggested the negative association between IQ and arrest masked more complex processes encompassing opposing indirect paths through aggressive (negative) and property (positive) offending, suggesting inconsistent mediation. As a result, the total indirect effect of IQ on arrest through offending was suppressed when an aggregated offending measure was used, highlighting the value of disaggregating offending categories.
Thesis
Dans cette thèse, nous avons voulu répondre à un questionnement : est-il possible d’observer un transfert d’apprentissage sur tablette numérique chez l’enfant, dans le cas de la Tour de Hanoï (TdH) ? Conformément au modèle de Klahr et Chen (2011), ce transfert numérique est-il soumis aux effets de similarités entre tâches et contextes, comme observé dans le transfert entre tâches tangibles ? De plus, au regard du rôle de l’inhibition dans le contrôle des interférences, l’inhibition contribue-t-elle à ce transfert ?Pour répondre à ces questions, trois études ont été menées auprès d’enfants de 6 à 7 ans français et gabonais. Les résultats ont montré que, comme pour le transfert entre tâches tangibles, le transfert numérique est caractérisé par une baisse de performances temporaire. Cette baisse de performances lors du passage à la tâche de transfert peut néanmoins être limitée par un entrainement spécifique (étude 1), une similarité des supports de présentation des tâches (étude 2), et une similarité conjointe des tâches et de leurs contextes numériques de présentation (étude 3). Dans les trois études, on note également que le contrôle inhibiteur (Diamond, 2013) contribue à limiter cette baisse.Cette thèse a en outre permis de tester la validité du modèle de Klahr et Chen (2011) dans le cas d’un transfert numérique. Dans cette perspective, ce travail a montré qu’apprendre sur une tablette ne garantit pas le transfert de l’apprentissage. Il faut que la nouvelle tâche soit également réalisée sur une tablette numérique, ce qui nuance l’hypothèse d’un effet facilitateur des tablettes numériques sur le transfert d’apprentissage. La contribution de l’inhibition au transfert d’apprentissage est attestée par les résultats obtenus, prolongeant des travaux antérieurs (Borst & Houdé, 2014). Les résultats de ce travail ébauchent des nouvelles perspectives pour une meilleure connaissance du rôle de l’inhibition dans le transfert d’apprentissage et des processus métacognitifs qui pourraient expliquer le pattern de regain de performances observé dans ce travail. Ils renseignent aussi les professionnels de l’éducation et des concepteurs des interfaces graphiques des tablettes numériques sur les conditions d’apprentissage et de transfert à partir de ces outils. Ainsi, les résultats sont discutés sur le plan théorique et en termes d’implications pratiques et pédagogiques.
Article
Full-text available
Bu araştırmada okul öncesi dönemdeki çocukların problem çözme becerileri ile bilişsel yetenekleri arasındaki ilişkileri belirlemek amaçlanmıştır. Araştırmanın örneklemi Aydın ilinde 9 farklı okul öncesi kurumda eğitim alan 83 çocuktan oluşmaktadır. Araştırmada çocukların bilişsel yeteneklerini belirlemek için Renkli Progresif Matrisler Testi, problem çözme becerilerini ölçmek için Problem Çözme Becerileri Ölçeği uygulanmıştır. Problem Çözme Becerileri ölçeği ile Renkli Progresif Matrisler Testi arasında doğrusal bir ilişki olup olmadığını belirlemek için Pearson Moment çift yönlü korelasyon analizi tekniği kullanılmıştır. Araştırma sonucunda okul öncesi dönemdeki çocukların bilişsel yetenekleri ile problem çözme basamakları ve problem çözme becerileri arasında anlamlı düzeyde ilişki tespit edilmiştir. //// In this study, it was aimed to determine the relationship between problem solving skills and cognitive abilities of preschool children. The sample of the study consisted of 83 children who were educated in 9 pre-schools in Aydın. Color Progressive Matrices Test was used to determine children’s cognitive abilities and Problem Solving Skills Scale was used to measure problem solving skills. The Pearson Product-Moment Coefficients analysis technique was used to examine whether there is a linear relationship between the Problem Solving Skills Scale and the Color Progressive Matrices Test. As a result of the research, the cognitive abilities of pre-school children were found to be significantly correlated with problem solving steps and problem solving skills.
Article
Two studies examined factors that would influence people's preferences for interaction with a perpetrator of sexism. In Study 1 (n = 348), participants preferred to interact (being friends or developing a relationship) with an intelligent person regardless of whether or not that person was sexist. Study 2 (n = 614) replicated this finding and confirmed that where a perpetrator had a high level of intelligence, people were more willing to interact with them, regardless of the perpetrator's sex and the perceived commission or non-commission of sexist behavior. Moreover, Study 2 provides evidence that participants' hostile sexism beliefs are a significant covariate of a willingness to interact with unintelligent women. The results are discussed in terms of their implications for the understanding of person perception.
Article
Full-text available
Complex problem solving (CPS) is considered to be one of the most important skills for successful learning. In an effort to explore the nature of CPS, this study aims to investigate the role of inductive reasoning (IR) and combinatorial reasoning (CR) in the problem-solving process of students using statistically distinguishable exploration strategies in the CPS environment. The sample was drawn from a group of university students (N = 1343). The tests were delivered via the eDia online assessment platform. Latent class analyses were employed to seek students whose problem-solving strategies showed similar patterns. Four qualitatively different class profiles were identified: (1) 84.3% of the students were proficient strategy users, (2) 6.2% were rapid learners, (3) 3.1% were non-persistent explorers, and (4) 6.5% were non-performing explorers. Better exploration strategy users showed greater development in thinking skills, and the roles of IR and CR in the CPS process were varied for each type of strategy user. To sum up, the analysis identified students’ problem-solving behaviours in respect of exploration strategy in the CPS environment and detected a number of remarkable differences in terms of the use of thinking skills between students with different exploration strategies.
Chapter
This chapter reviews assessment strategies for medical, psychological, and educational functioning for persons with intellectual disabilities. This includes a review of background information important to diagnostic assessment including (a) medical history, (b) family and home life, (c) educational history, (d) developmental history, (e) behavioural/mental health, and (f) employment/community. A review of indirect (i.e. rating scales and interviews) and direct assessment (i.e. behavioural observations) techniques is discussed. Finally, psychoeducational assessment is discussed to assist individualized educational plan teams in determining if a child requires and is eligible for special education assistance. Specific psychoeducational assessment practices are reviewed including (a) behavioural and assessment techniques, (b) cognitive assessment strategies, and (c) the evaluation of adaptive behaviour, language, basic psychological processes, academic achievement, and social/emotional functioning.
Article
Full-text available
In this paper we propose the multiple complex systems (MCS) approach for assessing domain-general complex problem-solving (CPS) skills and its processes knowledge acquisition and knowledge application. After defining the construct and the formal frameworks for describing complex problems, we emphasise some of the measurement issues inherent in assessing CPS skills with single tasks (i.e., fixed item difficulty, low or unknown reliability, and a large impact of random errors). With examples of the MicroDYN test and the MicroFIN test (two instances of the MCS approach), we show how to adequately score problem-solving skills by using multiple tasks. We discuss implications for problem-solving research and the assessment of CPS skills in general.
Article
Full-text available
Education in the twenty-first century must prepare students to meet the challenges of a dynamic and interconnected world. However, assessment of students' skills tends to focus primarily on static tasks. Therefore, it is not known whether knowledge about successful strategies displayed on static tasks can be transferred to interactive and dynamic environments. This study investigated whether students' knowledge of a certain strategy (i.e.; vary-one-thing-at-a-time, VOTAT) that was assessed in a paper-and-pencil-based scientific reasoning task as well as their fluid intelligence and learning orientation would be sufficient to explain variance in the application of the VOTAT strategy in solving an interactive complex problem solving (CPS) task (i.e.; CPS strategy). Furthermore, we analyzed whether CPS strategy mediated the relation between the predictors (i.e.; scientific reasoning, learning orientation, fluid intelligence) and CPS performance. The sample consisted of N = 3,191 Finnish students attending the 6th and 9th grades. Results revealed that all predictors were significantly related to CPS strategy, but a substantial amount of variance in CPS strategy remained unexplained (ΔR2 =.583). Furthermore, CPS strategy mediated the relation between the predictors and CPS performance. Three implications are discussed: Different demands on the problem solver, knowledge transfer from static versus interactive tasks, or metastrategic knowledge may explain the unexplained variance in CPS strategy. Additionally, the results of our mediation analyses emphasize the importance of measuring strategies via logfiles to gain a deeper understanding of determinants of students' CPS performance. Finally, fostering motivational factors such as students' learning orientation yields small improvements in CPS performance.
Article
Full-text available
Ziel des Projekts ist, die Unterschiede zwischen Personen beim Lösen komplexer, alltagsnaher Probleme aufzuklären. Dazu setzen wir computersimulierte, alltagsorientierte Planspiele ein. Kognitive und nicht-kognitive Variablen dienen als Prädiktoren der Problemlöseleistung: Intelligenz, Arbeitsgedächtnis, problemrelevantes Vorwissen, Computerkenntnisse und Persönlichkeitseigenschaften. Diese Variablen wurden in früheren Untersuchungen als relevante Einflußgrößen auf den Erfolg bei der Steuerung eines komplexen Systems identifiziert.
Article
Full-text available
Computer-based problem solving scenarios or "microworlds" are contemporary assessment instru-ments frequently used to assess students' complex problem solving behavior – a key aspect of today's educational curricula and assessment frameworks. Surprisingly, almost nothing is known about their (1) acceptance or (2) psychometric characteristics in student populations. This article introduces the Genetics Lab (GL), a newly developed microworld, and addresses this lack of em-pirical data in two studies. Findings from Study 1, with a sample of 61 ninth graders, show that acceptance of the GL was high and that the internal consistencies of the scores obtained were satisfactory. In addition, meaningful intercorrelations between the scores supported the instru-ment's construct validity. Study 2 drew on data from 79 ninth graders in differing school types. Large to medium correlations with figural and numerical reasoning scores provided evidence for the instrument's construct validity. In terms of external validity, substantial correlations were found between academic performance and scores on the GL, most of which were higher than those ob-served between academic performance and the reasoning scales administered. In sum, this research closes an important empirical gap by (1) proving acceptance of the GL and (2) demonstrating satisfactory psychometric properties of its scores in student populations.
Article
Full-text available
Im Zentrum des Beitrags steht die Bedeutung und Prognosekraft dynamischer Problemlösekompetenz (DPK) für das Kriterium der technischen Problemlöseleistung. Als weitere Prädiktoren wurden in der Untersuchung fluide Intelligenz und technisches Fachwissen berücksichtigt. Die Integration fluider Intelligenz ermöglichte auch eine Untersuchung der Frage, ob sich DKP von fluider Intelligenz unterscheiden lässt. Die auf einer Kfz-Mechatroniker- und Elektronikerstichprobe (n = 129, n = 88) basierenden Analysen belegten die empirische Eigenständigkeit der DKP gegenüber fluider Intelligenz. In der Elektronikerstichprobe erklärte DKP die technische Problemlöseleistung am besten, wobei fluide Intelligenz darüber hinaus inkrementelle Validität zeigte. Bei den Kfz-Mechatronikern korrelierte nur fluide Intelligenz mit dem Kriterium, allerdings schwach. Wurde Fachwissen einbezogen, verschwand sowohl die Bedeutung der DKP als auch fluider Intelligenz. Allerdings übte bei den Elektronikern fluide Intelligenz einen über Wissen vermittelten indirekten Einfluss auf das Kriterium aus. Die nicht ganz eindeutige Befundlage, stichprobenspezifische Unterschiede sowie theoretische Ansätze zur Erklärung der beobachteten Effekte werden diskutiert.
Article
Full-text available
This article addresses two unsolved measurement issues in dynamic problem solving (DPS) research: (a) unsystematic construction of DPS tests making a comparison of results obtained in different studies difficult and (b) use of time-intensive single tasks leading to severe reliability problems. To solve these issues, the MicroDYN approach is presented, which combines (a) the formal framework of linear structural equation models as a systematic way to construct tasks with (b) multiple and independent tasks to increase reliability. Results indicated that the assumed measurement model that comprised three dimensions, information retrieval, model building, and forecasting, fitted the data well (n = 114 students) and could be replicated in another sample (n = 140), showing excellent reliability estimates for all dimensions. Predictive validity of school grades was excellent for model building but nonexistent for the other two MicroDYN dimensions and for an additional measure of DPS. Implications are discussed.
Chapter
In der Informations- und Wissensgesellschaft, in der sich (nicht nur) Schülerinnen und Schüler zurechtfinden müssen, ist die Fähigkeit, Informationen gezielt und schnell zu finden oder abzurufen, zu verarbeiten und zu kommunizieren, eine Kompetenz, die in immer mehr schulischen und außerschulischen Bereichen immer stärker an Bedeutung gewinnt. Vor dem Hintergrund dieser alltäglichen Anforderungen werden Fähigkeiten im Umgang mit modernen Informations- und Kommunikationstechnologien wie dem Computer und dem Internet neben der Lesefähigkeit zu einer Schlüsselkompetenz (Schulz-Zander, 2000), die sich auch immer mehr als eine der zentralen Fähigkeiten im Berufsleben etabliert (Russon, Josefowitz, & Edmonds, 1994).
Article
There are 2 families of statistical procedures in meta-analysis: fixed- and random-effects procedures. They were developed for somewhat different inference goals: making inferences about the effect parameters in the studies that have been observed versus making inferences about the distribution of effect parameters in a population of studies from a random sample of studies. The authors evaluate the performance of confidence intervals and hypothesis tests when each type of statistical procedure is used for each type of inference and confirm that each procedure is best for making the kind of inference for which it was designed. Conditionally random-effects procedures (a hybrid type) are shown to have properties in between those of fixed- and random-effects procedures.
Article
Figurale Matrizenaufgaben stellen eine bedeutsames Aufgabenformat im Bereich der Diagnostik kognitiver Fähigkeiten dar. Während die Konstruktion der Aufgabenstämme in der Literatur relativ gut dokumentiert ist, finden sich nur wenige Hinweise zur Konstruktion der für die Vorgabe benötigten Distraktoren. Dies erweist sich als problematisch, da ein Einfluss der Distraktorgestaltung auf das Bearbeitungsverhalten von Testpersonen vermutet werden kann. Die Matrizenkonstruktionsaufgabe stellt ein alternatives Format zur Vorgabe figuraler Matrizen dar. In diesem müssen die Testpersonen die Lösung der Aufgaben in einer computerisierten Testumgebung selbst konstruieren, weshalb auf die Verwendung von Distraktoren verzichtet werden kann. Im Rahmen der Studie wurden einer Stichprobe von 318 Studierenden 40 Matrizenkonstruktionsaufgaben vorgegeben. Die Ergebnisse deuten auf eine gute psychometrische Eignung des Aufgabenformats hin: Es konnte eine große Bandbreite an Aufgabenschwierigkeiten realisiert werden. Die Aufgaben erfassen eine latente Fähigkeitsdimension und weisen eine gute Konstrukt- und Kriteriumsvalidität auf. In der Diskussion werden Perspektiven zur Weiterentwicklung des Aufgabenformats besprochen.
Article
Recent advancements in the assessment of Complex Problem Solving (CPS) build on the use of homogeneous tasks that enable the reliable estimation of CPS skills. The range of problems featured in established instruments such as MicroDYN is consequently limited to a specific subset of homogeneous complex problems. This restriction is problematic when looking at domain-specific examples of complex problems, which feature characteristics absent from current assessment instruments (e.g., threshold states). We propose to utilize the formal framework of Finite State Automata (FSA) to extend the range of problems included in CPS assessment. An approach based on FSA, called MicroFIN, is presented, translated into specific tasks, and empirically investigated. We conducted an empirical study (N = 576), (1) inspecting the psychometric features of MicroFIN, (2) relating it to MicroDYN, and (3) investigating the relations to a measure of reasoning (i.e., CogAT). MicroFIN (1) exhibited adequate measurement characteristics and multitrait-multimethod models indicated (2) the convergence of latent dimensions measured with MicroDYN. Relations to reasoning (3) were moderate and comparable to the ones previously found for MicroDYN. Empirical results and corresponding explanations are discussed. More importantly, MicroFIN highlights the feasibility of expanding CPS assessment to a larger spectrum of complex problems.
Chapter
Parathyroid hormone-like hormone (PTHLH) is an important chondrogenic regulator; however, the gene has not been directly linked to human disease. We studied a family with autosomal-dominant Brachydactyly Type E (BDE) and identified a t(8;12)(q13;p11.2) translocation with breakpoints (BPs) upstream of PTHLH on chromosome 12p11.2 and a disrupted KCNB2 on 8q13. We sequenced the BPs and identified a highly conserved Activator protein 1 (AP-1) motif on 12p11.2, together with a C-ets-1 motif translocated from 8q13. AP-1 and C-ets-1 bound in vitro and in vivo at the derivative chromosome 8 breakpoint [der(8) BP], but were differently enriched between the wild-type and BP allele. We differentiated fibroblasts from BDE patients into chondrogenic cells and found that PTHLH and its targets, ADAMTS-7 and ADAMTS-12 were downregu-lated along with impaired chondrogenic differentiation. We next used human and murine chondrocytes and observed that the AP-1 motif stimulated, whereas der(8) BP or C-ets-1 decreased, PTHLH promoter activity. These results are the first to identify a cis-directed PTHLH downregulation as primary cause of human chondrodysplasia.
Article
This paper reviews research on dynamic decision making, i.e., decision making under conditions which require a series of decisions, where the decisions are not independent, where the state of the world changes, both autonomously and as a consequence of the decision maker's actions, and where the decisions have to be made in real time. It is difficult to find useful normative theories for these kinds of decisions, and research thus has to focus on descriptive issues. A general approach, based on control theory, is proposed as a means to organize research in the area. An experimental paradigm for the study of dynamic decision making, that of computer stimulated microworlds, is discussed, and two approaches using this paradigm are described: the individual differences approach, typical of German work in the tradition of research on complex problem solving, and the experimental approach. In studies following the former approach, the behavior of groups differing in performance is compared, either with respect to strategies or with respect to performance on psychological tests. The results show that there are wideinterindividual differences in performance, but no stable correlations between performance in microworlds and scores on traditional psychological tests have been found. Experimental research studying the effects of system characteristics, such as complexity and feedback delays, on dynamic decision making has shown that decision performance in dynamic tasks is strongly affected by feedback delays and whether or not the decisions have side effects. Although neither approach has led to any well-developed theory of dynamic decision making so far, the results nevertheless indicate that we are now able to produce highly reliable experimental results in thelaboratory, results that agree with those found in field studies of dynamic decision making. This shows that an important first step towards a better understanding of these phenomena has been taken.
Article
Complex Problem Solving (CPS) describes skills frequently needed in everyday life such as the use of new technological devices. Therefore, CPS skills constitute an increasingly important individual ability that needs theoretically embedded, reliable and validated measurement devices. The present article shows that current tests do not sufficiently address the requirement of a theory-based assessment. An integrative approach, the Action Theoretical Problem Space Model by Rollett (2008), is introduced and used to demonstrate how a theoretical framework can influence and inform test development. Implications for the assessment of CPS and its potential are discussed.
Article
Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further, CPS assessments show incremental validity beyond tests of other cognitive abilities when predicting relevant outcomes. However, these empirical accounts have relied on single CPS assessment instruments. We do not know whether these findings will generalize to the construct level across different CPS assessment instruments. To answer this question, we tested a sample of N = 339 German university students who completed three CPS assessment instruments based on multiple complex systems (MicroDYN, the Genetics Lab, and MicroFIN) and the matrices subtest of the Intelligence Structure Test as measure of reasoning. Students further reported their school grades. Analyses including latent multitrait–multimethod models provided support for the conceptualization of CPS as a complex cognitive ability. Results indicated that different CPS assessment instruments showed sufficient convergent validity (with a consistency mostly between .50 and .60). In addition, we found evidence for the divergent validity of CPS from reasoning (reasoning predicted two CPS facets, knowledge and control, βKNOW = .49 and βCON = .53, respectively). In the prediction of academic achievement, CPS explained variance in natural science grades after we controlled for reasoning (βCPS = .22), whereas social science grades were not predicted. Our findings suggest that the validity of CPS generalizes across different measurement instruments.
Article
Meta-analysis is a statistical tool for estimating the mean and variance of underlying population effects from a collection of empirical studies addressing ostensibly the same research question. Meta-analysis has become an increasing popular and valuable tool in psychological research, and major review articles typically employ these methods. This article describes the process of conducting meta-analysis: selecting articles, developing inclusion criteria, calculating effect sizes, conducting the actual analysis (including information on how to do the analysis on popular computer packages such as IBM SPSS and R) and estimating the effects of publication bias. Guidance is also given on how to write up a meta-analysis.
Article
In recent years, computer-based complex scenarios have been increasingly used for personnel assessment. The simulation of important facets of real job demands promises high content validity, high acceptance, and the assessment of key qualifications for challenging management tasks such as aptitudes of thinking in complex systems in case of intransparency. The potential behind observing and analyzing the subject’s behavior when interacting with the complex system is discussed in the context of the principles of Brunswik symmetry. Besides more precision in predicting job performance, a description of the problem solver’s behavior facilitates a better tailoring of human resources development measures. The reflections and arguments are checked against empirical results from N = 110 subjects who processed the scenario FSYS. The relations between scenario scales and external criteria corresponded to the expectations. Both total performance and behavior in the scenario can explain additional variance of job-equivalent performance indicators in addition to general intelligence.
Article
The growing popularity of meta-analysis has focused increased attention on the statistical models analysts are using and the assumptions underlying these models. Although comparisons often have been limited to fixed-effects (FE) models, recently there has been a call to investigate the differences between FE and random-effects (RE) models, differences that may have substantial theoretical and applied implications (National Research Council, 1992). Three FE models (including L. V. Hedges & I. Olkin's, 1985, and R. Rosenthal's, 1991, tests) and 2 RE models were applied to simulated correlation data in tests for moderator effects. The FE models seriously underestimated and the RE models greatly overestimated sampling error variance when their basic assumptions were violated, which caused biased confidence intervals and hypothesis tests. The implications of these and other findings are discussed as are methodological issues concerning meta-analyses.
Article
Glass's estimator of effect size, the sample mean difference divided by the sample standard deviation, is studied in the context of an explicit statistical model. The exact distribution of Glass's estimator is obtained and the estimator is shown to have a small sample bias. The minimum variance unbiased estimator is obtained and shown to have uniformly smaller variance than Glass's (biased) estimator. Measurement error is shown to attenuate estimates of effect size and a correction is given. The effects of measurement invalidity are discussed. Expressions for weights that yield the most precise weighted estimate of effect size are also derived.
Article
For decades it has been discussed whether Complex Problem Solving (CPS) depends exclusively on general intelligence and problem-specific prior knowledge, or whether systematically gathering information, viably representing causal relations between variables, and effectively applying one’s knowledge depends on operative aspects of intelligence that are not sufficiently addressed by traditional tests of intelligence. Before this question can be answered, measurement issues in the assessment of CPS have to be solved and the constructs CPS, intelligence, and prior knowledge have to be defined more clearly. In this study, we propose a test for the most important aspects of CPS competency, which is based on multiple knowledge-neutral complex problems. This test (MicroDYN) is validated empirically using a sample of undergraduates (n = 140): Latent correlations between CPS and fluid intelligence are about .50 confirming empirical separability of constructs. Adding to this, CPS is incrementally valid to fluid intelligence when predicting school grades. This holds particularly for grade point average and science grades. Implications for CPS in assessment contexts are discussed.