No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Complex Problem Solving and Intelligence: A Meta-Analysis
Abstract
The purpose of this meta-analysis is to examine the nature and magnitude of the relation between
complex problem-solving skills (CPS) and intelligence, a topic that has been widely discussed
and that has instigated a vast array of partially contradicting findings in the past. Theoretically,
researchers have hypothesized the two constructs to be everything from completely separate to
identical. Over the course of almost four decades, empirical studies yielded results in support of
both arguments. Our meta-analysis of 47 studies containing 60 independent samples and a total
sample size of 13,740 participants revealed a substantial correlation of CPS and intelligence with
an average effect size of M(g) = .433. In addition, we investigated whether the operationalization
of CPS and intelligence moderated this correlation. Whereas there were no significant correlation
differences considering the operationalization of intelligence, the approach used to measure CPS
moderated the correlation of CPS and intelligence. Especially the most recent approach towards
the assessment of CPS yielded the strongest associations between the two constructs.
Implications for existing theories and future research are discussed.
... A broad range of tools exist to assess individuals' CPS skills (Dörner et al., 1983;Funke, 2003;Greiff et al., 2012;Jonassen, 2011;Sonnleitner et al., 2012). Today, the most common such tools are computer-based microworlds encompassing a number of complex problem tasks, each with a different set of input variables that can be actively manipulated by the participant, affecting several output variables (for an overview of CPS assessment tools, see Stadler et al., 2015). ...
... Thus, while MicroDYN can be considered largely suitable for teaching the underlying strategies and mechanisms of successful CPS under 'clean' conditions (e.g., Schoppek & Fischer, 2017), it should be complemented by additional assessment approaches that a) feature a higher number of input variables, and b) avoid arbitrary labels, such as Tailorshop or related approaches (Danner et al., 2011;Stadler et al., 2015). In addition, we simultaneously advocate for the inclusion of interaction effects between input variables in MicroDYN in order to better highlight such interaction effects. ...
... The fine-grained investigation of strategy UNSUCCESSFUL AND SUCCESSFUL COMPLEX PROBLEM SOLVERS 38 application as it relates to CPS performance lies at the heart of the present article, and we have discussed our findings primarily in relation to their implications for future CPS training programs in educational contexts. However, CPS, intelligence, and education are closely linked Greiff & Neubert, 2014;Mayer, 2000;Stadler et al., 2015), and it is an overarching goal of education to shape and foster students' intellectual abilities so that they are able to successfully deal with the multi-faceted and dynamic demands of 21 st -century life (Martinez, 2000;Ritchie & Tucker-Drob, 2018). As stated by Martinez (2000), intelligence can be defined as "the knowledge, skills, and strategies necessary to be effective in a world that is complex and information-rich" (p. ...
Complex problem solving (CPS) is a key competence in educational contexts with strong conceptual links to students' overall intelligence. However, the mechanisms underlying successful CPS are not fully understood. Therefore, this study investigated several factors presumed to be relevant to CPS success using log file data to code each individual student action during six CPS tasks with different characteristics (N = 1276). We coded individual strategy combinations per student for each item for different strategy combinations of vary-one-thing-at-a-time (VOTAT), hold-one-thing-at-a-time (HOTAT), vary-no-thing-at-a-time (NOTAT), and change-all (CA). Results from generalized linear mixed models showed that CPS success was likely to be achieved by using VOTAT. However, there was an increased chance of solving an item when additional strategies, such as NOTAT or NOTAT plus HOTAT were used. This result was moderated by the presence/absence of eigendynamics as an important determinant of item difficulty. Strategy combinations of VOTAT together with other strategies (all including NOTAT) showed higher chances of CPS success when eigendynamics were present. Additionally,
higher chances of solving an item when using VOTAT with additional strategies was demonstrated for items without eigendynamics. Overall, our results suggest that flexibility in strategy application is the driving force for successful CPS performance. Implications are discussed in light of the presumed benefit of pedagogically relevant
metacognitive skills, such as planning, monitoring and reflecting, for CPS success. Based on our findings, we provide specific recommendations for the development of computer-based learning simulations to train CPS and related competencies, ultimately enhancing students' skills in educational contexts.
... The two constructs intelligence and complex problem solving (CPS) are conceptually and empirically connected (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). Taking a theoretical stance, a high level of intelligence facilitates successful problem-solving behavior (Kröner, Plass, & Leutner, 2005;. ...
... However, research has shown that when intelligence and CPS are both reliably and validly measured, they correlate substantially. In their meta-analysis, Stadler et al. (2015) analyzed the intelligence-CPS-correlations extracted from 47 studies and found a weighted average correlation of r = .43, indicating that intelligence and CPS are closely related but not identical (Stadler et al., 2015). ...
... In their meta-analysis, Stadler et al. (2015) analyzed the intelligence-CPS-correlations extracted from 47 studies and found a weighted average correlation of r = .43, indicating that intelligence and CPS are closely related but not identical (Stadler et al., 2015). Descriptively reviewing correlations that were reported in studies that employed more recent CPS measures (that resolved psychometric weaknesses of earlier measures; Wüstenberg et al., 2012) and broader measures of intelligence in the sense of g instead of narrower measures such as (figural) reasoning, we observed that the coefficients tended to be even higher and ranged in the area of .69 ...
Intelligence and complex problem solving (CPS) correlate closely, but little is known about the mechanism that translates intelligence into successful CPS. Therefore, this study considered the strategic exploration behaviors VOTAT (vary-one-thing-at-a-time) and NOTAT (vary no-thing-at-a-time) as possible mediators. A sample of 495 high-school students worked on nine CPS tasks, six of which with solely direct effects and three with direct and eigendynamic effects. We expected substantial mediation effects if the applied strategic behaviors were optimal to identify the particular underlying effect types (i.e., direct effects: VOTAT; direct and eigendynamic effects: VOTAT and NOTAT). The model for tasks with only direct effects revealed VOTAT and NOTAT to be substantial mediators: Whereas VOTAT showed substantial positive relations to intelligence and CPS performance, NOTAT unexpectedly showed substantial negative relations. Both VOTAT and NOTAT resulted in significant indirect mediation effects. The model for tasks with direct and eigendynamic effects showed substantial positive relations of VOTAT and NOTAT to intelligence and CPS-performance and resulted in significant and positive indirect mediation effects. Moreover, the indirect effects differed between VOTAT and NOTAT and across the two facets of CPS performance. Overall, strategic exploration behaviors are relevant for explaining the g-CPS-relation.
... Alternatively, strategy games or complex simulations seem to be more suitable to assess general mental ability (GMA), reasoning, critical thinking, and decision-making. In line with this, scores from games that were designed to simulate complex problem solving usually correlate substantially with measures of GMA or reasoning (Stadler et al., 2015). ...
... This seems to be in contrast to more recent studies that found positive correlations between performance in complex problem solving simulations and cognitive ability. In a meta-analysis of research that used computer simulations to investigate complex problem solving, Stadler et al. (2015) found an (uncorrected) mean correlation of 0.43 between performance in these simulations and cognitive ability measures. However, for the older simulations from this domain, which are also more similar to the simulation game used in the present study, the mean correlation was only 0.34, whereas the mean correlation was even 0.59 for newer simulations, which were often also more reliable. ...
... The reason for this is that this specific GBA targeted complex problem solving, which can be defined as "the successful interaction with task environments that are dynamic (i.e., change as a function of the user's interventions and/or as a function of time) and in which some, if not all, of the environment's regularities, can only be revealed by successful exploration and integration of the information gained in that process" (Frensch & Funke, 1995, p. 14). Based on this definition, it has been argued that GMA-and also reasoning as a more specific ability-should contribute to performance in these simulation games (see Stadler et al., 2015). Even though previous research suggests that males and females do not differ concerning GMA (Ployhart & Holtz, 2008), recent meta-analytic research revealed a small difference favoring men over women for reasoning (Waschl & Burns, 2020). ...
Gamified and game-based assessments (GBAs) are increasingly used for personnel selection but there are concerns that males and younger applicants have an advantage in these assessments. However, hardly any research has addressed whether sex and age are related to GBA performance. Similarly, the criterion-related validity of GBAs is also not sufficiently confirmed. Therefore, we analyzed archival data from a high-stakes setting in which applicants completed a computer-based simulation game targeting complex problem solving. The analyses confirmed expectations for the present simulation game of better performance for males than for females and for younger than for older applicants. However, the effect sizes were small. Furthermore, performance in the current simulation game correlated with job-related performance as measured in an assessment center.
... If the causes are interdependent the solution may be unique; if the causes are independent, the formulation of a solution for each cause may have to be considered. The solution proposal is typically a faster exercise than the diagnosis of the problem, which when done with quality, allows the preparation of the solution to be a more administrative act [23]. ...
... The formulation of the solution focuses substantively on the analysis of causes [23]. With the causes rigorously identified, the solution is easier to work with. ...
... Only after a clear understanding of the problem should the proposition of solutions begin. Furthermore, there are different ways to identify the possible causes of a problem [23,27,28]; however, they need to be validated with facts afterwards. ...
Problem solving skills are increasingly important to be able to tackle the complex problems encountered in the business world. Nowadays is increasingly important to achieve sustainable development, focusing not only on economic profit but also on creating social value. It is widely agreed that the principles of scientific management can lead to more effective solutions for complex problems. Problems have to be looked at objectively, with methodology and intellectual integrity and modesty. Several techniques have been developed to help analyze the causes of the problem or formulate solutions. Although these business research techniques are important tools, they are presented as isolated measures. Pereira Problem Solving methodology presented provides guide to address business and management problems. It is an integrative and easy-to-use instrument that helps organizations adopt scientific management practices and will enhance the efficiency of the solutions encountered.
... The question of how complex problem solving (CPS) skills are to be integrated into the nomological network of intellectual abilities was and still is one of the most examined questions in CPS research (e.g., Dörner and Funke 2017;Dörner et al. 1983;Funke and Frensch 2007;Kretzschmar et al. 2016;Süß 1996;Süß and Kretzschmar 2018). While most studies have investigated the association between (subconstructs of) intelligence and CPS (for an overview, see Stadler et al. 2015), there are only a few studies that have additionally considered working memory as a relevant cognitive ability to solve complex problems (for an overview, see Zech et al. 2017). ...
... Whereas there is a consensus regarding the high correlation between intelligence and CPS (see, e.g., Kretzschmar et al. 2016;Stadler et al. 2015), the results regarding the effect of working memory on CPS are inconsistent. For example, in the studies of Wittmann and Süß (1999) and Greiff et al. (2016), both working memory and (subconstructs of) intelligence significantly explained variance in CPS. ...
... The relation of working memory and intelligence (e.g., Ackerman et al. 2005;Oberauer et al. 2005) and of intelligence and CPS (e.g., Kretzschmar et al. 2016;Stadler et al. 2015) within the nomological network of cognitive abilities have stimulated a great amount of research. However, only a few studies have simultaneously considered all three constructs with inconsistent results. ...
In order to investigate the nature of complex problem solving (CPS) within the nomological network of cognitive abilities, few studies have simultantiously considered working memory and intelligence, and results are inconsistent. The Brunswik symmetry principle was recently discussed as a possible explanation for the inconsistent findings because the operationalizations differed greatly between the studies. Following this assumption, 16 different combinations of operationalizations of working memory and fluid reasoning were examined in the present study (N=152). Based on structural equation modeling with single-indicator latent variables (i.e., corrected for measurement error), it was found that working memory incrementally explained CPS variance above and beyond fluid reasoning in only 2 of 16 conditions. However, according to the Brunswik symmetry principle, both conditions can be interpreted as an asymmetrical (unfair) comparison, in which working memory was artificially favored over fluid reasoning. We conclude that there is little evidence that working memory plays a unique role in solving complex problems independent of fluid reasoning. Furthermore, the impact of the Brunswik symmetry principle was clearly demonstrated as the explained variance in CPS varied between 4 and 31%, depending on which operationalizations of working memory and fluid reasoning were considered. We argue that future studies investigating the interplay of cognitive abilities will benefit if the Brunswik principle is taken into account.
... The so-called microworlds deal with, for example, the job of a small-town mayor (Lohhausen; or a business manager of a factory (Tailorshop; Putz-Osterloh, 1981). Such microworlds featuring a high degree of complexity and interconnectivity can claim a high degree of face validity (e.g., Stadler et al., 2015). Nevertheless, these early assessment procedures faced severe difficulties in meeting the standards of satisfactory psychometric measurement, as has been mentioned before by different authors (e.g., Greiff & Funke, 2010;Kröner, 2001). ...
... The theoretically derived CPS dimensions knowledge acquisition and knowledge application were supported in confirmatory factor analyses (Greiff & Neubert, 2014). Additionally and regarding the relations with other important variables, evidence for convergent construct validity of MCS with intelligence (see Stadler et al., 2015) as well as incremental predictive validity of MCS in explaining school grades beyond, for example, figural reasoning (as a narrow indicator for intelligence) was found (e.g., . To conclude, MCS scenarios allow psychometrically convincing assessments of CPS. ...
... Whereas intelligence is usually assessed with static, well defined, and transparent problems, CPS is assessed with computer based simulated scenarios that lack several of these static elements and include instead dynamic components (see e.g., Süß & Kretzschmar, 2018). The debate, whether CPS constitutes a construct distinct from intelligence or whether CPS is conceptualized as part of or can be integrated in an extended construct of intelligence has not yet been resolved (e.g., Kretzschmar et al., 2016;Stadler et al., 2015;Stadler, Niepel, & Greiff, 2019). On a conceptual level, intelligence and CPS overlap substantially regarding, for example, the ability to solve problems, as mentioned (see Gottfredson, 1997). ...
Although the relation between intelligence and complex problem solving (CPS) has been investigated repeatedly, the moderating effect of prior knowledge on this relation still remains an open question. The Elshout-Raaheim hypothesis (see Leutner, 2002) predicts a higher correlation between intelligence and CPS at a medium level of problem specific prior knowledge and lower correlations at higher as well as lower levels of prior knowledge, thus a curvilinear moderation. We investigated this hypothesis in a sample of N = 495 high school students by using general intelligence (g) and a minimal complex systems approach (MCS) of CPS. Strategic prior knowledge in the sense of the relative frequency of the vary-one-thing-at-a-time strategy (VOTAT; Tschirgi, 1980) increased across MCS tasks in the knowledge acquisition phase of CPS. With increasing prior knowledge, correlations followed the predicted inverted U-shaped pattern in the knowledge acquisition phase and the knowledge application phase of CPS, thus supporting the Elshout-Raaheim hypothesis. The moderating effect of strategic prior knowledge for the intelligence–CPS relation and its relevance are discussed.
... A natural question is what is the relationship between Gf and CPS? This question has been the subject of debate over the past several years [11][12][13][14][15][16][17]. In this paper, we demonstrate that there is a strong relationship between fluid ability (Gf) and complex problem solving (CPS) skill, both conceptually and empirically. ...
... Empirically, we examine the correlations between ratings of Gf and CPS importance for several hundred occupations in the U.S. Department of Labor's O*NET database. We believe that this analysis complements other approaches to the question of the relationship between CPS and Gf, which have examined the correlations between performance on CPS tests and performance on Gf tests (e.g., [15]). ...
... Beyond the German CPS tradition, there are categories of tasks referred to as microworlds, which date back to Pappert and the Logo language [63], game-based assessments [64][65][66], such as Space Fortress [67], complex dynamic control tasks [68]; even intelligent tutoring systems may be thought to fit into this category [69]. Dynamic Decision Making [70] and Systems Thinking have been described as constructs that "overlap greatly with CPS in their respective definitions" ( [15], p. 37). ...
Complex problem solving (CPS) has emerged over the past several decades as an important construct in education and in the workforce. We examine the relationship between CPS and general fluid ability (Gf) both conceptually and empirically. A review of definitions of the two factors, prototypical tasks, and the information processing analyses of performance on those tasks suggest considerable conceptual overlap. We review three definitions of CPS: a general definition emerging from the human problem solving literature; a more specialized definition from the "German School" emphasizing performance in many-variable microworlds, with high domain-knowledge requirements; and a third definition based on performance in Minimal Complex Systems (MCS), with fewer variables and reduced knowledge requirements. We find a correlation of 0.86 between expert ratings of the importance of CPS and Gf across 691 occupations in the O*NET database. We find evidence that employers value both Gf and CPS skills, but CPS skills more highly, even after controlling for the importance of domain knowledge. We suggest that this may be due to CPS requiring not just cognitive ability but additionally skill in applying that ability in domains. We suggest that a fruitful future direction is to explore the importance of domain knowledge in CPS.
... In such systems, many smart home appliances could be monitored remotely using technologies such as wearables and smart-home devices. In addition, if intelligence is embedded into these movable devices/appliances [7,8], it is possible to enhance the performances of energy systems in many ways, e.g., energy optimization based on real-time monitoring. This could also help cut costs in smart homes by reducing the requirement for the user to stay at home for controlling the home appliances. ...
... The optimization is formulated as per Equation (8). All dataset features are applied to the optimization formula to optimally reduce appliance energy consumption by considering the weights associated with each feature based on their coefficients. ...
Home appliances are considered to account for a large portion of smart homes’ energy consumption. This is due to the abundant use of IoT devices. Various home appliances, such as heaters, dishwashers, and vacuum cleaners, are used every day. It is thought that proper control of these home appliances can reduce significant amounts of energy use. For this purpose, optimization techniques focusing mainly on energy reduction are used. Current optimization techniques somewhat reduce energy use but overlook user convenience, which was the main goal of introducing home appliances. Therefore, there is a need for an optimization method that effectively addresses the trade-off between energy saving and user convenience. Current optimization techniques should include weather metrics other than temperature and humidity to effectively optimize the energy cost of controlling the desired indoor setting of a smart home for the user. This research work involves an optimization technique that addresses the trade-off between energy saving and user convenience, including the use of air pressure, dew point, and wind speed. To test the optimization, a hybrid approach utilizing GWO and PSO was modeled. This work involved enabling proactive energy optimization using appliance energy prediction. An LSTM model was designed to test the appliances’ energy predictions. Through predictions and optimized control, smart home appliances could be proactively and effectively controlled. First, we evaluated the RMSE score of the predictive model and found that the proposed model results in low RMSE values. Second, we conducted several simulations and found the proposed optimization results to provide energy cost savings used in appliance control to regulate the desired indoor setting of the smart home. Energy cost reduction goals using the optimization strategies were evaluated for seasonal and monthly patterns of data for result verification. Hence, the proposed work is considered a better candidate solution for proactively optimizing the energy of smart homes.
... Cognitive developmental priorities of successive developmental cycles constrain the situations presenting problems that may be solved and the solutions possible (Stadler et al., 2015). The respective column in Table 1 summarizes changes in problemsolving possibilities with development. ...
... Adolescents must understand that problem solving is a dynamic interaction with task situations which change as a function of the own interventions aiming to reveal their regularities emerging from ongoing explorations and their integration into rules and solutions (Frensch and Funke, 1995). Expectedly, problem solving is moderately related with general intelligence (Stadler et al., 2015) Frontiers in Psychology 11 frontiersin.org and critical thinking. ...
This paper summarizes a theory of cognitive development and elaborates on its educational implications. The theory postulates that development occurs in cycles along multiple fronts. Cognitive competence in each cycle comprises a different profile of executive, inferential, and awareness processes, reflecting changes in developmental priorities in each cycle. Changes reflect varying needs in representing, understanding, and interacting with the world. Interaction control dominates episodic representation in infancy; attention control and perceptual awareness dominate in realistic representations in preschool; inferential control and awareness dominate rule-based representation in primary school; truth and validity control and precise self-evaluation dominate in principle-based thought in adolescence. We demonstrate that the best predictors of school learning in each cycle are the cycle’s cognitive priorities. Also learning in different domains, e.g., language and mathematics, depends on an interaction between the general cognitive processes dominating in each cycle and the state of the representational systems associated with each domain. When a representational system is deficient, specific learning difficulties may emerge, e.g., dyslexia and dyscalculia. We also discuss the educational implications for evaluation and learning at school.
... In the 2000s, there was a belief that systems thinking can be an answer to complex systems problems [11][12][13], and there is convergence around their definitions [14,15], This belief was translated later into action, where some studies appeared to show the significance of systems thinking in the domain of complex systems and recruiting employees [16]. However, what remained unanswered is the relationship between an individual's systems thinking (ST) and his/her general perception of different stages in the CPS process-that is a current gap in the literature. ...
... Numerous studies have linked complex systems and issues to systems thinking (ST) (e.g., [1,16,[62][63][64][65]). Several researchers [14,15] stated that the definitions of CPS and ST have some overlap. Funke [14] stated that five attributes distinguish complex problems from simple problems, which include (1) the complexity of the problem situation, (2) the relationships between the variables involved, (3) the dynamics of the situation and developments within the system, and the role of time, (4) partial or complete lack of transparency, and (5) polytely (a Greek term for "many goals") and the possibility of conflict in the existence of several goals. ...
The ability to solve modern complex systems becomes a necessity of the 21st century. The purpose of this study is the development of an instrument that measures an individual’s perception toward solving complex problems. Based on literature and definitions, an instrument with four stages named perceived complex problem-solving (PCPS) was designed through exploratory and confirmatory stages. The instrument is validated and scaled through different models, and the final model is discussed. After completing validation and scale development of the PCPS instrument, the final model of the PCPS instrument was introduced to resolve the gap in the literature. The final model of the PCPS instrument is able to find and quantify the degree of perception an individual holds in dealing with complex problems and can be utilized in different settings and environments. Further research about the relationship between Systems Thinking and CPS revealed individuals with a high level of systems thinking have a better understanding of the characteristics of complex problems and so better perception of CPS.
... A recent meta-analysis [16], which included 60 studies from 1982 to 2014, showed that the correlations between successful complex problem-solving and intelligence varied from 0.339 to 0.585, depending on the type of the complex problem. The weakest correlation (0.339) was obtained for "classical" complex problem tasks (LOHHAUSEN, MORO, TAILORSHOP, etc.,) [6,13], while the strongest (0.585)-for minimal complex systems tasks (MicroDYN, MicroFIN, etc.,) [16]. ...
... A recent meta-analysis [16], which included 60 studies from 1982 to 2014, showed that the correlations between successful complex problem-solving and intelligence varied from 0.339 to 0.585, depending on the type of the complex problem. The weakest correlation (0.339) was obtained for "classical" complex problem tasks (LOHHAUSEN, MORO, TAILORSHOP, etc.,) [6,13], while the strongest (0.585)-for minimal complex systems tasks (MicroDYN, MicroFIN, etc.,) [16]. ...
Current studies of complex problem-solving do not commonly evaluate the regulatory role of such personality-based variables as tolerance for uncertainty, risk-readiness, and patterns for coping with decisional conflict. This research aims to establish the contribution of those traits into individual parameters of complex problem-solving strategies. The study was conducted on 53 healthy individuals 17 to 29 years old (M = 20.42; SD = 2.34). Our own computerized complex problem task “The Anthill” was developed for this research. We identified five measurable parameters of the participants’ problem-solving strategies: preferred orientational level (POL); orientational level variability (OLV); class quotas‘ range (R); mean and median quotas shift (MS and MeS); and abrupt changes of strategy (AC). Psychodiagnostic methods included: new questionnaire of tolerance/intolerance for uncertainty; personal decision-making factors questionnaire; Melbourne Decision Making Questionnaire; Subjective Risk Intelligence Scale; Eysencks’ Impulsiveness Scale. The study showed the role of tolerance for uncertainty, risk-readiness, negative attitude toward uncertainty, and decision-making styles in the regulation of complex problem-solving strategies. Specifically, procrastination, tolerance for uncertainty, and risk-readiness were significant predictors of individual strategy indicators, such as POL, OLV, and MeS. Thus, personality traits were shown to regulate resource allocation strategies and the required level of orientation in a complex problem.
... In predicting school achievement, two distinct research lines can be traced: one line has stressed the role of intelligence [4-7] and a second line the role of interpersonal relationships [e.g., 8, 9]. In the former case, studies have focused attention on the role of fluid intelligence [3,10,11], which refers to the ability to adapt and deal with new situations in a flexible way, without previous learning being a decisive help. It is fundamentally shaped by primary skills, such as induction and deduction, relationships and classifications, the breadth of operational memory or intellectual speed [3,[12][13][14][15][16]. ...
Several studies have shown the relevance among students of the quality of their interpersonal relationships for their academic achievement. Nevertheless, most studies available have explored the relation between the cognitive functioning and academic achievement without taking into account the quality of the relationships experienced in the school environment. Furthermore, the studies that have begun to consider the joint role of these factors in the prediction of academic achievement are scant. Therefore, it appears of relevance to deepen the relation between cognitive functioning and quality of school relationships in order to support students’ academic achievement and the potential of youth. In this paper, we examined the moderating role of the quality of student–teacher relationships and school bonding (STR-SB) in the associations of fluid intelligence ( Gf ) with academic achievement among adolescents (N = 219). A multiple-group structural equation modelling analysis revealed that STR-SB quality moderated unexpectedly only the link between Gf and mathematics. The findings support the idea that the quality of student–teacher relationships may be a relevant dimension to be considered to clarify the association between cognitive functioning and academic achievement.
... Assessing these competencies requires assessment tools that respond to the test-taskers' inputs to allow for adequately complex and realistic tasks. Unlike conventional tasks (such as multiple-choice questions), these interactive tasks change, while the test-taker is trying to solve them, providing feedback to interventions or new information (Stadler et al., 2015). ...
Background: Over the last 20 years, educational large-scale assessments have under-gone dramatic changes moving away from simple paper-pencil assessments to inno-vative, technology-based assessments. This comprehensive switch has led to somerather technical improvements such as identifying early guessing or improvingstandardization.
Objectives: At the same time, process data on student interaction with items hasbeen shown to carry value for obtaining, reporting, and interpreting additional resultson student skills in international comparisons. In fact, on the basis of innovative simu-lated assessment environments, news about student rankings, under- and overper-forming countries, and novel ideas on how to improve educational systems areprominently featured in the media. At the same time, few of these efforts have beenused in a sustainable way to create new knowledge (i.e., on a scientific level), to improve learning and instruction (i.e., on a practical level), and to provide actionableadvice to political stakeholders (i.e., on a policy level).
Methods: This paper will adopt a meta-perspective and discuss recent and currentdevelopments with a focus on these three perspectives. There will be a particularemphasis on new assessment environments that have been recently employed inlarge-scale assessments.Results and Conclusions:Most findings remain very task specific. We propose necessary steps that need to be taken in order to yield sustainable change from analysingprocess data on all three levels.
Implications: New technologies might be capable of contributing to the research-policy-practitioner gap when it comes to utilizing the results from large-scale assess-ments to increase the quality of education around the globe but this will require amore systematic approach towards researching them.
... Complex problem solving requires effective decision-making based on the manipulation of multiple pieces of information. It is thus not surprising that intelligence shows a substantial correlation with solving complex real-life problems (Stadler et al., 2015). Tests measuring complex problem-solving abilities ask participants to make decisions about leading a fictional tailor shop for 12 simulated months (Funke, 2010) or controlling a rocket ship and a vehicle to reach specific goals (e.g., landing on a planet; Wirth & Funke, 2005). ...
Intelligence, creativity, and wisdom have a long history in psychology, each describing a different set of traits and abilities. Each has been associated with prototypical outcomes—intelligence with educational and work attainment (Roth et al., 2015; Watkins et al., 2007), creativity with unique achievements and innovations (Torrance, 1988), and wisdom with balancing interests and goals (Brienza et al., 2018). However, emphasis on these typical life outcomes can obscure similarities among these constructs or a possibility of their integration. In this chapter we argue that emotions and emotion abilities are a major attribute contributing to positive outcomes of intelligence, creativity, and wisdom.KeywordsIntelligenceEmotional intelligencePersonal intelligenceCreativityWisdomPersonal wisdomDecision-makingEmotionsEmotion regulation
... There are several kinds of microworlds (for an overview, see Stadler et al., 2015) that differ in certain aspects. Classical microworlds, such as Tailorshop (e.g., Danner et al. 2011), traditionally consist of one item (i.e., one problem) that has a high level of complexity, given the number of interrelated variables, intransparency, and conflicting goals. ...
This chapter covers the roles that intelligence, creativity, and wisdom play during problem solving as an integral competency that is needed to master challenges in the twenty-first century. We suggest that problem solving requires a set of skills that are strongly intertwined with all three concepts: intelligence, creativity, and wisdom.
... However, it demonstrates the similarity between the processes assumed in the standard model of CPS and the induction-deduction cycle that is characteristic for many problems used in intelligence tests (Hunt, 2010). Therefore, it is coherent that the performance in controlling simple systems (as used in MicroDYN) is closely correlated with measures of intelligence Stadler et al., 2015). The assignment of these processes to Type 2 is justified by their high demands on working memory. ...
This paper reflects on Dietrich Dörner's observation that participants in complex dynamic control tasks exhibit a "tendency to economize", that is, they tend to minimize cognitive effort. I interpret this observation in terms of a dual processing approach and explore if the reluctance to adopt Type 2 processing could be rooted in biological energy saving. There is evidence that the energy available for the cortex at any point in time is quite limited. Therefore, effortful thinking comes at the cost of neglecting other cortical functions. The proposed dual processing approach to complex problem solving is explored in an experiment where we varied cognitive load by means of a secondary task in order to make Type 1 or Type 2 processing more likely. Results show that cognitive load had no effect on target achievement. Even in the single task condition, many participants prefer Type 1 processing, confirming Dörner's observation.
... What are characteristics of successful CPS? Research concerning successful CPS has shown mixed findings. Studies investigating CPS and the relationship with wisdom [18], intelligence [19][20][21], personality [22], "savants" [23], creativity [24], culture [25], and general strategic knowledge [26,27] couldn't predict successful CPS adequately. Only domain-specific knowledge was related to CPS performance in some instances [28], problem-solving strategies [19,29,30], as well as self-consciousness and self-reflection to some degree [31,32]. ...
Complex problem solving (CPS) research has focused on cognitive variables, but in recent years, the influential role of emotions and motivation during the CPS process has been highlighted. In the current study, we focus on emotion regulation during CPS. Eighty-three university students worked on a simulated chocolate-producing company. Initially, they completed a survey on emotion regulation and demographics. Then, they were randomly assigned to four conditions where emotions were induced with short video clips: anger, fear, happiness, and trust. A manipulation check assessed the successful priming of emotions. While working individually on the microworld, CPS behavior and performance were saved in log files. We hypothesized that happiness and trust would lead to better performance than fear and anger. We also hypothesized that emotion regulation would be positively related to performance. There were no differences in performance at the beginning and at the end of the simulation among the four emotion groups. Regression analyses showed that emotion-regulation strategies significantly predicted CPS performance. Aggression was positively related to performance. Results show that it is more the regulation of emotions than the emotion per se that influences CPS performance.
... Capacity is relevant for maintaining, combining, and processing interim results. Not surprisingly, WMC has been shown to predict success in experiential learning tasks (Schmi et al. 2020) and in complex scenarios (Stadler et al. 2015). Second, both cognitive capacity and flexibility may share certain basal cognitive process and functions. ...
The task-switching paradigm is deemed a measure of cognitive flexibility. Previous research has demonstrated that individual differences in task-switch costs are moderately inversely related to cognitive ability. However, current theories emphasize multiple component processes of task switching, such as task-set preparation and task-set inertia. The relations of task-switching processes with cognitive ability were investigated in the current study. Participants completed a task-switching paradigm with geometric forms and a visuospatial working memory capacity (WMC) task. The task-switch effect was decomposed with the diffusion model. Effects of task-switching and response congruency were estimated as latent differences using structural equation modeling. Their magnitudes and relations with visuospatial WMC were investigated. Effects in the means of parameter estimates replicated previous findings, namely increased non-decision time in task-switch trials. Further, task switches and response incongruency had independent effects on drift rates, reflecting their differential effects on task readiness. Findings obtained with the figural tasks employed in this study revealed that WMC was inversely related to the task-switch effect in non-decision time. Relations with drift rates were inconsistent. Finally, WMC was moderately inversely related to response caution. These findings suggest that more able participants either needed less time for task-set preparation or that they invested less time for task-set preparation.
... Moreover, complex problem-solving tasks require the performance of more complex mental processes than intelligence tests do, such as actively interacting with a problem to gain knowledge about the problem environment (M. Stadler, Becker, Gödker, Leutner, & Greiff, 2015). In general, complex problems of reducing the barrier between a given starting state and the intended goal state are solved with the help of cognitive activities and behavior (Seel, 2011). ...
There are few studies related to programs based on augmented reality in the educational field, so measuring the effectiveness of augmented reality technology programs on the educational environment in higher education needs to be explored. Therefore, this study aimed to know the effectiveness of a program based on augmented reality on enhancing the skills of solving complex problems for the students of the optimal investment diploma. The participants were 60 participants from the Optimal Investment Diploma students at the College of Education at Najran University. They were randomly divided into two groups (control and experimental), each group included 30 participants. So that the control group studies the course "Issues and Problems in Teaching Digital Skills" in the traditional way, and the experimental group studies the same course through a program based on augmented reality. And 7 tasks of the Micro-DYN test battery were used as a tool to achieve the objective of the study. The study found the effectiveness of the program based on augmented reality in enhancing the skills of solving complex problems, mainly the acquisition and application of knowledge.
... [-0.21, -0.05], p = .002; see column 1 of Table 2), it reduced the within-study heterogeneity to levels typical to other three-level meta-analyses (e.g., Cheung, 2019;Nam et al., 2003;Stadler et al., 2015) (I 2 level-2 = 0.40, [0.28, 0.51]; I 2 level-3 = 0.49 [0.35, 0.64]). Model fit was also significantly improved when these outliers were excluded (see Table S2). ...
Special education services provided in U.S. schools should be designed to enable students with disabilities (SWD) to make “appropriately ambitious” educational progress (Endrew F., 2017). Yet the field lacks consensus on whether business-as-usual (BAU) special education services effectively support outcomes for SWD. We conducted a moderated multilevel multivariate meta-analysis of 40 studies and 1,721 effect sizes from more than 14 million participants to evaluate the effectiveness of BAU special education services. Effects ranged from null-to-positive (z = -0.07 to 0.12) for achievement and negative-to-null (z = -0.13 to -0.01) for behavior. We observed positive outcomes for educational attainment including graduation outcomes (z = 0.12 to 0.24) and postsecondary outcomes (z = 0.10 to 0.23), as well as for other outcomes (z = 0.21 to 0.38). Studies that used a more rigorous research design or comparison condition reported more positive effects across outcomes. Additional casual work is needed in this field.
... Computer-based microworlds have become the current standard for assessing students' CPS skills (e.g., Stadler et al., 2015). These microworlds involve different hypothetical scenarios as complex problems that must be solved, such as creating a lemonade using different ingredients with arbitrary labels or evaluating the impact of changing the dosage of several unknown types of medication on different body functions (Greiff et al., 2013a,b). ...
In recent years, the collection and analysis of technology-based process data in the educational measurement realm has steadily increased. Process data are usually generated automatically while a student is working on a computer-based task or exercise, and contain all relevant pieces of information on how the student interacts with this task. Such information includes but is not limited to task duration, number of clicks, and task performance. Given the informational richness of computer-based process data, its analysis is of great interest to educational stakeholders and researchers alike. Notably, scientific results obtained from process data analysis have been used to adapt assessment techniques and institutional curricula, and have uncovered areas for intervention to foster integral skills that provide an educational benefit to students. This article presents a thorough overview of historical and contemporary applications of technology-based process data analysis in the educational field. Specifically, it provides in-depth coverage of the role of process data analysis with respect to psychometrical considerations, formative and summative assessment. Moreover, we describe process data analysis for large-scale educational assessments such as PISA and PIAAC, as well as publicly available process data repositories. Additionally, process data analysis in educational measurement is showcased using the example of complex problem solving, a skill that has proven to be relevant for students' educational success and beyond. Finally, this article addresses several current challenges to consider when engaging in computer-based process data analysis for educational purposes, before concluding with an outlook on potential future developments in the field.
... CPS tasks having conceptual links with intelligence and decades of successful application in training and education (Wood et al. 2009). However, they are often discounted as intelligence measures because of the challenge in extracting psychometrically reliable and valid performance indicators that correlate sufficiently with static tests of intelligence (Beckmann and Guthke 1995;Greiff et al. 2015;Stadler et al. 2015). Consistent with others , we argue that emphasis on classic psychometric qualities has led to an advocacy for MCS-like tasks, a reduction in multi-phasic task complexity, and questionable validity as tests of "true" CPS ability . ...
Despite substantial evidence for the link between an individual’s intelligence and successful life outcomes, questions about what defines intelligence have remained the focus of heated dispute. The most common approach to understanding intelligence has been to investigate what performance on tests of intellect is and is not associated with. This psychometric approach, based on correlations and factor analysis is deficient. In this review, we aim to substantiate why classic psychometrics which focus on between-person accounts will necessarily provide a limited account of intelligence until theoretical considerations of within-person accounts are incorporated. First, we consider the impact of entrenched psychometric presumptions that support the status quo and impede alternative views. Second, we review the importance of process-theories, which are critical for any serious attempt to build a within-person account of intelligence. Third, features of dynamic tasks are reviewed, and we outline how static tasks can be modified to target within-person processes. Finally, we explain how multilevel models are conceptually and psychometrically well-suited to building and testing within-individual notions of intelligence, which at its core, we argue is cognitive flexibility. We conclude by describing an application of these ideas in the context of microworlds as a case study.
... Komplexes Problemlösen korreliert jedoch relativ eng mit Intelligenz: In einer Metaanalyse resultierte ein mittlerer Zusammenhang von M(g) = .43 (Stadler et al., 2015); bei breiter Operationalisierung der Intelligenz und psychometrisch überzeugender Erfassung komplexen Problemlösens liegen die latenten Korrelationen teils noch höher (r = .85 bei Kretzschmar, Neubert, Wüstenberg & Greiff, 2016; r ≈ .70 ...
Zusammenfassung. Die vorliegende Standortbestimmung zeigt die hohe wissenschaftliche Qualität der Intelligenzforschung und von Intelligenztests. Es werden aber auch mögliche Missverständnisse und Einseitigkeiten der Ergebnisrezeption und -interpretation thematisiert. Im Einzelnen werden (1) die hohe prognostische und kriterienbezogene Validität bei gleichzeitigen Vorbehalten wie teils niedriger Akzeptanz bzw. Augenscheinvalidität, (2) die Darstellung empirischer Befunde aus der Perspektive ausgewählter Theorien sowie (3) die Bedeutung von Umwelteinflüssen und hohen Erblichkeitskoeffizienten eingehender betrachtet. Für jeden dieser Bereiche wird verdeutlicht, dass vor allem Präzision bei der Rezeption und Darstellung von Forschungsergebnissen notwendig ist, um Einseitigkeiten, Missverständnisse und Instrumentalisierungen zu vermeiden. Der vorliegende Beitrag zeigt, dass einiges, was als Problem der Intelligenzforschung und von Intelligenztests kritisiert wird, letztendlich auf die dargestellten Missverständnisse zurückzuführen ist. Vor diesem Hintergrund wird der Unterschied zwischen der qualitativ hochwertigen Intelligenzforschung und Intelligenztestung einerseits sowie den Missverständnissen und Einseitigkeiten bei der Rezeption andererseits herausgearbeitet. Weiterhin werden berechtigte Kritikpunkte an der Intelligenzforschung und an Intelligenztests sowie Forschungsdesiderata benannt.
... In a meta-analysis, an unadjusted mean relationship between the two constructs of M(g) = .43 was reported (Stadler et al., 2015). In further studies with a broad operationalization of intelligence and a psychometrically strong assessment of CPS, substantially higher correlation coefficients were reported (Lotz et al., 2016: r = .69 ...
Human beings are frequently confronted with complex systems that change dynamically over time. Correspondingly, so called eigendynamic effects are an important feature in complex problem solving (CPS) tasks. Eigendynamic effects describe increases or decreases in outcome variables over time without the individual's intervention. Identifying the eigendynamic effects from the start while exploring the system is reasonably an effective strategic behavior when systems are comprised of different effect types (i.e., eigendynamic effects alongside other effects). In this study, we hypothesized that the strategic behavior to identify eigendynamic effects early when exploring the system was related to CPS performance and mediated the relationship between intelligence and CPS performance. In a sample of N = 262 German high school students, the investigated strategic behavior predicted CPS performance when exploring the system (knowledge acquisition phase) and when controlling the system (knowledge application phase) in single CPS tasks and in a set of CPS tasks comprising of different effect types. In mediation models the strategic behavior to identify eigendynamic effects early mediated the intelligence-CPS performance relation in the knowledge acquisition phase, but not in the knowledge application phase. We discuss these findings in the light of how students interact with complex and dynamic systems, pointing to the importance of this strategic behavior for the relationship between intelligence and CPS.
... In addition, when studies need to be cross-compared, there are often no adequate metrics to refer to. Hence, only general comparisons can be made using the number of variables (Funke, 1991) or structure of the task (Stadler et al., 2015). ...
Complex problem solving (CPS) has been a field that uses computer-simulated scenarios and has been applied in problem-solving-related studies. However, the problem scenario has not been thoroughly discussed as an essential factor in determining the reliability of the studies. Consequently, there are no systematic principles for scenario design in CPS studies. This study was performed to establish fundamental standards for complexity analysis in the CPS scenario design. We created a high-fidelity problem scenario to investigate the cognitive processes in CPS discussions. The reliability of the system and scenario was validated by five industrial experts. The findings of this study can be applied to future experiment designs, meta-analysis methods, and study replications.
... However, there are also concerns that essential skills for solving problems in our current society are not fully reflected in narrow definitions of intelligence and, thus, are not part of the standardized intelligence tests (Halpern and Dunn 2021;Stanovich 2009;and Stanovich 2014). For instance, although problem-solving skills and intelligence are both theoretically and empirically related (e.g., Greiff et al. 2014 andStadler et al. 2015), there are certain problem-solving skills, which are essential for solving consequential world problems but not reflected in traditional intelligence tests (for instance, detecting and controlling autonomous changes in problems; see Stadler et al. 2019). In the current article, we review the role of higher-order thinking (HOT) skills as one promising candidate reflecting integral, established problem-solving skills. ...
The development of a vaccine marks a breakthrough in the fight against infectious diseases.
However, to eradicate highly infectious diseases globally, the immunization of large parts of the
population is needed. Otherwise, diseases, such as polio, measles, or more recently COVID-19, will
repeatedly flare-up, with devastating effects on individuals and, in the worst case, on significant
shares of the world population. For example, polio has been almost eradicated over the past three
decades through an unprecedented global effort, but complete immunization has not yet been
achieved. In this article, we use polio as an example to show how the global effort of developing and
administering a vaccine can be understood as solving a complex problem since it involves cultural,
political, and geographical barriers that demand solutions in dynamically changing and highly
versatile environments. Referring to the literature on problem-solving, higher-order thinking, and
complex reasoning, we demonstrate how the ability to deal with real-world environments that are
complex and dynamically changing, adapting initial solutions to new circumstances and collaborate
efficiently with others, has been essential for this endeavor. We argue that problem-solving abilities
form one basis for solving consequential world problems.
... challenging problem scenarios (Ifenthaler, 2012;Jonassen, 2011;Milbourne & Wiebe, 2017;Stadler et al., 2015), and high achievement is strongly linked to aspects of self-regulation (Tortop, 2015). In other studies, Klassen (2010) reported that young adolescent learners with LD rated their self-regulatory skills at a lower level than students without learning difficulties, and low self-regulation has been shown to predict low achievement levels (Zimmerman & Kitsantas, 2014). ...
Problem-based learning (PBL) is increasingly referenced in secondary education as a teaching strategy. This quasi-experimental, field-based study is unique in that it examined whether PBL was effective for all students, including identified groups of exceptional learners within 10 mainstream, Grade 8, Australian classrooms in two schools. Each class completed the same three-week PBL unit. Pre- and post-unit indicators of key capacities - topic knowledge, understanding of the problem-solving process, and self-regulatory skills essential to achieving PBL goals - were measured. Significant differences both in initial capacity and in pre-post-assessment changes across the cohort and between identified groups of learners were found. The implications of these findings are discussed.
... The exercise should start with a profound and meticulous appreciation of the problem in order to ensure that it is fully understood and correctly defined (Pereira et al., 2021a). In line with this approach, problem-solving literature recommends dedicating more time in the analysis of the problem, instead of in the analysis of the solution (Bhardwaj et al., 2018;Stadler et al., 2015;Awudu et al., 2020). The first step in the Pereira problem solving model is the problem definition step, followed by the identification of its main business impacts (e.g., in costs, revenue, efficiency) and by a trend analysis of these impacts. ...
... Suboptimal performance on a DDM task may result from limitations in cognitive capacities such as fluid intelligence and abstract thinking skills, as well as limited task-specific context knowledge (Kerstholt and Raaijmakers, 1997;Goode and Beckmann, 2010;Gonzalez et al., 2017). Working memory capacity , fluid intelligence (Stadler et al., 2015;Kyllonen et al., 2017), along with external factors, such as cognitive workload (Nicholson & O'Hare, 2014), time pressure (Gonzalez, 2004(Gonzalez, , 2005 and task complexity (Fu et al., 2019) have also been found to influence DDM performance. Cognitive workload has been shown to affect DDM regardless of prior task experience (Nicholson & O'Hare, 2014). ...
The aim of this research was to examine how broadband noise which is present in many workplaces affects dynamic decision-making. The effect of potential moderating factors, cognitive workload and sex, were also examined. Forty-eight participants (24 females) with an average age of 27.38 years (SD = 12.34) were asked to complete a dynamic decision-making task over three consecutive-days. Independent variables were Noise (Broadband - 0dBA vs. 75dBA above background) and Cognitive Workload (Low vs. High, manipulated via presence of a secondary task). Among females, broadband noise significantly impaired performance in low workload, but significantly improved performance in high workload. In contrast, among males broadband noise had no significant effect on overall performance. From an applied perspective, understanding the interaction between noise, cognitive workload and sex allows for the design of a training environment to ensure maximum performance by all staff.
... Intelligence has often been shown to be one of the strongest predictors for long-term learning, information retrieval, or academic achievement, and it is also argued to be especially valuable and predictive for difficult and stimulating learning environments and complex materials (e.g., Gottfredson, 1997;Kuncel et al., 2004;Fergusson et al., 2005;Bornstein et al., 2013;Roth et al., 2015;Stadler et al., 2015;Stern, 2015Stern, , 2017Strenze, 2015). Moreover, intelligence is even defined as the ability to learn, to reason, and to solve problems and has also often been found to be associated with successful information processing, successful retrieval from long-term memory, and higher working memory capacities (see, e.g., Gottfredson, 1997;Sternberg, 1997;Oberauer et al., 2005;Bornstein et al., 2013;Stern, 2015Stern, , 2017Wang et al., 2017). ...
It has often been shown that tests as intentionally hindered and difficult learning tasks increase long-term learning compared to easier tasks. Previous work additionally indicated that higher intelligence might serve as a prerequisite for such beneficial effects of tests. Nevertheless, despite their long-term learning effects, tests were also found to be evaluated as more negative and to lead to more stress and anxiety compared to easier control tasks. Stress and anxiety, in turn, often yield detrimental effects on learning outcomes. Hence, we hypothesized that tests increase later learning outcomes but simultaneously also lead to more stress perceptions. Such increased stress was, in turn, hypothesized to reduce later learning outcomes (thus, stress might serve as a mediator of the beneficial effects of tests on learning). All these assumed effects should further be moderated by intelligence, insofar as that higher intelligence should increase beneficial effects of tests on learning, should decrease stress perceptions caused by tests, and should reduce detrimental effects of stress on learning outcomes. Higher intelligence was also assumed to be generally associated with higher learning. We conducted a laboratory study ( N =89) to test these hypotheses: Participants underwent an intelligence screening, then worked on either a test or a re-reading control task, and reported their immediate stress perceptions. Later learning outcomes were assessed after 1week. The results supported all assumed main effects but none of the assumed interactions. Thus, participants using tests had higher long-term learning outcomes compared to participants using re-reading tasks. However, participants using tests also perceived more immediate stress compared to participants that only re-read the materials. These stress perceptions in turn diminished the beneficial effects of tests. Stress was also generally related to lower learning, whereas higher intelligence was linked to higher learning and also to lower stress. Hence, our findings again support the often assumed benefits of tests—even when simultaneously considering learners’ intelligence and and when considering the by tests caused stress perceptions. Notably, controlling for stress further increases these long-term learning benefits. We then discuss some limitations and boundaries of our work as well as ideas for future studies.
... 95% CI [.26, .48]), which is also in line with previous literature (e.g., Stadler et al., 2015). More importantly, the short measure based on Model 43 is related with the respective construct to a similar extent (WMC, r = .23, ...
Rationale: Assessing complex problem-solving skills (CPS) is of great interest to many researchers. However, existing assessments require long testing times making them difficult to include in many studies and experiments. Here, we propose a specific composition of microworlds based on the MicroDYN approach, which allows for valid estimation of CPS in a substantially reduced amount of time (<20 min). Methods: Based on the reanalysis of a sample of N = 232 university students who worked on 11 microworlds of increasing difficulty, we conducted multiple confirmatory factor analyses to test all possible combinations of microworlds, which were theoretically justified in advance. Results/ Discussion: We demonstrate one best fitting set with five microworlds, which shows excellent factorial validity and relates to both conventional measures of intelligence and to school grades. We hope that this will allow other researchers to include CPS into their study designs even when testing time is limited.
... Also, complex problem solving and intelligence showed significant and substantial correlations. Brain skills play critical parts in doing problemsolving (19) . ...
... Overall, individuals with higher mental abilities are generally better at complex analytical problem-solving tasks such as optimisation (e.g. Burgoyne, Sala, Gobet, et al., 2016;Stadler et al., 2015). In other words, they are relatively more likely to find a good solution when searching for the best bundles of slack resources to employ. ...
The management literature has been investigating teams’ human capital resources as a predictor of their task performance. However, our knowledge regarding the precise structure of the human capital-performance relationship, as well as the resource orchestrator role managers play in this relationship, remains limited. In this study, we relax the assumption that human capital resources are used effectively, and conceptually extend the human capital resources construct by distinguishing between gross and active human capital resources. Doing so both helps to better understand the human capital-performance link and clarify the exact role that managers play in this link. Using 98 teams’ data over 2 years (5492 sets of player-level data aggregated to 196 sets of team-level data) from European Big Five football (soccer) leagues, we test our predictions. Our study has implications for the human capital literature as well as for the resource-based view literature on organisational slack.
... A number of explanations exist for the inconsistency of the findings on the relation between CPS and intelligence. Stadler et al. (2015) concluded that the zero-correlation found in early studies could be explained by using broader measures of intelligence that include a variety of different cognitive tasks (e.g., factual knowledge). Since more recent studies use more specific latent factors of intelligence that are conceptually closer to the construct of CPS (e.g., fluid intelligence), this relationship was found to be stronger. ...
The main aim of this study was to (a) test the construct validity of complex problem solving (CPS); (b) examine the ability to acquire knowledge as a mediator of the relationship between intelligence and CPS performance; and (c) investigate the personal need for structure as a moderator of the relationship between intelligence and knowledge acquisition. A total of 128 participants completed the self-report Personal Need for Structure scale; the Vienna Matrix Test to assess intelligence; and a new multiple complex systems approach method to assess CPS skills. When analyzing the internal structure of CPS, we found that a two-dimensional model consisting of knowledge acquisition and knowledge application best fitted the data. We also found that the relationship between intelligence and CPS performance was partially mediated by the ability to acquire knowledge. Finally, personal need for structure did not moderate the relationship between intelligence and the ability to acquire knowledge. Our results indicate a need to further investigate other cognitive abilities in interaction with contextual situational factors that could additionally explain variance in CPS performance. Moreover, we also highlight the importance of deeper observation of the knowledge application phase of CPS process.
... Teachers in the 21st century are required not only to be able to teach and manage classroom activities effectively but also to be creative in making learning more interesting, developing variations in learning by elaborating learning using technology to support improving the quality of teaching, and reflecting and evaluating their learning practices continuously [8]. Increasing students' CPS abilities can be obtained by applying a learning model that guides in solving problems [9]. The CPS of students can be skilled through computer-based training and evaluation, which requires completion steps quickly and precisely [10]. ...
One of the competencies needed currently is non-routine competency in solving a complex problem. Students with complex problem solving will be able to identify the complex problem, understand to review related information, and create solutions for the problems. However, the improvement of complex problem-solving abilities lacks attention. This phenomenon occurs when students are given problem-solving, students have not been able to understand the problem, plan solutions, and provide solutions to problems. To overcome these problems, the LAPS-Talk-Ball learning model integrated with interactive games that guide the students in solving problems was implemented by using interactive games and speech balloons to stimulate each student to express their ideas. This research is quantitative experimental research to determine the success of the LAPS-Talk-Ball learning model integrated with android-based interactive games in training complex problem-solving students using saturated sampling and non-equivalent control group research designs. From the results of gain test results high improvement category, which means that the LAPS-Talk-Ball learning model integrated with android-based interactive games can train the complex problem-solving abilities of students.
... It is an individual skill that has gained importance in educational and psychological research, since it requires skills beyond routine reasoning (Scherer, Greiff, & Hautamäki, 2015). Further, complex problem solving tasks demand the performance of more complex mental processes than intelligence tests do, such as the active interaction with the problem to acquire knowledge about the problem environment (Stadler, Becker, Gödker, Leutner, & Greiff, 2015). Overall, complex problem solving takes place for reducing the barrier between a given start state and an intended goal state with the help of cognitive activities and behavior (Funke, 2012). ...
The present study aimed to determine the effect of vodcast utilization in the enhancement the skills of Canadian smocking and its role in complex problem solving. The sample included (120) learners who were in their fourth academic level in the department of home economics. They were randomly divided into experimental and control group, with 60 learners in each group. A product assessment card and 7 Micro-DYN test battery tasks were used as main study instruments. Results proved showed that performance of students who learned through video lectures of handmade embroidery or vodcasts was better than the performance of colleagues in the control group with regard to the development of Canadian smocking skills and complex problem solving and mainly knowledge acquisition and application. The results provide significant proof of the vodcast advantages in educational technology studies and development.
... One final point is worth making about potential contributions to a science of practical wisdom, which is that the concept clearly overlaps with other more traditional foci of psychological research such as judgment and problem-solving, and it may be valuable to mine these literatures to enhance the understanding of phronesis. For example, decision-making competence (Fischhoff, 2010) and complex problem-solving skills (Stadler et al., 2015) have both been found to correlate about 0.50 with measures of cognitive ability, a substantial relationship. At the same time, Fischhoff reported competence was also associated with higher socioeconomic status, absence of paternal substance use, and a more positive peer environment even after controlling for cognitive variables, suggesting better environmental circumstances can contribute to better decisionmaking skills (also see Odom, 1967). ...
The VIA Classification of Character Strengths and Virtue has received substantial attention since its inception as a model of 24 dimensions of positive human functioning, but less so as a potential contributor to a psychological science on the nature of virtue. The current paper presents an overview of how this classification could serve to advance the science of virtue. Specifically, we summarize previous research on the dimensional versus categorical characterization of virtue, and on the identification of cardinal virtues. We give particular attention to the three-dimensional model of cardinal virtues that includes moral, self-regulatory, and intellectual domains. We also discuss the possibility that these three clusters be treated as fundamental elements of a virtue model, meaning that they clearly and directly contribute to both individual and communal flourishing across various cultures. This discussion includes a summary of previous speculations about the evolution of adaptations underlying the human capacity for using behavioral repertoires associated with the three virtues, as well as discussing ways in which they simultaneously enhance community and individual, in the last case focusing particularly on evidence concerning mating potential. We then discuss the relationship between the evolutionary perspective on virtues and Aristotle’s concept of the reciprocity of the virtues. Finally, we provide speculations about the nature of practical wisdom. While accepting the potential value of future revisions to the VIA model, that model even under its current conditions has the potential to generate a number of intriguing and testable hypotheses about the nature of virtue.
... Teachers in the 21st century are required not only to be able to teach and manage classroom activities effectively but also to be creative in making learning more interesting, developing variations in learning by elaborating learning using technology to support improving the quality of teaching, and reflecting and evaluating their learning practices continuously [8]. Increasing students' CPS abilities can be obtained by applying a learning model that guides in solving problems [9]. The CPS of students can be skilled through computer-based training and evaluation, which requires completion steps quickly and precisely [10]. ...
Case-Based Games Learning (CBGL) is a strategy of active learning in which the students implement knowledge and analytical skill related to the real complex situation (contextual) and relevant with subject lessons. Give case-based task or question routinely can help students to solve case-based mathematics problems. This study aims to analyze the effectiveness of CBGL using Quizizz dan Case Base Learning (CBL) without Quizizz based on students’ conceptual understanding in mathematics by finding out the percentage of the correct answer. Students were grouping by using a purposive sampling method based on the conceptual understanding indicator. The result of this study is the average conceptual understanding CBGL using Quizizz application strategy is more effective than CBL without Quizizz application in evaluating student work results quickly, precisely, and accurately. Therefore the use of Quizzes application with CBGL was more effective than CBL in assessing the work result quickly, right, and accurately.
... On an empirical level, research over the last decades has shown CPS to be related to other cognitive performance measures, including reasoning, working memory, and school achievement (e.g., Kretzschmar et al. 2016;Stadler et al. 2015). Some personality aspects also exhibit relations with CPS, albeit weakly (for instance, with the Big 5; Greiff and Neubert 2014). ...
Complex problem solving (CPS) is an up-and-coming twenty-first century skill that requires test-takers to solve dynamically changing problems, often assessed using computer-based tests. The log data that users produce when interacting with a computer-based test provide valuable information about each individual behavioral action they undertake, but such data are rather difficult to handle from a statistical point of view. This paper addresses this issue by building upon recent research focused on decoding log data and aims to identify homogeneous student profiles with regard to their ability to solve CPS tasks. Therefore, we estimated a discrete two-tier item response theory model, which allowed us to profile units (i.e., students) while taking into account the multidimensionality of the data and the explanatory effect of individual characteristics. The results indicate that: (1) CPS can be thought of as a three-dimensional latent variable; (2) there are ten latent classes of students with homogenous profiles regarding the CPS dimensions; (3) students in the higher latent classes generally demonstrate higher cognitive and non-cognitive performances; (4) some of the latent classes seem to profit from learning-by-doing within tasks, whereas others seem to exhibit the reverse behavior; (5) cognitive and non-cognitive skills, as well as gender and to some extent age, contribute to distinguishing among the latent classes.
... Finally, as hypothesized, controlling GPA for general intelligence reduced the relation between the log data and GPA substantially. Since previous research consistently reported substantial overlap between CPS ability and general intelligence, this provides additional support for the validity of the assumption that log data actually reflects CPS ability (Kretzschmar et al., 2016;Stadler et al., 2015). The relation between general intelligence and GPA (r = .57) ...
Analyzing test-taking behavior allows researchers to investigate the steps and actions resulting in the specific test outcome. The underlying assumption is that test-taking behavior is a valid indicator of the tested ability. The aim of this paper is to scrutinize this assumption in the context of complex problem solving (CPS) by analyzing individual differences in test-taking behavior and their relation to individual differences in established correlates of CPS ability. We investigated a sample of Finnish students who achieved the maximum score on five CPS tasks and worked on an additional intelligence measure. We logged the number of interactions with the CPS tasks and time-on-task. Students showed significant variance in both time-on-task (s² = 260.09, p = .005) and the number of interactions (s² = 0.381, p < .001) despite having no variance in their CPS test scores. Using structural equation modeling, both time-on-task (β = 0.17, p = .015) and the number of interactions (β = −0.17, p < .001) significantly predicted students' GPA, which has frequently shown to be associated with CPS ability. When adding intelligence – another established correlate of CPS – as predictor of GPA (β = 0.55, p < .001), the relation both between time-on-task (β = −0.09, p = .053) and the number of interactions (β = 0.09, p = .100) and students’ GPA was reduced to negligible effects. Taken together, our study supports the assumption of test-taking behavior actually representing differences in CPS ability – over and above test scores.
We conducted a systematic literature review searching for articles discussing complex problem-solving (CPS) in education. We used keywords such as “complex problem solving,” “education,” and “mathematics” to search for articles in Harzing’s Publish or Perish Scopus source. The findings revealed 44 articles published between 2013 and 2022. The majority of these articles were from Germany, and most focused on assessing and measuring CPS. Research topics included the process of cognition, metacognition, working memory, intelligence, motivation, CPS ability, and intention. Although CPS is closely related to the abilities needed for learning mathematics, there is currently limited research on complex problem-solving in education.
In this chapter we discuss the link between intelligence and problem-solving. To preview, we argue that the ability to solve problems is not just an aspect or feature of intelligence – it is the essence of intelligence. We briefly review evidence from psychometric research concerning the nature of individual differences in intelligence, and then review evidence for how intelligence relates to complex problem-solving. We also consider the question of what mechanisms might underlie both problem-solving and intelligence, focusing on fluid intelligence and some of our own research on placekeeping ability. We then discuss the predictive validity of intelligence as it relates to job performance, mortality, expertise, and academic achievement. We also discuss practical uses of intelligence tests. Finally, we consider the question of whether intelligence as problem-solving ability can be improved through training. We close with directions for future research.
Complex problem solving (CPS) is considered to be one of the most important skills for successful learning. In an effort to explore the nature of CPS, this study aims to investigate the role of inductive reasoning (IR) and combinatorial reasoning (CR) in the problem-solving process of students using statistically distinguishable exploration strategies in the CPS environment. The sample was drawn from a group of university students (N = 1343). The tests were delivered via the eDia online assessment platform. Latent class analyses were employed to seek students whose problem-solving strategies showed similar patterns. Four qualitatively different class profiles were identified: (1) 84.3% of the students were proficient strategy users, (2) 6.2% were rapid learners, (3) 3.1% were non-persistent explorers, and (4) 6.5% were non-performing explorers. Better exploration strategy users showed greater development in thinking skills, and the roles of IR and CR in the CPS process were varied for each type of strategy user. To sum up, the analysis identified students’ problem-solving behaviours in respect of exploration strategy in the CPS environment and detected a number of remarkable differences in terms of the use of thinking skills between students with different exploration strategies.
Dans cette thèse, nous avons voulu répondre à un questionnement : est-il possible d’observer un transfert d’apprentissage sur tablette numérique chez l’enfant, dans le cas de la Tour de Hanoï (TdH) ? Conformément au modèle de Klahr et Chen (2011), ce transfert numérique est-il soumis aux effets de similarités entre tâches et contextes, comme observé dans le transfert entre tâches tangibles ? De plus, au regard du rôle de l’inhibition dans le contrôle des interférences, l’inhibition contribue-t-elle à ce transfert ?Pour répondre à ces questions, trois études ont été menées auprès d’enfants de 6 à 7 ans français et gabonais. Les résultats ont montré que, comme pour le transfert entre tâches tangibles, le transfert numérique est caractérisé par une baisse de performances temporaire. Cette baisse de performances lors du passage à la tâche de transfert peut néanmoins être limitée par un entrainement spécifique (étude 1), une similarité des supports de présentation des tâches (étude 2), et une similarité conjointe des tâches et de leurs contextes numériques de présentation (étude 3). Dans les trois études, on note également que le contrôle inhibiteur (Diamond, 2013) contribue à limiter cette baisse.Cette thèse a en outre permis de tester la validité du modèle de Klahr et Chen (2011) dans le cas d’un transfert numérique. Dans cette perspective, ce travail a montré qu’apprendre sur une tablette ne garantit pas le transfert de l’apprentissage. Il faut que la nouvelle tâche soit également réalisée sur une tablette numérique, ce qui nuance l’hypothèse d’un effet facilitateur des tablettes numériques sur le transfert d’apprentissage. La contribution de l’inhibition au transfert d’apprentissage est attestée par les résultats obtenus, prolongeant des travaux antérieurs (Borst & Houdé, 2014). Les résultats de ce travail ébauchent des nouvelles perspectives pour une meilleure connaissance du rôle de l’inhibition dans le transfert d’apprentissage et des processus métacognitifs qui pourraient expliquer le pattern de regain de performances observé dans ce travail. Ils renseignent aussi les professionnels de l’éducation et des concepteurs des interfaces graphiques des tablettes numériques sur les conditions d’apprentissage et de transfert à partir de ces outils. Ainsi, les résultats sont discutés sur le plan théorique et en termes d’implications pratiques et pédagogiques.
The current study assessed the role of differential offense category involvement in the link between cognitive ability and arrest. Using structural equation modeling, the current study relied on data from the Pathways ( n = 1,354) and Add Health ( n = 3,605) to address three research questions on the differential etiology of offending category (aggressive and property), the mediating effects of disaggregated offending on the relationship between IQ and arrest, and how those effects vary across samples. The results suggested the negative association between IQ and arrest masked more complex processes encompassing opposing indirect paths through aggressive (negative) and property (positive) offending, suggesting inconsistent mediation. As a result, the total indirect effect of IQ on arrest through offending was suppressed when an aggregated offending measure was used, highlighting the value of disaggregating offending categories.
The accuracy of learning results relies on the evaluation and assessment. The learning goals, including problem solving ability must be aligned with the valid standardized measurement tools. The study on exploring the nature of problem-solving, framework, and assessment in the Indonesian context will make contributions to problem solving assessment in Indonesian educational learning system. This review involved 32 studies that focus on problem-solving test development conducted in Indonesia and have the Indonesian version of the test. All tests are in the scope of certain subjects (mathematics, science, physics, and chemistry) and administered grade 7 to undergraduate level. Each test revealed a good value of reliability. Most of them have acceptable reliability score (r-value between .60 and .80) and high-reliability score (r > .80). Besides, they also showed content and construct validity (acceptable r value in Pearson product moment analysis and INFIT MNSQ index), but additional analysis is needed to fully develop the tests’ empirical evidence. All the tests are categorized as domain specific problem solving which focus on mathematics, science for junior high school, physics, chemistry, and biology. In addition, the topic coverage in the test should be improved and further studies about the measurement of problem-solving and test development are needed in the case of the Indonesian context.
David Sherry's pioneering work on the neuroecology of spatial memory has three characteristics that could inspire studies on other cognitive processes: it was grounded in a robust prior literature in psychology and neuroscience; it identified several natural history contexts in which repeated independent evolution of spatial memory differences had occurred in different clades; it involved a precise cognitive ability with a precise neural substrate. We discuss the application of these three principles to a more domain-general trait—innovation. We argue that targeting the caudolateral nidopallium and its connected areas, favoring problem-solving over reversal learning as an experimental assay, and focusing on situations that involve environmental change, such as urbanization and invasion, can help the study of innovation progress, like the field of spatial memory has since 1989.
Two studies examined factors that would influence people's preferences for interaction with a perpetrator of sexism. In Study 1 (n = 348), participants preferred to interact (being friends or developing a relationship) with an intelligent person regardless of whether or not that person was sexist. Study 2 (n = 614) replicated this finding and confirmed that where a perpetrator had a high level of intelligence, people were more willing to interact with them, regardless of the perpetrator's sex and the perceived commission or non-commission of sexist behavior. Moreover, Study 2 provides evidence that participants' hostile sexism beliefs are a significant covariate of a willingness to interact with unintelligent women. The results are discussed in terms of their implications for the understanding of person perception.
In this paper we propose the multiple complex systems (MCS) approach for
assessing domain-general complex problem-solving (CPS) skills and its processes
knowledge acquisition and knowledge application. After defining the construct
and the formal frameworks for describing complex problems, we emphasise
some of the measurement issues inherent in assessing CPS skills with single tasks
(i.e., fixed item difficulty, low or unknown reliability, and a large impact of
random errors). With examples of the MicroDYN test and the MicroFIN test
(two instances of the MCS approach), we show how to adequately score
problem-solving skills by using multiple tasks. We discuss implications for
problem-solving research and the assessment of CPS skills in general.
Recent advancements in the assessment of Complex Problem Solving (CPS) build on the use of homogeneous tasks that enable the reliable estimation of CPS skills. The range of problems featured in established instruments such as MicroDYN is consequently limited to a specific subset of homogeneous complex problems. This restriction is problematic when looking at domain-specific examples of complex problems, which feature characteristics absent from current assessment instruments (e.g., threshold states). We propose to utilize the formal framework of Finite State Automata (FSA) to extend the range of problems included in CPS assessment. An approach based on FSA, called MicroFIN, is presented, translated into specific tasks, and empirically investigated. We conducted an empirical study (N = 576), (1) inspecting the psychometric features of MicroFIN, (2) relating it to MicroDYN, and (3) investigating the relations to a measure of reasoning (i.e., CogAT). MicroFIN (1) exhibited adequate measurement characteristics and multitrait-multimethod models indicated (2) the convergence of latent dimensions measured with MicroDYN. Relations to reasoning (3) were moderate and comparable to the ones previously found for MicroDYN. Empirical results and corresponding explanations are discussed. More importantly, MicroFIN highlights the feasibility of expanding CPS assessment to a larger spectrum of complex problems.
Education in the twenty-first century must prepare students to meet the challenges of a dynamic and interconnected world. However, assessment of students' skills tends to focus primarily on static tasks. Therefore, it is not known whether knowledge about successful strategies displayed on static tasks can be transferred to interactive and dynamic environments. This study investigated whether students' knowledge of a certain strategy (i.e.; vary-one-thing-at-a-time, VOTAT) that was assessed in a paper-and-pencil-based scientific reasoning task as well as their fluid intelligence and learning orientation would be sufficient to explain variance in the application of the VOTAT strategy in solving an interactive complex problem solving (CPS) task (i.e.; CPS strategy). Furthermore, we analyzed whether CPS strategy mediated the relation between the predictors (i.e.; scientific reasoning, learning orientation, fluid intelligence) and CPS performance. The sample consisted of N = 3,191 Finnish students attending the 6th and 9th grades. Results revealed that all predictors were significantly related to CPS strategy, but a substantial amount of variance in CPS strategy remained unexplained (ΔR2 =.583). Furthermore, CPS strategy mediated the relation between the predictors and CPS performance. Three implications are discussed: Different demands on the problem solver, knowledge transfer from static versus interactive tasks, or metastrategic knowledge may explain the unexplained variance in CPS strategy. Additionally, the results of our mediation analyses emphasize the importance of measuring strategies via logfiles to gain a deeper understanding of determinants of students' CPS performance. Finally, fostering motivational factors such as students' learning orientation yields small improvements in CPS performance.
Ziel des Projekts ist, die Unterschiede zwischen Personen beim Lösen komplexer, alltagsnaher
Probleme aufzuklären. Dazu setzen wir computersimulierte, alltagsorientierte Planspiele
ein. Kognitive und nicht-kognitive Variablen dienen als Prädiktoren der Problemlöseleistung:
Intelligenz, Arbeitsgedächtnis, problemrelevantes Vorwissen, Computerkenntnisse
und Persönlichkeitseigenschaften. Diese Variablen wurden in früheren Untersuchungen
als relevante Einflußgrößen auf den Erfolg bei der Steuerung eines komplexen
Systems identifiziert.
Computer-based problem solving scenarios or "microworlds" are contemporary assessment instru-ments frequently used to assess students' complex problem solving behavior – a key aspect of today's educational curricula and assessment frameworks. Surprisingly, almost nothing is known about their (1) acceptance or (2) psychometric characteristics in student populations. This article introduces the Genetics Lab (GL), a newly developed microworld, and addresses this lack of em-pirical data in two studies. Findings from Study 1, with a sample of 61 ninth graders, show that acceptance of the GL was high and that the internal consistencies of the scores obtained were satisfactory. In addition, meaningful intercorrelations between the scores supported the instru-ment's construct validity. Study 2 drew on data from 79 ninth graders in differing school types. Large to medium correlations with figural and numerical reasoning scores provided evidence for the instrument's construct validity. In terms of external validity, substantial correlations were found between academic performance and scores on the GL, most of which were higher than those ob-served between academic performance and the reasoning scales administered. In sum, this research closes an important empirical gap by (1) proving acceptance of the GL and (2) demonstrating satisfactory psychometric properties of its scores in student populations.
Im Zentrum des Beitrags steht die Bedeutung und Prognosekraft dynamischer Problemlösekompetenz (DPK) für das Kriterium der technischen Problemlöseleistung. Als weitere Prädiktoren wurden in der Untersuchung fluide Intelligenz und technisches Fachwissen berücksichtigt. Die Integration fluider Intelligenz ermöglichte auch eine Untersuchung der Frage, ob sich DKP von fluider Intelligenz unterscheiden lässt. Die auf einer Kfz-Mechatroniker- und Elektronikerstichprobe (n = 129, n = 88) basierenden Analysen belegten die empirische Eigenständigkeit der DKP gegenüber fluider Intelligenz. In der Elektronikerstichprobe erklärte DKP die technische Problemlöseleistung am besten, wobei fluide Intelligenz darüber hinaus inkrementelle Validität zeigte. Bei den Kfz-Mechatronikern korrelierte nur fluide Intelligenz mit dem Kriterium, allerdings schwach. Wurde Fachwissen einbezogen, verschwand sowohl die Bedeutung der DKP als auch fluider Intelligenz. Allerdings übte bei den Elektronikern fluide Intelligenz einen über Wissen vermittelten indirekten Einfluss auf das Kriterium aus. Die nicht ganz eindeutige Befundlage, stichprobenspezifische Unterschiede sowie theoretische Ansätze zur Erklärung der beobachteten Effekte werden diskutiert.
This article addresses two unsolved measurement issues in dynamic problem solving (DPS) research: (a) unsystematic construction of DPS tests making a comparison of results obtained in different studies difficult and (b) use of time-intensive single tasks leading to severe reliability problems. To solve these issues, the MicroDYN approach is presented, which combines (a) the formal framework of linear structural equation models as a systematic way to construct tasks with (b) multiple and independent tasks to increase reliability. Results indicated that the assumed measurement model that comprised three dimensions, information retrieval, model building, and forecasting, fitted the data well (n = 114 students) and could be replicated in another sample (n = 140), showing excellent reliability estimates for all dimensions. Predictive validity of school grades was excellent for model building but nonexistent for the other two MicroDYN dimensions and for an additional measure of DPS. Implications are discussed.
The growing popularity of meta-analysis has focused increased attention on the statistical models analysts are using and the assumptions underlying these models. Although comparisons often have been limited to fixed-effects (FE) models, recently there has been a call to investigate the differences between FE and random-effects (RE) models, differences that may have substantial theoretical and applied implications (National Research Council, 1992). Three FE models (including L. V. Hedges & I. Olkin's, 1985, and R. Rosenthal's, 1991, tests) and 2 RE models were applied to simulated correlation data in tests for moderator effects. The FE models seriously underestimated and the RE models greatly overestimated sampling error variance when their basic assumptions were violated, which caused biased confidence intervals and hypothesis tests. The implications of these and other findings are discussed as are methodological issues concerning meta-analyses. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the "file drawer problem" is that journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show nonsignificant results. Quantitative procedures for computing the tolerance for filed and future null results are reported and illustrated, and the implications are discussed. (15 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
There are 2 families of statistical procedures in meta-analysis: fixed- and random-effects procedures. They were developed for somewhat different inference goals: making inferences about the effect parameters in the studies that have been observed versus making inferences about the distribution of effect parameters in a population of studies from a random sample of studies. The authors evaluate the performance of confidence intervals and hypothesis tests when each type of statistical procedure is used for each type of inference and confirm that each procedure is best for making the kind of inference for which it was designed. Conditionally random-effects procedures (a hybrid type) are shown to have properties in between those of fixed- and random-effects procedures. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The ability to make decisions within a few seconds to a couple of minutes is the hallmark of many applied situations ranging from air traffic control to stock market trading. The 5 studies reported here explore the hypothesis that there are abstract psychological demands common to many rapid decision-making situations. The authors suggest that these characteristics can be used to identify people who are good rapid decision makers. People were trained to operate computer simulations of the job skills involved in air traffic control and 2 aspects of public safety dispatch. Although these tasks are very different on the surface, they all require decision making under time pressure. Performance was successfully predicted from performance on an abstract decision-making task. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In der Informations- und Wissensgesellschaft, in der sich (nicht nur) Schülerinnen und Schüler zurechtfinden müssen, ist die Fähigkeit, Informationen gezielt und schnell zu finden oder abzurufen, zu verarbeiten und zu kommunizieren, eine Kompetenz, die in immer mehr schulischen und außerschulischen Bereichen immer stärker an Bedeutung gewinnt. Vor dem Hintergrund dieser alltäglichen Anforderungen werden Fähigkeiten im Umgang mit modernen Informations- und Kommunikationstechnologien wie dem Computer und dem Internet neben der Lesefähigkeit zu einer Schlüsselkompetenz (Schulz-Zander, 2000), die sich auch immer mehr als eine der zentralen Fähigkeiten im Berufsleben etabliert (Russon, Josefowitz, & Edmonds, 1994).
There are 2 families of statistical procedures in meta-analysis: fixed- and random-effects procedures. They were developed for somewhat different inference goals: making inferences about the effect parameters in the studies that have been observed versus making inferences about the distribution of effect parameters in a population of studies from a random sample of studies. The authors evaluate the performance of confidence intervals and hypothesis tests when each type of statistical procedure is used for each type of inference and confirm that each procedure is best for making the kind of inference for which it was designed. Conditionally random-effects procedures (a hybrid type) are shown to have properties in between those of fixed- and random-effects procedures.
Figurale Matrizenaufgaben stellen eine bedeutsames Aufgabenformat im Bereich der Diagnostik kognitiver Fähigkeiten dar. Während die Konstruktion der Aufgabenstämme in der Literatur relativ gut dokumentiert ist, finden sich nur wenige Hinweise zur Konstruktion der für die Vorgabe benötigten Distraktoren. Dies erweist sich als problematisch, da ein Einfluss der Distraktorgestaltung auf das Bearbeitungsverhalten von Testpersonen vermutet werden kann. Die Matrizenkonstruktionsaufgabe stellt ein alternatives Format zur Vorgabe figuraler Matrizen dar. In diesem müssen die Testpersonen die Lösung der Aufgaben in einer computerisierten Testumgebung selbst konstruieren, weshalb auf die Verwendung von Distraktoren verzichtet werden kann. Im Rahmen der Studie wurden einer Stichprobe von 318 Studierenden 40 Matrizenkonstruktionsaufgaben vorgegeben. Die Ergebnisse deuten auf eine gute psychometrische Eignung des Aufgabenformats hin: Es konnte eine große Bandbreite an Aufgabenschwierigkeiten realisiert werden. Die Aufgaben erfassen eine latente Fähigkeitsdimension und weisen eine gute Konstrukt- und Kriteriumsvalidität auf. In der Diskussion werden Perspektiven zur Weiterentwicklung des Aufgabenformats besprochen.
Parathyroid hormone-like hormone (PTHLH) is an important chondrogenic regulator; however, the gene has not been directly linked to human disease. We studied a family with autosomal-dominant Brachydactyly Type E (BDE) and identified a t(8;12)(q13;p11.2) translocation with breakpoints (BPs) upstream of PTHLH on chromosome 12p11.2 and a disrupted KCNB2 on 8q13. We sequenced the BPs and identified a highly conserved Activator protein 1 (AP-1) motif on 12p11.2, together with a C-ets-1 motif translocated from 8q13. AP-1 and C-ets-1 bound in vitro and in vivo at the derivative chromosome 8 breakpoint [der(8) BP], but were differently enriched between the wild-type and BP allele. We differentiated fibroblasts from BDE patients into chondrogenic cells and found that PTHLH and its targets, ADAMTS-7 and ADAMTS-12 were downregu-lated along with impaired chondrogenic differentiation. We next used human and murine chondrocytes and observed that the AP-1 motif stimulated, whereas der(8) BP or C-ets-1 decreased, PTHLH promoter activity. These results are the first to identify a cis-directed PTHLH downregulation as primary cause of human chondrodysplasia.
This paper reviews research on dynamic decision making, i.e., decision making under conditions which require a series of decisions, where the decisions are not independent, where the state of the world changes, both autonomously and as a consequence of the decision maker's actions, and where the decisions have to be made in real time. It is difficult to find useful normative theories for these kinds of decisions, and research thus has to focus on descriptive issues. A general approach, based on control theory, is proposed as a means to organize research in the area. An experimental paradigm for the study of dynamic decision making, that of computer stimulated microworlds, is discussed, and two approaches using this paradigm are described: the individual differences approach, typical of German work in the tradition of research on complex problem solving, and the experimental approach. In studies following the former approach, the behavior of groups differing in performance is compared, either with respect to strategies or with respect to performance on psychological tests. The results show that there are wideinterindividual differences in performance, but no stable correlations between performance in microworlds and scores on traditional psychological tests have been found. Experimental research studying the effects of system characteristics, such as complexity and feedback delays, on dynamic decision making has shown that decision performance in dynamic tasks is strongly affected by feedback delays and whether or not the decisions have side effects. Although neither approach has led to any well-developed theory of dynamic decision making so far, the results nevertheless indicate that we are now able to produce highly reliable experimental results in thelaboratory, results that agree with those found in field studies of dynamic decision making. This shows that an important first step towards a better understanding of these phenomena has been taken.
Complex Problem Solving (CPS) describes skills frequently needed in everyday life such as the use of new technological devices. Therefore, CPS skills constitute an increasingly important individual ability that needs theoretically embedded, reliable and validated measurement devices. The present article shows that current tests do not sufficiently address the requirement of a theory-based assessment. An integrative approach, the Action Theoretical Problem Space Model by Rollett (2008), is introduced and used to demonstrate how a theoretical framework can influence and inform test development. Implications for the assessment of CPS and its potential are discussed.
Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further, CPS assessments show incremental validity beyond tests of other cognitive abilities when predicting relevant outcomes. However, these empirical accounts have relied on single CPS assessment instruments. We do not know whether these findings will generalize to the construct level across different CPS assessment instruments. To answer this question, we tested a sample of N = 339 German university students who completed three CPS assessment instruments based on multiple complex systems (MicroDYN, the Genetics Lab, and MicroFIN) and the matrices subtest of the Intelligence Structure Test as measure of reasoning. Students further reported their school grades. Analyses including latent multitrait–multimethod models provided support for the conceptualization of CPS as a complex cognitive ability. Results indicated that different CPS assessment instruments showed sufficient convergent validity (with a consistency mostly between .50 and .60). In addition, we found evidence for the divergent validity of CPS from reasoning (reasoning predicted two CPS facets, knowledge and control, βKNOW = .49 and βCON = .53, respectively). In the prediction of academic achievement, CPS explained variance in natural science grades after we controlled for reasoning (βCPS = .22), whereas social science grades were not predicted. Our findings suggest that the validity of CPS generalizes across different measurement instruments.
Meta-analysis is a statistical tool for estimating the mean and variance of underlying population effects from a collection of empirical studies addressing ostensibly the same research question. Meta-analysis has become an increasing popular and valuable tool in psychological research, and major review articles typically employ these methods. This article describes the process of conducting meta-analysis: selecting articles, developing inclusion criteria, calculating effect sizes, conducting the actual analysis (including information on how to do the analysis on popular computer packages such as IBM SPSS and R) and estimating the effects of publication bias. Guidance is also given on how to write up a meta-analysis.
In recent years, computer-based complex scenarios have been increasingly used for personnel assessment. The simulation of important facets of real job demands promises high content validity, high acceptance, and the assessment of key qualifications for challenging management tasks such as aptitudes of thinking in complex systems in case of intransparency. The potential behind observing and analyzing the subject’s behavior when interacting with the complex system is discussed in the context of the principles of Brunswik symmetry. Besides more precision in predicting job performance, a description of the problem solver’s behavior facilitates a better tailoring of human resources development measures. The reflections and arguments are checked against empirical results from N = 110 subjects who processed the scenario FSYS. The relations between scenario scales and external criteria corresponded to the expectations. Both total performance and behavior in the scenario can explain additional variance of job-equivalent performance indicators in addition to general intelligence.