PreprintPDF Available

Which Divergent Thinking Index is More Associated with Problem Construction Ability? The Role of Flexibility and Task Nature

Preprints and early-stage research may not have been peer reviewed yet.
Which Divergent Thinking Index is More Associated with Problem
Construction Ability? The Role of Flexibility and Task Nature
Authors: Ahmed M. Abdulla Alabbasi, Roni Reiter-Palmon, Zainab M. Sultan
Arabian Gulf University
University of Nebraska at Omaha
Correspondence concerning this article should be addressed to Ahmed M. Abdulla Alabbasi,
PhD, Department of Gifted Education, Arabian Gulf University, Email:
Draft version: October 1st, 2020. This paper is under review
Problem construction and divergent thinking (DT) are considered to be indicators of creative
potential. Previous studies, with different goals, suggest a positive correlation between problem
construction and DT. However, none of these works have explicitly examined which index of
DT is more associated with problem construction. The current investigation examined the
association between problem construction and three main indexes of DT: fluency, flexibility, and
originality. It also tested whether such a relation differs based on task nature (verbal versus
figural). The sample consisted of 90 sixth graders who completed three tests: (a) a verbal DT
test, (b) a figural DT test, and (c) a problem construction test. Correlational analysis showed that
flexibility was highly correlated with problem construction in the verbal DT test, whereas
originality was highly correlated with problem construction in the figural test. The association
between problem construction and verbal versus figural DT significantly differed in all DT
indexes. This finding suggests that figural and verbal tasks assess DT in a different way, which
was confirmed by a canonical correlation analysis. Finally, results of a multiple regression
analysis showed that verbal DT significantly explained 59% of the variance in fluency scores in
problem construction and 60% in originality scores in problem construction. Meanwhile, figural
DT explained 8% to 9% of the variability in fluency and originality scores in problem
construction. As suggested by experts in the field of problem construction, the role of flexibility
in problem construction is a fertile area to be considered in future studies.
Keywords: problem construction, divergent thinking, task nature, ill-defined problems
1. Introduction
The significance of problem construction is widely recognized, and it is considered to be
the first step in any creative problem-solving effort (e.g., Mumford et al., 1991; Osborn, 1953;
Treffinger & Isaksen, 2005; Wallas, 1926). Some scholars have considered problem construction
to be more important than problem solving (e.g., Csikszentmihalyi, 1988; Einstein & Infeld,
1938; Mackworth, 1965; Wertheimer, 1945). A recent meta-analysis showed that problem
construction and creativity are significantly positively correlated (Abdulla et al., 2020), and the
effect size is the nearly same as that reported in a meta-analysis (Kim, 2008) on the correlation
between divergent thinking (DT) and creative achievement.
The history of studying problem construction started with the seminal work of
Csikszentmihalyi and Getzels (1971) that aimed to assess problem construction in art students.
Since then, the field of creativity research has shown a growing interest in studying problem
construction. However, this important cognitive process has not receiving as much attention from
researchers in the creativity field relative to DT or ideation. To date, there is only one edited
book on problem construction (Runco, 1994a). Figures 1 and 2 show a comparison between the
number of hits when three terms (problem solving, DT, and problem construction) were searched
from 1960 to 2020 in the following databases: ERIC, Academic Search Complete, Academic
Search Premium, and ProQuest Central. Advanced searched was used, and the following
combinations of keywords were used: (a) problem solving AND creativity OR creative, (b) DT
AND creativity OR creative, and (c) all terminologies of problem construction reported in
Abdulla and Cramond (2018; see page 202) AND creativity OR creative. The search included
titles only, and redundant works were eliminated.
Figures 1 and 2 around here
While the results indicate that there is indeed growth for research on problem
construction, and it is increasing at a higher rate relative to other research, the sheer number of
papers is still relatively small. This is problem for a number of reasons. First, problem
construction is an essential element in all models of creative process. Wallas (1926), Osborn
(1953), and Merrifield et al. (1962) called such a step preparation, whereas others used different
terminologies, such as problem finding (PF), problem identification, and idea finding (Amabile,
1988; Mumford et al., 1991; Reiter-Palmon et al., 1997; Runco, 1994a; Silverman, 1985).
Regardless of the terms used, they all refer to a situation in which the problem is ill-defined. In
addition, some definitions of creativity explicitly include problem construction (e.g., Cropley,
2011; Runco, 2014; Torrance, 1966). Second, advancing innovation and creativity for the benefit
of business and society is depends on those who can sense gaps in knowledge and discover ill-
defined problems to solve them in a novel way. A meta-analysis by Ma (2009) indicated that
problem construction was the strongest predictor of creativity providing support for this
However, this is not to say that other cognitive processes are less important than problem
construction for creativity and creative thinking. The argument here is that problem construction
is an essential cognitive ability that deserves more attention from researchers in the creativity
Nonetheless, after six decades of systematic study of problem construction, we have a
better understanding of the relation between problem construction and some cognitive processes,
such as inter and intrapersonal evaluation (Basadur, 1995; Mumford et al., 1991; Runco, 1994b),
problem solving (Arlin, 1975; Bouchard & Drauden, 1976; Carson & Runco, 1999), selective
encoding and combination, selective comparison (Brugman, 1995), and DT (e.g., Arlin, 1975;
Artley et al., 1980; Chand & Runco, 1993; Csikszentmihalyi & Getzels, 1971; Hoover, 1994;
Wakefield, 1985).
Among these cognitive processes, creativity literature suggests DT to be associated with
problem construction. Many researchers have sought to explore such an association (Ambrosio,
1993; Arlin, 1975; Chand & Runco, 1993; Hoover, 1994; Hoover & Feldhusen, 1990; Reiter-
Palmon et al., 1997; Runco & Acar, 2010; Wakefield, 1985); however, these studies have
yielded conflicting findings (see Table 1). Such conflict might be due to (a) the DT and problem
construction assessments used in the studies, (b) the DT skills targeted (i.e., fluency, flexibility,
and originality), and (c) the task nature (verbal vs. figural tasks).
Table 1 around here
For example, Ambrosio (1993) used the figural form of the Torrance Test of Creative
Thinking (TTCT) and reported a negative relation between DT and problem construction quality
(r= -.08), and a significant positive relation between DT and problem construction quantity (r =
.41). Arlin (1975) reported a positive relation between problem construction quality and adaptive
flexibility (r = .26), and a negative relation between problem construction quality and
spontaneous flexibility (r = -.09) and fluency (r= -.18). Runco and Acar (2010) showed a
significant positive relation between verbal DT test and problem construction (r ranged from .29
to .44).
As shown in Table 1, only one study looked at the relation between problem construction
and elaboration (i.e., Arlin, 1975), and only two studies examined the relation between problem
construction and flexibility (Arlin, 1975; Hoover, 1994). It has been suggested that the specific
DT measure and index used (verbal or figural; fluency, originality or flexibility) can affect the
results (Hornberg & Reiter-Palmon,2017; Reiter-Palmon et al., 2019). Further, much of the
research utilizing DT tasks tends to use fluency. Therefore, one of the contributions of this study
is to evaluate flexibility and originality in addition to fluency. Flexibility in particular may be
critical for problem construction. According to Mumford et al. (1994), “Flexibility in applying
selection and screening strategies may contribute to individual differences in problem finding
skills.” In addition to evaluating the relationships with different indices of DT, our study also
determined whether the relationship between DT and problem construction will differ based on
the nature of the DT task, verbal or figural.
2. Methods
2.1. Participants and Procedures
The participants included 90 students in the sixth grade (42 boys and 48 girls). We
randomly collected data from two public schools after receiving the official approval from the
Directorate for Scientific Research at the Ministry of Education. Prior to data collection, each
participant was asked to read and sign a consent form prepared by the authors of this study.
Before the tests were administered, the first author visited the two schools and spent one
class (40 minutes) with the students to create familiarity with them (no DT or problem
construction activities were discussed). By the end of this meeting, the author informed the study
participants of a subsequent visit in the next week to administer some activities that could show
the students’ creativity. The test was administered in the second class/session (8:45 am), as
recommended by the teachers. The participants were informed that they will obtain the results of
their “creative activity score” after the authors had scored them. The tests were conducted in a
game-like condition (untimed), following the results of a recent meta-analysis that showed that
the untimed condition significantly improves students’ performance in DT tests (Said-Metwaly
et al., 2020).
2.2. Instruments
2.2.1. Uses Test
We administered three tasks from the Uses test (Wallach & Kogan, 1965): (a) uses for a
spoon, (b) uses for a wheel, and (c) uses for a toothbrush. The directions for the task are as
In this game, I am going to name an object—any kind of object, like a light bulb
or the floor—and it will be your job to tell lots of different ways that the object
could be used. Any object can be used in a lot of different ways. Remember, think
of all the different ways you could use the object that I mentioned (Wallach &
Kogan, 1965, p. 31).
Responses to the Uses test were scored for fluency, flexibility, and originality. A total of
199 different ideas were generated for the spoon task, 124 different ideas for the wheel task, and
181 different ideas for the toothbrush task. Fluency was defined as the number of different ideas
related to a given stimulus. Flexibility was defined as the number of different categories or shifts
between ideas. Based on participants’ responses, 12 categories were created for the spoon task,
11 for the wheel task, and eight for the toothbrush task (see Table 2). Finally, originality was
scored based on a 3% cutoff criterion.
2.2.2. Figural Test
We administered the Figural test from the Runco Creativity Assessment Battery, which
consists of three tasks, similar to the Wallach and Kogan Line Meaning test (Wallach & Kogan,
1965, p. 36). The Figural test includes three shapes: Spiral, Blocks, and Lines. The directions are
as follows:
Look at the figure below. What do you see? List as many things as you can that
this figure might be or represent. This is NOT a test. Think of this as a game and
have fun with it! The more ideas you list, the better (Runco et al., 2016, p. 6).
The Figural test was also scored for fluency, flexibility, and originality. A total of
105 different ideas were generated for the first task (Spiral), 111 for the second task
(Blocks), and 53 for the third task (Lines). For flexibility, 10 categories were created in
the Spiral task, 11 categories in the Blocks task, and 11 categories in the Lines task.
Originality was scored using the same method as the Uses test (i.e., 3% cutoff).
Table 2 around here
2.2.3. Problem Generation Test
We used the Problem Generation (PG) test to assess participants’ problem construction
ability (Okuda et al., 1991; The PG test consists of three
open-ended tasks that require participants to list as many problems as they can. This kind of
assessment can be considered as a problem discovery test given than participants are asked to
generate as many problems as they can (real or hypothetical) without any instruction to evaluate
any of their ideas, or that their ideas will be evaluated. The three PG tasks are related to (a) home
and school, (b) life situations, and (c) health and well-being. An example of a PG task is as
List problems along with your friends, peers, or schoolmates (any individual who
is approximately the same age as yourself). These problems can be real, or they
can be hypothetical and imaginary. Do not limit yourself; the more problems you
can list, the better. You can think of problems that exist now or those that might
exist in future.
The PG test was scored for fluency and originality. The reason for not scoring for
flexibility was that the nature of the tasks was extremely diverse. PG tasks asked participants to
list real and hypothetical problems, which made it difficult to score for flexibility. The
participants generated 235, 253, and 269 different ideas in the first, second, and third tasks,
respectively. The same method was utilized for scoring fluency and originality in the PG test.
Finally, the students were asked to fill out a demographic questionnaire with items on
their age and sex.
3. Results
3.1. Reliability
Both inter-class correlation (ICC) and internal consistency reliability were calculated for
the three tests used. As mentioned above, DT verbal and figural tests were scored for fluency,
flexibility, and originality, whereas problem construction was scored for fluency and originality.
Table (3) shows the reliability coefficients for the three tests. For the ICC, a two-way mixed
model absolute agreement was calculated.
Table 3 around here
As moderate reliability coefficients were observed in the Figural test, we looked at each
task of the Figural test. The Lines task showed lowered reliability coefficients in the Figural test.
After removing the Lines test, the reliability coefficients in the Figural test improved. Therefore,
the Lines task scores were eliminated in all subsequent analyses.
3.2. Correlation Analysis
As shown in Table 4, the pattern of correlations between DT and problem construction
differed in the verbal versus figural tests. In the verbal DT, the highest correlation was found
between flexibility and (a) problem construction fluency (r = .75, p <.001) and (b) problem
construction originality (r = .74, p <.001), followed by the correlation between (c) verbal fluency
and problem construction fluency and originality (r = .73, p <.001). Originality in DT was also
significantly correlated with (a) fluency in problem construction (r = .67, p <.001) and (b)
originality in problem construction (r = .68, p <.001).
Meanwhile, the magnitude of the correlation between figural DT and problem
construction was much lower compared with verbal DT, which ranged from .22 to .30. We tested
whether the relation between problem construction and DT was statistically different based on
the task nature (i.e., problem construction and verbal DT vs. problem construction and figural
DT; Lee & Preacher, 2013; Steiger, 1980, pp. 237–248). In transforming the Pearson correlation
coefficients to Fisher’s z, the results showed that the relation between fluency in problem
construction and DT (verbal vs. figural) was significantly different in terms of fluency (z = 4.48,
p <.001), flexibility (z = 4.47, p <.001), and originality (z = 3.31, p <.001); and originality in
problem construction and DT (verbal vs. figural) was significantly different in terms of fluency
(z = 4.26, p <.001), flexibility (z = 4.18, p <.001), and originality (z = 3.28, p <.001). Thus, it can
be concluded that the task nature significantly influenced the relationship between problem
construction and DT such that verbal tasks were more strongly related to problem construction.
A canonical correlation analysis (CCA) was conducted to examine the relation between
the verbal and figural DT tests. The results showed no significant correlation between the verbal
and figural DT in all DT indexes: (a) fluency [Wilks’ Lambda = .850, Rc = .35, p = .13], (b)
flexibility [Wilks’ Lambda = 970, Rc = .17, p = .60], and originality [Wilks’ Lambda = 998, Rc =
.04, p = .70].
Table 4 around here
3.3. Regression Analysis
Multiple regression analysis was conducted to evaluate the prediction of the dependent
variables (i.e., figural and verbal DT) on problem construction scores (see Table 5). Verbal DT
indices were significantly related to problem construction fluency [F(3,86) = 41.94, p <.001, R2
= .59] and problem construction originality [F(3,86) = 42.21, p < .001, R2 = .60]. Meanwhile,
figural DT was not statistically related to problem construction fluency [F(3,86) = 2.34, p = .079,
R2 = .08] but significantly related to problem construction originality [F(3,86) = 2.78, p = .046,
R2 = .09].
Table 5 around here
3.4. CCA
We performed a CCA for each DT index and problem construction. The first set included
fluency in the Uses and Figural tests, and the second set included fluency and originality in
problem construction. The results showed that fluency in the Uses test significantly correlated
with problem construction fluency and originality [Rc = .74, p <.001], whereas fluency in the
Figural test was not significantly correlated with problem construction fluency and originality
[Rc = .16, p = .13].
In the second CCA, flexibility in the Uses and Figural tests was entered as set 1. The
results showed that flexibility in the Uses test significantly correlated with problem construction
fluency and originality [Rc = .75, p <.001], whereas flexibility in the Figural test did not
significantly correlate with problem construction fluency and originality [Rc = .14, p = .20].
Finally, in the third CCA, originality scores in the Uses and Figural tests were entered as
set 1. The results indicated a significant correlation between originality in the Uses test and
problem construction [Rc = .69, p <.001]. We found a non-significant relation between
originality scores in the Figural test and between problem construction fluency and originality
[Rc = .12, p = .25].
4. Discussion
Studies on problem construction have suggested that some cognitive processing skills are
related to problem construction, and have also paid attention to DT along with problem
construction. Previous research has examined the association between problem construction and
DT, however, none of these studies have explicitly aimed to examine multiple DT indices and
tasks and the relative association with problem construction. The results of our study showed that
flexibility, which was neglected in previous works on problem construction and DT, was highly
correlated with problem construction compared with fluency and originality, especially when
verbal tasks were administered. Arlin (1975) described problem finders as “flexible thinkers.”
Our findings support this claim and suggest that good problem finders are characterized by an
ability to shift between ideas and keep formulating different ideas to find a novel problem that
needs to be solved. However, more studies are needed to elucidate fully the role of flexibility in
problem construction.
The other important finding in the present study is the difference in the relation between
problem construction and DT based on the task nature. We found that problem construction can
be better assessed using verbal tasks than figural ones. This finding is partly supported by
Abdulla et al. (2020), in which problem construction in the writing domains is highly correlated
with creativity (r = .36) compared with art (r = .20). One issue that future studies might consider
is whether highly capable problem finders are overrepresented in writing versus in other
domains. It is possible that the strong correlations between problem construction and DT in the
writing domain is in part a result of both requiring writing as the mode of generated responses.
Previous research suggests that the task and index of measurement is important in understanding
these types of relationships (Reiter-Palmon et al., 2019; Reiter-Palmon & Schoenbeck, in press).
Additional research evaluating whether these relationships hold with other verbal DT measures
such as consequences as well as other figural DT measures would help clarify this issue.
It is also important to note problem construction is not a single process. Many scholars
have differentiated known problem construction skills, such as problem definition, problem
identification, and problem discovery (e.g., Getzels, 1982; Mumford et al., 1994; Runco, 1994b).
The differences between these and other problem construction-related skills lie in (a) how well-
or ill-defined a problem is and (b) the degree to which ideation and evaluation are required
(Abdulla & Cramond, 2018). In our study, problem construction was operationalized as problem
discovery, which has been considered as the highest level of PF hierarchy (Abdulla & Cramond,
2018). We asked the participants to produce as many ideas as they could think of for different
and novel problems related to home and school, life situations, and health and well-being. They
were explicitly instructed to think of real or hypothetical problems, and to think of problems that
exist or those that might exist in the future. Previous work on instructions indicates that the focus
of instructions can have a significant impact on the outcomes associated with DT measures
(Acar, Runco, & Park, 2020; Nusbaum, Silvia, & Beaty, 2014; Runco, Illies, & Reiter-Palmon,
2005). The focus of the instructions is on both different and novel ideas may be one reason why
flexibility was such an effective predictor. This point is crucial because different results might be
attributed to the level of ideation and evaluation required in the problem construction task (Lee
& Cho, 2007). Thus, our findings might not be generalized to all problem construction tasks,
except those that require problem discovery where there is limited role for evaluation.
Abdulla, A. M., & Cramond, B. (2018). The creative problem finding hierarchy: A suggested
model for understanding problem finding. Creativity. Theories-Research-Applications, 5,
Abdulla, A. M., Paek, S. H., Cramond, B., & Runco, M. A. (2020). Problem finding and
creativity: A meta-analytic review. Psychology of Aesthetics, Creativity, and the Arts, 14,
Acar, S., Runco, M. A., & Park, H. (2020). What should people be told when they take a
divergent thinking test? A meta-analytic review of explicit instructions for divergent
thinking. Psychology of Aesthetics, Creativity, and the Arts, 14, 39–
Amabile, T. (1988). A model of creativity and innovation in organizations. Research in
Organizational Behavior, 10, 123–169.
Ambrosio, A. L. (1994). Social problem finding: A cognitive-developmental perspective.
Dissertation Abstracts International Section A, 54, 3700.
Arlin, P. K. (1975). A Cognitive process model of problem finding. Educational Horizons,
Winter, 99-106.
Artley, N. L., Van Horn, R., Friedrich, D. D., & Carroll, J. L. (1980). The relationship between
problem finding, creativity and cognitive style. Creative Child & Adult Quarterly, 5, 20–
Basadur, M. (1995). Optimal ideation-evaluation ratios. Creativity Research Journal, 8, 63–
Bouchard, T. J., & Drauden, G. (1976). Discovery-oriented behavior and problem solving. The
Journal of Psychology: Interdisciplinary and Applied, 92(2), 243–
Brugman, G. (1995). The discovery and formulation of problems. European Education, 27(1),
Carson, D. K., & Runco, M. A. (1999). Creative problem solving and problem finding in young
adults: Interconnections with stress, hassles, and coping abilities. The Journal of Creative
Behavior, 33, 167–190.
Chand, I., & Runco, M. A. (1993). Problem finding skills as components in the creative
process. Personality and Individual Differences, 14(1), 155–
Cropley, A. J. (2011). Definitions of creativity. In M. A. Runco & S. R. Pritzker (Eds.),
Encyclopedia of creativity (pp. 511-524). San Diego, CA: Academic Press.
Csikszentmihalyi, M. (1988). Solving a problem is not finding a new one: A reply to Simon. New
Ideas in Psychology, 6, 183-186.
Csikszentmihalyi, M., & Getzels, J. W. (1971). Discovery-oriented behavior and the originality
of creative products: A study with artists. Journal of Personality and Social Psychology,
19, 47–52.
Einstein, A., & Infeld, L. (1938). The evolution of physics. New York: Simon & Schuster.
Frederiksen, N., & Evans, F. R. (1974). Effects of models of creative performance on ability to
formulate hypotheses. Journal of Educational Psychology, 66, 67–
Getzels, J. W. (1982). The problem of the problem. In R. Hogarth (Ed.), New directions for
methodology of social and behavioral science: Question framing (pp. 37-49). San
Francisco, CA: Jossey-Bass.
Hoover, S. M. (1994). Scientific problem finding in gifted fifth-grade students. Roeper Review,
16, 156–159.
Hoover, S. M., & Feldhusen, J. F. (1990). The scientific hypothesis formulation ability of gifted
ninth-grade students. Journal of Educational Psychology, 82(4), 838–
Jaarsveld, S., Lachmann, T., & van Leeuwen, C. (2012). Creative reasoning across
developmental levels: Convergence and divergence in problem creation. Intelligence, 40,
Kim, K. H. (2008). Meta-analyses of the relationship of creative achievement to both IQ and
divergent thinking test scores. The Journal of Creative Behavior, 42, 106–
Lee, H., & Cho, Y. (2007). Factors affecting problem finding depending on degree of structure
of problem situation. Journal of Educational Research, 101(2), 113–123.
Lee, I. A., & Preacher, K. J. (2013, September). Calculation for the test of the difference between
two dependent correlations with one variable in common [Computer software]. Available
Ma, H.-H. (2009). The effect size of variables associated with creativity: A meta-
analysis. Creativity Research Journal, 21, 30–
Mackworth, N. H. (1965). Originality. American Psychologist, 20(1), 51–
Merrifield, P. R, Guilford, J. P., Christensen, P. R., & Frick, J. W. (1962). The role of intellectual
factors in problem solving. Psychological Monographs: General and Applied, 76, 1–
Mumford, M. D., Mobley, M. I., Uhlman, C. E., Reiter-Palmon, R., & Doares, L. M. (1991).
Process analytic models of creative capacities. Creativity Research Journal, 4, 91–
Mumford, M. D., Reiter-Palmon, R., & Redmond, M. R. (1994). Problem construction and
cognition: Applying problem representations in ill-defined domains. In M. A. Runco
(Ed.), Problem finding, problem solving, and creativity (pp. 3–39). Ablex Publishing.
Nusbaum, E. C., Silvia, P. J., & Beaty, R. E. (2014). Ready, set, create: What instructing people
to “be creative” reveals about the meaning and mechanisms of divergent
thinking. Psychology of Aesthetics, Creativity, and the Arts, 8, 423–
Okuda, S. M., Runco, M. A., & Berger, D. E. (1991). Creativity and the finding and solving of
real-world problems. Journal of Psychoeducational Assessment, 9, 45–
Osborn, A. F. (1953). Applied imagination. Scribner's.
Reiter-Palmon, R., Forthmann, B., & Barbot, B. (2019). Scoring divergent thinking tests: A
review and systematic framework. Psychology of Aesthetics, Creativity, and the Arts, 13,
Reiter-Palmon, R., Mumford, M. D., O'Connor Boes, J., & Runco, M. A. (1997). Problem
construction and creativity: The role of ability, cue consistency and active
processing. Creativity Research Journal, 10, 9-23.
Runco Creativity Assessment Battery (2020, April). Creativity Testing Services. Retrieved from
Runco, M. A. (1994b). Conclusions concerning problem finding, problem solving, and creativity.
In M. A. Runco (Ed.), Problem finding, problem solving, and creativity (pp. 272–290).
Ablex Publishing.
Runco, M. A. (2014). Creativity: Theories and themes: Research, development, and practice.
San Diego, CA: Academic Press.
Runco, M. A., & Acar, S. (2010). Do tests of divergent thinking have an experiential
bias? Psychology of Aesthetics, Creativity, and the Arts, 4, 144–
Runco, M. A., Abdulla, A. M., Peak, S., Aljasim, F. A., & AlSuwaidi, H. N. (2016). Which test
of divergent thinking is best? Creativity. Theories-Research-Applications, 3, 4-18.
Runco, M. A., Illies, J. J., & Reiter-Palmon, R. (2005). Explicit instructions to be creative and
original: A comparison of strategies and criteria as targets with three types of divergent
thinking tests. Korean Journal of Thinking & Problem Solving, 15, 5–15.
Said-Metwaly, S., Fernández-Castilla, B., Kyndt, E., & Van den Noortgate, W. (2020). Testing
conditions and creative performance: Meta-analyses of the impact of time limits and
instructions. Psychology of Aesthetics, Creativity, and the Arts, 14, 15–
Silverman, B.G. (1985). The use of analogs in the innovation process: A software engineering
protocol analysis. IEEE Transactions on Systems, Man, & Cybernetics, 15, 30-44.
Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological
Bulletin, 87, 245–251.
Torrance, E. P. (1966). Torrance Tests of Creative Thinking: Norms technical manual (Research
Edition). Princeton, NJ: Personnel Press.
Treffinger, D. J., & Isaksen, S. G. (2005). Creative Problem Solving: The history, development,
and implications for gifted education and talent development. Gifted Child Quarterly, 49,
Wakefield, J. F. (1985). Towards creativity: Problem finding in a divergent-thinking
exercise. Child Study Journal, 15, 265–270.
Wallach, M. A., & Kogan, N. (1965). Modes of thinking in young children: A study of the
creativity-intelligence distinction. Holt, Rinehart & Winston.
Wallas, G. (1926). The art of thought. New York: Harcourt Brace and World.
Wertheimer, M. (1945). Productive thinking. Harper.
Table 1
Selected Studies on the Relation Between DT and Problem Construction
DT Test
DT Skills
Frederiksen & Evans
Consequences Test
Arlin (1975)
Puzzles Test
Expressional Fluency
Associational Fluency
Ideational Fluency
Adaptive Flexibility
Artley et al. (1980)
Wakefield (1985)
DT Test developed by
the author
Ambrosio (1993)
Chand & Runco
Presented Problems
Test (PP)
Problem Generation
Test (PG)
Discovered DT Test
Table 1 Continued
Selected Studies on the Relation Between DT and Problem Construction
DT Test
DT Skills
Hoover (1994)
Reiter-Palmon et al.
Consequences Test
(Guilford, 1965)
Ideational Fluency
Lee & Cho (2007)
Runco & Acar
Uses Test
Jaarsveld et al.
Test for Creative
Production (TCT-DP)
Note: TTCT = Torrance Test of Creative Thinking; DT = Divergent Thinking; PF = Problem Finding
Table 2
Flexibility Categories for Verbal and Figural Tests
Task Nature
(1) carry or move things, (2) care and beauty, (3) design and decoration, (4) artistic uses, (5) scientific uses, (6) recycling, (7)
food and kitchen uses, (8) clean/cleaning, (9) stationery material, (10) game/use it in a game, (11) school/study uses, and (12)
other uses*
(1) carry or move things, (2) recycling and manufacturing, (3) decoration, (4) blocks, (5) shelter for pets, (6) container, (7) sport
uses, (8) artistic uses, (9) musical instrument, (10) play and games, and (11) other uses*
(1) cleaning, (2) care and beauty, (3) recycling, (4) artistic uses, (5) food and kitchen uses, (6) stationery material, (7) hanger,
and (8) other uses*
(1) ideas/thoughts, (2) ropes, (3) nature (e.g., air, steam, smoke), (4) path/road, (5) food, (6) letters/numbers, (7) games, (8)
tools, (9) trees, and (10) others*
(1) places, (2) games, (3) geometric figures, (4) mathematics, (5) roads/crossroads, (6) machines, (7) part of the human body,
(8) tools, (9) phone/tablets, (10) letter or words, and (11) others*
(1) road, (2) sorting things, (3) architecture, (4) stick/cane, (5) artistic and design uses, (6) furniture, (8) games, (9) musical
instruments, (10) levels, and (11) others*
Note: *Others = all responses that did not fit any of the mentioned categories.
Table 3
Interclass and Internal Consistency Reliability Coefficients of Study Instruments
Internal Consistency
Fluency - Uses
Flexibility - Uses
Originality - Uses
Fluency - Figural
.53 (.71)*
.61 (.73)*
Flexibility - Figural
.50 (.68)*
.64 (.69)*
Originality - Figural
.51 (.62)*
.55 (.62)*
Problem Generation - Fluency
Problem Generation - Originality
Note: * = After removing the Lines task; ICC = Interclass Correlation
Table 4
Correlations Between Study Variables (n= 90)
Note: Uses = Uses Test; PG = Problem Generation Test
*p <.05 **p <.01.
Table 5
Regression Analysis of Divergent Thinking Skills with Problem Construction
95% CI
Problem Construction (Fluency)
DT Verbal
R2 = .59
DT Figural
R2 = .08
Problem Construction (Originality)
DT Verbal
[-.787 - .648]
R2 = .60
DT Figural
R2 = .09
Note: DT = Divergent Thinking
Figure 1. Line Chart of the Comparison Between the Number of Publications on Problem Construction, DT, and
Problem Solving in the Past Five Decades.
Figure 2. Histogram of the Comparison Between the Number of Publications on Problem Construction, DT, and
Problem Solving in the Past Five Decades.
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Although generation of novelty remains central, the definition of creativity has broadened from aesthetics to encompass practical products in many areas, not just in fine arts, while relevance and effectiveness are also necessary. Creative products range from sublime to everyday. Creativity also involves aspects of thinking, personality, and motivation and feelings. These are partly domain-specific but are also partly transferable across domains. Creativity emerges in an environment which does not block the processes, and at least tolerates the personal properties. The generation of effective novelty involves a number of paradoxes, but a stage model helps to resolve these.
Full-text available
This paper proposes a model, which hopefully will allow researchers in the psychology of creativity to confirm that the different levels and different labels for problem finding can be unified under one construct – problem finding (PF). Although no clear distinctions are made among the levels and terms used in the PF literature, the current efforts suggest that there are important differences that can be explained by (a) how well- or ill-defined a problem is, and (b) the degree to which ideation and evaluation are required. Based on these two criteria, a rubric is presented that allows distinctions to be made among five the PF processes: (a) problem discovery, (b) problem formulation, (c) problem construction, (d) problem identification, and (e) problem definition. The authors examined the literature on PF in English from 1960 to 2015 using the following databases: (a) Academic Search Premier, (b) PsycARTICLES, (c) PsycINFO, (d) Dissertation Abstract, (e) Educational Resources Information Center (ERIC), (f) Psychology & Behavioral Science Collection, and (g) the Google Scholar. This search resulted in 199 articles in which at least 13 different terms were used to describe the process of finding a problem. Only a few articles endeavored to distinguish among the terms used in the literature. This paper concludes by suggesting that one term (i.e., problem finding) is to be used to avoid confusion. If this is not possible, for whatever reason, the term used instead should be defined and the reasons for the choice of terms clearly stated.
Full-text available
Divergent thinking (DT) tests are probably the most common-ly used measures of creative potential. Several extensive batteries are available but most research relies on one or two specific tests rather than a complete battery. This may limit generalizations because tests of DT are not equivalent. They are not always highly inter-correlated. Additionally, some DT tests appear to be better than others at eliciting originality. This is critical because originality is vital for creativity. The primary purpose of the present study was to determine which test of DT elicits the most originality. Seven measures of DTwere administered on a sample of 611 participants in eight Arabic countries. The tests were Figural, Titles, Realistic Pre-sented Problems, Realistic Problem Generation, Instances, Uses, and Similarities. The Quick Test of Convergent Think-ing, Runco’s Ideational Behavior Scale, and a demographic questionnaire were also administered. A linear mixed model analysis confirmed that the originality scores in the DT tests differed by test. Post-hoc tests indicated that the Titles and Realistic Problem Generation tests produced the highest mean originality scores, whereas the Realistic Presented Problems test produced the lowest mean originality scores. These differences confirm that research using only one DT test will not provide generalizable results.
Full-text available
This study examined the relationship among creative problem solving (PS) and problem generation (PG) abilities, stress and daily hassles, and coping skills in a sample of college undergraduates. Heirarchical regression analyses indicated that separate sets of both PS and PS task scores were predictive of scores on certain coping scales even after the variance accounted for by indices of stress and hassles was removed. Specifically, PG and PS abilities were negatively related to such coping processes as confrontation, distancing, escape-avoidance tendencies, and excessive acceptance of responsibility, and positively associated with more general adaptive qualities. The findings strongly suggest that PS and PG abilities are important components of an individual's overall capacity to cope with both major and minor stresses of life.
Full-text available
The “be creative effect”—instructing people to “be creative” before a divergent thinking task makes the responses more creative—is one of the oldest findings in creativity science. The present research suggests that this seemingly simple effect has underappreciated implications for the assessment of divergent thinking and for theories of the executive and controlled nature of creative thought. An experiment measured fluid intelligence and gave two divergent thinking tasks: people completed one with “be creative” instructions and the other with “be fluent” (generate as many ideas as possible) instructions. The responses were scored for creativity (using subjective scoring) and for fluency (the total number of ideas). Multilevel latent variable models found, not surprisingly, a large effect of task instructions: asking people to be creative yielded better but fewer responses, suggesting a focus on quality over quantity. Notably, the study replicated both sides of the contentious intelligence-and-creativity literature. When people were told to come up with a lot of ideas and creativity was scored as fluency, intelligence weakly predicted creativity. But when people were told to be creative and creativity was scored as subjective ratings, intelligence more strongly predicted creativity. We conclude with implications for the assessment of creativity, for models of the intentional and controlled nature of creative thought, and for the ongoing debate over creativity and intelligence.
Divergent thinking (DT) tests are often used for creativity assessment. They differ from many other tests in that they are open-ended. A great deal of research has examined the influence of test instructions on the number and nature of responses to DT tests. Most instructions explicitly emphasize quantity (e.g., "give as many ideas as you can"). Others target additional features, such as creativity, originality, or idea quality. Do such alternative explicit instructions make any difference? The present meta-analysis examined studies that compared the explicit instructions emphasizing creativity, originality, and quality to quantity instructions. Using a 3-level multilevel approach, analyses with all 204 effect sizes from 31 studies indicated that creativity and quality instructions increased performance on DT when added to quantity instructions (gs = .243 and .271, respectively), more than quantity instructions alone. However, the originality instructions did not change DT performance (g=-.159). Thus, explicit instructions may increase or decrease DT performance, depending on which alternative explicit instructions are used and how they are presented. Practical implications of the findings are discussed, as are limitations of this research.
Divergent thinking tests are often used in creativity research as measures of creative potential. However, measurement approaches across studies vary to a great extent. One facet of divergent thinking measurement that contributes strongly to differences across studies is the scoring of participants’ responses. Most commonly, responses are scored for fluency, flexibility, and originality. However, even with respect to only one dimension (e.g., originality), scoring decisions vary extensively. In the current work, a systematic framework for practical scoring decisions was developed. Scoring dimensions, instruction-scoring fit, adequacy of responses, objectivity (vs. subjectivity), level of scoring (response vs. ideational pool level), and the method of aggregation were identified as determining factors of divergent thinking test scoring. In addition, recommendations and guidelines for making these decisions and reporting the information in papers have been provided.
This study examined the relationship between problem finding (PF) and creativity. A search of published and unpublished studies in English from 1960 to 2015 resulted in 40 studies with a total of 6,649 male and female participants, with ages ranging from childhood to adulthood. Accordingly, this meta-analysis estimated the population correlations between PF and creativity within the 40 studies; it examined whether the correlations varied according to the sample characteristics or other aspects of the individual investigations. We also examined how various labels are used in PF literature and how the terms differ from one another. Using the random-effects model, the results showed that PF and creativity were significantly correlated, r = .22 (95% confidence interval [.11, .32], p = < .001), but with high heterogeneity. Moreover, 3 of 5 moderators were significant and explained the variation in the mean effect size: (a) the various indices of divergent thinking, (b) the PF domain, and (c) participants’ age. An analysis of variance showed that using different labels in PF and creativity research did not significantly alter study results. Still, the findings suggest that PF consists of various processes that evolve when individuals discover, identify, or define problems. This study also calls for measures that can assess ill-defined problems to complement existing divergent thinking measures that assess presented problems.
There is disagreement among researchers about whether IQ tests or divergent thinking (DT) tests are better predictors of creative achievement. Resolving this dispute is complicated by the fact that some research has shown a relationship between IQ and DT test scores (e.g., Runco & Albert, 1986; Wallach, 1970). The present study conducted meta-analyses of the relationships between creative achievement and both IQ and DT test scores. The analyses included 17 studies (with 5,544 participants) that established the correlation coefficients between IQ and creative achievement and 27 studies (with 47,197 participants) that established the correlation coefficients between DT test scores and creative achievement. Marginal, but statistically significant, Fisher's Z-transformed correlation coefficients were revealed. The analysis found a significantly higher relationship between DT test scores and creative achievement (r = .216) than between IQ test scores and creative achievement (r = .167). The differences in the correlation coefficients were explained by differences in DT tests, creative achievement types, predicted time periods, and creativity subscales. The significant independent moderator effect for different DT tests indicates that the Torrance Tests of Creative Thinking (TTCT) predict creative achievement better than any other DT test included in this study. Among the creative achievement types, music is predicted the best by IQ and all others are predicted best by DT tests. Among the time periods evaluated, the relationship between DT test scores and creative achievement had the highest correlation at the period of 11–15 years.
Although a number of factors condition the success of creative efforts, most investigators recognize the fundamental importance of novel problem solutions. As a result, a number of systems intending to describe the processes contributing to the generation of innovative problem solutions have been proposed. In the present article, earlier models describing the processes contributing to creative problem solutions are reviewed. The common themes appearing in these models are then considered in relation to the use of extant information structures. Certain implications of cognitive information processing for understanding the nature and ontogeny of the creative act are then discussed, along with their potential contributions to the identification and development of creative potential.