Conference PaperPDF Available

Abstract

In the field of education, especially in primary school, we can favor the development of creative thinking and written expression of the first grades pupils, using teaching (didactic) practices of selected fairy tales, with creative dynamics. Through fairy tales we encouraged children to produce original ideas, free their imagination and express themselves through creative writing. The selected texts written by the pupils as well as the criteria for creativity evaluation digitized were submitted to judges-educators and subjected linguistic processing. In this paper we present a methodology, correlating judges-teachers’ evaluation with the linguistic processing statistics, comparing fairy tales as for their creative potential and, finally, comparing creativity of different aged pupils based on the same fairy tales. The creativity criteria used are seven: the four of the [18], [11] rubric plus the three classic ones [5],[16], [12]. Results of our statistics comprise a k-means clustering of tales, using all seven creativity indices (rubric), designating their relevant didactic practices. An important result is that a very similar clustering of tales is produced based on either the seven creativity criteria mentioned or the percentage of the part of speech (POS) used by the pupils. The above results confirm our initial hypothesis that some aspects of creative writing can be fairly successfully evaluated using linguistic features, [10]. Moreover, our research results can be further exploited in the development of automatic assessment systems and automated content analysis of fairy tales, written by primary school pupils. Keywords: Creative writing assessment, fairy tales, linguistic features
Children’s Creative Writing using lexical diversity indices
Ioanna Tyrou
Department of Italian Language and Literature, National and Kapodistrian University of Athens,
Greece, (iotyrou@ill.uoa.gr)
Abstract
In the field of education, especially in primary school, we can favor the development of creative thinking
and written expression of the first grades pupils, using teaching (didactic) practices of selected fairy
tales, with creative dynamics. Through fairy tales we encouraged children to produce original ideas,
free their imagination and express themselves through creative writing. The selected texts written by the
pupils as well as the criteria for creativity evaluation digitized were submitted to judges-educators and
subjected linguistic processing. In this paper we present a methodology, correlating judges-teachers’
evaluation with the linguistic processing statistics, comparing fairy tales as for their creative potential
and, finally, comparing creativity of different aged pupils based on the same fairy tales. The creativity
criteria used are seven: the four of the [18], [11] rubric plus the three classic ones [5],[16], [12]. Results
of our statistics comprise a k-means clustering of tales, using all seven creativity indices (rubric),
designating their relevant didactic practices. An important result is that a very similar clustering of tales
is produced based on either the seven creativity criteria mentioned or the percentage of the part of
speech (POS) used by the pupils. The above results confirm our initial hypothesis that some aspects
of creative writing can be fairly successfully evaluated using linguistic features, [10]. Moreover, our
research results can be further exploited in the development of automatic assessment systems and
automated content analysis of fairy tales, written by primary school pupils.
Keywords: Creative writing assessment, fairy tales, linguistic features
1. Creative Writing and Expression in the Classroom
When children come to school, they "carry" with them an inner need to express themselves orally,
which combined with children's love of fairy tales and stories, a strong foundation for emotional
engagement and spiritual stimulation could be established.
The child in Primary School has to work creatively, to early cultivate a personal relationship with the
words and the way to combine them to form a homogeneous whole, without missing the joy from other
actions such as storytelling, theatrical or drawing representation of the fairy tale. Of course, children
enjoy benefit in many ways when they get actively involved with children's literature: sharing experiences
from a book in small or large groups may lead to essential conversations and writing applications.
As a main procedural approach in primary school, it offers liberation of thought and flexibility of
personal experimentations on writing ways, it frees self-expression and presents pupil's personal ideas,
emotions and feelings [16], [17]. Creative writing as an educational activity should be "guided by the
instructor" [15], who has to be flexible and to provide a trigger for the "liberation" of the imagination, to
encourage pupils, to suggest and also motivate them in creative written expression [9].
2. Learning through fairy tales
The full of exaggerations and contradictions fairy tale is to entertain, to charm, to teach in an indirect
way, above all to reach the Aristotelian catharsis by escalating tensions: to relieve and to purify. It has
a simple style, stereotype expressions of beginning and end, deliberate repetitions, it provides its interior
for explosive ventures and turnovers.
We can say that fairy tale school reading develops creativity, extends thought, train memory and
exploits children potentiality, nourishing dialogue, language expression and semantic abundance of
words, fostering pluralism of opinions, cultivating a co-operation and teamwork climate. In other words,
it contributes to the child's overall psychokinesis, cognitive, perceptual and emotional development [4],
[14], [6].
3. Assessment of Creative Writing by Judges and Language Tools
3.1 Creative Tales
In order to confirm and support some theoretical approaches, we conducted a field research in the
classroom. Through specific creative tales, using established teaching practices, we encouraged
children to express themselves through creative writing. Fairy tales written by B and E grade pupils were
selected, from different teaching practices.
We compared full of fantasy children's stories such as "the dog that did not know how to bark". The
aim was to find out whether pupils would contribute to the story creating images, sounds,
characterization of the hero, how well formed and aesthetic their narrative could be, whether quantity
and types of ideas would be important, enriching their text, and how original their story could be, written
at the beginning of the school year. Under the same aim we presented a fairy tale "the clean wolf and
the dirty pigs", reverting the well-known story, aiming at bringing about new elements to it.
Furthermore, children of the B and E grade working as a group on an invented narrative of the Rodari
"the planet of truth", presented ideas on how to give life to an invented "small-human" [17], made up of
material other than that of a human (such as glass, plastic, wood, ice cream, paper, iron, etc) and guide
the humanoid to act, to develop relationships and hit by accidents appropriate to the material chosen.
Finally, responses of the B and E grade pupils relevant to the Rodari fairy tale "Rinaldo adventure"
were organized at the end of the school year. Without an end for this story pupils were asked to change
parts of the story, using their imagination, trying to act as writers and giving their own end version of the
story.
3.2 Judges and Creativity Criteria
The use of judges for the evaluation of creative writing, writing in general, is considered as an
established methodological approach [1], [2], [20], [7], [8], [3]. Following this practice, six experienced
school teachers were selected to act as judges. Creative criteria with sample assessing examples were
presented to them and explained in detail.
Some of the pupils written story versions were digitized and given to judges for an assessment,
based on the following creativity criteria: Quantity of Ideas (Cognitive Ability), Types of Ideas (Cognitive
Flexibility), Rarity of Ideas (Originality of Thought), Picture / Verbal Iconification, Sound / Verbal Audio
Description, Character Description / Characterization, Narration / Story Telling. The former three are
referred to by various works [5], [16], [12], [13], [19]. The four latter criteria [18], [11], despite their relative
difficulty in utilization, provided interesting data.
Each judge separately processing the material, recorded his evaluation giving one of the four grades
for each fairy tale and for each student: Excellent (4), Good (3), Reasonable (2), Poor (1).
To integrating judges' assessment for statistical processing, we tested inter-rater agreement by taking
into account assessments which are in agreement more than or equal to 70%. For each student, every
fairy tale and each creative criterion separately, the median value of the resulting evaluation of these
judges was calculated. Following the same inter-rater agreement process, for the "collective creation"
of pupils of the same grade (age group), we calculated the median of the assessments of judges who
agree 70% or more for all pupils of that grade, per creativity criterion and per tale. The resulting numbers
were normalized from 0 to 1.
3.3 Linguistic Processing
The digitized texts were also tested by linguistic processing to record for each pupil and every fairy
tale under consideration the following language characteristics [18], [11]:
Absolute number and percentage of lemmata per part of speech (names, verbs, adjectives,
adverbs, conjunctions, pronouns, prepositions, articles, etc.), where all lemma types are taken
into account.
Percentage of lemma types per part of speech (names, verbs, adjectives, adverbs, etc), i.e.
each lemma is recorded once, irrespective of its number of occurrences.
Percentage of apax lemmata (the words that appear only once), per part of speech (names,
verbs, adjectives, adverbs, etc.).
In this paper we present results based on both content words (names, verbs, adjectives and adverbs)
and functional words (prepositions, conjunctions, articles, etc.), but the apax lemmata.
For calculating the percentage of the word types in the text of each pupil for each fairy tale, the lemma
of each word was found (using the ILSP lemmatizer) and the unique types were identified, where each
lemma is recorded once irrespective of the number of its occurrences. Then, the percentage of the types
and the unique types was calculated for all the words in their relevant grammatical category (ILSP-
http://nlp.ilsp.gr/soaplab2-axis/).
The values of all parameters, in tables, were normalized to the number of students in each age group
and, if applicable, to the number of words or sentences per age group, for comparison.
3.4 The Research Questions
Central questions that initiated and led this research, for which an initial answer is attempted in this
paper, is the correlation of Fare Tales and Age Groups by means of creativity criteria and linguistic
parameters. More specifically:
Similarities between Fairy Tales / Teaching Practices. Namely, whether there is a difference
in pupils' creativity depending on the kind (subject, teaching practice) of the fairy tale. (Figure
1, Figure 2)
Correlating Age Groups. Namely to compare creativity of different age pupils, using creativity
criteria and linguistic parameters. (Figure 3)
Clustering of Fairy Tales, using creativity criteria and linguistic parameters. (Figure 4)
4. Conclusions
Observing Figure 1 we conclude that characteristic is the "Small Human" fairy tale written by the
pupils of the Grade B (BTA12), whose evaluation is just as excellent in character description and story
(narration) as that of the Grade E pupils (ETA17), while it is assessed as average only in the image and
voice criteria. Almost identical pattern is found for the "Rinaldo" fairy tale (BTR13, ETR19)! The
evaluation of the fairy tale "Wolf" (BAL10) is found below average except in story (narration) in which is
assessed as 0,7. Τhe "Dog" (BAS11) in image and story (narration) is assessed over 0,7 and average
in voice and character description.
Figure 1: A noticeable difference is presented in assessing tales of all age groups (B and E) using
image (picture) and voice (audio) criteria. Similar figures are related to the criteria story (narration)
and mostly to characterization (character description).
Based on Figure 2 we observe that the fairy tale "Dog" (BAS11) is assessed as almost nil, whereas
the "Wolf" (BAL10) is assessed as average as for the Rarity of Ideas. The assessment of the fairy tale
"Small Human" for both age groups (BTA12, ETA17) and that of the “Dog” (BAS11) are assessed above
0,7, reaching the maximum value in all the criteria, as is found for the “Rinaldo” of B grade (BTR13)!
Figure 2: A comparative assessment of almost all tales is presented using these 3 criteria
Comparing the assessment of the same tale (“small human”), using all 7 creative criteria along with
percentage of lemmata, we observe (Figure 3) a high similarity between age groups. A comparative
result emerged comparing all relevant tales using all creativity criteria and the linguistic parameters
lemmata types.
Figure 3: High similarity in the assessment of tale “small human” of the two age groups, using all
creativity criteria and lemmata types.
Considering clustering of all fairy tales based on all 7 creativity criteria in combination with linguistic
parameters (lemmata types) we find (Figure 4) that Fairy tale "Wolf" forms a distinct cluster (cluster 3),
while fairy tale "Dog" belongs to the same cluster as "Rinaldo" and “Small Human” (LITTLE_HUMAN)
fairy tales of the E Grade (cluster 2). “Rinaldo’ and "Small Human" of Grade B form their own group
(cluster 1).
Figure 4: 3 distinct clusters emerged, differentiating tales, using all 7 creativity criteria and either
percentage of lemmata or lemmata types.
As we can see differentiation between fairy tales and ages is depended on the creative criteria and /
or the linguistic parameters applied. The above results confirm our initial hypothesis that some aspects
of creative writing can be fairly successfully evaluated including some linguistic features.
Correlations
1
0.97 1
B-average
ETA E-average
ETA
B-average
ETA
E-average
ETA
References
[1] Amabile, T. M. Social psychology of creativity: A consensual assessment technique”, Journal of
Personality and Social Psychology, 43, 1982, pages 9971013.
[2] Amabile, T. M. Creativity in context”, Boulder, CO, Westview Press,1996.
[3] Baer, J. & S. S. McKool. Assessing creativity using the consensual assessment technique”, S.
Schreiner (ed.), Handbook of research on assessment technologies, methods and applications in
higher education, Hershey, PA: Information Science Reference, 2009, pages 65-77. 10.4018/978-
1-60566-667-9.ch004
[4] Budihastuti, Dra. Exti. Short Story Text in Project-Based Learning Model on 7th Graders”,
American Journal of Educational Research 3.2, 2015, pages 109-115.
http://pubs.sciepub.com/education/3/2/1/index.html
[5] Crossley, S.A., Muldner, K., McNamara, D.S. Idea Generation in Student Writing: Computational
Assessments and Links to Successful Writing”, Written Communication, Vol. 33 (3), SAGE
Publications, 2016, pages 328354, https://doi.org/10.1177/0741088316650178
[6] Hung, C.M., Hwang, G.J., & Huang, I. “A Project-based Digital Storytelling Approach for
Improving Students' Learning Motivation, Problem-Solving Competence and Learning
Achievement”, Educational Technology & Society, Volume 15 (4), 2012, pages 368-379.
https://pdfs.semanticscholar.org/ce24/85df8c9f48a964b36f7f9fe36030192cad3d.pdf?_ga=2.24988
05.430862470.1577960894-1975653073.1577960894
[7] Kaufman, J. C., & Baer, J. Beyond new and appropriate: Who decides what is
creative?”, Creativity Research Journal, 24(1), 2012, pages 83-91,
https://doi.org/10.1080/10400419.2012.649237
[8] Kaufman, J. C., Gentile, C. A., & Baer, J. Do gifted student writers and creative writing experts
rate creativity the same way? Gifted Child Quarterly, 49, 2005, pages 260-265. http://www-
usr.rider.edu/~baer/KaufmanGentileBaer.pdf
[9] McNamara S. Danielle, Scott A. Crossley b , Rod D. Roscoe, Laura K. Allen, Jianmin Dai. A
hierarchical classification approach to automated essay scoring”, Assessing Writing 23, 2015, pages
3559. https://doi.org/10.1016/j.asw.2014.09.002.
[10] Mozaffari, H. “An Analytical Rubric for Assessing Creativity in Creative Writing.” Theory and
Practice in Language Studies 3 (12), 2013, pages 22142219.
https://pdfs.semanticscholar.org/fd69/adb2f145b923e94596cc0f734e6e653191d1.pdf.
[11] Runco, M.A., Acar, S. Divergent Thinking as an Indicator of Creative Potential”, Creativity
research journal, 24 (1), 2012, pages 110.
http://www.researchgate.net/publication/8902778_Creativity
[12] Slabon, A.W., Richards, L. R., Denneny, P.V. Learning by restorying”, Instructional Science,
Volume 42, Issue 4, 2014, pages 505-521.
[13] Souliotis, M. Creative writing”, Athens: Ministry of Education and Culture, Δημιουργική Γραφή.
Οδηγίες Πλεύσεως, 2012. (In Greek).
[14] Τyrou, I. Creative Writing with Fairy Tales - Evaluating Creativity in Primary School”, 4th
Panhellenic Conference "Neos Paigagogos", Eugenides Foundation, Athens, 2017. (In Greek).
http://users.sch.gr/synedrio/Praktika_Synedriou_04_Synedrio_Neos_Paidagogos_2017.pdf
[15] Tyrou, I., Karakitsou,V., Koutli, M., Larios, N. Creative Writing and Activities Based On Multiple
Intelligences Upon Gianni Rodari Fairy Tales 4th International Conference of Education, Research
and Innovation (ICERI2011), 2011, Madrid, Spain.
[16] Vaezi, M & Rezaei, S. Development of a rubric for evaluating creative writing: a multi-phase
research, New Writing, 16:3, 2019, pages 303-
317, https://doi.org/10.1080/14790726.2018.1520894
https://www.academia.edu/37495741/Development_of_a_rubric_for_evaluating_creative_writing_
A_multi-phase_research
[17] Zedelius, C.M., Mills, C. & Schooler, J.W. Beyond subjective judgments: Predicting evaluations
of creative writing from computational linguistic features”, Behavior Research Methods, 2018.
https://link.springer.com/content/pdf/10.3758%2Fs13428-018-1137-1.pdf
[18] Kotopoulos, Τ. Teaching Creative Writing. The case of writing literary works for adolescents,
Θέματα παιδικής λογοτεχνίας, Εκδ. Πανεπιστημίου Μακεδονίας, 2012, pages 93-162. (In Greek).
[19] Xanthakou, G. “Creativity in School”, Athens, Ελληνικά Γράμματα,1998. (In Greek).
[20] Sioutas, N., Zimianitis, K., Koutaleli, K. & Panagopoulou, E. “Creative Thinking - Producing
Innovative and Original Ideas, Ministry of National Education and Religious Affairs, General
Secretariat for Adult Education, 2008. (In Greek).
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Although project-based learning is a well-known and widely used instructional strategy, it remains a challenging issue to effectively apply this approach to practical settings for improving the learning performance of students. In this study, a project-based digital storytelling approach is proposed to cope with this problem. With a quasiexperiment, the proposed approach has been applied to a learning activity of a science course in an elementary school. A total of 117 Grade 5 students in an elementary school in southern Taiwan were assigned to an experimental group (N = 60) and a control group (N = 57) to compare the performance of the approach with that of conventional project-based learning. A web-based information-searching system, Meta-Analyzer, was used to enable the students to collect data on the Internet based on the questions raised by the teachers, and Microsoft's Photo Story was used to help the experimental group develop movies for storytelling based on the collected data. Moreover, several measuring tools, including the science learning motivation scale, the problem-solving competence scale and the science achievement test, were used to collect feedback as well as evaluate the learning performance of the students. The experimental results show that the project-based learning with digital storytelling could effectively enhance the students' science learning motivation, problem-solving competence, and learning achievement. © International Forum of Educational Technology & Society (IFETS).
Article
Full-text available
The Consensual Assessment Technique is a powerful tool used by creativity researchers in which panels of expert judges are asked to rate the creativity of creative products such as stories, collages, poems, and other artifacts. Experts in the domain in question serve as judges; thus, for a study of creativity using stories and poems, a panel of writers and/or teachers of creative writing might judge the creativity of the stories, and a separate panel of poets and/or poetry critics might judge the creativity of the poems. The Consensual Assessment Technique is based on the idea that the best measure of the creativity of a work of art, a theory, a research proposal, or any other artifact is the combined assessment of experts in that field. Unlike other measures of creativity, such as divergent-thinking tests, the Consensual Assessment Technique is not based on any particular theory of creativity, which means that its validity (which has been well established empirically) is not dependent upon the validity of any particular theory of creativity. This chapter explains the Consensual Assessment Technique, discusses how it has been used in research, and explores ways it might be employed in assessment in higher education.
Article
Full-text available
Divergent thinking (DT) tests are very often used in creativity studies. Certainly DT does not guarantee actual creative achievement, but tests of DT are reliable and reasonably valid predictors of certain performance criteria. The validity of DT is described as reasonable because validity is not an all-or-nothing attribute, but is, instead, a matter of degree. Also, validity only makes sense relative to particular criteria. The criteria strongly associated with DT are detailed in this article. It also summarizes the uses and limitations of DT, conceptually and psychometrically. After the psychometric evidence is reviewed, alternative tests and scoring procedures are described, including several that have only recently been published. Throughout this article related processes, such as problem finding and evaluative thinking, are linked to DT.
Article
Full-text available
Little research has been conducted on how gifted novices compare to experts in their judgments of creative writing. If novices and experts assign similar ratings, it could be argued that gifted novices are able to offer their peers feedback of a similar quality to that provided by experts. Such a finding would support the use of collaborative feedback in gifted classrooms. We asked gifted high school creative writers and three groups of experts (cognitive psychologists, creative writers, and teachers) to rate a set of 27 short stories and 28 poems for creativity using a scale of 1 to 6. The interrater agreement among the novices was within acceptable standards, and the agreement among the experts was very strong. When the ratings of novices were compared to the ratings of experts, a strong degree of correlation was found, supporting the use of peer feedback among gifted novice creative writers.
Drawing on a modified version of Delphi technique, the researchers in this study tried to develop a rubric comprising the main criteria to be considered in the evaluation of works of fiction. Review of the related literature, as well as the administration of a Likert scale questionnaire, and a series of unstructured interviews with experts in the fields of literature and creative writing, led to the identification of ten elements which were used in the construction of the first version of the rubric. To ensure its validity, a number of distinguished creative writing professors were asked to review this assessment tool and comment on its appropriateness for measuring the intended construct. Some revisions were made based on these comments, and following that, the researchers came up with an analytical rubric consisting of nine elements, namely narrative voice, characterisation, story, setting, mood and atmosphere, language and writing mechanics, dialogue, plot, and image. The reliability of this rubric was also established through the calculation of both interrater and intrarater reliability. Finally, the significance of the development of this valid and reliable rubric is discussed and its implications for teaching and assessing creative pieces of writing are presented.
Article
The question of how to evaluate creativity in the context of creative writing has been a subject of ongoing discussion. A key question is whether something as elusive as creativity can be evaluated in a systematic way that goes beyond subjective judgments. To answer this question, we tested whether human evaluations of the creativity of short stories can be predicted by: (1) established measures of creativity and (2) computerized linguistic analyses of the stories. We conducted two studies, in which college students (with and without interest and experience in creative writing) wrote short stories based on a writing prompt. Independent raters (six in Study 1, five in Study 2) assessed the stories using an evaluation rubric specifically designed to assess aspects of creativity, on which they showed high interrater reliability. We provide evidence of convergent validity, in that the rubric evaluations correlated with established creativity measures, including measures of divergent thinking, associative fluency, and self-reported creative behavior and achievements. Linguistic properties of the short stories were analyzed with two computerized text analysis tools: Coh-Metrix, which analyzes aspects of text cohesion and readability, and Linguistic Inquiry and Word Count, which identifies meaningful psychological categories of the text content. Linguistic features predicted the human ratings of creativity to a significant degree. These results provide novel evidence that creative writing can be evaluated reliably and in a systematic way that captures objective features of the text. The results further establish our evaluation rubric as a useful tool to assess creative writing.
Article
Idea generation is an important component of most major theories of writing. However, few studies have linked idea generation in writing samples to assessments of writing quality or examined links between linguistic features in a text and idea generation. This study uses human ratings of idea generation, such as idea fluency, idea flexibility, idea originality, and idea elaboration, to analyze the extent to which idea generation relates to human judgments of essay quality in a corpus of college student essays. In conjunction with this analysis, linguistic features extracted from the essays are used to develop a predictive model of idea generation to further understand relations between the language features in an essay and the idea generation scores assigned to that essay. The results indicate that essays rated as containing a greater number of ideas that were flexible, original, and elaborated were judged to be of higher quality. Two of these features (elaboration and originality) were significant predictors of essay quality scores in a regression analysis that explained 33% of the variance in human scores. The results also indicate that idea generation is strongly linked to language features in essays. Specifically, the use of unique multiword units, more difficult words, semantic but not lexical similarities between paragraphs, and fewer word repetitions explained 80% of the variance in human scores of idea generation. These results have implications for writing theories and writing practice.
Article
Assessment is a crucial component of teaching creative writing. However, the discipline lags far behind its composition and literary counterparts to develop a plausible method for evaluating creativity. Therefore, after reviewing the past and current literature on the topic, this study describes the design and implementation of an analytical rubric which encompasses four major qualities of creative language (image, characterization, voice and story). The reliability and validity of the proposed rubric are also empirically evaluated. The result reveals that the rubric provides the sufficient reliability and validity value reported in the literature for rubrics. Besides, the article includes some sample analyses to explicate its application.
Article
In this paper, we introduce restorying, a pedagogical approach based on social constructivism that employs successive iterations of rewriting and discussing personal, student-generated, domain-relevant stories to promote conceptual application, critical thinking, and ill-structured problem solving skills. Using a naturalistic, qualitative case study design, this study describes and analyzes how restorying promoted learning and transfer for master’s level students in two sections of a conflict management class. Data sources included course documents, course assignments, class observations, interviews, focus groups, and surveys. An inductive analytic approach using constant comparative methods was employed for analyzing qualitative data, and course section assignment performance averages were calculated for analyzing quantitative data. The restorying approach emphasizes learning new content through personal story application and story sharing among course participants. The restorying approach offers a potentially viable alternative for those not satisfied with conventional case studies. Conventional case studies may fall short in providing the desired range or complexity of problem space elements or may embed problems in contexts that are not learner relevant. Moreover, the sustained analysis of a past personal experience may result in deeper internalization of domain content and transfer of learning. While restorying may not be appropriate for some learning contexts, it holds promise for settings that can incorporate the key elements and address the adoption considerations described in this paper. The restorying approach invites one to consider how personal, student-generated, domain-relevant stories may be employed, shared, reflected upon, revised, expanded upon, and redeployed to promote achievement of desired learning outcomes.