No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
An Empirical Evaluation of Visual Metaphors in the Animation of Roles of Variables
Abstract
Roles of variables, which describe stereotypic usages of variables, can be exploited to facilitate teaching introductory programming. This paper describes the evaluation of visual metaphors for roles used in a role-based program animator. The evaluation is based on several criteria: properties of the images, metaphor recognition and grading, and effects on learning. The study demonstrates that as a whole the role metaphors facilitate learning. The results also identify ideas for further elaboration of the individual metaphors. Furthermore, the study suggests that the evaluation of animated metaphors may require special measures.
... In a series of experiments, we use the content of visualizations and the level of viewer's engagement as the varied factor, and gather information of the effects of this variation on the visual attention and the mental models. So far, we have studied the effect of the content of visualization on programming knowledge [23] and the effect of the content of visualizations on visual attention and mental model of the studied program [15]. The experiment on the effect of the content of visualizations on short-term mental model of roles [23] was carried out by comparing empirically the effects of PlanAni's original role images versus neutral control images. ...
... So far, we have studied the effect of the content of visualization on programming knowledge [23] and the effect of the content of visualizations on visual attention and mental model of the studied program [15]. The experiment on the effect of the content of visualizations on short-term mental model of roles [23] was carried out by comparing empirically the effects of PlanAni's original role images versus neutral control images. In this experiment , engagement was not varied and visual attention was not measured. ...
... Locations of gaze were recorded using an eye-tracking camera [24]. Knowledge on variable roles was measured using a post-test from [23]. The program summaries were analyzed using Good's program summary analysis scheme [7] with the additional categories presented in [2]. ...
Empirical evaluation of visualizations has so far been typically carried out by measuring the performance of participants that have been shown the visualization in relation to control group by grading programming tasks. Such studies tell little about what effects take place during the visualizations and how these effects build up into the learning effects. To address these issues, we are carrying out a series of experiments using visualization tools whose long-term effects are known.This paper presents a model of the cognitive phenomena that take place during visualization viewing sessions and describes an experiment where two visualization tools that differed in the amount of animation were used. The results show that even when participants were provided with rich visual information in the form of animations, they resorted heavily to the textual cues present on the screen. As the animation proceeded, participants started to follow visual cues more closely. However, if the names of the variables involved in the current operation were not in the visually cued area, participants tended to search for them in the program code. In the absence of animation, participants were more eager to browse the code in general.
... Mental representations of PlanAni users were, in fact, similar to that of good code comprehenders. The effects of the content of visualizations on short-term mental model of roles were investigated by Stützle and Sajaniemi [19]. In their experiment, PlanAni's original role images were compared with neutral control images; engagement was not varied and visual attention was not measured. ...
... In the earlier experiments [11, 12, 19], the type of engagement has been kept fixed and only the representation used for the variables has been varied. Collected information has been related to the location and size of the visualizations, and to presence or absence of role animation. ...
... Group TC (Text, Control flow) used PlanAni without role images and role animation to perform a task which requires information mainly of program' s control flow. The pre-test score on role knowledge (applied from [19]) was used to divide the participants evenly into the groups. The dependent variables were the locations of the participant's gaze (visual attention ), the participant's post-test score on role knowledge (short-term programming knowledge), and the program summary provided by the participant (short-term program knowledge). ...
When visualization tools utilized in computer science education have been evaluated empirically, the results have been controversial. The extent to which the tools have benefited learning has remained unclear, as well as the ways through which the benefit has been achieved. In our research, we have chosen to vary type of students' engagement and representation of the visualization tool in a series of experiments, in order to investigate the phenomena taking place during individual viewing of visualizations. In the current experiment, we varied student engagement using two different tasks to perform during viewing; data flow task (D) and control flow task (C). Repre-sentation of visualization was varied by using two versions of the program ani-mator; one with special images and animation, and one without. The results show that while the distribution of visual attention of the participants performing task D was steady throughout the time, the participants performing task C focused their visual attention at the beginning almost solely on the code, and increased their visual attention to the other parts of the visualization on the second half of the viewing. The participants performing task D also benefited most from the tool, at least regarding programming knowledge.
... To test the lightweight structural techniques described in the previous section, we conducted an investigation by studying animated metaphors used in a computer program animator. In an earlier study (StützleSt¨Stützle and Sajaniemi, 2005), these metaphors have been studied using the operational and pragmatic approaches, i.e., evaluated from the users' perspective. These user acceptance results are now used as a measure of the mental model accuracy that will be compared with judges' evaluations of the similarity of subjects' perceived properties of images with properties of the target. ...
... ThisFigure 5: Original and control metaphors for fixed value, follower, gatherer, and temporary (Stützle and Sajaniemi, 2005).Table 1: Roles of variables used in the investigation. ...
Visual metaphors in the form of still or animated pictures have been used in user interfaces with the hope of enhancing learning and use of computer applications. This paper studies animated metaphors with the intent to understand how they relate to human cognition and how their quality can be measured. We present a model of the relationships within metaphors, suggest lightweight evaluation techniques based on this model, and test these techniques in an empirical investigation. The results indicate that a lightweight analysis based on still images and made by domain-aware but metaphor-unaware judges can be used as a first step in deciding which metaphors are worthy of further study, and to direct animation efforts to overcome the most crucial problems. Furthermore, the results show that animation may increase or decrease the quality of a metaphor by considerable amounts; hence the final evaluation must be based on actual use of fully implemented metaphors. The results also confirm earlier suggestions to use rich metaphors and provides evidence that richness of the still image is important for the effectiveness of animation.
... (Nash, 1962; Stott et al., 2010), computer science (Carroll & Mack, 1985; Petre & Blackwell, 1999; Stützle & Sajaniemi, 2005), and films (Forceville, 2015). ...
Competent assessment and management of the risk of harm is a core competency that mental health professionals are expected to possess. However, despite this expectation, adequate training programs have been lacking for decades and, even when risk assessment training is provided, it is often reported as being insufficient. The literature indicates that training delivery methods often include passive and didactic methods during supervision or seminar sessions. To help enhance the learning of suicide risk factors, some authors proposed a visual metaphor that visually and metaphorically depicts all suicide risk factors. The main purpose of this study was to examine the efficacy of the proposed visual metaphor.
A pilot RCT was undertaken to test several hypotheses, all of which predicted that the visual metaphor would demonstrate superior effects when compared with the conventional textual teaching methods. A group of 22 psychology students were randomized into either a control group (who learnt suicide risk factors via the conventional textual teaching methods) or a treatment group (that learnt the risk factors using the visual metaphor in addition to the conventional textual teaching methods). Memory recall, knowledge transfer, cognitive load, and satisfaction were all tested at the end of the learning sessions.
Independent samples t tests indicated that the visual metaphor was effective in improving memory recall and knowledge transfer and reducing the cognitive load. The differences between the two groups’ post-learning scores were significant in each of these outcome measures. The treatment group also expressed higher satisfaction levels in comparison to the control group. Overall the visual metaphor of suicide risk factors was found to be superior to the conventional teaching methods in teaching suicide risk factors to university psychology students. Limitations, implications of this study and directions for future research are discussed.
... They found that the visualization tool used had various subtle effects on the visualizations that students tended to draw, but document no differences on the impact of the tools on learning. Stützle and Sajaniemi [21] experimentally evaluated a set of visual metaphors designed to be evocative and intended to facilitate the learning of common patterns of variable use in programs; they found that learners recalled the visual metaphors and the corresponding concepts better using the in 429 429 ICSE 2015, Florence, Italy Joint SE Education and Training the evocative metaphors condition than in a control condition that used a set of neutral images instead. Emotional design relates to these efforts in that, as implemented in studies to date, it is partially based on metaphorical , anthropomorphic illustrations. ...
A small number of recent studies have suggested that learning is enhanced when the illustrations in instructional materials are designed to appeal to the learners' emotions through the use of color and the personification of key elements. We sought to replicate this emotional design effect in the context of introductory object-oriented programming (OOP). In this preliminary study, a group of freshmen studied a text on objects which was illustrated using anthropomorphic graphics while a control group had access to abstract graphics. We found no significant difference in the groups' scores on a comprehension post-test, but the experimental group spent substantially less time on the task than the control group. Among those participants who had no prior programming experience, the materials inspired by emotional design were perceived as less intelligible and appealing and led to lower self-reported concentration levels. Although this result does not match the pattern of results from earlier studies, it shows that the choice of illustrations in learning materials matters and calls for more research that addresses the limitations of this preliminary study.
... Metaphor is the creative expression of a point of view (Boostrom, 1998). Metaphors stimulate the thought system and encourage students to use their existing knowledge and improve their competences (Stützle and Sajaniemi, 2005:90). Lakoff and Johnson (1980) opine that metaphors are reflected in the perceptions and thoughts of a person. ...
The purpose of the research was to reveal elementary 8th grade students' opinions concerning democracy with the aid of metaphors. The students were asked to produce metaphors about the concept of democracy. 140 students from 3 public schools in Ankara (Turkey) participated in the research. 55% of the students were females and 45% were males. The students who participated in the research were asked to produce a metaphor about the concept of democracy and to explain these metaphors. To realize the purpose of the study, forms were distributed to the students asking them to complete the sentence, "Democracy is like… because…" and 56 students out of 140 produced 32 valid metaphors. Phenomenology was used for the purposes of the present research. Data obtained from the participants were interpreted via content analysis. The produced metaphors were divided into 4 categories: "Democracy as the symbol of equality", "Democracy which exists with rules", "Democracy which teaches living together" and "Democracy as a dream". The findings were discussed in the light of the related literature.
... These two sections are included for readers unfamiliar with one or both of these concepts; other readers could skip them. More elaborate presentation of roles of variables1718192021 and RoboLab [6,9] could be found in the cited sources. In the Section 4 main results of the conducted case-study are presented. ...
This paper describes a case study of using narratives to motivate non-technology inclined children, 11-15 years old, to learn programming, using LEGO Mindstorms robots and RoboLab graphical programming language. Case study was done during 2004/2005 and 2005/2006 school years, following two different school teams participating in FIRST LEGO League competitions. Using narrative concept and a concept of roles of variables, it was possible to explain several searching and sorting algorithms to children, including an algorithm of finding minimal/maximal value from the set of input values. Results are encouraging and could be generalized to other programming languages but RoboLab.
... The results also identify ideas for further elaboration of the individual metaphors. Furthermore, the study suggests that the evaluation of animated metaphors may require special measures (Stutzle & Sajaniemi, 2005). v) We interpret the rules governing an interactive system as the rules specifying a game. ...
To sum up our studies, let's consider some propositions of this chapter. The sign nature of HCI and visualization allows applying the semiotics analysis to them. The basic idea of similarities between application domain entities with visual objects is required when visual language is designed. It is possible to speak about some idea of metaphoric representation. A metaphor is a framework of visual language which provides both depicting by an author of the visual text (that is a designer of the interactive [visualization] system) and user interpretation of the visual text. The main approaches to Computer Metaphor Theory are based on the cognitive approach to metaphors linked with G. Lakoff and his coauthors. In frameworks of these approaches the
In this study, the effects of metaphorical learning method which was applied to the gifted and talented students' foreign language teaching (English) was examined. The study was applied to 38 gifted and talented students in Afyonkarahisar Science and Art Center in 2005-2006 education years. In this study, pretest-posttest control group model was applied. Pretest, posttest, English Lesson Activity Observation Form and English Lesson Performance Evaluation Measure (rubric) are used as a measure device. To gather data which is necessary for the study, to measure students' English information and to balance groups (pretest) at the beginning of the study and to see groups' situation (posttest) at the end of the study, a measure device was formed. English lesson activity observation form was prepared to determine student's attitude to English learning and to learn his foreign language method. It will be filled by student's parents, English teacher and researcher. English lesson performance evaluation measure was also prepared with the aim of observing control and experiment group students' sentence structuring, using metaphors when learning foreign language. The scores of means which were obtained from measure devices were subjected to the statistical evaluation. According to the study, the experiment group students', who learned with metaphoric learning strategies, academic success mean is 83,05 and the control group students', who learned with traditional instruction method, academic success mean is 68,98.
This paper presents results of an experiment in which Random Access Memory (RAM) diagrams were used to teach novice students C programming. Students were divided into two groups that were differently instructed. The control group was instructed in the traditional way while the experiment group was instructed with the aid of RAM diagrams employed throughout the course. Examination results from the two groups were compared. Statistical analysis was done and the Z value was calculated. The results suggest that the use of RAM diagrams improves programming comprehension and programming skills.
In this paper, we present an approach for introducing programming concepts using visual tiling patterns (tilings for short). Tilings exhibit regularities and their construction can be achieved through the use of simple textual commands. We show that fundamental programming concepts, such as sequencing of commands (i.e., the right commands in the right order), conditional structures (i.e., yes/no decisions), grouping and repetition of sets of commands (i.e., subroutines that are executed more than once), can be communicated effectively using tilings and in the process support students' learning and mastering of these concepts. The use of concrete, visual tilings represents a potentially effective approach to introduce programming to students.
This article is a survey of program visualization systems intended for teaching beginners about the runtime behavior of computer programs. Our focus is on generic systems that are capable of illustrating many kinds of programs and behaviors. We inclusively describe such systems from the last three decades and review findings from their empirical evaluations. A comparable review on the topic does not previously exist; ours is intended to serve as a reference for the creators, evaluators, and users of educational program visualization systems. Moreover, we revisit the issue of learner engagement which has been identified as a potentially key factor in the success of educational software visualization and summarize what little is known about engagement in the context of the generic program visualization systems for beginners that we have reviewed; a proposed refinement of the frameworks previously used by computing education researchers to rank types of learner engagement is a side product of this effort. Overall, our review illustrates that program visualization systems for beginners are often short-lived research prototypes that support the user-controlled viewing of program animations; a recent trend is to support more engaging modes of user interaction. The results of evaluations largely support the use of program visualization in introductory programming education, but research to date is insufficient for drawing more nuanced conclusions with respect to learner engagement. On the basis of our review, we identify interesting questions to answer for future research in relation to themes such as engagement, the authenticity of learning tasks, cognitive load, and the integration of program visualization into introductory programming pedagogy.
When visualization tools utilized in computer programming education have been evaluated empirically, the results have remained controversial. To address this problem, we have developed a model of short-term effects of program animation, and used it in a series of experiments. In the current experiment, we varied visual representation of an animation tool and the type of students' engagement. Results of the current experiment analyzed together with the results from the earlier experiments provided support for the hypothesis that what a student does plays a more central role in the usefulness of a visualization than representation used by the tool. Moreover, the levels of engagement as they are generally used in the research literature seem not to be the best possible indicators of the effectiveness of a visualization. (Contains 11 tables and 4 figures.)
In 3 studies, 176 nonprogrammers (undergraduates) learned a computer programming language by a method (model) that emphasized a diagram model of a computer expressed in familiar terms or by a method with no model (nonmodel) and then practiced on exercises and took a posttest. In learning and posttest performance, model Ss performed best on interpretation of programs and on problems requiring looping, while nonmodel Ss excelled on straightforward generation of programs. The model was especially helpful for low ability Ss. Practice in interpretation helped nonmodel Ss most and practice in writing simple programs helped model Ss most. The roles of the model in establishing a meaningful learning set, and of practice on mathemagenic activity, are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In 2 experiments, a total of 160 nonprogrammers (undergraduates) learned a computer programming language and took a multileveled transfer posttest. Ss who were given pretraining with a concrete model of the computer before learning performed better on novel transfer and worse on near transfer relative to no-pretraining Ss including Ss who were given posttraining with the same model after learning. Similar differences in the pattern of transfer were noted for Ss who controlled the order of presentation of frames relative to E-controlled Ss. Results suggest that availability and activation of assimilative sets influence the structure of learning outcomes. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Interface metaphors facilitate the learning of new computer systems by supporting the transformation of existing knowledge
in order to improve the comprehension of novel situations. However, there is very little guidance for software designers on
how to select, implement and evaluate interface metaphors. This paper, which is based upon extensive work in developing metaphors
for telecommunications systems, provides a framework for software designers who wish to exploit the use of interface metaphors.
The paper proposes a set of six design steps, to provide designers with a practical approach to the application of metaphor
in the design of interactive systems. An explanation of the activities required in each step is given and justified from experience
gained in developing a number of interface metaphors. A pragmatic model of the use of metaphor in human centred system design
is introduced, and a technique for eliciting metaphor characteristics is developed from ethnomethodology. The approach has
been discussed with software designers at two workshops, and the final content has been influenced by their input.
Roles of variables is a new concept that captures tacit expert knowl-edge in a form that can be taught in introductory programming courses. A role describes some stereotypic use of variables, and only ten roles are needed to cover 99 % of all variables in novice-level programs. This paper presents the results of an experiment where roles were introduced to novices learning Pascal programming. Students were divided into three groups that were instructed differently: in the tra-ditional way with no treatment of roles; using roles throughout the course; and using a role-based program animator in addition to using roles in teaching. The results show that students are not only able to understand the role concept and to apply it in new situations but—more importantly— that roles provide students a new conceptual framework that enables them to mentally process program information in a way demonstrating good programming skills. Moreover, the use of the animator seems to foster the adoption of role knowledge.
Computer programming is a difficult skill for many students and visualizations may be used to foster learning. This paper presents a program animation system, PlanAni, that is based on the concept of the roles of variables. Roles represent schematic uses of variables that occur in programs over and over again, and a set of nine roles covers practically all variables in novice-level programs. PlanAni has been tested in a teaching experiment comparing traditional teaching with role-based teaching and animation. The results of a semi-structured interview with the teacher indicate that students like to work with the animator and that the system clarifies many concepts in programming.
Previous research on metaphor and a collection of cases where metaphors have been consciously or unconsciously used are reviewed. Based on these cases, a set of guidelines for metaphorical design is offered. Finally, based on the cases and the guidelines, a set of characteristics of metaphorical design is presented as a theoretical basis for understanding what metaphors are and how they work.