No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
The Story/Test/Story Method: A Combined Approach to Usability Testing and Contextual Inquiry
Abstract
The Story/Test/Story Method is a new method that represents a combined approach to usability testing and contextual inquiry that can be used in the classroom and beyond. Usability testing has been used by both rhetoricians and technical communicators to assess the effectiveness of digital technologies for specific user groups. Contextual inquiry is a method with its roots in ethnography that examines the contexts of users to ensure that technologies support their needs. In classroom assignments, as well as in UX research projects beyond the classroom, it can be difficult to incorporate both methods, however. At the same time, it is essential for students and researchers to assess both a user’s context and their actions while using a technology in order to learn how that technology might be improved. In this article I introduce the Story/Test/Story Method, explain its precedents in composition and technical communication literature, give examples of its use in the classroom, and discuss its limitations when compared to both usability testing and contextual inquiry as standalone methods. Ultimately, I hope that writing researchers, teachers, and consultants can use the Story/Test/Story Method in their own work to assess, critique, and develop digital technologies within their own specific contexts.
... This balancing act that relies on communication is one possible reason why TPC scholars have shown increasing interest in UX. This interest has taken several forms, including pedagogical [5], [6], [7]; methodological [7], [8], [9], [10], [11]; observational [8], [10], [11], [12]; and disciplinary [1], [2], [3], [4], [5], [13], [14]. This scholarship is too complex to neatly summarize, but we can draw some boundaries: ...
... This balancing act that relies on communication is one possible reason why TPC scholars have shown increasing interest in UX. This interest has taken several forms, including pedagogical [5], [6], [7]; methodological [7], [8], [9], [10], [11]; observational [8], [10], [11], [12]; and disciplinary [1], [2], [3], [4], [5], [13], [14]. This scholarship is too complex to neatly summarize, but we can draw some boundaries: ...
As the technical and professional communication (TPC) field has evolved in response to broader changes in the world economy, numerous professions have arisen within its ranks that coexist with the traditional roles of technical writer and technical editor. These include instructional design, content strategy, and user-experience (UX) design [1], [2]. A unique challenge for TPC is to include training in numerous professions within a single college major or program. Some programs have chosen to focus on specific professions, even going so far as to rename their program around that profession. Others have continued to focus broadly on the overall field while updating their curricula as needed to serve students seeking training in a particular career path.
... This is in accordance with the basis for the preparation of the WCT test, namely elevating the evaluation of critical thinking skills from a general domain that is specified into a construct with physics content, namely waves so that the product of students' thinking in the form of essays emphasizes understanding physical concepts compared to mathematical concepts. However, mathematical concepts are not completely eliminated, because physical concepts, especially waves, still require mathematical relationships (Aghvinian et al., 2021;Getto, 2020). ...
... In addition, the presentation of phenomena in the WCT test is made in the form of story telling and modeling of children's activities so that it will be more memorable when students read it. This is an adjustment to the generalization of several studies related to contextual approaches (Aghvinian et al., 2021;Getto, 2020). ...
p style="text-align: justify;">This study aims to produce a product to evaluate students' critical thinking skills that departs from physics content where students often have misconceptions. This research is a development research with research stages covering a) research and review literature; (b) planning chapter objectives; (c) developing a preliminary form; (d) field-testing the preliminary form; (e) Revise the preliminary form; (f) conducting a main field-test. The Waves Critical Thinking (WCT) test developed consists of 7 questions with 15 specific domains. Total percentage of content validity test was obtained 87.98% with appropriate criteria and based on the construct validity WCT test, the Goodness of Fit criteria were obtained which were classified as fit. The test instrument being tested consists of 15 objective items. The reliability of WCT test results 0.597 as a Cronbach's alpha score with the medium category and all the components have a good level of composite reliability. The outcome of the study was the WCT test with a valid state for measuring students' CT in a specific domain of physics wave material.</p
... One notable strength of this study lies in its qualitative design, which utilized serial semi-structured interviews, allowing participants to share their experiences with the self-reflection tool in depth rather than conducting limited-scale user experience testing [40]. Additionally, it is worth emphasizing that all participants completed three interviews, and a single investigator conducted all the interviews to ensure the credibility and consistency of the data collection process. ...
Objectives
This study aims to explore how the design of a self‐reflection tool can assist patients with type 2 diabetes (T2D) by facilitating and sustaining their reflective practices in the context of diabetes care. Specifically, the study seeks to examine how patients utilize self‐reflection tools, identify aspects they feel improve diabetes management, and uncover the opportunities and challenges they encounter when integrating such tools into their daily lives.
Methods
The research involved two phases: first, the preliminary development of a self‐reflection tool prototype and second, the exploration of user experience. During the second phase, five patients with T2D participated in three interview sessions bi‐weekly over the period of 6 weeks. The interviews were transcribed and thematically coded, and affinity diagramming was then used to synthesize the data.
Results
Four main themes emerged from the interviews. The designed self‐reflection tool was found to have the potential to enhance participants’ motivation for engaging in diabetes management by improving diabetes management, initiating self‐reflection, facilitating new thoughts and meanings, and providing opportunities for the patients to share their reflections.
Conclusions
The designed self‐reflection tool helped patients with T2D by encouraging them to adopt a more positive mindset and supporting them in addressing challenges related to their diabetes management. The study suggests that there is promising potential for the self‐reflection tool to evolve into a reflection‐sharing tool that can be shared with other patients.
Patient or Public Contribution
Participants with T2D took part in the designed interviews, and their experiences and comments meaningfully contributed to enriching the co‐design of the self‐reflection tool. The suggested potential for this tool to be expanded into a reflection‐sharing tool is also significant.
... The power of storytelling lies in its ability to immerse users within a narrative context, effectively bridging the gap between their experience and measurement tools [33]. Designers often leverage the immersive nature of storytelling to enhance usability testing [20], aiding in the development of high fidelity and their preparation for market launch as products or services [12] [13]. Storytelling proves especially beneficial in children's application design, significantly increasing engagement [29] [2]. ...
ABSTRACT The increasing application of conversational agents as assistants and playmates for children brings about the need for evaluation methods tailored for children. The System Usability Scale (SUS) for kids is a well-established instrument for measuring subjective aspects of the usability for children. However, using the scale for children presents difficulties relating to children's concentration and the biases observed when they respond to questionnaires. To make the questionnaire completion enjoyable for children and improve their engagement with the questionnaire, we rendered the adaptation of SUS for children in a game-board format. This paper motivates the presentation of the questionnaire in this way and discusses the use of this instrument in a case study with 35 children aged eight to twelve. We argue that this presentation helped engage with children and there was little evidence of extreme response bias. We conclude that this board format is a more appropriate way to present the SUS questionnaires to children in usability tests, which may also apply to different surveys including rating scales.
... The process involves the u�liza�on of observa�on and interviews as means of collec�ng qualita�ve data and quan�ta�ve measures iden�fying the components that have effect on a specific issues (Merrill et al., 2021). Also, Contextual inquiry method inves�gates the various contexts in which users operate to determine if technologies effec�vely meet their requirements (Geto, 2020). ...
Interactive instructional videos have become vital pedagogical tools to promote active learning, boost student engagement, and encourage independent learning. However, effectively developing interactive multimedia often poses challenges for educators who lack advanced technical expertise. This study investigated the usability of the Edpuzzle platform for creating interactive videos through qualitative techniques including task-based usability assessments, semi-structured interviews, and analysis of user reviews. 7 educators participated in think-aloud protocol and system usability assessments on core tasks like adding videos, inserting questions, and utilizing the video editor. Interviews also examined user perspectives on Edpuzzle's workflow, features, and areas needing improvement. Additionally, supplementary data from user reviews on Capterra were analysed to identify common pain points. The synthesized findings revealed notable learnability issues such as unclear onboarding processes and lack of guidance on key features. Users also expressed frustrations with restrictive creation capabilities in the video editor, desiring more advanced editing options. The data indicated needs to expand interactive features like diversifying question types and incorporating artificial intelligence to boost relevance Overall, the results spotlight critical needs to refine and clarify the onboarding process, optimize the interface design, expand creative capabilities and interactivity features, and smooth out workflows to enhance Edpuzzle's usability. This research provides valuable qualitative insights to inform the user- centred, iterative development of interactive video tools to better empower educators in effectively leveraging this emerging educational technology.
... There has recently been a significant uptick in research at the intersections of UX and TPC. This work has taken various forms, including the development of pedagogies for teaching UX [12][13][14], the development of research methods for doing UX [15][16][17][18][19][20], the report of findings of UX studies [19][20][21], and explorations of the state of UX as 230 ...
... The complexity of this process also lends itself to thinking of UX as a matter of communication that happens within a design process, which is probably why scholars of TPC have been increasingly interested in UX as of late. This interest has taken many forms, including the development of pedagogies for teaching UX [5,[22][23], the development of research methods for doing UX [20][21][24][25][26][27], the report of findings of UX studies [26][27][28], and explorations of the state of UX as a field, including its central skill sets and disciplinary home [1][2][3][4][5][29][30]. ...
Few studies related open design to the clothing sector, but none explored how users would experience it. The study reported here aimed to investigate how people with some or without prior sewing knowledge - advanced and amateur users – experience an open design-clothing product. Following four fashion design heuristics, a garment was created and distributed as DIY kits among advanced and amateur users. Data were collected in two stages: assembly and personalization. The results indicate that although skills play a significant role during assembly, other factors, like cross-generational differences and personal taste, influence how users experience an open design product. Furthermore, the study shows that given the necessary support, the open design can be used by a heterogeneous public, amplifying the participation of users with little or without prior sewing skills in clothing co-creation.
ICL2021 was the 24th edition of the International Conference on Interactive Collaborative Learning and the 50th edition of the IGIP International Conference on Engineering Pedagogy.
This interdisciplinary conference aims to focus on the exchange of relevant trends and research results as well as the presentation of practical experiences in Interactive Collaborative Learning and Engineering Pedagogy.
ICL2021 has been organized by Technische Universität Dresden and University of Applied Science Dresden, Germany, from September 22 to 24, 2021, as a hybrid event.
This year’s theme of the conference was “Mobility for Smart Cities and Regional Development – Challenges for Higher Education”.
This research project tries to investigate the influence of the Moth stories in the development of the speaking English skill. This is an experimental representation of the learning in form of storytelling method. This study had the participation of 39 students from the second semester “A” of Pedagogía de los Idiomas Nacionales y Extranjeros program at Technical University of Ambato. The whole experimental group (EG) was allowed to use online resources such as stories taken from The Moth web page to practice and complete some speaking tasks. To assess the speaking skill, a mock-up Key English Test (KET) speaking part was used as a pre and post-test. The speaking criteria were stated in the rubric provided by Cambridge which consisted of Grammar and Vocabulary, Pronunciation and Interactive Communication. The experiment was designed with the use of with the use of online stories according to student’s age and needs and obviously taking into consideration the COVID-19 pandemic. In order to analyze the experimental data that were represented in the paper, the SPSS program was used to apply a T-Student test. To conclude, the results showed that online stories taken from The Moth web page had positive effects on the development of the students’ speaking skill. Its interface, resource, structure, and methodology were engaging for students so they improved mainly the pronunciation and its branches which are the intelligibility, word-stress, intonation and phonological features. Thus, the positive effects of using online storytelling strategies were obvious in this research.
The motivation to stimulate children’s learning engagement could be found in the fact that learning is not always motivational in itself. This is particularly true when learning is obligatory and based upon material that has not been chosen by the children themselves. A Lean UX approach to the co-design of an educational serious videogame (MOBI) is proposed in this paper. The core idea is that children's natural interest in playing can be stimulated by engendering the feeling that they are participating in the creation of something. The hypothesis is that this approach can increase the children's level of engagement and can facilitate their awareness of their learning perception. With the aim of testing this hypothesis, this paper describes an experience with 50 children with ages between 10 and 12 years old. The results indicate that the children’s satisfaction grew significantly during the process, with an important reduction in the requests for changes and that 60% had the perception of having learned. It can be concluded that the co-design based upon a Lean UX methodology, of a children’s educational serious videogame increases their level of product engagement and facilitates their awareness of their learning perception.
Written Communication 24 (3), 223-249, 2007
This study examines the factors influencing language and script choice in instant messaging (IM), a form of real-time computer-mediated communication, in a multilingual setting. Grounded in the New Literacy Studies, the study understands IM as a social practice involving texts, encompassing a range of literacy practices, within which a subset called “text-making practices” is highlighted in this article. Drawing on results from an analysis of chat texts, interviews, and logbooks collected from 19 young people, the author suggests that the text-making practices related to language and writing system choice are guided by the perceived affordances of the IM technology and the available linguistic resources. Seven ecological factors influencing these perceptions have been identified: perceived expressiveness of the language, perceived functions of IM , user familiarity with the language, user identification with the language, technical constraints of inputting methods, speed , and perceived practicality of the writing system. The author argues that these factors often co-occur in real use.
The authors, an instructor and students, describe our practice of user-centered design on three levels: in the design and structure of an advanced undergraduate course in which we all participated, in student projects designed during the course, and in our reflections on the course presented here. We argue that principles of user-centered design can and should be more than course concepts and assignments; they can be core practices of the course that hold both students and teachers accountable for the impacts of their rhetorical choices. We offer a model for other teacher-scholars looking to involve students in the design of their courses and in writing together about their work.
New-media writing exerts pressure in ways that writing instruction typically has not. In this article, we map the infrastructural dynamics that support-or disrupt-new-media writing instruction, drawing from a multimedia writing course taught at our institution. An infrastructural framework provides a robust tool for writing teachers to navigate and negotiate the institutional complexities that shape new-media writing and offers composers a path through which to navigate the systems within and across which they work. Further, an infrastructural framework focused on the when of new-media composing creates space for reflection and change within institutional structures and networks.
Service-learning projects that connect students in new media writing courses with community organizations are typically framed as opportunities for students to develop production skills through “real-world” projects. This article proposes a different model and rationale for digital-age service-learning projects, in which students teach organizational staff and/or the constituents they serve both how to produce new media texts and how to use these texts for rhetorical action. A pedagogical framework for this model is presented, which outlines three dimensions that constitute the capacity for rhetorical action with new media: functional competence with particular technologies, a critical perspective on technology, and a rhetorical approach to technology and media creation. Two iterations of a course called Digital Storytelling in Organizations, which used this service-learning model and pedagogical framework, are described. A rationale for translating this service-learning model to courses that feature other new media technologies is also discussed.
To build upon user-centered design methods, we used a collaborative and multi-modal approach to involve users early in the design process for a website. This article presents our methods and results and addresses the benefits and limitations of the Collaborative Prototype Design Process (CPDP), including ways in which this new method can be implemented. The CPDP is an innovative approach to user-centered website design that empha-sizes collaboration, iterative testing, and data-driven design. The CPDP balances the power and needs of users with those of designers and, thus, enables design teams to test more tasks and involve more users. We divided our initial team into three independent design teams to separately profile users, test usability of low-fidelity paper prototypes, and then create and test usability of resulting wireframes. After completing the user-centered design and usability testing, the three teams merged to analyze their diverse findings and create a final prototype.
We all use stories to communicate, explore, persuade, and inspire. In user experience, stories help us to understand our users, learn about their goals, explain our research, and demonstrate our design ideas. In this book, Quesenbery and Brooks teach you how to craft and tell your own unique stories to improve your designs.
This experience report outlines an experiential social justice-focused client project from an upper-level document design course embedded within a larger technical and professional communication (TPC) program. As our TPC classrooms continue to address the precarious social exigencies and cultural issues of writing for the public, our students need experience responding to such needs in their careers and communities. This report addresses this emerging trend by turning to the design thinking process as a scheme to aid TPC instructors in bringing social justice issues into their TPC assignments and curricula, particularly in working with client projects. The author provides lessons learned for TPC instructors involved with social justice-focused client projects, including the successes and shortcomings of employing the design thinking process alongside typical TPC genres such as white paper reports and usability testing of documentation.
This study examines how the layout of syllabi in first-year writing courses can aid or obscure content that is key to student success. Understanding the function of a syllabus as a charter document in a genre ecology helps to underscore the importance of the syllabus as a regulatory document that can have a significant impact for students. This study makes the case for considering students as expert end-users that have valuable knowledge useful to composition programs interested in redesigning first-year writing course syllabi using a human-centered design approach.
This article argues that writing programs have an opportunity to create a new playing field in their online composition courses. A playing field that conceives of students and content differently than does a typical iteration of an online course (a course that traditionally migrates materials and practices from a f2f context and reimagines them for an online setting). The focus of this article emphasizes how readers can use user-centered design in their online courses to accommodate all students with varying learning styles. Readers will gain a better understanding of how significant user-centered design can be for maintaining student enrollments, promoting learning and avoiding attrition. Further readers will understand that specific moves made by the instructor will have very real repercussions on whether or not a course, or even elements of a course, are accessible by all.
The CCCC Position Statement of Principles and Example Effective Practices for Online Writing Instruction can be viewed as a set of principles for user-centered design in online writing classrooms. However, operationalizing the principles and practices can be overwhelming. Our article identifies a set of principles that we introduce to new online writing instructors. We describe how we build a user-experience mindset into the foundation of online writing instruction using the CCCC Position Statement as well as principles from UX and user-centered design; we draw on work by key figures in UX and usability, including Goodwin (2009), Klein (2016), and Buley (2013; see also Howard & Greer, 2011). Our article describes how we introduce basic principles of user-centered design to new instructors, apply those principles to core topics in online writing instruction, and model a process of student feedback to promote an iterative design philosophy for online courses.
The global diffusion of online media means individuals are increasingly composing web texts for international readers. To do so effectively, writers need to understand the expectations and preferences cultural and national audiences associate with using online materials – particularly where they are used, when, and how. The usability concept of personas – or archetypes of an audience – can help writing students research and understand such aspects. Personas do so by focusing research on viewing readers/users as distinct groups within greater social and cultural contexts. This article explains how writing instructors can user persona-related activities to help composition students address contextual factors when writing online texts for international and intercultural audiences.
Following the emerging importance of networks to digital rhetoric, we theorize user experience (UX) design as a form of networked rhetoric. By networked rhetoric, we mean the act of tracing rhetorical impacts between human and nonhuman actors within networks. While researching emerging trends in this area, we were asked to conduct a usability study of an online coastal atlas. During this research project, we found that the atlas was not usable for many of its core user base, and were able to help the atlas’s development team to rethink their application based on the relationships they wanted users to be able to form with this emerging system. Through our discussion of this research within the purview of networked rhetoric, we provide implications for other digital rhetoricians interested in helping to craft better user experiences for their colleagues, students, and community partners.
This essay argues that new authoring environments fundamentally change the authorship paradigm. Four kinds of critical literacy may be useful to produce computational media that responds appropriately to the larger rhetorical context of software culture. These critical literacies include not only writing code, but also learning a range of user interfaces, participating in design practices for debugging programs, and recognizing the norms of digital labor workflows and systems of credit.
In this article I explore how the tenets of design thinking as described by Richard Buchanan, Nigel Cross, Lucy Kimbell, Tim Brown and others might be applied to writing instruction. I argue that design thinking, a human-centered approach to designing innovative solutions in response to wicked problems, can help close the gap between writing inside and outside of school and prepare students to participate in a future of writing. More specifically, I recommend that we apply design thinking to writing classes by 1) teaching writing as a design process, 2) creating wicked writing assignments, 3) encouraging writing in teams, and 4) fostering experimentation through prototyping.
In a changing information economy of images and multimodal composition practices, writing programs face new challenges and opportunities for creating spaces that encourage students to explore processes in high-tech, low-stakes environments. This article suggests that composing in multimodal spaces is always a negotiation. Therefore, a one-size-fits-all approach for designing such spaces may not be as useful as portable, flexible approaches. In this article, I apply Joddy Murray's (2009) values of non-discursive rhetoric as a generative heuristic for space design that highlights important perspectives for programs that teach multimodal composition.
Not unlike pedagogies that have as their goal the need to redefine literacy as a type of critical citizenship necessary for social change, much work in computers and composition pedagogy argues for this democratizing potential. As writing teachers become more familiar with electronic networks, however, in place of our initial utopian labels, we are increasingly turning to labels of heterotopic conflict, particularly terms such as electronic contact zones. Through the discussion of a first-year writing class composed of both Hispanic American and Anglo students, this article argues that rather than regard online conflicts between students as mere flaming, such conflicts can be seen as a way of helping students develop as literate citizens more aware of difference.
As technical communication gains the technology to deliver dynamic custom documents, the importance of audience analysis increases. As a major factor in supporting dynamic adjustment of document content, the audience analysis must clearly capture the range of user goals and information needs in a flexible manner. Replacing a linear audience analysis model with a multidimensional model provides one method of achieving that flexibility. With a minimum of three separate dimensions to capture topic knowledge, detail required, and user cognitive ability, this model provides the writer a means of connecting content with information requirements and ensuring the dynamic document fits varying audience needs.
The Elements of User Experience: User-Centered Design for the WebJesse James GarrettNew Riders, 2003 ISBN 0735712026 $29.99
The User experience team of one: A research and design survival guide
- Leah Buley
Buley, Leah. (2013). The User experience team of one: A research and design survival guide. Brooklyn, NY: Rosenfeld Media.
Designing around the whole story with user narratives Retrieved from:. UX Magazine
- Marc Cajolet
- Bloomer
- Sarah
- Alexandra Stevens
Cajolet, Marc, Bloomer, Sarah, & Stevens, Alexandra. (2014). Designing around the whole story with user narratives Retrieved from:. UX Magazine.
http://uxmag.com/articles/designing-around-the-whole-story-with-user-narratives
A closer look at personas: What they are and how they work (part 1) Retrieved from:. Smashing Magazine
- Shlomo Goltz
Goltz, Shlomo. (2014). A closer look at personas: What they are and how they work (part 1) Retrieved from:. Smashing Magazine.
http://www.smashingmagazine.com/2014/08/06/a-closer-look-at-personas-part-1/
The role of user experience in the product development process Retrieved from. UX Matters
- Rex Hartson
- Pyla
- Pardha
Hartson, Rex, & Pyla, Pardha. (2012). The UX book: Process and guidelines for ensuring a quality user experience. Boston, MA: Morgan Kaufmann.
Hoober, Steven. (2014). The role of user experience in the product development process Retrieved from. UX Matters.
http://www.uxmatters.com/mt/archives/2014/05/the-role-of-user-experience-in-the-product-development-process.php
Task analysis: The key UX design step everyone skips
- Larry Marine
Marine, Larry. (2014). Task analysis: The key UX design step everyone skips Retrieved from:. Search Engine Watch.
http://searchenginewatch.com/article/2336547/Task-Analysis-The-Key-UX-Design-Step-Everyone-Skips
Why you only need to test with 5 users Retrieved from
- Peter Morville
Morville, Peter. (2007). User experience strategy Retrieved from:. Semantic Studios. http://semanticstudios.com/user experience strategy/
Nielsen, Jakob. (2000). Why you only need to test with 5 users Retrieved from:. Nielsen Norman Group. http://www.nngroup.com/articles/
why-you-only-need-to-test-with-5-users/
First rule of usability? Don't listen to users Retrieved from
- Jakob Nielsen
Nielsen, Jakob. (2001). First rule of usability? Don't listen to users Retrieved from:. Nielsen Norman Group. http://www.nngroup.com/
articles/first-rule-of-usability-dont-listen-to-users/
The most important usability activity Retrieved from
- Jakob Nielsen
Nielsen, Jakob. (2012a). The most important usability activity Retrieved from:. Nielsen Norman Group. https://www.nngroup.com/articles/
the-most-important-usability-activity/
<Methods> experience design </methods>
- Potts
Potts, Liza, & Bartocci, Gerianne. (2009). <Methods> experience design </methods>. Proceedings of the 27th ACM international conference on
design of communication, 17-21.