No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
The Use of Think-aloud Methods in Qualitative Research An Introduction to Think-aloud Methods
Abstract
Think-aloud is a research method in which participants speak aloud any words in their mind as they complete a task. A review of the literature has shown that think-aloud research methods have a sound theoretical basis and provide a valid source of data about participant thinking, especially during language based activities. However, a researcher needs to design a process which takes into account a number of concerns, by selecting a suitable task, a role for the researcher, a source of triangulation, and, most importantly, an appropriate method of interpretation. This paper argues that think-aloud research can be effectively interpreted through a qualitative lens. A qualitative approach also has implications for the choice of participant(s) and the treatment of the data. Participants should be treated as quasi-researchers, and their efforts rewarded with reciprocity.
... For instance, participants may inadvertently omit certain aspects of their cognitive processes while verbalizing, resulting in incomplete data (Roth et al., 2013). Moreover, verbalizations from human participants often fail to capture implicit cognitive mechanisms and strategic reasoning, as these processes may remain unspoken or unconscious during task execution (Charters, 2003;Jacobse & Harskamp, 2012;Nielsen et al., 2002). This limitation hinders researchers' ability to obtain a comprehensive understanding of learners' cognitive processes. ...
... Participants may unconsciously filter their thoughts, resulting in incomplete data. This issue arises when subtle cognitive processes or those perceived as irrelevant during the task are omitted from verbal reports (Charters, 2003). Additionally, because not all cognitive processes are active within working memory, significant aspects of cognition may remain unreported (Roth et al., 2013). ...
In the field of education, the think‐aloud protocol is commonly used to encourage learners to articulate their thoughts during the learning process, providing observers with valuable insights into learners' cognitive processes beyond the final learning outcomes. However, the implementation of think‐aloud protocols faces challenges such as task interference and limitations in completeness and authenticity of verbal reports. This study proposes a method called Cognitive Echo, which leverages large language models (LLMs) trained with simulated student experiences to enhance the completeness and authenticity of think‐aloud verbalizations. LLMs have been demonstrated to simulate human‐like behaviour more effectively by memorizing experiences. In this work, we introduce specific learner roles and train the LLMs to act as distinct learners. Our method involves integrating transaction data from learners' interactions with a tutoring system and the tutor's content to create interactive experiences between learners and teachers, thereby training the model to become simulated students with learning experiences. To investigate the effectiveness of this approach, we designed a test playground based on the retrospective think‐aloud protocol and examined how LLM‐trained simulated students improve cognitive process transparency and generalization of learning strategies. The study found that Cognitive Echo not only reveals what simulated students genuinely think about their learning experiences but also enables them to transfer their different cognitive strategies to new tasks. By training simulated students on real learning behaviour data to ensure their cognitive processes reflect authentic learner experiences, this approach will extend think‐aloud protocols to more practice‐oriented applications.
Practitioner notes
What is already known about this topic Think‐aloud protocols are widely used in educational settings to explore students' cognitive processes by asking them to verbalize their thoughts while solving problems, but they are prone to issues like task interference and incomplete data reporting.
Existed applications of simulating student cognition in educational research are rigid and less adaptive to individual learner characteristics.
Artificial intelligences, especially large language models, have shown promise in educational contexts, particularly for simulating human‐like behaviours.
What this paper adds This paper introduces the concept of Cognitive Echo, a method that integrates LLM‐powered simulated students into think‐aloud protocols, which addresses the limitations of traditional verbalization‐based methods by leveraging retrospective data.
The study shows that LLMs, when fine‐tuned with authentic learner experiences, can replicate distinct human‐like cognitive processes, enabling a more complete and authentic simulation of how students think and solve problems.
It demonstrates how the use of LLMs to simulate students' cognitive processes can enhance the transparency and completeness of think‐aloud protocols by allowing researchers to capture cognitive strategies and behaviours that would otherwise go unspoken.
Implications for practice and/or policy Teacher training programmes can benefit from integrating LLM‐based simulated students, which enable preservice teachers to practice responding to a wide range of cognitive processes and challenges without the constraints of real‐time think‐aloud tasks.
The Cognitive Echo method, by offering a more authentic and less intrusive way of capturing student cognition, can be applied in teacher training scenarios where simulation of real‐world classroom dynamics is crucial for developing pedagogical skills.
The use of Cognitive Echo could help in the creation of digital twins of educational scenarios, facilitating research into complex educational issues (eg, bullying and learning disabilities) through simulations that model real‐world interactions.
... Ethical approval was received from the local commission for minimal-risk studies of the Department of Psychology of the University of Milan-Bicocca (# RM-2024-779). Participants were invited to take part in a 1-hour long testing session, which included filling out questionnaires and participating in one of two mutually exclusive procedures for usability testing: either the Cognitive Walkthrough (CW; Polson et al., 1992;Jaspers, 2009, n = 11) or the Think-Aloud (TA; Charters, 2003;Jaspers, 2009, n = 8). ...
... The Think Aloud (TA) task was employed to assess the functionality, navigability, and overall ease of use of the technology as end-users interact with it. Participants were encouraged to verbalize their thoughts, intentions, and reasoning as they engaged with the app, helping to identify usability issues and specific challenges, along with the underlying reasons for these difficulties (Charters, 2003;Jaspers, 2009). Participants were instructed to complete 8 designated tasks using the app, as per manual, without guidance from the examiner. ...
... To complement these quantitative measures, we incorporated qualitative analysis to deeply explore user experiences, e.g., Remi-Haven's overall strengths and weaknesses to be improved. During the system use, participants employed the think-aloud protocol [22], articulating their thoughts and reactions in real-time while engaging with the system. Then participants discussed both the system's strengths and areas for improvement in the subsequent semi-structured interviews. ...
... They then selected their preferred reminiscence mode and engaged in 1-2 rounds of reminiscence using RemiHaven on a PC, with researcher's assistance. During the trial, we closely monitored interactions and requested participants to use the "think aloud" method [22] to provide realtime feedback. Afterward, they completed a questionnaire that corresponded to the two parts of the evaluation metrics using a 5-point Likert scale, followed by semi-structured interviews to gain deeper insights into their overall impressions of the system and expectations for further development. ...
With increasing social mobility and an aging society, more older adults in China are migrating to new cities, known as "older drifters." Due to fewer social connections and cultural adaptation challenges, they face negative emotions such as loneliness and depression. While reminiscence-based interventions have been used to improve older adults' psychological well-being, challenges such as the lack of tangible materials and limited social resources constrain the feasibility of traditional reminiscence approaches for older drifters. To address this challenge, we designed RemiHaven, a personalized reminiscence support tool based on a two-phase formative study. It integrates "In-Town" and "Out-of-Town" peer agents to enhance personalization, engagement, and emotional resonance in the reminiscence process, powered by Multimodal Large Language Models (MLLMs). Our evaluations show RemiHaven's strengths in supporting reminiscence while identifying potential challenges. We conclude by offering insights for the future design of reminiscence support tools for older migrants.
... Language acquisition research often grapples with the challenge of capturing a comprehensive understanding of learners' language knowledge as data derived solely from language production can be insufficient to draw conclusions. Think-aloud protocols (TAPs) have emerged as valuable tools in educational research by providing insights into the often invisible cognitive processes that underlie learning activities, which production data alone cannot reveal (Aradi, 2023;Bowles, 2010;Charters, 2003;Fan & Zhu, 2021;van Someren et al., 1994). TAPs involve participants verbalizing their thought processes while engaging in specific tasks, such as reading or problem solving (Bowles, 2008;Chin & Abd Ghani, 2021;Ericsson & Simon, 1984;Pattapong, 2022). ...
... TAPs involve participants verbalizing their thought processes while engaging in specific tasks, such as reading or problem solving (Bowles, 2008;Chin & Abd Ghani, 2021;Ericsson & Simon, 1984;Pattapong, 2022). This method allows researchers to capture real-time data related to learners' strategic approaches, cognitive load, and problem-solving tactics (Aradi, 2023;Bilici et al., 2023;Bowles, 2010;Charters, 2003;van Someren et al., 1994). By analyzing these verbalizations, researchers can gain a deeper understanding of the strategies that are employed by learners, the intricacies of their decision-making processes, and the challenges they encounter. ...
This study aimed to investigate the differences in the vocabulary learning strategies (VLSs) employed by beginner and advanced Saudi learners of English as a foreign language (EFL). The research involved 60 participants who were equally divided between beginner and advanced proficiency levels. The data were collected using a demographic and background questionnaire, think-aloud protocols (TAPs) for real-time assessment of cognitive processes during vocabulary learning tasks, and semi-structured interviews for validation and deeper exploration of the TAP data. The findings revealed that the beginner learners predominantly relied on basic memory and cognitive strategies, such as repetition, rote memorization, and the use of bilingual dictionaries, which facilitated immediate results and foundational vocabulary building. In contrast, the advanced learners employed more complex strategies, including the use of monolingual dictionaries, contextual guessing, and morphological analysis, which promoted a deeper understanding and retention of new vocabulary words. Several factors that influenced VLS selection were identified: the beginners prioritized familiarity, lower cognitive load, and the need for tangible outcomes, whereas the advanced learners sought strategies that provided cognitive challenges and deeper comprehension. The study underscores the need for tailored pedagogical approaches to address the distinct strategy needs and preferences of EFL learners at different proficiency levels.
... La seconde phase du protocole expérimental a engagé les participant·es dans la réalisation en ligne d'une tâche scolaire de littératie informationnelle tout en verbalisant leurs réflexions. La méthode de pensée à voix haute (think aloud protocol), utilisée depuis plus d'un siècle par des chercheur·ses en sciences cognitives et en sciences de l'éducation (Charters, 2003), permet l'étude des processus rattachés à la compréhension du sens en environnement numérique (Anmarkrud et al., 2013;Coiro, 2011;Coiro et Dobler, 2007;Falardeau et al., 2014;Kaakinen et Hyona, 2005;Salmerón et al., 2017). Elle offre un accès valide et fiable aux pensées et aux actions du·de la lecteur·rice (Charters, 2003;Coiro et Dobler, 2007;Pressley et Afflerbach, 1995) tout en offrant la possibilité d'inférer les processus et les stratégies cognitives de manière contextualisée (Falardeau et al., 2014;Hilden et Presley, 2011), ce que l'observation directe du comportement et la mesure de performance ne permettent que partiellement (Forget, 2013). ...
... La méthode de pensée à voix haute (think aloud protocol), utilisée depuis plus d'un siècle par des chercheur·ses en sciences cognitives et en sciences de l'éducation (Charters, 2003), permet l'étude des processus rattachés à la compréhension du sens en environnement numérique (Anmarkrud et al., 2013;Coiro, 2011;Coiro et Dobler, 2007;Falardeau et al., 2014;Kaakinen et Hyona, 2005;Salmerón et al., 2017). Elle offre un accès valide et fiable aux pensées et aux actions du·de la lecteur·rice (Charters, 2003;Coiro et Dobler, 2007;Pressley et Afflerbach, 1995) tout en offrant la possibilité d'inférer les processus et les stratégies cognitives de manière contextualisée (Falardeau et al., 2014;Hilden et Presley, 2011), ce que l'observation directe du comportement et la mesure de performance ne permettent que partiellement (Forget, 2013). Pour répondre aux objectifs de cette étude, le logiciel de montage Camtasia Screen Capture a sauvegardé, sous forme de vidéo, les activités de navigation des participant·es, leurs verbalisations et leurs expressions faciales (Coiro, 2007;2011). ...
Résumé
Le présent article décrit une étude de cas multiples qui s'est penchée sur l'interaction des compé-tences de navigation, d'évaluation et d'intégration au sein d'un processus menant à la compréhension d'informations consultées en ligne, et ce, lors de la réalisation d'une tâche de littératie informationnelle par des lecteur·rices internautes de 6 e année. L'étude combine l'entretien direct, la verbalisation de la pensée, l'entretien libre de verbalisation de la compréhension et l'entretien rétrospectif semi-dirigé. Elle permet l'induction d'une mobilisation autorégulée des compétences, en plus de permettre, au départ des convergences et des divergences observées entre les cas, la formulation d'une hypothèse de profils de lecteur·rices internautes.
Abstract
This article describes a multiple case study that investigated the interaction of navigation, evaluation and integration competencies within a process leading to the comprehension of online informations during the realization of an information literacy task by 6 th grade readers. In combining direct interviews, think-aloud protocol, free interviews of comprehension verbalization and semi-directed retrospective interviews, this study allows the induction of an autoregulated mobilization of the competencies while learning from multimodal and non-linear informations consulted online. It also leads to the hypothesis of five online reader's profiles.
Mots-clés : littératie informationnelle en ligne, multimodalité, compréhension, étude de cas multiples, lecteur·rices internautes de 6 e année.
Keywords : online information literacy, multimodality, comprehension, multiple case study, 6th grade online readers
... From her review of the literature, Charters (2010) has found that thinkaloud research methods are supported by a strong theoretical foundation and give a valid data source about the participants' thought processes, particularly within language-oriented activities. However, due to possible variations in research purposes and, hence, data collection procedures, there can be differences in issues confronted. ...
... The varied implementations of the think-aloud reports in this article show that despite some controversies arising, this way of reporting the thought processes has high acceptance and beneficial initially for research but later on for teaching as well. Most of the studies reviewed have shown the accuracy of the think-aloud reports in revealing what is actually going on in the participants' thought processes (Charters, 2010;Samson, et al., 2017;Goldstein, 2011). The reviews on the implementation of think-aloud reports collected by Bowles (2015) show that 90% of the studies implementing think-aloud protocols have no problem with veridicality. ...
Thinking-Aloud or Think-Aloud Protocols have been used to study thinking processes in varied fields such as psychology, business, sports, industry, and education. Recently, studies have tried to see its application in teaching in both offline and online modes. Despite its success, as demonstrated by these studies, controversies concerning the believability of thinking aloud and the process of reactivity have been raised. However, no explicit solution has been provided. In response to the issues, this paper critically reviewed examples of think-aloud protocols using Bowles’s guide to TAP (2008). The results show that (1) problems of the reactivity process stem from the nature of the tasks given, and (2) to minimize the problems of reactivity, the TAP data collection procedure should move from the least to the most intervention of the researcher.
... After each gesture was recorded, the expert reviewed its animated 3D visualisation, from which they could choose to re-improvise if needed. Throughout the improvisation session, the researcher encouraged the expert to continuously reflect on their design choices using the think-aloud method [71]. Once all six gestures were recorded, the expert viewed their 2D animations side-by-side in the same order that they designed while completing a questionnaire for each gesture. ...
The vision of adaptive architecture proposes that robotic technologies could enable interior spaces to physically transform in a bidirectional interaction with occupants. Yet, it is still unknown how this interaction could unfold in an understandable way. Inspired by HRI studies where robotic furniture gestured intents to occupants by deliberately positioning or moving in space, we hypothesise that adaptive architecture could also convey intents through gestures performed by a mobile robotic partition. To explore this design space, we invited 15 multidisciplinary experts to join co-design improvisation sessions, where they manually manoeuvred a deactivated robotic partition to design gestures conveying six architectural intents that varied in purpose and urgency. Using a gesture elicitation method alongside motion-tracking data, a Laban-based questionnaire, and thematic analysis, we identified 20 unique gestural strategies. Through categorisation, we introduced architectonic gestures as a novel strategy for robotic furniture to convey intent by indexically leveraging its spatial impact, complementing the established deictic and emblematic gestures. Our study thus represents an exploratory step toward making the autonomous gestures of adaptive architecture more legible. By understanding how robotic gestures are interpreted based not only on their motion but also on their spatial impact, we contribute to bridging HRI with Human-Building Interaction research.
... User testing sessions were guided by the Think-Aloud Testing protocol method (Charters, 2003), which encouraged participants to articulate their thoughts and actions in real-time as they interacted with the mHealth tool. During the think-aloud testing session, we asked a series of questions to understand CHWs' experiences with the mHealth supervision tool. ...
... Qualitative think-aloud interviews allowed for exploring students' thoughts with minimal prompting questions [12]. Though we used purposeful sampling to identify Latinas, we encountered difficulties during recruitment, highlighting the difficulty of conducting equitycentered research with underrepresented communities. ...
... Once the first versions of the educational activities with Oqtant were completed, we began testing them with students using think-aloud interviews [75]. These interviews occurred over Zoom while Oqtant was online, so the students could submit jobs and receive the results during the interview. ...
As quantum technologies transition from the research laboratory into commercial development, the opportunities for students to begin their careers in this new quantum industry are increasing. With these new career pathways, more and more people are considering the best ways to educate students about quantum concepts and relevant skills. In particular, the quantum industry is looking for new employees with experimental skills, but the instructional labs, capstone projects, research experiences, and internships that provide experiences where students can learn these skills are often resource intensive and not available at all institutions. The quantum company, Infleqtion, recently made its online quantum matter machine, Oqtant, publicly available, so people around the world could send commands to create and manipulate Bose-Einstein condensates and receive back real experimental data. Making a complex quantum experiment accessible to anyone has the potential to extend the opportunity to work with quantum experiments to students at less-resourced institutions. As a first step in understanding the potential benefits of using such a platform in educational settings, we collected data from instructors and students who were interested in using, or had used, Oqtant. In this study, we investigate instructors’ views about reasons they would like to use Oqtant and the challenges they would face in doing so. We also provide a concrete example of how Oqtant was used in an upper-division undergraduate quantum mechanics course and the instructor’s perception of its benefits. We complement this with the student perspective, discussing student experiences interacting with Oqtant in their course or through think-aloud interviews outside of a course. This allows us to investigate the reasons students perceive Oqtant to be a real experiment even though they never physically interact with it, how Oqtant compares to their other experimental experiences, and what they enjoy about working with it. These results will help the community consider the potential value of creating more opportunities for students to access remote quantum experiments.
Published by the American Physical Society 2025
... Although generally used in studies attempting to understand problem-solving and learning, think aloud methods have been used to gain understanding about a wide range of research goals including students' writing skills (Charters, 2003), the thought processes of distance runners (Samson et al., 2017), as well as a means to validate survey instruments (Pepper et al., 2018). ...
Over the past two decades there has been significant reform to the post-primary mathematics curriculum in Ireland. The introduction of Project Maths in 2010 placed an increased emphasis on problem-solving within the curriculum and encouraged teachers to adopt problem-solving approaches to the teaching and learning of mathematics. A further reform at Junior Cycle again stressed the importance of problem-solving within mathematics and introduced classroom-based assessments, one of which is explicitly focused on problem-solving. Despite these extensive reforms, there is little evidence that much has changed in post-primary classrooms and teachers have expressed concerns regarding the reform and the new pedagogical approaches promoted within them (Berry et al., 2021; Byrne & Prendergast, 2020; Jeffes et al., 2013). This suggests that learners may not be experiencing the types of learning environments that build their capacities as problem-solvers.
This research aims to support young learners to become proficient and confident problem solvers by working with their teachers to develop and strengthen their problem-solving pedagogy. If teachers are struggling to enact the changes to their practices as envisaged by the reform, it is important to understand why so that appropriate support can be offered. The first phase of this research therefore investigated teachers’ perceptions of problem-solving and concerns related to incorporating it in the classroom. Data from semi-structured interviews with teachers show that teachers need clear guidance on how to enact problem-solving approaches in their classroom along with opportunities to understand how their role as a teacher is impacted, and to critically reflect on their current practices and beliefs around the teaching and learning of mathematics. In addition, teachers expressed a desire for more collaboration with colleagues and for suitable resources to use with problem-solving in their classrooms. These findings informed the second phase of the research. A professional development intervention was developed which comprised specifically designed educative curriculum materials alongside lesson study. A case-study was conducted to explore the impact of this professional development intervention on teachers’ knowledge, beliefs, and practices related to problem-solving. Findings from the case-study revealed that engagement with the educative curriculum materials and lesson study impacted on teachers mathematical problem-solving knowledge for teaching, in particular their knowledge of Structured Problem Solving practices. Moreover, it supported the teachers in their journeys of teacher change, through surfacing tensions in their practice, encouraging reflection on tensions, and assisting them in seeking resolutions to these tensions.
... Ride-hailing services already provide several responsible consumption nudges, such as carbon offset, tree planting, and using electric vehicles. In this research, we combined a quantitative study using the Socially Responsible Purchase and Disposal (SRPD) questionnaire [26] to measure the participant's attitude towards responsible consumption and a qualitative study using a loud protocol [27] and interviews to observe the participant's behavior when using the ride-hailing services. The participant objectives of our study are individuals belonging to the young generation, namely within the age range of 18-29 years. ...
This study thoroughly examined responsible consumption implemented in Indonesia's ride-hailing services. We employed a mixed-methods approach that integrated quantitative data from the Socially Responsible Purchase and Disposal (SRPD) questionnaire with qualitative insights derived from think-aloud protocols and interviews. Our objective was to examine data and identify discrepancies between attitudes and behaviors among young individuals aged 18-29 to provide directions for future research. The results of our study uncovered substantial disparities between the attitudes that participants expressed and their subsequent actions. Although individuals acknowledge the importance of responsible consumption, the testing shows that they rarely use features related to responsible consumption, such as carbon funds, electric vehicle options, and tipping. Several key factors they stated are increased cost, lack of awareness of the features of responsible consumption, lack of perceived trust, and transparency of funds. Based on the interview, the personal connection between participants and ride-hailing drivers encourages them to tip more. Our research highlights the critical gap between attitudes and actions in responsible ride-hailing services in Indonesia's younger generation, which can be solved by implementing proper digital nudges. This research emphasizes the importance of further research to bridge this gap and encourage more accountable digital consumption.
... 3.4.1 Semi-structured interview and Think-aloud method. After each improvisation session, dancers were encouraged to express their feelings and thoughts when dancing with different avatars (using the Think aloud method [31]). Participants were asked after interacting with each avatar whether they noticed the change of avatar and what the change was as an awareness check. ...
The use of motion capture in live dance performances has created an emerging discipline enabling dancers to play different avatars on the digital stage. Unlike classical workflows, avatars enable performers to act as different characters in customized narratives, but research has yet to address how movement, improvisation, and perception change when dancers act as avatars. We created five avatars representing differing genders, shapes, and body limitations, and invited 15 dancers to improvise with each in practice and performance settings. Results show that dancers used avatars to distance themselves from their own habitual movements, exploring new ways of moving through differing physical constraints. Dancers explored using gender-stereotyped movements like powerful or feminine actions, experimenting with gender identity. However, focusing on avatars can coincide with a lack of continuity in improvisation. This work shows how emerging practices with performance technology enable dancers to improvise with new constraints, stepping outside the classical stage.
... The participants were able to easily refer back to scenarios throughout the interview, which led to their reading, arranging, comparing, rereading, and talking through the underpinnings of their beliefs about the acceptability of each writing assistance scenario. Similar to the gains achieved through a think-aloud protocol or technique, the study's implementation of scenarios provided insight into the participants' thought processes (Charters, 2003). When asked about their consideration of certain writing assistance activities, some participants explained their views by comparing scenarios, such as: ...
Scenarios, or short descriptions of hypothetical situations, can serve as a methodological tool to gather insights and develop an understanding about participants’ perceptions. Studies have made use of scenarios to gauge student beliefs about the acceptability of various writing activities that may be considered help, plagiarism, or something else. These studies often employed scenarios within surveys comprised of single- or multiple-choice queries, leaving little space for participants’ views beyond binary responses and no room for their detailed explanations. Using scenarios as a way to start an in-depth discussion, however, offers opportunities to explore students’ perspectives and reasonings on using writing assistance from third parties. This article describes a unique implementation of a bespoke set of scenarios in a qualitative study that examined undergraduate students’ perceptions about third-party writing assistance. Using these distinctive scenarios represents a contribution to how we collect data on students’ perspectives. This method is particularly applicable for qualitative research studies exploring complex participant perceptions about nuanced phenomena within education.
... The guides also related topics crucial to women's decision-making process when contemplating various healthcare access options, including their prior experiences with VCs, telemedicine, smartphone usage patterns, and other relevant factors influencing their healthcare choices. We also employed the think-aloud protocol [34,35] in interviews to gain deeper insights into the decision-making processes of UO women regarding their choice between VCs and ICCs. This method allowed women to verbalize their thoughts as they considered various healthcare scenarios, providing data on the cognitive processes underlying their preferences. ...
Video consultations in primary care settings demonstrate substantial benefits, including improved accessibility, reduced waiting times, and enhanced health management. These services could particularly benefit ultra-Orthodox women in Israel, who typically manage large families and face unique healthcare access challenges as primary caregivers. However, eliciting preferences within this closed religious community presents distinct methodological challenges because of cultural sensitivities and religious restrictions regarding technology use.
We aimed to develop and validate a culturally sensitive, discrete choice experiment questionnaire for eliciting ultra-Orthodox women’s preferences regarding video versus in-clinic consultations in primary care settings.
A three-stage mixed-methods approach was employed: (1) 33 semi-structured interviews with key stakeholders (women, men, rabbis, and healthcare providers) to identify attributes and levels; (2) an attribute-ranking exercise with 88 ultra-Orthodox women to refine attributes; and (3) cognitive interviews with 15 women to validate the discrete choice experiment questionnaire.
Four key attributes emerged as most important for ultra-Orthodox women when choosing between video and in-clinic consultations: (1) consultation timing (regular hours/after 20:00); (2) travel time; (3) waiting time; and (4) familiarity with the healthcare provider. Importantly, the study revealed the necessity for a dedicated device exclusively for healthcare provider communication, closed to open Internet networks, as a fundamental prerequisite for implementing video consultations in this community. Additional unique findings emerged through this methodological process, contributing to the understanding of technological adoption in closed religious patients’ communities.
This study provides a comprehensive example of implementing pre-discrete choice experiment stages while addressing unique considerations of a special population. The findings provide a framework for developing inclusive telemedicine services for traditionally underserved populations.
... Visual Information, Satisfaction and Preferences. We used the think-aloud method [8] to evaluate the visual information participants observed when viewing the artifacts and to collect participants' subjective ratings about the artifact displays. Specifically, we guided participants to evaluate the visual information observed in the front of the artifact, and when they were grabbing artifacts using their hands. ...
The viewing experience of 3D artifacts in Virtual Reality (VR) museums is constrained and affected by various factors, such as pedestal height, viewing distance, and object scale. User experiences regarding these factors can vary subjectively, making it difficult to identify a universal optimal solution. In this paper, we collect empirical data on user-determined parameters for the optimal viewing experience in VR museums. By modeling users' viewing behaviors in VR museums, we derive predictive functions that configure the pedestal height, calculate the optimal viewing distance, and adjust the appropriate handheld scale for the optimal viewing experience. This led to our novel 3D responsive design, ResponsiveView. Similar to the responsive web design that automatically adjusts for different screen sizes, ResponsiveView automatically adjusts the parameters in the VR environment to facilitate users' viewing experience. The design has been validated with two popular inputs available in current commercial VR devices: controller-based interactions and hand tracking, demonstrating enhanced viewing experience in VR museums
... This approach helped identify misunderstandings, ambiguities and contextual associations. The Think-Aloud approach complemented the semi-structured interviews and provided real-time data on how patients interpreted and processed survey questions [21,22]. ...
Objective
The 11‐item Patients' Perception of Safety Culture Scale (PaPSC) measures patients' perceptions of the safety culture within healthcare organizations. While patients can respond to these items, factors such as limited knowledge, unclear roles and insufficient information may influence their assessments. Despite previous research on the PaPSC, no qualitative validation has been conducted. This study addresses this gap by exploring patients' perspectives on safety culture.
Methods
A qualitative, exploratory approach was adopted, employing problem‐centred interviews with patients from the cardiology and cardiothoracic surgery wards of a tertiary care university hospital. Thematic coding combining deductive and inductive methods was used to identify predefined and emergent themes.
Results
Data saturation was reached after 22 interviews, providing a comprehensive account of patients' experiences. Most patients were able to respond to the PaPSC items, confirming its robustness in assessing safety culture. However, they emphasized interpersonal aspects such as empathy, trust and clear communication, and contextual elements such as the care environment and patient–staff interactions, which are less well represented in standardized instruments. Several items were perceived as ambiguous, highlighting the need for explanatory text to enhance clarity and response accuracy.
Conclusions
This study underscores the value of integrating qualitative methods with standardized tools such as the PaPSC to reveal nuanced aspects of safety culture. Active patient involvement in tool development can improve the comprehensiveness and effectiveness of safety interventions. Ensuring that safety culture assessments accurately reflect patients' experiences and needs contributes to more patient‐centred healthcare practices.
Patient Contribution
Problem‐centred interviews and the Think‐Aloud method were used in this study to ensure active patient participation. Patients contributes by identifying areas where survey items needed further clarification or contextualization, thereby enhancing the validity and usability of the Patients' Perception of Safety Culture Scale (PaPSC). Their feedback also led to the refinements in the study design and tools, underscoring the importance of patient‐centred approaches in healthcare safety research. Although patients were not directly involved in the study design, since the PaPSC scale items were predefined and derived from a prior critical review of existing instruments, their role in this validation study was crucial. The study aimed not to develop new items but to assess the applicability and clarity of an established instrument. The study empowered the patients to share their views openly in a supportive and respectful environment, offering valuable insights for improving safety culture assessment.
... (47)(48)(49)(50)(51)(52)(53)(54) Qualitative Data: A Think-Aloud approach was used for qualitative interviews in which participants were asked to verbalize their experiences around a specific task -in this case the use of TRACH-Support. (55)(56)(57) Participants were asked to focus on domains of Usability, Acceptability, and potential modifications (eTable 2). A matrix technique was used to analyze the data.(58-60) ...
Background: Tracheostomy and prolonged mechanical ventilation decision-making is one of the most emotionally difficult decisions facing surrogate decision-makers in health care. Often, surrogates face decisions between the potential for prolonged life support verses transitions to comfort measures and possible death. Despite more than two decades of research, major gaps exist in improving the decision-making process.
Objective: Develop and pilot testing a novel shared decision-making tool for tracheostomy and prolonged mechanical ventilation.
Methods: Development of the novel web-based conversation tool called TRACH-Support was an iterative process engaging key stakeholders (patients, surrogates, critical care providers, and shared decision-making experts) at multiple points. Development of the website used a Human-Centered Design approach with modern graphics and website interfaces. Pragmatic pilot testing was a mixed methods approach recruiting surrogates, providers, nurses, and respiratory therapists. Primary quantitative outcomes included Usability (System Usability Scale (SUS)) and Acceptability (Acceptability of Intervention (AIM)) measures with multiple secondary outcomes. Qualitative interviews used a Think Aloud approach and matrix analysis methodology.
Results: A total of 86 participants were recruited for the quantitative survey with 10 surrogates and 10 providers completing qualitative interviews. Mean SUS score among all participants was 68.2/100 (SD=10.7) but surrogates specifically had a mean SUS=74.2/100 (74.2). The overall mean AIM score was 4.2/5 (SD=0.8) and 79.4% of all participants viewed TRACH-Support as Acceptable or Very Acceptable. Qualitative interviews indicated that TRACH-Support had high Usability and Acceptability. Customizability, pictures, novel outcomes, and the organization were all features that contributed to participant views. Participants also suggested several modifications including reducing the word count, adding video testimonials, and adding information on how faith/religion may play a role in decision-making for some.
Conclusions: TRACH-Support, a novel, web-based, customizable and personalizable conversation tool for tracheostomy and prolonged mechanical ventilation was developed according to the most rigorous standards for decision-support tools. It had high Usability and Acceptability as assessed by quantitative and qualitative measures. Future large-scale testing is needed to assess real-world effectiveness and implementation.
... The Rehabilitation Science graduate students underwent formal coursework in qualitative methods, in addition to training by the senior authors. The transcripts of the think-aloud protocol and subsequent interviews were segmented into sensible chunks or communication units, which-after calibration-were thematically coded independently by two graduate student authors (E.A.G., E.S., S.B., J.H., A.E., and R.P.C.) in accordance with best practices [23][24][25]. The coding of the think-aloud utterances focused on thoughts reflecting ways in which the education met their needs as women, or not, and could be improved. ...
Patient education is a core component of cardiovascular rehabilitation (CR). Recent guidelines call for tailored materials to support women, given their unique needs and preferences. This study investigated women patient's perspectives of online CR educational materials (Cardiac College, Toronto). Through a cross-sectional design, a think-aloud protocol followed by semistructured qualitative interviews was used to collect data on women's perceptions of the comprehensive educational webpages with regard to content, format, visuals, length/volume, difficulty, applicability, implementability and suggestions for improvement. Thirty-eight current and previous CR program participants completed 1-h Zoom interviews. Transcripts were coded thematically using best practices in NVivo by two researchers independently. Four themes emerged: (I) materials met their needs (content, presentation, and empowerment), (II) suggestions for improvement in presentation (website design, text, and visuals), (III) content (volume, additions, and difficulty), and (IV) optimizing reach and implementability (inclusiveness, barriers, and dissemination). Overall, the education content met women's needs and was relatable, but should be updated for currency, visual appeal, and searchability. Cardiac College for Women may meet these needs and preferences.
... Once the first versions of the educational activities with Oqtant were completed, we began testing them with students using think-aloud interviews [76]. These interviews occurred over Zoom while Oqtant was online, so the students could submit jobs and receive the results during the interview. ...
As quantum technologies transition from the research laboratory into commercial development, the opportunities for students to begin their careers in this new quantum industry are increasing. With these new career pathways, more and more people are considering the best ways to educate students about quantum concepts and relevant skills. In particular, the quantum industry is looking for new employees with experimental skills, but the instructional labs, capstone projects, research experiences, and internships that provide experiences where students can learn these skills are often resource-intensive and not available at all institutions. The quantum company, Infleqtion, recently made its online quantum matter machine Oqtant publicly available, so people around the world could send commands to create and manipulate Bose-Einstein condensates and receive back real experimental data. Making a complex quantum experiment accessible to anyone has the potential to extend the opportunity to work with quantum experiments to students at less-resourced institutions. As a first step in understanding the potential benefits of using such a platform in educational settings, we collected data from instructors and students who were interested in using, or had used, Oqtant. In this study, we investigate instructors' views about reasons they would like to use Oqtant and challenges they would face in doing so. We also provide a concrete example of how Oqtant was used in an upper-division undergraduate quantum mechanics course and the instructor's perception of its benefits. We complement this with the student perspective, discussing student experiences interacting with Oqtant in their course or through think-aloud interviews outside of a course. These results will help the community consider the potential value for students of creating more opportunities to access remote quantum experiments.
... Yet, its analysis can be approached from both quantitative and qualitative perspectives, especially in mixed-methods research (Creswell & Plano-Clark, 2018). As Charters (2003) explains, both approaches are valuable but yield different insights: the quantitative approach extracts numerical patterns from the codes to generalize participants' experiences, while the qualitative approach provides a naturalistic perspective, detailing individual experiences. In our research, we have also used the quantitative approach to extract those numerical patterns (Panadero et al., 2020 and the qualitative approach to create general profiles of student self-assessment methods (Panadero et al., under review), as will be discussed later. ...
... Aufgrund der aktuellen überschaubaren Anzahl an Erhebungsinstrumenten im Bereich KI, ML und DL greifen wir auf das Instrument von Rodríguez-García, Moreno-León, Román-González und Robles (2021) zurück das bereits validiert und in einer weiteren Studie im schulischen Kontext eingesetzt wurde. Da die Erfassung von prozessbezogenen Kompetenzen im NwT-Unterricht ein generelles Forschungsdesiderat darstellt (Brändle, 2023) und bislang wenig zu den Prozessen bei Schüler*innen während dem Data Lifecycle bekannt ist (vgl. A. Grillenberger, 2019) soll nach den Lerngelegenheiten mit der think-aloud (Charters, 2003) oder der teachback-Methode (Fincher & Robins, 2019, S. 468) den Schüler*innen eine Aufgabenstellung ähnlich zu dem Kontext der Lerngelegenheiten gegeben werden, um die prozessbezogenen Kompetenzen zu beobachten. Somit soll mit zwei empirischen Zugängen die Wirkung der Intervention auf die Data Literacy Fähigkeiten der Schüler*innen durch die Lerngelegenheiten überprüft und damit erste Ansätze für die empirische Erfassung der prozessbezogenen Facetten generiert werden. ...
In den aktuellen Bildungsplänen der MINT-Fächer in Baden-Württemberg finden zukunftsrelevante Technologien wie maschinelles Lernen bislang kaum Berücksichtigung. Es stellt sich daher die Frage, wie maschinelles Lernen und zugehörige Grundlagen wie z. B. Data Literacy in bestehende Curricula im MINT-Unterricht integriert werden können. Konzeptideen finden sich momentan hauptsächlich im Fachbereich Informatik. Um mit dem dynamischen Fortschritt von (digitalen) Technologien Schritt zu halten, ist es wichtig, dass technische Allgemeinbildung diese Entwicklungen berücksichtigt. Im Unterricht müssen jedoch einerseits bestehende Inhalte, deren allgemeinbildender Charakter bereits zur Diskussion stand, aus den Curricula durchgeführt und andererseits neueste technologische Entwicklungen integriert werden. Der vorliegende Beitrag befasst sich aus diesem Grund mit Integrationsmöglichkeiten zukunftsrelevanter Technologien, hier maschinelles Lernen und Data Literacy, in den MINT-Unterricht der Sekundarstufen I und II im Schulfach Naturwissenschaft und Technik in Baden-Württemberg.
The current curricula for STEM subjects in Baden-Württemberg do not yet include future-oriented technologies such as machine learning. The question therefore arises as to how machine learning and associated fundamentals such as data literacy can be integrated into existing STEM curricula. Concept ideas can currently be found mainly in the field of computer science. To keep pace with the dynamic progress of (digital) technologies, it is important that general technical education adapts flexibly to these developments. However, existing content from the curricula must be implemented in lessons on the one hand and the latest technological developments must be integrated on the other. For this reason, this article looks at ways of integrating future-relevant technologies into STEM lessons in the subject Science and Technology at lower and upper secondary level in Baden-Württemberg.
... Think-aloud methods are widely used in usability research to gain insights into users' cognitive processes during task performance [18,21]. This method involves participants verbalizing their thoughts while completing a task, providing researchers access to unobservable mental processes during user interactions [8]. Think-aloud has been applied in various fields in HCI, such as education and qualitative research [11]. ...
Think-alouds are a common HCI usability method where participants verbalize their thoughts while using interfaces. However, their utility in cross-cultural settings, particularly in the Global South, is unclear, where cultural differences impact user interactions. This paper investigates the usability challenges teachers in rural C\^ote d'Ivoire faced when using a chatbot designed to support an educational program. We conducted think-aloud sessions with 20 teachers two weeks after a chatbot deployment, analyzing their navigation, errors, and time spent on tasks. We discuss our approach and findings that helped us identify usability issues and challenging features for improving the chatbot designs. Our note summarizes our reflections on using think-aloud and contributes to discussions on its culturally sensitive adaptation in the Global South.
... Qualitative research methods (e.g., open-ended questions and Think Aloud methods (Charters, 2010)) were chosen alongside quantitative analysis (e.g., success task rates Nielsen & Levy, 1994) to offer a more comprehensive understanding of the usability within the PEnguIN app. The incorporation of both qualitative and quantitative research methods was driven by several considerations. ...
... After each gesture was recorded, the expert reviewed its animated 3D visualisation, from which they could choose to re-improvise if needed. Throughout the improvisation session, the researcher encouraged the expert to continuously reflect on their design choices using the think-aloud method [71]. Once all six gestures were recorded, the expert viewed their 2D animations side-by-side in the same order that they designed while completing a questionnaire for each gesture. ...
The vision of adaptive architecture proposes that robotic technologies could enable interior spaces to physically transform in a bidirectional interaction with occupants. Yet, it is still unknown how this interaction could unfold in an understandable way. Inspired by HRI studies where robotic furniture gestured intents to occupants by deliberately positioning or moving in space, we hypothesise that adaptive architecture could also convey intents through gestures performed by a mobile robotic partition. To explore this design space, we invited 15 multidisciplinary experts to join co-design improvisation sessions, where they manually manoeuvred a deactivated robotic partition to design gestures conveying six architectural intents that varied in purpose and urgency. Using a gesture elicitation method alongside motion-tracking data, a Laban-based questionnaire, and thematic analysis, we identified 20 unique gestural strategies. Through categorisation, we introduced architectonic gestures as a novel strategy for robotic furniture to convey intent by indexically leveraging its spatial impact, complementing the established deictic and emblematic gestures. Our study thus represents an exploratory step toward making the autonomous gestures of adaptive architecture more legible. By understanding how robotic gestures are interpreted based not only on their motion but also on their spatial impact, we contribute to bridging HRI with Human-Building Interaction research.
... The researchers chose this mixed method approach because it could "broaden the scope of the investigation and enrich the researcher's ability to draw conclusions" (Dörnyei 2007: 186). The main source of data was concurrent think-aloud protocol as it provided the best access to participants' cognitive processes during the execution of a writing task (Charters 2003). With regard to reliability, sample composition and severity ratings of think-aloud data, the worldwide study results are still conflicting. ...
Foreign language (FL) proficiency assumes effective application of all four language skills in order to communicate properly in the target language. The development of writing in particular has been acknowledged as one of the main challenges in FL learning and teaching. Against the background of increasing L2 writing strategy research in recent years (Leki 1998; Sasaki 2000; Casanave 2004), scholars have been focusing on learners’ writing strategies in various writing tasks. However, empirical research on FL writing competence in the Croatian context has yet to receive sufficient attention. In response to this need, the present study sought to explore the writing processes and products of eight Croatian EFL university students during their collaborative work on argumentative essays. We opted for a case study design. The aim of this case study is threefold: (1) to identify the writing strategies that EFL students predominantly employ in the process of essay writing within the traditional cognitive framework, (2) to explore participants’ attitudes toward collaborative writing, and (3) to investigate if there are any distinctions in the use of writing strategies by skilled and less-skilled student writers. We analysed and compared data collected from a think-aloud protocol, immediate retrospective interviews, and written drafts of papers. The results indicated that the participants utilized a relatively high level and diverse repertoire of writing strategies. The findings also showed that there are no significant differences in the number of writing strategies used by skilled and less-skilled writers. However, the former displayed a more effective strategy usage than the latter. Accurate application of writing strategies, explicit teaching in class, awareness of different rhetorical conventions of academic writing in other languages, and individual factors are identified as the key elements of discourse competence. The findings are discussed in light of FL writing competence and implications for future teaching pedagogy and practice.
... To explore how participants interact with the co-constructive tool PSLVIS and the resulting PSL, and to identify challenges encountered during their application, we employ the think-aloud method. This qualitative research method is used to elicit cognitive processes by requiring participants to verbalize their thoughts while performing tasks, with these verbalizations recorded for subsequent analysis [4,24]. The think-aloud method serves multiple purposes, including documenting decision-making processes [19,23] and assessing the usability and perception of products such as software [1,8,22,25]. ...
This study explores the co-construction of probabilistic scoring systems. Using a self-developed web-based tool, called PSLVIS, participants were able to create their own decision-support models through an interactive interface. Seven academic advising experts participated, assessing the probability of student success both with and without the assistance of a Probabilistic Scoring List (PSL). The results indicate that while the co-constructed models slightly improved the experts’ accuracy, they also increased decision time. Experts interacted with PSLVIS and PSL in diverse ways, displaying different levels of algorithmic aversion and appreciation. This study underscores the potential of decisionsupport systems that integrate data-driven algorithms with human expertise, while also revealing the wide range of challenges that need to be addressed for successful co-construction and practical implementation.
... Field testing data was collected by the first author (JL) through cognitive debriefing via one-on-one semistructured interviews, utilizing the think-aloud technique [31]. The test leader was a licensed audiologist with previous experience in qualitative interviewing for research purposes. ...
Objective
Translating the newly developed Empowerment Audiology Questionnaire, EmpAQ-15 to Swedish, and performing content validation on the Swedish version.
Design
Best-practice principles using forward and back translations which were revised by a committee prior to field testing. Field testing was conducted by cognitive interviews with hearing-aid users talking through and rating the items in the translated questionnaire. Content validation was assessed by examining equivalence, accessibility, acceptability, comprehensiveness, and relevance of interview data. Questionnaire introduction and scoring instructions were evaluated by Swedish audiologists.
Study sample
Ten adult native speaking Swedish hearing aid users, recruited with purposive sampling. Maximum variation based on age, gender, hearing aid usage, and degree of hearing loss. Seven Swedish audiologists assessing instructions for result calculations.
Results
The conceptual equivalence between the Swedish translation and the English original questionnaire was judged to be high overall. The instructions and majority of items were experienced as accessible, acceptable, comprehensive, and relevant. The audiologists showed that they could follow scoring instructions and reason about the results.
Conclusions
This content validity study was the first step towards a Swedish version of a self- report measure of Empowerment for people with hearing loss.
... As part of the think-aloud research technique, the examiner speaks the thoughts that come to his/her mind. Charters [67] stated that the think-aloud technique is one of the most effective ways to assess higher-level thinking processes. ...
With the increased number of mobile apps, authentication processes play a key role in verifying users’ identities and protecting data from security threats. Utilizing proper authentication techniques is key to protecting computer apps from being hacked. In this paper, we aimed to compare the authentication methods of the sign-up, sign-in, and password recovery processes of 50 e-commerce apps. To ensure accurate data analysis, we checked every app in a separate session and used the “think-aloud” technique while recording the screen. The researchers prepared a list of items that were checked during each session to identify the similarities and differences between tested apps regarding the authentication process. The results of this security analysis unequivocally demonstrated how different apps’ designs for authentication processes are. Users’ memory and comprehension are burdened by these variances, and no app can ensure that they adhere to recommended standards. The results of this study confirmed the necessity for unified and user-friendly authentication processes. This can be possible by following a usable security framework for the authentication process.
... As Zuckerberg [8] puts it, one might debate whether these modifications are for the better or if Facebook's impact on society is producing unintended consequences. Under the umbrella of social media research, particularly Facebook, this study examines potential impacts and issues on users' continuous interpersonal communication as a result of new technological advancements [9], [10], [11], [2]. This study employs a qualitative interview approach with sixteen distinct Facebook users to better understand their usage and perspectives on the topic. ...
In recent years, the evolution of social media, particularly Facebook, has introduced tools that significantly accelerate social interactions. This study investigates the usage of Facebook’s reactionbutton and its impact on interpersonal communication, focusing on immediacy, media richness, and the societal drive for speed. This paper argues that, while the reaction-button enhances the speed and ease of communication, it may reduce the depth and clarity of interactions, potentially leading to miscommunication. This study employed a qualitative method involving in-depth interviews with 16 Facebook users. The participants provided insights into their experiences using the reaction-button and its effects on their social relationships. Results showed the widespread preference for the reaction-button over the traditional 'like' feature. Participants regard reactions as a timeefficient tool that enhances online engagement, though the immediacy of reactions sometimes undermines deeper communication. This suggests that, despite promoting faster communication, the reaction-button may undermine meaningful engagement. These findings have significant implications for understanding the evolving nature of digital interactions and highlight the need for social media platforms to balance speed with clarity.
... This empirical study employed a mixed-methods approach, combining both qualitative and quantitative methods and data analysis. The two central research methods in this study were: 1) a think-aloud protocol, as developed by Ericsson and Simon (1984) and described by Charters (2003), and 2) a retrospective interview, which was conducted with each participant after they completed indexing and tagging tasks. The think-aloud protocol was conducted during the subject indexing and subject tagging portion of the study. ...
Controlled vocabularies are widely used in Knowledge Organization Systems (KOS); however; they are criticized for perpetuating biases, being slow to change, and not reflecting the language of many groups and cultures. This paper examines and challenges these criticisms by identifying five forms of vocabulary control in use today (minimal control, passive control, post hoc control, flexible control, and rigid control), and studying their effect on the subject indexing and subject tagging processes from the perspective of novice indexers and taggers. The study uses a mixed methods approach, including a survey, a think-aloud protocol that was employed while participants indexed and tagged documents, and a retrospective interview, to better understand participants’ actions, thoughts, and reactions during the indexing and tagging process. The study explores how controlled vocabularies make indexers and taggers feel controlled, in control, or out of control while they index and tag; measures how different forms of vocabulary control affect coextensiveness between aboutness statements and indexing terms; and analyzes how these experiences inform the criticisms of controlled vocabularies, particularly in relation to how vocabulary control aids and hinders racial and social justice in indexes. The study finds that the form of vocabulary control has a significant effect on the subject indexing and subject tagging processes, including an effect on participants’ feelings of control during the processes, an effect on how participants transformed concepts during subject representation, and an effect on how participants navigate complex issues related to racial and social justice.
BACKGROUND
Many patients are unaware of their stroke risk. The purpose of this research was to compare the effect of behaviorally tailored mailed messages on patient activation to reduce stroke risk.
METHODS
Randomized parallel group clinical trial. We used electronic health records to construct Framingham Stroke Risk Scores in primary care patients from 1 Veterans Health Administration (VA) and 1 non-VA healthcare system, Eskenazi Health System (EHS). Four stroke risk messages were developed through patient interviews: standard, incentive (5 incentive was more effective than a standard message in engaging high-risk patients with their healthcare system, including a salience message may also be important in some patient populations. Many primary care patients are unaware of their stroke risk.
REGISTRATION
URL: https://www.clinicaltrials.gov ; Unique identifier: NCT02721446.
Since 2010, nine states in the United States have passed legislation supporting K-12 ethnic studies. However, we know little about how K-12 teachers respond to ethnic studies mandates. Through semistructured interviews with six White teachers, this article focuses on the perspectives and experiences of high school teachers establishing ethnic studies in Indiana. The study reveals that educators’ interpretation and translation of ethnic studies policy is a complex and dynamic process that is influenced by three primary factors: (a) teaching experience, (b) equity consciousness, and (c) sociopolitical and local contexts.
I denne artikel undersøges, hvordan elever i læseforståelsesvanskeligheder konstruerer mening, når de læser litterære tekster. Studiets forskningsinteresse retter sig således mod elevers kognitive læseforståelsesproces under læsning af en særlig teksttype (litterære tekster) i en særlig kontekst (litteraturundervisningen). Følgelig er studiets teoretiske rammeværk flerfokalt og omfatter dels kognitionspsykologisk teori om læseforståelsesprocessen samt sociokognitiv teori om læsning af litterære tekster i en skolekontekst. Gennem metoden Verbal Protocol udskilles tre læserprofiler: infererende læsere, parafraserende læsere og associative læsere, som tilsammen tegner et nuanceret billede af, hvordan læseforståelsesprocessen kommer til udtryk hos elever med læseforståelsesvanskeligheder uden afkodningens mellemkomst. English abstract Constructing Meaning While Reading Literary Texts – Verbal Reports From Pupils With Reading Difficulties This article examines how pupils with reading comprehension difficulties construct meaning as they read literary texts. The study is directed at pupils’ reading comprehension process while reading a particular type of text (literary texts) in a particular context (literature tuition). As a consequence, the study’s theoretical framework includes a cognitive approach regarding the reading process and a socio-cognitive approach regarding literary texts in a school setting. Applying Verbal Protocol as a research method, three reader profiles have been distinguished: inferring readers, paraphrasing readers and associative readers. These provide an opportunity for qualifying our understanding of the comprehension process in pupils with reading comprehension difficulties as they comprehend without decoding.
The International Regulations for the Prevention of Collisions at Sea (IRPCS) provide a comprehensive set of instructions for watchkeeping officers to follow and prevent collisions at sea. This study compares how six newly qualified deck officers and six Master Mariners, who were all trained at the same college, applied the IRPCS. Individual, semi-structured interviews were used to uncover how the 12 participants applied and interpreted the rules for three authentic scenarios. Phenomenography was used to capture the qualitatively different means by which participants interpreted the IRPCS. For basic collision avoidance situations, the results indicated little difference between the cohorts' ability to interpret and apply the IRPCS. However, when the scenarios became more complicated, Master Mariners outperformed newly qualified deck officers. In these cases, Master Mariners displayed a greater capacity to assess the overall situation, whereas newly qualified deck officers tended to simplify by focusing on a single rule. These findings indicate that training needs to focus on developing situational awareness; and training scenarios need to incorporate multiple vessels in authentic scenarios to enhance newly qualified deck officers' capacities to interpret the IRPCS.
L’alliance des méthodes mobiles et des méthodes visuelles retrouve une actualité renouvelée dans l’étude des mobilités futures. Impératifs écologiques, politiques publiques et annonces d’innovation technologique questionnent les pratiques (auto)mobiles et rendent nécessaire d’appréhender conjointement les mobilités quotidiennes du présent et les imaginaires de l’avenir, et cela en mouvement . Cet article propose la méthode du convoi audiovisuel et automobile pour réaliser une sociologie située et collaborative des mobilités quotidiennes, qui permet d’offrir des prises heuristiques au-delà même de l’analyse sociologique. Le visuel y devient un site d’interaction qui rend explicites les connaissances tacites et compétences situées liées à la mobilité quotidienne ; il permet de documenter et de rendre présente l’épaisseur relationnelle et expérientielle de l’espace du proche. La coconstruction du récit automobile filmé permet de mettre en évidence les modalités spécifiques des socialités locales en tant qu’elles sont imbriquées dans les pratiques de mobilité, y compris par les difficultés et imperfections qui émergent lors de ces embarquements.
Fostering thinking abilities, such as critical and creative thinking, has become an increasing societal demand in the digital age. Effective education necessitates grasping learners’ thought processes, yet gaining insights into these internal processes through external observations is extremely difficult. The think-aloud method, a concurrent verbal reporting technique where participants vocalize their thoughts, has been traditionally employed but exhibits certain limitations such as evoking heightened tension, and prompts the need for improvements. This pilot study focused on the modality used for reporting, comparing cases where participants reported verbally (think-aloud) versus via typing (type-thought). The aim was to examine potential differences based on participants’ subjective reports between these two reporting methods. After engaging in both methods, participants rated various aspects of each method. They also answered questions regarding their general interpersonal anxiety tendencies and were finally interviewed to gather qualitative feedback. Results indicated that the type-thought method elicited less resistance to reporting or anxiety during performance compared to think-aloud. However, no differences emerged between the two methods regarding perceived reporting difficulty or ease of engagement with the primary task. Notably, an interaction with interpersonal anxiety tendencies suggested that for those prone to such anxiety, the think-aloud method tends to increase resistance to reporting their thoughts. This study demonstrated that the type-thought method possesses distinct characteristics from the traditional think-aloud method, suggesting its potential as an alternative concurrent verbal reporting technique. These findings are highly significant, implying the need for further investigations into the relationships between various individual factors such as personality traits, cognitive styles, and reporting methods. It also highlights the importance of tailoring the reporting method to participants based on their traits and preferences for more accurate insights into thought processes.
In this paper, sufficient LMI-based conditions for general Quadratic Supply Rate (QSR) functions, known as QSR-dissipativity of nonlinear systems in Takagi- Sugeno form are proposed. To determine the stability of negative feedback loops of two systems, it is sufficient to prove the passivity of each individual system. This property can be used to ensure the stability of even globally distributed systems if the individual local systems are passive. This is particularly relevant for the stabilization of massively distributed power systems, for example. The passivity can either be a property of non-regulated systems or non-passive systems becomes passive through feedback controller. Both cases are analyzed for the class of Takagi-Sugeno fuzzy systems.
Aim
The aim of this study was to explore young people’s experiences of resources and strategies for promoting their mental health.
Methods
Individual interviews with 33 people aged 16–25 years were conducted using a method inspired by cognitive interviewing, which combines think aloud techniques with probing questions. The interviews were based on the young people’s reflections of the questions in the Swedish national public health survey. Data were analysed with reflexive thematic analysis.
Results
The resources and strategies that the young people in this study described as important for promoting mental well-being are related to societal prerequisites needed to navigate life; to their immediate surroundings, including social interactions; and to the young people themselves. These resources and strategies are presented under the following three themes: prerequisites for navigating life, social interactions on one’s own terms, and who I am and what I can do.
Conclusions
Young people have a variety of resources and strategies of their own available for promoting mental well-being. However, they cannot promote mental health just on their own; access to social networks and the opportunity to share thoughts and feelings are central. Further, to promote mental health, young people need good social conditions, knowledge, and support from adults so that the existence they struggle to navigate feels comprehensible, manageable and meaningful.
The rise of generative artificial intelligence (GAI) applications, epitomized by ChatGPT, has reshaped design processes by enhancing idea generation and conceptual depth for designers. However, the facilitating effects on novice designers' thinking remain uncertain, particularly in the context of sustainable service concept generation. This study examines these impacts of ChatGPT on design thinking process and outcomes through controlled experiments with 36 novice designers with ChatGPT, Tiangong AI, and no tools under a sustainable service design task. Through the protocol analysis, this study visualizes the design thinking by network-based cognitive maps, then evaluates design outcomes and systematically analyzes characteristics of design thinking development under different tool interventions. Findings indicate that ChatGPT enhances design concept novelty and systematicity but has limited impact on originality and sustainability. Furthermore, ChatGPT plays an active role in fostering thinking divergence and fluency, especially in providing relevant guides for developed ideas and accelerating the evaluation and creation process. The network-based cognitive maps reveal distinct shifts and styles influenced by ChatGPT, providing references for novice designers using such tools to enhance inspiration and design fluency, and also effectively employ diverse tools during specific concept generation stage. The study also provides insights for enhancing the relevance of educational curricula and enabling bottom-up sustainable service innovations.
The current debate about quantitative and qualitative methods focuses on whether there is a necessary connection between method-type and research paradigm that makes the different approaches incompatible. This paper argues that part of the connection is rhetorical. Quantitative methods express the assumptions of a positvisit paradigm which holds that behavior can be explained through objective facts. Design and instrumentation persuade by showing how bias and error are eliminated. Qualitative methods express the assumptions of a phenomenological paradigm that there are multiple realities that are socially defined. Rich description persuades by showing that the researcher was immersed in the setting and giving the reader enough detail to “make sense” of the situation. While rhetorically different, the results of the two methodologies can be complementary. Examples are drawn from two studies using different methodologies to study the same problem.
Proposes that verbal reports are data and that accounting for them, as well as for other kinds of data, requires explication of the mechanisms by which the reports are generated, and the ways in which they are sensitive to experimental factors (instructions, tasks, etc). Within the theoretical framework of human information processing, different types of processes underlying verbalization are discussed, and a model is presented of how Ss, in response to an instruction to think aloud, verbalize information that they are attending to in short-term memory (STM). Verbalizing information is shown to affect cognitive processes only if the instructions require verbalization of information that would not otherwise be attended to. From an analysis of what would be in STM at the time of report, the model predicts what could be reliably reported. The inaccurate reports found by other research are shown to result from requesting information that was never directly heeded, thus forcing Ss to infer rather than remember their mental processes. (112 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Anthropological research does not have subjects. We work with informants in an atmosphere of trust and mutual respect. (Mead, 1969, p. 371)
This study investigated the on-line processing strategies used by a sample of nonnative speakers of English who were asked to give the meanings of selected common idioms presented in a written context. Data were collected by means of the think-aloud procedure: Participants were asked to verbalize their thoughts as they arrived at the meanings of the idioms. Analysis revealed that most of the participants engaged in a heuristic approach to idiom comprehension, employing a variety of strategies through trial and error to find the meanings of the idioms. Models of L1 idiom acquisition did not apply well to the comprehension of idioms by the L2 users. Some pedagogical suggestions derived from the findings are included.
This article takes issue with a question sometimes heard at workshops on interview research: "How shall I find a method to analyze the 1,000 pages of interview transcripts I have collected?" Rather than merely dismiss the question as one posed too late and leading in unproductive directions or attempt to answer the question directly, the article seeks to bring into the open the empiristic presuppositions about qualitative research implied by the question. The question is itself treated as a text to be analyzed. Seven critical key terms are selected for interpretation: how, have, method, analyze, 1,000 pages, transcripts, and collected. The interpretation focuses on both what is said and what is not said in the question, as well as what could have been said to lead interview analyses in more constructive directions. Finally, the present interpretation of the question is discussed as an example of qualitative analysis.
This article develops the idea of consciously using different lenses for the purpose of expanding the perceptual efficacy of the researcher. Lenses provide a means of opening into or looking into. They alert and arouse us to mobilize resources of interest, energy, time, and attention. To be so mobilized is to foreground and background what is out there and thus to construct what is available to see. The author assumes that although researchers always use one or another lens while engaged in the process of observation, they can learn to see in unexpected but useful ways if they are aware of a range of possible focal points. Nine such focal points are developed in this article.
This paper aims to investigate the factors that influence the language-switching behaviour in the thinking processes of a bilingual person engaged in second language composing tasks, based on a review of related research studies and a report of an empirical case study. The study uses a multitask within-subject design. A participant is asked to do three sets of second language composing tasks which include text composition in English as a second language, written translation from Chinese (first language) into English, and problem-solving in math in English. Each set consists of two tasks: one with a low level of knowledge demands, and the other with a high level of knowledge demands. An analysis of the think-aloud protocols reveals several factors which may affect language-switching; the findings of the study suggest that levels of knowledge demands may play an important role in influencing language-switching in second language composing processes.
A study examined the reading strategies of a "reluctant reader," a bright and accomplished fifth grader whose achievements had not as yet included a love of reading or very good comprehension skills. Subject of the study, a female, was an excellent student at a private school where whole language, reading and writing groups, and individualized reading are stressed. A read-aloud, think-aloud protocol was used to determine which of the student's reading processes were serving her well and which were in need of development. Results indicated that her lack of involvement with the text and ignoring of grammatical clues led to her inability to fully comprehend the meaning of the text. Findings suggest, after some practice with the protocol, the student progressed toward increased confidence in her own meaning-making capabilities and took more pleasure in reading. (Contains 8 references.) (CR)
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Despite scant empirical evidence and questionable theoretical support, sentence-combining continues to be one of the most widely used instructional alternatives to formal grammar instruction in second language writing instruction. This study explored the cognitive strategies that second language writers engaged in during sentence-combining tasks in order to determine: 1) the cognitive demands of sentence-combining tasks, 2) if different types of sentence-combining tasks require different levels of cognitive strategies, and 3) the extent to which sentence-combining tasks require second language writers to attend to aspects of cohesion and evaluation.Nine advanced-level second language writers participated in think-aloud protocols (Ericsson & Simon, 1980, 1984) as they completed both controlled and open sentence-combining tasks. The protocols were analyzed according to the type of cognitive strategies used during sentence-combining tasks. The results showed that these second language writers engaged in restating content, constructing meaning, and higher and lower-level planning as they completed sentence-combining tasks. Between-task comparisons indicated that open sentence-combining tasks required significantly more higher-level planning than controlled sentence-combining tasks. Finally, these second language writers evaluated the appropriateness of their constructions but did not attend to aspects of cohesion during sentence-combining tasks. Relevant theoretical and pedagogical implications for second language writing instruction ore discussed.
Thesis (M.Ed.)--Brock University, 2003. Brock University. Faculty of Education. Thesis
Composing in first and second languages: Possible effects of EFL writing instruction. Paper presented at the Balkan Conference on English Language Teaching of the International Association of Teachers of English as a Foreign Language
- A Akyel
- S Kamisli
Akyel, A. & Kamisli, S. (1996). Composing in first and second languages:
Possible effects of EFL writing instruction. Paper presented at the
Balkan Conference on English Language Teaching of the International
Association of Teachers of English as a Foreign Language, Istanbul,
Turkey. (ERIC Document Reproduction Service No. ED 401 719)
Philosophical and theoretical frameworks
- J W Creswell
Creswell, J. W. (1998) Philosophical and theoretical frameworks. In J. S. Creswell,
(Ed.), Qualitative inquiry and research design: Choosing among five
traditions (pp. 73-91). Thousand Oaks, CA: Sage.
Researching teaching through collaborative inquiry with students
- J Dalton
- M Lantaigne-Richard
- M Quattrocchi
Dalton, J., Lantaigne-Richard, M., & Quattrocchi, M. (2000). Researching teaching
through collaborative inquiry with students. In A.L. Cole & J.G. Knowles
(Eds.), Researching teaching: Exploring teacher development through
reflexive inquiry (pp. 151-165). Boston: Allyn & Bacon.
Taking the test: Using verbal report data in looking at the processing of cloze tasks. Edinburgh Working Papers In Applied Linguistics
- B Gibson
Gibson, B. (1997). Taking the test: Using verbal report data in looking at the
processing of cloze tasks. Edinburgh Working Papers In Applied
Linguistics, 8, 54-62. (ERIC Document Reproduction Service No. ED
409 713)
The case study approach to research problems
- S B Merriam
Merriam, S. B. (1988). The case study approach to research problems. In S.B.
Merriam (Ed.), Case study research in education: A qualitative
approach (pp. 5-21). San Francisco: Jossey Bass.
Exploring the comprehension strategies of EFL readers: A multi-method study. Paper presented at an International Workshop on Written Language Processing at the University of New South Wales
- Sugirin
Sugirin. (1999, February). Exploring the comprehension strategies of EFL
readers: A multi-method study. Paper presented at an International
Workshop on Written Language Processing at the University of New
South Wales, Sydney, Dec. 9, 1998. (ERIC Document Reproduction
Service No. ED 428 548)
Case study research: Design and methods
- R K Yin
Yin, R. K. (1984). Case study research: Design and methods. Beverley Hills,
CA: Sage Publications.