Fig 3 - uploaded by Oscar Díaz
Content may be subject to copyright.
DScaffolding RCA map with three purpose nodes. Available at https://www. mindmeister.com/830267652. (Color figure online)
Source publication
Reading literature is important, but problematic. In Quora and other PhD forums, students moan about their frustrating reading and literature review experiences. Strategic reading might help. This term is coined to conceive of reading as a process of constructing meaning by interacting with text in a targeted way. The fact that strategic reading is...
Contexts in source publication
Context 1
... seamlessness. RCA nodes are turned into "Purpose nodes" through adding two possible children: the "Supporting Evidences?" node and the "Who else addresses it?" node. Introducing such nodes turns the father into a Purpose node. DScaffolding decorates Purpose nodes with one of up to eight different background colours (see Fig. 3). -Process seamlessness. "Supporting Evidences?" and "Who else addresses it?" are created as any other node. However, DScaffolding constraints these nodes to hang from the appropriate fathers, i.e. cause/consequence nodes and means nodes, respectively (see Fig. ...
Context 2
... decorates Purpose nodes with one of up to eight different background colours (see Fig. 3). -Process seamlessness. "Supporting Evidences?" and "Who else addresses it?" are created as any other node. However, DScaffolding constraints these nodes to hang from the appropriate fathers, i.e. cause/consequence nodes and means nodes, respectively (see Fig. ...
Context 3
... example in Fig. 3 shows three current RCA reading concerns: the problem statement (in green), "Poor reference recoverability" (in pink), and "Uncoupled RCA and reading tools" (in purple). Some evidence has already been collected for these concerns drawn from the literature. Note that the automatically generated background colours will later be mapped to ...
Context 4
... the namesake MindMeister map. In so doing, DScaffolding realizes the annotation pipe. But what is meant by "enrich"? Enrich refers to DScaf- folding automatically creating Annotation nodes out of annotations coming from Mendeley repositories. An Annotation node addresses an RCA issue, and as such, it hangs from the corresponding RCA node (see Fig. 3). Node properties include: a label, an attached comment and a background colour. For Annotation ...
Citations
... Generally, there are several approaches to do this type of analysis [2], [3]: systematic reviews, scoping reviews, or even meta-reviews of multiple review papers, among many others. The task of reviewing literature in a particular field is usually done through ''strategic reading'' [4], where researchers consider several publications to identify those that address The associate editor coordinating the review of this manuscript and approving it for publication was James Harland. tasks, methods, resources, and concepts of interests, and then read only a selection of those in detail. ...
Nowadays, the use of technology in continuously increasing, making a significant impact in almost every area, including education. New areas have gained much popularity in the last years in educational technology (EdTech), such as Massive Open Online Courses (MOOCs) or computer-supported collaborative learning. In addition, research and interest in this area have also been growing over the years. The quantity of research and scientific publications in EdTech is constantly increasing, and trying to analyze and extract information from a set of research papers is often a very time-consuming task. To make this process easier and solve these limitations, we present
Fontana
, a framework that can quickly perform trend and social network analysis using any corpus of documents and its metadata. Specifically, the framework can: 1) Discover the latest trends given any corpus of documents, using Natural Language Processing (NLP) analysis and keywords (bibliometric approach); 2) Discover the evolution of the trends previously identified over the years; 3) Discover the primary authors and papers, along with hidden relationships between existing communities. To test its functionality, we evaluated the framework using a corpus of papers from the EdTech research field. We also followed an open science methodology making the entire framework available in Open Science Framework (OSF) easy to access and use. The case study successfully proved the capabilities of the framework, revealing some of the most frequent topics in the area, such as “EDM,” “learning analytics,” or “collaborative learning.” We expect our work to help identifying trends and patterns in the EdTech area, using natural language processing and social network analysis to objectively process large amounts of research.
... One group was tasked with analyzing the data using the analysis approach of our method, hereinafter referred to as QDAcity group. The other group was tasked to perform an analysis of the same data using a method utilizing strategic reading [14,15], hereinafter referred to as control group. The participant sampling is described in more detail in Sect. ...
... Strategic reading is a term coined in the educational domain [33] and has been suggested for design science research and domain modeling [14,15]. At its core, strategic reading requires the reader to actively employ all of the following seven reading strategies: ...
Using qualitative data analysis (QDA) to perform domain analysis and modeling has shown great promise. Yet, the evaluation of such approaches has been limited to single-case case studies. While these exploratory cases are valuable for an initial assessment, the evaluation of the efficacy of QDA to solve the suggested problems is restricted by the common single-case case study research design. Using our own method, called QDAcity-RE, as the example, we present an in-depth empirical evaluation of employing qualitative data analysis for domain modeling using a controlled experiment design. Our controlled experiment shows that the QDA-based method leads to a deeper and richer set of domain concepts discovered from the data, while also being more time efficient than the control group using a comparable non-QDA-based method with the same level of traceability.
... In the experiment we evaluate our method, used in the QDAcity group, against a different method [50,51] for domain modeling used by the control group. The method employed by the control group is based on strategic reading and features a similar level of pre-RS traceability between the elements of the domain model and the underlying (source) text on which the analysis is based [50]. ...
... One group was tasked with analyzing the data using the analysis approach of our method. The other group was tasked to perform an analysis of the same data using a strategy utilizing strategic reading [50,51]. The participant sampling is described in more detail in section 6.1.2. ...
... Strategic reading is a term coined in the educational domain [129], and has been suggested for design science research [50,51]. At its core, strategic reading requires the reader to actively employ all of the following seven reading strategies: ...
The creation of domain models from qualitative input relies heavily on experience. An uncodified ad-hoc modeling process is still common and leads to poor documentation of the requirements analysis. In this thesis, we present a novel method for domain analysis based on qualitative data analysis (QDA). The method helps identifying inconsistencies, ensures a high degree of completeness, and inherently provides traceability from analysis results back to stakeholder input. In our approach, the QDAcity-RE method, the research process of theory building facilitates domain analysis within the requirements elicitation phase of a software development project. We show how an iterative process of concurrent data collection and analysis can be applied to requirements engineering (RE), including open, axial, and selective coding of qualitative data. The traceability of domain model elements back to original statements by stakeholders, generated by our method, does not have to be created and maintained separately after the fact. The traces are documented in an analysis artifact called the code system, which evolves iteratively with the analysis process. The code system can act as a universal model from which a variety of artifacts can be derived, describing both behavioral and structural aspects. This thesis focuses on the creation of conceptual domain models using QDAcity-RE, but peripherally also demonstrates this capability through the generation of behavioral models and a software requirements specification. We applied and evaluated our method for domain modeling in four exploratory projects in the domains of medical imaging diagnostics, railway systems, HR development, and qualitative research. We show that by applying QDA to domain analysis, structural elements and relationships needed to derive a UML class diagram can be extracted from a code system based on interviews with domain experts. Constant comparison and theoretical sampling assist in integrating differing domain descriptions into an abstract model. While the analysis process still requires interpretations and modeling decisions, our method provides more guidance than existing domain analysis approaches and a thorough documentation of these decisions. In addition, codes and memos ensure traceability between the original data and the derived model and assist in connecting several RE artifacts, ensuring a high degree of inter-model consistency. We validated our claim that QDAcity-RE helps an analyst gain a deeper understanding of a problem domain through a controlled experiment.
... One review (in RRD) reports that "grey literature is a major input, especially in SE, where practitioners may face problems that scholars don't know". The practitioner's experience should be targeted and contextualized to detect potential researcher's bias or "too narrow questions" [18]. Therefore, GL can be a good representative of the state-of-the-practice. ...
Context: Following on other scientific disciplines, such as health sciences, the use of Grey Literature (GL) has become widespread in Software Engineering (SE) research. Whilst the number of papers incorporating GL in SE is increasing, there is little empirically known about different aspects of the use of GL in SE research.
Method: We used a mixed-methods approach for this research. We carried out a Systematic Literature Review (SLR) of the use of GL in SE, and surveyed the authors of the selected papers included in the SLR (as GL users) and the invited experts in SE community on the use of GL in SE research. Results: We systematically selected and reviewed 102 SE secondary studies that incorporate GL in SE research, from which we identified two groups based on their reporting: 1) 76 reviews only claim their use of GL; 2) 26 reviews report the results by including GL. We also obtained 20 replies from the GL users and 24 replies from the invited SE experts. Conclusion: There is no common understanding of the meaning of GL in SE. Researchers define the scopes and the definitions of GL in a variety of ways. We found five main reasons of using GL in SE research. The findings have enabled us to propose a conceptual model for how GL works in SE research lifecycle. There is an apparent need for research to develop guidelines for using GL in SE and for assessing quality of GL. The current work can provide a panorama of the state-of-the-art of using GL in SE for the follow-up research, as to determine the important position of GL in SE research.
... This includes theorization and empirical evidence on the metacognition of reading (Grabe, 2009), aspects related to self-regulation (Paris and Myers, 1981), and negative effects of mind wandering and fatigue (Unsworth and McMillan, 2013). Insights from this literature have been taken up by methodologists of literature reviews (e.g., Díaz et al. 2017, Ridley 2012, vom Brocke et al. 2105, Hart 1998. For example, Hart (1998, p.54) advises readers to "skim through the text, get information about the structure and the general idea, read the preface and introduction, and read the parts of the text that are marked as important by yourself". ...
Understanding a new literature corpus can be a grueling experience for junior scholars. Nevertheless, corresponding guidelines have not been updated for decades. We contend that the traditional strategy of skimming all papers and reading selected papers afterwards needs to be revised. Therefore, we design a new strategy that guides the overall exploratory process by prioritizing influential papers for initial reading, followed by skimming the remaining papers. Consistent with schemata theory, starting with in-depth reading allows readers to acquire more substantial prior content schemata, which are representative for the literature corpus and useful in the following skimming process. To this end, we develop a prototype that identifies the influential papers from a set of PDFs, which is illustrated in a case study in the IT business value domain. With the new strategy, we envision a more efficient process of exploring unknown literature corpora.
... Awareness of Problem came from the literature, specially the survey conducted in [26]. Suggestion was derived from our previous work on Strategic Reading for students [5] (Section 2). Development was undertaken in an iterative approach of prototyping different aspects of the purposeful artifact (Section 3). ...
... The paper seems to overlook the 'why' and focus too much on the 'what'. * (Page 1): "Different causes can be blamed for this situation: (1) lack of transparency in the process [18,5], (2) lack of agreement about what constitutes good reviewing [18,16,24,8], (3) lack of skills and reviewing experience [11,8], or (4) lack of time". The problem should be analysed in more detail. ...
Peer review is under pressure. Demand for reviews is outstripping supply where reviewers tend to be busy people who contribute voluntarily. Authors highly value reviews, yet complain about the time it takes to get feedback to the point of putting research timeliness at stake. Though part of the review process has been moved to the Web, the review itself is still often conducted with the only help of a yellow highlighter, physical or digital. This work looks for more performant highlighters that account for the review specifics. Peer review does not stop at spotting the manuscript (de)merits, it also strives for manuscript improvement and gatekeeping. These functions are conducted within an often tacit research-quality framework, and frequently in a discontinuous way. Unfortunately, when it comes to support review practices, current facilities fall short. This work introduces a set of requirements for review-dedicated highlighters. These requirements are instantiated and evaluated through Review&Go, a color-coding highlighter that generates a review draft out of the reviewer’s highlighting activities. The aim is to offer representational guidance to enhance context/cognitive awareness so that reviewers can exert less effort while offering valuable and timely reviews.
... And then the bulb lighted up: if a pivotal skill for researchers is that of asking the right questions then, we can conjecture that RCA could be the means to find these questions. This paper reports on how this idea was developed 1 DSR requires a profound understanding of the problem to be solved, the consequences to be alleviated, and the causes to be prevented. This in turn usually implies extracting evidence from the literature that warrants the project's RCA. ...
... Next sections elaborates on this question, illustrating the case for MindMeister (as the RCA tool) and Mendeley (as the reference manager). For details about how this example is factored out into general meta-requirements refer to [1]. ...
... We built DScaffolding to assess the extent to which this theory holds. First evaluations indicate that not only reading but also RCA might benefit from a tight coupling between these two processes (refer to [1] for further insights). We do hope Antoni like the approach! ...
“Strategic reading” is a term coined to conceive reading as a process of constructing meaning by interacting with text. While reading, individuals use their prior knowledge along with clues from the text to construct meaning, and place the new knowledge within this frame. Strategic reading is then a pivotal ability for conceptual modelers, more so if domain knowledge needs to be acquired mainly from the literature as it is the case for research projects. But this might turn problematic. In Quora and other PhD forums, students moan about their frustrating reading and literature review experiences. Traditionally, students are encouraged to annotate while reading. Digital annotations are expected to be useful for supporting comprehension and interpretation. Our belief is that strategic reading (and hence, conceptual modeling) can be more effective if annotation is conducted in direct relationship to a main research activity: root-cause analysis (RCA). RCA can provide the questions whose answers should be sought in the literature. Unfortunately, this process is not supported by current tools. When reading papers, researchers might not be all aware of the issues being raised during RCA. And the other way around, when it comes to RCA, evidences found in the literature might not be promptly accessible. This paper reports on research to develop a technical solution to this problem: a plug-in for Google Chrome that provides seamless integration between a RCA platform (i.e. MindMeister) and a reading platforms (i.e. Mendeley). The aim: improving RCA awareness while reading so that annotations can be traced back to the RCA issues.
Posing research questions is a fundamental step to guide and direct knowledge development in research. In design science research (DSR), research questions are important to define the scope and the modes of inquiry, characterize the artifacts, and communicate the contributions. Despite the importance of research questions, there are few guidelines on how to construct suitable DSR research questions. We fill this gap by exploring ways of constructing DSR research questions and analyzing the research questions in a sample of 104 DSR publications. The results show that about two thirds of the analyzed DSR publications actually use research questions to link their problem statements to research approaches and that most of the questions are aimed at problem-solving. Based on our analysis, we derive a typology of DSR question formulation to provide guidelines and patterns that help researchers formulate research questions during their DSR projects' duration.
