Lloyd Montgomery’s research while affiliated with Hamburg University and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (17)


Formalization of a requirements specification R2 using passive voice
Formalization of a requirements specification R3 using an ambiguous pronoun
Reduced version of the activity-based requirements quality theory (Frattini et al. 2023)
Causal assumptions about the impact of passive voice
Domain modeling task example for requirement 4

+9

Applying bayesian data analysis for causal inference about requirements quality: a controlled experiment
  • Article
  • Full-text available

November 2024

·

22 Reads

·

3 Citations

Empirical Software Engineering

·

Davide Fucci

·

·

[...]

·

It is commonly accepted that the quality of requirements specifications impacts subsequent software engineering activities. However, we still lack empirical evidence to support organizations in deciding whether their requirements are good enough or impede subsequent activities. We aim to contribute empirical evidence to the effect that requirements quality defects have on a software engineering activity that depends on this requirement. We conduct a controlled experiment in which 25 participants from industry and university generate domain models from four natural language requirements containing different quality defects. We evaluate the resulting models using both frequentist and Bayesian data analysis. Contrary to our expectations, our results show that the use of passive voice only has a minor impact on the resulting domain models. The use of ambiguous pronouns, however, shows a strong effect on various properties of the resulting domain models. Most notably, ambiguous pronouns lead to incorrect associations in domain models. Despite being equally advised against by literature and frequentist methods, the Bayesian data analysis shows that the two investigated quality defects have vastly different impacts on software engineering activities and, hence, deserve different levels of attention. Our employed method can be further utilized by researchers to improve reliable, detailed empirical evidence on requirements quality.

Download

Requirements Quality Research Artifacts: Recovery, Analysis, and Management Guideline

June 2024

·

30 Reads

Requirements quality research, which is dedicated to assessing and improving the quality of requirements specifications, is dependent on research artifacts like data sets (containing information about quality defects) and implementations (automatically detecting and removing these defects). However, recent research exposed that the majority of these research artifacts have become unavailable or have never been disclosed, which inhibits progress in the research domain. In this work, we aim to improve the availability of research artifacts in requirements quality research. To this end, we (1) extend an artifact recovery initiative, (2) empirically evaluate the reasons for artifact unavailability using Bayesian data analysis, and (3) compile a concise guideline for open science artifact disclosure. Our results include 10 recovered data sets and 7 recovered implementations, empirical support for artifact availability improving over time and the positive effect of public hosting services, and a pragmatic artifact management guideline open for community comments. With this work, we hope to encourage and support adherence to open science principles and improve the availability of research artifacts for the requirements research quality community.



Excerpt from the activity-based quality model for maintainability
Concepts of the requirements quality theory
Exemplary instantiation of the theory
Survey results depicting the distribution of codes
Architectural overview of the proposed tool-support
Requirements quality research: a harmonized theory, evaluation, and roadmap

August 2023

·

125 Reads

·

7 Citations

Requirements Engineering

High-quality requirements minimize the risk of propagating defects to later stages of the software development life cycle. Achieving a sufficient level of quality is a major goal of requirements engineering. This requires a clear definition and understanding of requirements quality. Though recent publications make an effort at disentangling the complex concept of quality, the requirements quality research community lacks identity and clear structure which guides advances and puts new findings into an holistic perspective. In this research commentary, we contribute (1) a harmonized requirements quality theory organizing its core concepts, (2) an evaluation of the current state of requirements quality research, and (3) a research roadmap to guide advancements in the field. We show that requirements quality research focuses on normative rules and mostly fails to connect requirements quality to its impact on subsequent software development activities, impeding the relevance of the research. Adherence to the proposed requirements quality theory and following the outlined roadmap will be a step toward amending this gap.


Let's Stop Building at the Feet of Giants: Recovering unavailable Requirements Quality Artifacts

April 2023

·

68 Reads

·

2 Citations

Requirements quality literature abounds with publications presenting artifacts, such as data sets and tools. However, recent systematic studies show that more than 80% of these artifacts have become unavailable or were never made public, limiting reproducibility and reusability. In this work, we report on an attempt to recover those artifacts. To that end, we requested corresponding authors of unavailable artifacts to recover and disclose them according to open science principles. Our results, based on 19 answers from 35 authors (54% response rate), include an assessment of the availability of requirements quality artifacts and a breakdown of authors' reasons for their continued unavailability. Overall, we improved the availability of seven data sets and seven implementations.





An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projects

August 2022

·

15 Reads

[Background] In large open-source software projects, development knowledge is often fragmented across multiple artefacts and contributors such that individual stakeholders are generally unaware of the full breadth of the product features. However, users want to know what the software is capable of, while contributors need to know where to fix, update, and add features. [Objective] This work aims at understanding how feature knowledge is documented in GitHub projects and how it is linked (if at all) to the source code. [Method] We conducted an in-depth qualitative exploratory content analysis of 25 popular GitHub repositories that provided the documentation artefacts recommended by GitHub's Community Standards indicator. We first extracted strategies used to document software features in textual artefacts and then strategies used to link the feature documentation with source code. [Results] We observed feature documentation in all studied projects in artefacts such as READMEs, wikis, and website resource files. However, the features were often described in an unstructured way. Additionally, tracing techniques to connect feature documentation and source code were rarely used. [Conclusions] Our results suggest a lacking (or a low-prioritised) feature documentation in open-source projects, little use of normalised structures, and a rare explicit referencing to source code. As a result, product feature traceability is likely to be very limited, and maintainability to suffer over time.



Citations (8)


... As can be observed in Fig. 5, the distributions of the priors are generally quite wide, and they overlap with the dashed line indicating zero. However, this uncertainty is not unexpected or uncommon in Bayesian analysis, especially in studies with smaller samples (Ghorbani et al. 2023;Frattini et al. 2024). Because we adjusted the models for the Perception of the participants, the effect of sentiment on the outcome can vary for each of the three levels of Perception. ...

Reference:

Negativity in self-admitted technical debt: how sentiment influences prioritization
Applying bayesian data analysis for causal inference about requirements quality: a controlled experiment

Empirical Software Engineering

... Software requirements specifications (SRS), the explicit manifestation of requirements as an artifact (Méndez Fernández et al. 2019), serve as input for various subsequent software engineering (SE) activities, such as deriving a software architecture, implementing features, or generating test cases (Méndez Fernández and Penzenstadler 2015). As a consequence, the quality of an SRS impacts the quality of requirements-dependent activities (Frattini et al. 2023;Femmer and Vogelsang 2018;Femmer et al. 2015). A quality defect in an SRS-for example, an ambiguous formulation-can cause differing interpretations and result in the design and implementation of a solution that does not meet the stakeholders' needs (Méndez et al. 2017). ...

Requirements quality research: a harmonized theory, evaluation, and roadmap

Requirements Engineering

... Finally, the attributes from the material group record to what degree both the data obtained by the experiment and the script(s) used to perform the analysis are available. We recorded the availability attribute based on a previously established, categorical scale of research artifact availability [14] which includes levels like archived, ...

Let's Stop Building at the Feet of Giants: Recovering unavailable Requirements Quality Artifacts

... Empirical research on software documentation quality is an active field that focuses on various artifacts, like API reference documentation [40] or README files [68], and the perspectives of documentation writers [1]. Studies on the evaluation of AI-generated documentation usually focus on automated metrics like BLEU, ROUGE, and METEOR [29], [64], [69]- [71]. ...

An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projects

... These smells can signify ambiguous, incomplete, inconsistent, or overly complex requirements, resulting in increased costs, delays, or defects in the final product [8]. Frattini et al. [9] published a catalog of 206 requirements quality indicators (aka smells) extracted from a systematic mapping study on 105 relevant primary studies [10]. The authors also categorized requirements smells into three categories: (1) lexical smells describe issues in single words or terms, e.g., code = "program source" or "set of rules"?; (2) syntactic smells describe issues in word or sentence structures, e.g., When the system sends a message to the receiver, it shall provide an acknowledgment (it = "system" or "receiver"?); and (3) semantic smells describe issues in interpreting the requirements within its context, e.g., The system shall generate a report at the end of each day, (strictly at midnight or at the end of the business hours?). ...

A Live Extensible Ontology of Quality Factors for Textual Requirements

... Table 1 shows a simple COSMIC data movement counting rule for the given requirements. Advancements in machine learning especially in the area of deep learning revolutionized the domain of Natural Language Processing (NLP) [9][10] towards efficient text analysis. Another advancing field of research is the introduction of domain-specific contextual generative AI pre-trained language models such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformer (GPT) families. ...

An alternative issue tracking dataset of public Jira repositories
  • Citing Conference Paper
  • October 2022

... These smells can signify ambiguous, incomplete, inconsistent, or overly complex requirements, resulting in increased costs, delays, or defects in the final product [8]. Frattini et al. [9] published a catalog of 206 requirements quality indicators (aka smells) extracted from a systematic mapping study on 105 relevant primary studies [10]. The authors also categorized requirements smells into three categories: (1) lexical smells describe issues in single words or terms, e.g., code = "program source" or "set of rules"?; (2) syntactic smells describe issues in word or sentence structures, e.g., When the system sends a message to the receiver, it shall provide an acknowledgment (it = "system" or "receiver"?); and (3) semantic smells describe issues in interpreting the requirements within its context, e.g., The system shall generate a report at the end of each day, (strictly at midnight or at the end of the business hours?). ...

Empirical research on requirements quality: a systematic mapping study

Requirements Engineering

... In contrast, code review suggestions, depicted in Figure 1, have the advantage of making the reviewer's feedback more structured and actionable by letting the reviewer implement their suggestion directly in the source code and enabling the submitter to integrate it with a single or bundled commit. Code review suggestions also make project best practices explicit through concrete code examples, which may provide guidance for future contributors lacking project experience [7], [8]. Furthermore, code review suggestions can be leveraged in the future to offer support for the reviewers, for example by automating recurrent suggestions. ...

A Simple NLP-Based Approach to Support Onboarding and Retention in Open Source Communities
  • Citing Conference Paper
  • September 2018