Sira Vegas's research while affiliated with Universidad Politécnica de Madrid and other places

Publications (83)

Conference Paper
Full-text available
Software Engineering (SE) experiments typically have small, hard to acquire sample sizes. Recruiting subjects is an issue for SE progress. However, we argue that characterization, i.e., understanding the population traits, is the key problem. We propose a roadmap to address the problem. CCS CONCEPTS • General and reference → Empirical studies.
Conference Paper
Full-text available
Background. Software Engineering (SE) researchers extensively perform experiments with human subjects. Well-defined samples are required to ensure external validity. Samples are selected purposely or by convenience, limiting the generalizability of results. Objective. We aim to depict the current status of participants selection in empirical SE, i...
Preprint
Full-text available
Background. Software Engineering (SE) researchers extensively perform experiments with human subjects. Well-defined samples are required to ensure external validity. Samples are selected \textit{purposely} or by \textit{convenience}, limiting the generalizability of results. Objective. We aim to depict the current status of participants selection i...
Article
Full-text available
Context Test-driven development (TDD) is an agile software development approach that has been widely claimed to improve software quality. However, the extent to which TDD improves quality appears to be largely dependent upon the characteristics of the study in which it is evaluated (e.g., the research method, participant type, programming environme...
Article
Full-text available
ContextIt has been argued that software engineering replications are useful for verifying the results of previous experiments. However, it has not yet been agreed how to check whether the results hold across replications. Besides, some authors suggest that replications that do not verify the results of previous experiments can be used to identify c...
Article
Context: Test-driven development (TDD) is an agile software development approach that has been widely claimed to improve software quality. However, the extent to which TDD improves quality appears to be largely dependent upon the characteristics of the study in which it is evaluated (e.g., the research method, participant type, programming environm...
Preprint
Full-text available
Context: Test-driven development (TDD) is an agile software development approach that has been widely claimed to improve software quality. However, the extent to which TDD improves quality appears to be largely dependent upon the characteristics of the study in which it is evaluated (e.g., the research method, participant type, programming environm...
Preprint
Context: It has been argued that software engineering replications are useful for verifying the results of previous experiments. However, it has not yet been agreed how to check whether the results hold across replications. Besides, some authors suggest that replications that do not verify the results of previous experiments can be used to identify...
Preprint
Full-text available
Empirical Standards are brief public document that communicate expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for common research methods in software engineering. These living documents, which should be continuously revised to reflect ev...
Conference Paper
Full-text available
Background. Most Mining Software Repositories (MSR) studies cannot obtain causal relations because they are not controlled experiments. The use of cohort studies as defined in epidemiology could help to overcome this shortcoming. Objective. Propose the adoption of cohort studies in MSR research in particular and empirical Software Engineering (SE)...
Article
Full-text available
A recurring problem in software development is incorrect decision making on the techniques, methods and tools to be used. Mostly, these decisions are based on developers’ perceptions about them. A factor influencing people’s perceptions is past experience, but it is not the only one. In this research, we aim to discover how well the perceptions of...
Article
Full-text available
Software engineering (SE) experiments suffer from threats to validity that may impact their results. Replication allows researchers building on top of previous experiments’ weaknesses and increasing the reliability of the findings. Illustrating the benefits of replication to increase the reliability of the findings and uncover moderator variables....
Preprint
Context: Researchers from different groups and institutions are collaborating on building groups of experiments by means of replication (i.e., conducting groups of replications). Disparate aggregation techniques are being applied to analyze groups of replications. The application of unsuitable techniques to aggregate replication results may undermi...
Preprint
Context: Software Engineering (SE) experiments suffer from threats to validity that may impact their results. Replication allows researchers building on top of previous experiments' weaknesses and increasing the reliability of the findings. Objective: Illustrating the benefits of replication to increase the reliability of the findings and uncover m...
Article
Reviews on test-driven development (TDD) studies suggest that the conflicting results reported in the literature are due to unobserved factors, such as the tasks used in the experiments, and highlight that there are very few industry experiments conducted with professionals. The goal of this study is to investigate the impact of a new factor, the c...
Article
Context As a software quality characteristic, usability includes the attributes of efficiency, effectiveness and user satisfaction. There are several recommendations in the literature on how to build usable software systems, but there are not very many empirical studies that provide evidence about their impact. Objective We report an experiment ca...
Article
Full-text available
Context: Researchers from different groups and institutions are collaborating on building groups of experiments by means of replication (i.e., conducting groups of replications). Disparate aggregation techniques are being applied to analyze groups of replications. The application of unsuitable techniques to aggregate replication results may undermi...
Article
Full-text available
Context Confirmation bias is a person’s tendency to look for evidence that strengthens his/her prior beliefs rather than refutes them. Manifestation of confirmation bias in software testing may have adverse effects on software quality. Psychology research suggests that time pressure could trigger confirmation bias. Objective In the software industr...
Article
Context Replication is a key component of experimentation for verifying previous results and findings. Experiment replication requires products like documentation describing the baseline experiment and a version of the experimental material. When replicating an experiment, changes may have to be made to some of the products, leading to new or modif...
Article
Context: There is a lack of empirical evidence on the differences between model-driven development (MDD), where code is automatically derived from conceptual models, and traditional software development method, where code is manually written. In our previous work, we compared both methods in a baseline experiment concluding that quality of the soft...
Conference Paper
Experimentation is a key issue in science and engineering. But it is one of software engineering's stumbling blocks. Quite a lot of experiments are run nowadays, but it is a risky business. Software engineering has some special features, leading to some experimentation issues being conceived of differently than in other disciplines. The aim of this...
Article
Full-text available
Existing empirical studies on test-driven development (TDD) report different conclusions about its effects on quality and productivity. Very few of those studies are experiments conducted with software professionals in industry. We aim to analyse the effects of TDD on the external quality of the work done and the productivity of developers in an in...
Article
Context Experiment replications play a central role in the scientific method. Although software engineering experimentation has matured a great deal, the number of experiment replications is still relatively small. Software engineering experiments are composed of complex concepts, procedures and artefacts. Laboratory packages are a means of transfe...
Conference Paper
Full-text available
Background: Although most Computer Science (CS) programs offered by higher education institutions usually include a software engineering course, some works report a lack of formal training in software testing. Aim: With the aim of studying the possible impact of knowledge acquired from CS programs on software testing, this paper reports an investig...
Conference Paper
Experimentation is a key issue in science and engineering. But it is one of software engineering's stumbling blocks. Quite a lot of experiments are run nowadays, but it is a risky business. Software engineering has some special features, leading to some experimentation issues being conceived of differently than in other disciplines. The aim of this...
Conference Paper
Controlled experiments in laboratory settings are relatively commonplace in software engineering, but experiments in industry are thin on the ground. Of the few existing cases, most are 1-1 (running one experiment at one company), just a few are n-1 (running n experiments at one company) and still fewer are 1-n (running one and the same experiment...
Article
In experiments with crossover design subjects apply more than one treatment. Crossover designs are widespread in software engineering experimentation: they require fewer subjects and control the variability among subjects. However, some researchers disapprove of crossover designs. The main criticisms are: the carryover threat and its troublesome an...
Article
Several empirical studies have explored the benefits of software design patterns, but their collective results are highly inconsistent. Resolving the inconsistencies requires investigating moderators-i.e., variables that cause an effect to differ across contexts. Objectives. Replicate a design patterns experiment at multiple sites and identify suff...
Conference Paper
Context: Juristo et al. [7] published a literature review about testing technique experiments. The goal was to provide a picture of which techniques and aspects of techniques had been studied experimentally, and try to compile a body of knowledge on testing techniques. Goal: In this paper, we extend Juristo et al.'s study to cover the years from 20...
Article
Full-text available
Replication is an essential part of the experimental paradigm and is considered the main component of scientific knowledge. There are many open issues that must be addressed before the replication process can be successfully formalized in empirical software engineering research. The software engineering community learns a great deal from performing...
Article
An accepted fact in software engineering is that software must undergo verification and validation process during development to ascertain and improve its quality level. But there are too many techniques than a single developer could master, yet, it is impossible to be certain that software is free of defects. So, it is crucial for developers to be...
Article
A replication is the repetition of an experiment. Several efforts have been made to adopt replication as a common practice in software engineering. There are different types of replications, depending on their purpose. Similar replications keep the experimental conditions as alike as possible to the original ones. External similar replications, whe...
Conference Paper
Some verification and validation techniques have been evaluated both theoretically and empirically. Most empirical studies have been conducted without subjects, passing over any effect testers have when they apply the techniques. We have run an experiment with students to evaluate the effectiveness of three verification and validation techniques (e...
Article
Replication of software engineering experiments is crucial for dealing with validity threats to experiments in this area. Even though the empirical software engineering community is aware of the importance of replication, the replication rate is still very low. The RESER'11 Joint Replication Project aims to tackle this problem by simultaneously run...
Article
In no science or engineering discipline does it make sense to speak of isolated experiments. The results of a single experiment cannot be viewed as representative of the underlying reality. Experiment replication is the repetition of an experiment to double-check its results. Multiple replications of an experiment increase the confidence in its res...
Conference Paper
Full-text available
Experiment replication is a key component of the scientific paradigm. The purpose of replication is to verify previously observed findings. Although some Software Engineering (SE) experiments have been replicated, yet, there is still disagreement about how replications should be run in our field. With the aim of gaining a better understanding of ho...
Article
Full-text available
The verification of a previously observed finding is impor-tant. Being able to verify a finding provides some guarantee that it is not product of chance but that the observed phe-nomenon is stable or regular. The most well done way of verifying experimental findings is through replication. But, is this the only method for verifying the observations...
Conference Paper
In no science or engineering discipline does it make sense to speak of isolated experiments. The results of a single experiment cannot be viewed as representative of the underlying reality. The concept of experiment is closely related to replication. Experiment replication is the repetition of an experiment to double-check its results. Multiple rep...
Article
Classification makes a significant contribution to advancing knowledge in both science and engineering. It is a way of investigating the relationships between the objects to be classified and identifies gaps in knowledge. Classification in engineering also has a practical application; it supports object selection. They can help mature software engi...
Conference Paper
Software is one of the most important drivers of innovation. As organizations are becoming more dependent on software, the improvement of software quality and productivity becomes of essential importance for the competitiveness of an organization. Continuing the success of the LSO Workshop series since 1999, this workshop will provide a communicati...
Article
Is 25 years enough time to build up a coherent body of knowledge that can help point to useful principles? As a testbed for helping us answer this question, software testing techniques are a good place to start. Few software practices are as important as testing, and testing techniques are amenable to measurement and reasoning about their effective...
Article
Goals of replication in empirical software engineering. Experimental replications are performed for the same reasons that independent experiments are conducted, to better understand software engineering phenomena and how to improve the practice of software development. One important benefit of replications is that they help mature software engineer...
Article
Full-text available
RESUMEN RESUMEN Software engineering is a relatively young field and experimental software engineering is even younger , so undisputed facts are few and far between . Nevertheless , experimental results can help practitioners make better decisions , and the authors have aggregated results derived from individual unit - testing experiments publish...
Article
One of the major problems within the software testing area is how to get a suitable set of cases to test a software system. A good set of test cases should assure maximum effectiveness with as few cases as possible. There are now numerous testing techniques available for generating test cases. However, many are never used, while just a few are used...
Article
Full-text available
Replication packages are a means for researchers to share artefacts that improve experiment replication. There is, however, still not very much external replication of software engineering experiments. Researchers taking delivery of replication packages find it difficult to evaluate their content for both adaptation to the new replication setting a...
Article
This chapter summarizes the discussions that took place during the Measurement and Model Building session of the Dagstuhl seminar on Empirical Software Engineering (ESE). The goal of this session was to address questions concerning two topics: data sharing and effective data interpretation.
Conference Paper
Full-text available
The replication of experiments is a key undertaking in SE. Successful replications enable a discipline's body of knowledge to grow, as the results are added to those of earlier replications. However, replication is extremely difficult in SE, primarily because it is difficult to get a setting that is exactly the same as in the original experiment. C...
Article
In this paper, we report the experiences of a working group who met, as part of the 2006 Dagstuhl Seminar on Empirical Software Engineering, to discuss the teaching of empirical methods to undergraduate students. The nature of the discussion meant that the group also indirectly considered teaching empirical methods to postgraduate students, mainly...
Article
One of the major problems within the software testing area is how to get a suitable set of cases to test a software system. This set should assure maximum effectiveness with the least possible number of test cases. There are now numerous testing techniques available for generating test cases. However, many are never used, and just a few are used ov...
Article
Full-text available
Testing technique-related empirical studies have been performed for 25 years. We have managed to accumulate a fair number of experiments in this time, which might lead us to think that we now could have a sizeable empirically backed body of knowledge (BoK) on testing techniques. However, the experiments in this field have some flaws, and, consequen...
Article
Full-text available
Mature knowledge allows engineering disciplines the achievement of predictable results. Unfortunately, the type of knowledge used in software engineering can be considered to be of a relatively low maturity, and developers are guided by intuition, fashion or market-speak rather than by facts or undisputed statements proper to an engineering discipl...
Article
Mature knowledge allows engineering disciplines the achievement of predictable results. Unfortunately, the type of knowledge used in software engineering can be considered to be of a relatively low maturity, and developers are guided by intuition, fashion or market-speak rather than by facts or undisputed statements proper to an engineering discipl...
Article
Domain models describe common and variant requirements for a family of similar systems. Although most of the notations, such as UML, are meant for modeling a single system, they can be extended to model variants. We have done that and applied such extended ...
Article
One major issue in managing software engineering knowledge is the construction of information repositories for software development artefacts (techniques, products, processes, tools, etc.). But how does one package each artefact so that the package contains the appropriate information to understand and use the artefact? What is the appropriate char...
Chapter
The objective of this first evaluation of the characterisation schema is to study some of the hypotheses formulated in Chapter 3. In this case, the characterisation schema will be verified to evaluate static aspects of the schema, like feasibility from the viewpoint of producers and consumers and schema flexi- bility. The primary objective of this...
Conference Paper
The origin of the study described here is the experiment performed by Basili and Selby, further replicated by Kamsties and Lott, and once again by Wood et al. These experiments investigated the effectiveness and efficiency of different code evaluation techniques (functional and structural testing and code reading). The working hypotheses are the sa...
Chapter
The main difference between scientific and what is termed common knowledge lies in the methods used to generate new knowledge. The result of using this kind of methods is that the propositions formulated by a science cannot be called into question as easily as those formulated using other methods (Judd et al., 1991).
Chapter
At this point, we have two characterisation schemas (a theoretical and an empirical schema) that reflect different viewpoints or perspectives of the problem of selecting testing techniques in a software projects. These are: theory, represented by the authors, and practice, represented by software developers and researchers in the testing area. The...
Chapter
At this point, a schema has been built that reflects, on the one hand, the authors’ view of the testing technique selection problem and, on the other, both the information needs of consumers for the purposes of selection and the provision of information required according to producers to define the properties of a testing technique.
Chapter
At this point, we have a characterisation schema that is the fruit of the personal reflection of the authors to determine the information relevant for selecting testing techniques in a given project. In the absence of a solid theory on software testing upon which to build the characterisation schema, this first schema was obtained by studying the s...
Conference Paper
A survey is an empirical research strategy for the collection of information from heterogeneous sources. In this way, survey results often exhibit a high degree of external validity. It is complementary to other empirical research strategies such as controlled experiments, which usually have their strengths in the high internal validity of the find...
Chapter
This chapter addresses the construction of the theoretical characterisation schema. As mentioned earlier, the characterisation schema reflects only information relevant for selecting testing techniques and its purpose is to help users to make the selection without having to have an exhaustive and practical knowledge of all the techniques. As there...
Chapter
As discussed in Chapter 1, the problem of testing technique selection arises due to the following circumstances: The processes, techniques and tools used in the construction of software systems are not universally applicable (Basili and Rombach, 1991). And this also applies to testing techniques, which are not all equally applicable for validating...
Chapter
The goal of this second evaluation is to conclude the study of the hypotheses formulated in this book. This time, the characterisation schema is tested to evaluate aspects of interest to consumers, like completeness, effectiveness, efficiency, usability or satisfaction.
Chapter
As this chapter shows, there is no approach as yet that specifically deals with the problem of testing technique characterisation. However, there are approaches that are closely related to this subject of research. On the one hand, testing techniques have been studied comparatively, that is, there are investigations that endeavour, by a variety of...
Article
One of the main problems in software testing is the development of a suitable set of test cases so that the effectiveness of the test is maximised with a minimum number of test cases. A lot of testing techniques are now available for developing test cases. However, some of them are misused, others are never used and only a few are applied again and...
Conference Paper
Engineering disciplines are characterised by the use of mature knowledge by means of which they can achieve predictable results. Unfortunately, the type of knowledge used in software engineering can be considered to be of a relatively low maturity, and developers are guided by reasoning based on intuition, fashion or market-speak rather than by fac...