ArticlePDF Available

What Do We Know about Test-Driven Development?

  • Fraunhofer Center for Experimental Software Engineering


What if someone argued that one of your basic conceptions about how to develop software was misguided? What would it take to change your mind? That's essentially the dilemma faced by advocates of test-driven development (TDD). The TDD paradigm argues that the basic cycle of developing code and then testing it to make sure it does what it's supposed to do-something drilled into most of us from the time we began learning software development- isn't the most effective approach. TDD replaces the traditional "code then test" cycle. First, you develop test cases for a small increment of functionality; then you write code that makes those tests run correctly. After each increment, you refactor the code to maintain code quality.
A preview of the PDF is not available
... Some of the benefits of AUT include reduced time to test, discover bugs, fix bugs, and implement new features; wider and measurable test coverage; reproducibility, reusability, consistency, and reliability of tests; improved accuracy, regression testing, parallel testing, faster feedback cycle, reduced cost, and higher team morale [15,31,17,40,27]. Likewise, TDD has been shown to result in benefits such as flexible and adaptive program design, cleaner interfaces, higher code quality, maintainability, extensibility, reliability, detailed evolving specification, reduced time on bug fixes and feature implementation, reliable refactoring and code changes, reduced cost, reduced development time, and increased programmer productivity [28,10,19,6,1,3,14,36,20,5,33]. Unfortunately, most of the undergraduate computer science and engineering introductory programming courses do not expose students to AUT and TDD. ...
... The results indicate that, in general, TDD has a small positive effect on quality but little to no discernible effect on productivity. The authors in [33] perform a systematic literature review from 1999 till 2010 about TDD's impact on delivered quality, internal quality, and productivity. The review focuses on 22 published articles that describe 33 unique studies. ...
... From a code testability and correctness point of view, TDD with AUT provides numerous advantages such as immediate and continuous feedback regarding the correctness of code, ability to identify, localize, and fix bugs quickly, traceability and localization of code that got broken due to the addition of new functionality, reduced number of bugs, and confidence in code quality based on test coverage. Moreover, TDD with AUT provides additional benefits which result in a positive environment for developers such as improved maintainability and extensibility of the code, reduced development time, higher programmer productivity, and reduced development costs [28,10,19,6,1,3,14,36,20,5,33]. ...
The best practices of agile software development have had a significant positive impact on the quality of software and time-to-delivery. As a result, many leading software companies employ some form of agile software development practices. Some of the most important best practices of agile software development, which have received significant attention in recent years, are automated unit testing (AUT) and test-driven development (TDD). Both of these practices work in conjunction to provide numerous benefits. AUT leads to reduced time to test, discover bugs, fix bugs, and implement new features; wider and measurable test coverage; reproducibility, reusability, consistency, and reliability of tests; improved accuracy, regression testing, parallel testing, faster feedback cycle, reduced cost, and higher team morale. The benefits of TDD include flexible and adaptive program design, cleaner interfaces, higher code quality, maintainability, extensibility, reliability, detailed evolving specification, reduced time on bug fixes and feature implementation, reliable refactoring, and code changes, reduced cost, reduced development time, and increased programmer productivity. Unfortunately, students in introductory programming courses are generally not introduced to AUT and TDD. This leads to the development of bad programming habits and practices which become harder to change later on. By introducing the students earlier to these industry-standard best practices, not only the motivation and interest of students in this area can be increased but also their academic success and job marketability can be enhanced. This paper presents the detailed design and efficacy study of an introductory C++ programming course designed using the principles of AUT and TDD. The paper presents the pedagogical techniques employed to build industry-proven agile software development practices in the students. As part of the paper, all the course material including the source code for the labs and the automated unit tests are being made available to encourage people to incorporate these best practices into their curricula.
... TDD, BDD, and ATDD and where they do apply to a software product TDD refers to unit tests to assist developers validating specifications. It focuses on small parts of the system that typically are part of the black box of a system [42], [43], [44]. BDD is is the testing of the behavior of the system, based on related input and expected output 20 [45], [46]. ...
Full-text available
It is common practice for practitioners in industry as well as for ICT/CS students to keep writing – and reading ¬– about software products to a bare minimum. However, refraining from documentation may result in severe issues concerning the vaporization of knowledge regarding decisions made during the phases of design, build, and maintenance. In this article, we distinguish between knowledge required upfront to start a project or iteration, knowledge required to complete a project or iteration, and knowledge required to operate and maintain software products. With `knowledge', we refer to actionable information. We propose three approaches to keep up with modern development methods to prevent the risk of knowledge vaporization in software projects. These approaches are `Just Enough Upfront' documentation, `Executable Knowledge', and `Automated Text Analytics' to help record, substantiate, manage and retrieve design decisions in the aforementioned phases. The main characteristic of `Just Enough Upfront' documentation is that knowledge required upfront includes shaping thoughts/ideas, a codified interface description between (sub)systems, and a plan. For building the software and making maximum use of progressive insights, updating the specifications is sufficient. Knowledge required by others to use, operate and maintain the product includes a detailed design and accountability of results. `Executable Knowledge' refers to any executable artifact except the source code. Primary artifacts include Test Driven Development methods and infrastructure-as-code, including continuous integration scripts. A third approach concerns `Automated Text Analysis' using Text Mining and Deep Learning to retrieve design decisions.
... On the other hand, the system's design is also influenced, since a major part of TDD is its decomposition into the smallest reasonable pieces. This reduced complexity also helps to avoid errors and increases maintainability (Crispin 2006;Shull et al. 2010). Even though the primary application area of TDD, and also the one that is relevant for the remainder of this paper, is in software development, it is also used in other contexts, such as process modelling (Slaats et al. 2018) or ontology development (Davies et al. 2019;Keet and Ławrynowicz 2016). ...
Conference Paper
Big data has emerged to be one of the driving factors of today’s society. However, the quality assurance of the corresponding applications is still far from being mature. Therefore, further work in this field is needed. This includes the improvement of existing approaches and strategies as well as the exploration of new ones. One rather recent proposition was the application of test driven development to the implementation of big data systems. Since their quality is of critical importance to achieve good results and the application of test driven development has been found to increase the developed product’s quality, this suggestion appears promising. However, there is a need for a structured approach to outline how the corresponding endeavors should be realized. Therefore, the publication at hand applies the design science research methodology to bridge this gap by proposing a process model for test driven development in the big data domain.
... On the one hand, it aims to increase test coverage, which helps to find and subsequently fix problems that occurred during the implementation of the artifact in question. On the other hand, TDD also influences the design process itself by leading to a more manageable and pre-planned structure that helps to avoid bugs and incompatibilities (Crispin 2006;Shull et al. 2010). The main application area is software development but process modelling (Slaats et al. 2018), the special case of implementing BD applications , and developing ontologies (Davies et al. 2019;Keet and Ławrynowicz 2016) are also found in the literature. ...
Conference Paper
The concept of big data hugely impacts today’s society and promises immense benefits when utilized correctly, yet the corresponding applications are highly susceptible to errors. Therefore, testing should be performed as much and rigorous as possible. One of the solutions proposed in the literature is the test driven development (TDD) approach. TDD is a software development approach with a long history but has not been widely applied in the big data domain. Nevertheless, a microservice-based test driven development concept has been proposed in the literature, and the feasibility of applying it in actual projects is explored here. For that, the fraud detection domain has been selected and a proof-of-concept online fraud detection platform is implemented, which processes real-time streaming data and filters fraudulent and legitimate transactions. After the implementation, an evaluation was carried out regarding test coverage and code quality. The automatic code analysis reports reveale d that TDD had produced very reliable, maintainable, and secure code at the first attempt that is ready for production. Finally, the evaluation revealed that it is highly feasible to develop big data applications using the concept mentioned. However, choosing suitable services, tools, frameworks, and code coverage solutions can make it more manageable.
... On the one hand, the test coverage shall be increased, helping in finding and subsequently remedying issues that occurred during the implementation of the respective artifact. On the other hand, TDD also influences the design process itself, leading to a more manageable and better preplanned structure that helps in avoiding mistakes and incompatibilities [19,20]. While its main application area is the software development domain, it is also applicable to the special case of the implementation of big data applications [21], process modeling [22], or the development of ontologies [23,24]. ...
With the significance, ubiquity, and complexity of information technology continuously rising, the corresponding quality assurance becomes increasingly important and challenging. Consequently, numerous tools, techniques, paradigms, and strategies that facilitate the quality assurance of the developed applications have emerged. Test-driven development (TDD) is one of those strategies. In contrast to the traditional approach, it places testing ahead of the actual implementation, which, in turn, also heavily influences the system’s design. With the approach being rather complex, naturally, guidance on its correct application is one of the most relevant aspects for many practitioners. To provide assistance to them, but also to researchers concerned with the domain, the publication at hand identifies twenty guidelines for the practical application of TDD in software engineering, by conducting a structured literature review. In doing so, it provides a solid foundation for the implementation of corresponding endeavors as well as a starting point for further discussion.KeywordsTest-driven developmentSoftware engineeringLiterature reviewGuidelineBest practiceGuidancePattern
... The latter effect is caused by TDD heavily relying on the decomposition of the developed application into possibly small pieces. Due to the correspondingly decreased complexity, it is easier to avoid errors and, additionally, the maintainability is also increased (Crispin 2006;Shull et al. 2010). ...
Conference Paper
Knowledge, information, and modern technologies have become some of the most influential drivers of today’s society, consequently leading to a high popularity of the concepts of big data (BD). However, their actual harnessing is a demanding task that is accompanied by many barriers and challenges. To facilitate the realization of the corresponding projects, the (big) data science engineering process (BDSEP) has been devised to support researchers and practitioners in the planning and implementation of data intensive projects by outlining the relevant steps. However, the BDSEP is only geared towards a test last development approach. With recent works suggesting the application of test driven development (TDD) in the big data domain, it appears reasonable to also provide a corresponding TDD focused equivalent to the BDSEP. Therefore, in the publication at hand, using the BDSEP as a foundation, the test driven big data science engineering process (TDBDSEP) is proposed, facilitating the application of TDD in the big data domain and further enriching the discourse on BD quality assurance.
... In practice, custom 'home-made' scripts of varying quality are employed to solve these tasks. It is often found that fundamental software engineering concepts, such as Test-Driven-Development (Shull et al., 2010), or the use of state-of-the-art version control tools (e.g. Git) and practices (e.g. ...
... Following this strategy, the test coverage is increased, which helps to find errors and, further, the system's design is changed, since emphasis is given to breaking it down into the smallest sensible pieces. This helps to avoid issues and mistakes that are caused by high complexity and increases maintainability (Crispin 2006;Shull et al. 2010). Besides software development, applications of TDD can also be found in other domains such as ontology development (Davies et al. 2019;Keet and Ławrynowicz 2016) and process modelling (Slaats et al. 2018). ...
Conference Paper
As a consequence of the ongoing digitalization in today’s society, the amount of data that is being produced is rapidly increasing. Moreover, not only the volume of the data is growing, but there are also more complex types of data and, depending on the use case, it is also necessary to integrate heterogenous data into one analysis. Since traditional ways of dealing with data are oftentimes overstrained by those new challenges, novel approaches and technologies have been developed. In its entirety, this phenomenon is summarized under the term big data. However, quality assurance in the big data realm is still not mature and this even more applies to the actual testing. Therefore, it is necessary to explore new approaches. One rather recent proposition was the application of the test driven development methodology to the big data domain. To further evaluate its feasibility and go beyond a purely theoretical point of view, the publication at hand discusses the test driven implementatio n of a movie recommendation system as an exemplary case. In doing so, it facilitates the general understanding of the topic, helps in judging the approach’s feasibility and provides some practical insights concerning its actual application.
Test driven development (TDD) is a practice that aims to improve product quality and maintainability by interweaving the design and implementation with its testing. It is most prominent in the software development domain. However, its usefulness is not undisputed, making it a somewhat controversial topic. Besides giving a short introduction regarding the principles of TDD, the publication at hand motivates and outlines a structured literature review to obtain new insights regarding the perception of TDD in computer science, hopefully contributing to the corresponding discourse. Furthermore, by already conducting the first steps of the review, it provides a first impression regarding the vastness of the potentially relevant literature base and gives a rough indication regarding the extend that is to be expected for the completed work.
Full-text available
Test-driven development is a discipline of design and programming where every line of new code is written in response to a test the programmer writes just before coding. This special issue of IEEE Software includes seven feature articles on various aspects of TDD and a Point/Counterpoint debate on the use of mock objects in applying it. The articles demonstrate the ways TDD is being used in nontrivial situations (database development, embedded software development, GUI development, performance tuning), signifying an adoption level for the practice beyond the visionary phase and into the early mainstream. In this introduction to the special issue on TDD, the guest editors also summarize selected TDD empirical studies from industry and academia.