Conference PaperPDF Available

Abstract and Figures

A software must do what it intends to do. The quality is core factor in IT Industry and in software products. The quality of the product is main concern of the producer and the main requirement of a customer. Software testing is the core activity of quality assurance and very important phase of software development life cycle. Quality of any system is known by testing that product. With the advancement of IT industry many new verification and validation technique are introduced over a time. New trends are well adopted in developed countries like USA, UK etc. But not as well in IT of developing countries like Pakistan. The need of this paper is due to a large number of outsourced projects and also multinational IT companies in Pakistan. Due to high demand of outsourced projects the software testing become an integral part of IT Industry of Pakistan. Our team has collected data about testing in IT Industry of Pakistan which will be benefit for the testers in IT Industry not just in Pakistan but worldwide. These results could be use by researchers to identify the strengths and weakness of testing in IT industry.
Content may be subject to copyright.
Software Testing Practices in IT Industry of Pakistan
Muhammad Shah Jahan
Department of Computer & Software Engineering,
College of Electrical & Mechanical Engineering (CEME),
National University of Science & Technology (NUST)
Islamabad, Pakistan
shah.jahan18@ce.ceme.edu.pk
Kashif
Department of Computer & Software Engineering,
College of Electrical & Mechanical Engineering (CEME),
National University of Science & Technology (NUST)
Islamabad, Pakistan
muhammad.kashif18@ce.ceme.edu.pk
Muhammad Talha Riaz
Department of Computer & Software Engineering,
College of Electrical & Mechanical Engineering (CEME),
National University of Science & Technology (NUST)
Islamabad, Pakistan
talha.riaz18@ce.ceme.edu.pk
Muhammad Abbas
Department of Computer & Software Engineering,
College of Electrical & Mechanical Engineering (CEME),
National University of Science & Technology (NUST)
Islamabad, Pakistan
abbasamir@hotmail.com
ABSTRACT
A software must do what it intends to do. The quality is core
factor in IT Industry and in software products. The quality of the
product is main concern of the producer and the main requirement
of a customer. Software testing is the core activity of quality
assurance and very important phase of software development life
cycle. Quality of any system is known by testing that product.
With the advancement of IT industry many new verification and
validation technique are introduced over a time. New trends are
well adopted in developed countries like USA, UK etc. But not as
well in IT of developing countries like Pakistan. The need of this
paper is due to a large number of outsourced projects and also
multinational IT companies in Pakistan. Due to high demand of
outsourced projects the software testing become an integral part of
IT Industry of Pakistan. Our team has collected data about testing
in IT Industry of Pakistan which will be benefit for the testers in
IT Industry not just in Pakistan but worldwide. These results
could be use by researchers to identify the strengths and weakness
of testing in IT industry.
CCS CONCEPTS
Software and its engineering → Software testing and
debugging
KEYWORDS
Software Testing in Pakistan, Software testing practices, Software
testing, Software testing in developing country
1 Introduction
IT industry is the backbone of every industry and the software is
the backbone of IT industry. The failure of software can produce
catastrophic results. The cost of software failure is another
headache as in US only $62 billion were wasted due to software
failure. The cost of software failure is now more catastrophic as
the software is being used in real time, finance and health industry
where the effect of single error can have significant effects[1, 2].
Due to high use of software, the cost and effect of failure
increases, so there is a rise in software testing [3].
The software testing is very-crucial phase of software
development life cycle. Today IT industry acknowledge that
software testing is the core activity to produce defect free software
product[4]. Software testing is very wide fields and the focus of
research is on software testing as a generic activity and there is
need to explore this field in more depth. There is a difference
between literature and industry practices about software
testing[5].
Software testing are built upon domain experience gained in
industry; the software testing problems are technical. The
practices should be built upon rigorous and systemic approach
suggested by[6] who presents three case studies from industry
which shows that organizational structure of IT organization has
very significant effect on software testing and results[2]. Ahonen
and et al. [7] shows that the most accurate way of retrieving
software testing results is by observing the professional testers.
IT projects are complex due to involvement of hardware, network
and software. The need of software testing increased and also the
cost of testing. 90% of projects are outsourced so the cost of
software failure increases as it requires more money to fix the
errors in distributed projects[8].
To achieve global success in international market the Pakistani
companies must succeed the quality standards of leading software
organizations. Due to rapid change in world two fields are rising,
ECBS '19, September 23, 2019, Bucharest, Romania
© 2019 Copyright is held by the owner/author(s). Publication
rights licensed to ACM.
ACM ISBN 978-1-4503-7636-5/19/09$15.00
https://doi.org/10.1145/3352700.3352724
web-enabled and mobile-enabled e-commerce. Software testing is
core for these fields due to fast turnout and short releases. The
customer demands higher quality standards. The gap between
where IT industry of Pakistan stands now and where should stand
and where could stand is very large. The above scenario describes
the perspective of Pakistani IT Industry but most of the other
developing nations are facing same problems in IT industry.
Every nation needs strong IT industry to succeed internationally.
A survey was conducted about software testing from the industry
and from the individual professionals. Online survey technique
was being used as Google Form and in some cases, interview was
conducted. This survey showed how mature Pakistani IT industry
is and how much organizations are paying attention to testing and
testers as well. This survey covers many aspects of software
testing from different testing-techniques from training of
individuals to education of testers. This survey will be equally
beneficial for the testing team, quality assurance team, researchers
and academia.
Rest of the paper is organized as follows. An extensive literature
review regarding survey of software testing is provided in Section
2. In Section 3 the survey results will be analyzed and discussed
and section 4 summarizes the findings and discusses the lessons
learned.
2 Literature Review
Performed very intensive survey of different key factors of
software quality in developing countries which consist upon 35
questions. This search exhibit that the products were not tested as
per test plan [9].
Torkar and et al. [10] Performed a similar survey about software
testing and reuse. There is a rise in reused code but testing of this
reused code is very limited. 52% of developers test their code,
more than 40% coder test most valued units. The open source and
business developers test their code equally. Developers like KISS
means keep it simple stupid whenever a new tool is released.
Geras and et al [11] Performed a survey in Alberta. The industry
in Alberta test less than the IT industry in United States. One of
the big causes of this less adoption of extreme programming. This
survey showed that most of the organizations produces error
prone test cases and a lack of criteria to stop the testing phase.
Due to this, these organizations release defective software
products. The most popular technique is Unit testing and there is
an increase in automated unit testing in development community.
Only 30% of the organizations uses automated testing with Junit,
NUnit or CPPUnit.
Kasurinen and et al. [12] Did an extensive industry study of 31
organizations. 55 industry specialists were interviewed about
software test automation. The testing process only receive three
fourth resources they required, and access of these resources is
limited. The adoption of automated testing tools is difficult in
industry. Even testing efficiency increases there are still 25%
effort dedicated to testing which is half of which is required. Only
26% of the participants use automated test cases. There is an
ambiguity that it depends upon that which organization is using
automation and how automation should be implemented.
Deak and et al. [13] Performed a research which was combination
of interviews and surveys with mix of qualitative and quantitative
questions. This research provides the current trends of organizing
software testing activities in IT industry. A new trend is that the
tester is part of the development team or a secondary role for
developer. The existence of testing department in small or
medium size organizations is nullable. Automated testing required
more budget and become complex that is why small automation is
being achieved.
Performed a nationwide survey about software testing in Canada.
This survey is the extension of surveys they performed in 2004
and 2009. This survey showed that
(I) There is an increase in training related to testing
(II) Unit testing and functional testing still the most
popular
(III) There is an increase in interest in mutation testing
(IV) Traditional test-last development has the most use
in IT industry of Canada[14-17].
Mailewa and et al [18] Purposed a novel testing technique for
testing of large set of test cases. A combinational testing. Test
automation is the solution of time problems in software testing.
Pairwise testing for combinational testing and selenium tool for
test automation of web projects. Also emphasize the advantages of
automation testing over manual testing.
The use of formal testing methods reduced even in real time
systems. Now organizations use more automation testing. The
cause of automated testing is high if there is use of agile software
development. They performed a survey of software testing and the
results showed that use of capability maturity models decreases
whilst automate testing become increasing common and more
sophisticated. Testing perform in design phase is decreasing while
acceptance testing increases[3].
Testing is one of the most important Phase of SDLC for quality
assurance but neglected as a result of time and budget. Sub-
optimizing activities of testing could lead to failure of software
projects. The testing should not only consider in testing phase of
SDLC (software development life cycle). The study found lack of
personal competences of testing is one of the major cause for
defected software product[19].
Performed an extensive study about stakeholder’s satisfaction and
advancement and penetration of agile methodologies in IT
industry of South Africa. This study based upon self-reported data
from 59 organizations. Ferreira and et al. [20] performed a survey
about unit testing of software process improvement network
(SPIN) which compress 50 representatives from different
companies. Runeson and et al. [21] a first time ever survey on the
subject of software testing in Australian ICT industry which
contains results collected from 65 organizations. These
organizations responded that they use at least one structured
software testing methodology. 65% of these organizations use
black-box testing while other use white-box testing. 65% of these
42 organizations carried out testing on 80% of their projects.[22]
Sneha and et al. [23]Performed and extensive research about
software testing methods and automated testing tools. This study
explores the testing techniques and tools and also automation
verses manual testing. The main goal of a testing activity is to
deliver software product with no or few errors. This study covered
unit testing, integration testing etc. Testing methodologies like
white-box testing, black-box testing etc.
With emergence of agile the need of testing increases. The
traditional methodologies has a separate phase for testing while in
agile it is very limited.[24]in agile factory model the defects
occurs are of severity of 3.26% of defects are defects during
acceptance integration and performance. While 17% in factory
acceptance testing[25]. The software automation test generation
can be modernized by agile development methods. This method
purpose team meetings of teams that develop the software product
and automated test software. The use of automation increases in
later sprints of Scrum which will speed up the delivery[26].
3 Methodology
The questioner-based survey methodology was used to get
responses from different IT organizations of Pakistan. We used
google forms, interviews and hard copy for this survey. In
interviews we asked questions of google form directly to the
participants. The GQM [27]approach was used in this survey. The
goal of this survey is to provide the details of the practices used
for software testing in IT Industry of Pakistan, to identify what
testing methods, tools and metrics are used by professionals and
researchers in Pakistan as well as worldwide. The goal of this
survey also is to identify the latest trends in software testing and
direction where we are going right now. This will identify the
strengths and weakness and also encourage the collaborations
between industry and academia. Following are the research
questions we raise in this research:
RQ 1: How much effort is spent on different levels of software
testing.
RQ 2: Which are the most and least used test technique.
RQ 3: What specific testing tools are used and how much
automation is used in software testing.
RQ 4: What type of test-metrics are used in Software testing.
RQ 5: How the teams and processes are managed during testing.
RQ 6: How much software testing training is provided by
organizations to their employees.
RQ 7: What are the challenges faced by industry.
Industry professionals and Authors did a peer-review on this set of
questions. This consultation was done to reduce the complexity in
form and to increase the similar terminology so that it could
match with action of respondents. As software testing terminology
in academia is slightly different from the industry where it is used
as (system testing, functional testing etc.). The feedback of
professionals was used to finalize the survey. Most of these
questions have quantitative predefined multiple-choice answers.
Few questions have qualitative answers.
The first category of questions (1-10) collected the information of
respondents and respondent’s organizations. The second section
(11-13) questions collected the response\es for test type/levels.
This section will show what type of testing (unit testing, system
testing etc.) is being perform in the industry and also shows which
organization are making the testing levels to identify the
characteristics of product under test. The third category (14-15)
questions showed what software testing technique are being used
and about a new and rising software testing technique which is
mutation testing.
In forth category (16-17) questions asked about automated testing
and tools/framework use for software automated testing. In fifth
category (18-19) we asked about how they measure software
testing and what metrics they use. In sixth category (20-22)
questions asked about the criteria used to terminate the testing
activities, what barriers prevent from adopting systematic testing
methodology and testing tools, and time spend on software
testing. In the seventh category (23) questions asked about the
software testing training and this question does not count for self-
study. And in eight and last category (24) questions asked about
involvement of organizations into any research and the three most
testing challenges they want academia to solve.
We send this form to 80 organizations and 450 individuals from
IT Industry of Pakistan.53 organizations while 185 individuals
participate in this survey the responses are 238 with 80% om
google form and 20% from interviewing and hardy copy. Many
organizations are CMMI and ISO ranked. These organizations are
large, medium and small sized. The responses from medium sized
and emerging companies is very satisfying.
3.1 DATA ANALYSIS & SURVEY RESULTS
3.1 Profile of participants
3.1.1 Respondents designation
The Figure-1 shows the position of the participant.This question
was a multiple-choice because a developer may be a tester at same
time and vice versa. Means one participants could hold more then
one position. People in diffrent positions see and rate the different
problems diiferently that is why this survey allow other positions
holders other than software tetsre are allowed to take part in this
survey.
Figure. 1. Respondents position
We analyzed the frequency in this question in which participants
holds more then one position. 4 reposndants were tester and
devleoper at same time while 4 people were bussiness analyst and
developer at same time. The ratio of joint position is rather low
3.8%. The highest joint position ware tester and QA Lead which
could be inter-related.
3.1.2 Participants Experience
Figure-2 shows the participants work experience in overall IT
industry and Figure-3 particularly in software testing. 15% and
5% have over 5-10 years of experience in software development
like developer, business analyst etc., respectively. 10% and 4%
have over 5-10 years’ experience in software testing. These results
are because when a newcomer enters in software industry, they
start working in software development phase and after experience
they enter in software testing phase.
Figure. 2. Working Experience
Figure. 3. Testing Experience
43% of the participants has 2-5 years of experience in overall
software development which is why more than the 26% of
participants are in their start of career who has 0 to 2 years of
experience of software development while 60% of the respondents
are in start of testing career. It shows more and more people are
choosing software testing as a career.
3.1.3 Organization Category
In this question we asked for the category of the participant’s
organization. We provide six choices and an option of others also.
But all the responses fall into one of the six provided categories.
Figure-4 shows the results.
Figure. 4. Organization Categories
Most of the responses are from software contractor, developing
customize software and developing software for internal use. The
implementation of testing activities is different for different
categories. The client of customize software has one firm while
package software used by many clients.
3.1.4 Academic Degree
To asked participants educational background, we asked for their
highest degree. Figure-5 shows the results
Figure. 5. Academic Degree
The results are very satisfying as 60% of the respondents holds
BS degree while 36% of the respondents have master’s degree
which is quite high ratio for a developing IT Industry. Just 1% of
the respondents have PHD degree as most of PHD’s prefer
academia as a profession.
3.1.5 University Degree
As a follow up of previous questions we asked the participants
about their major degree (software engineering, Computer
Engineering, computer science e.g.). the results are showed in
Figure-6
Figure. 6. University Degree
Software testing and computer science was the major degree of
most of the participants. 61% of the participants have software
engineering as their major while 29% have computer science and
8% have MBA as their major Subject. These business participants
hold position of business analyst.
3.1.6 Testing Certification
In this question we asked the participants about the testing
certification they have. The results are not very impressive
compare to develop countries but impressive for a developing
country. The Figure-7 displays the results.
Figure. 7. Testing Certification
51% of the participants take at least one type of testing certificate.
While 49% have no certificate. These are those people who was
developer and then turned to tester and have a lot of development
experience. The trend of certification could be seen in newcomers
who chose software testing as a profession. 19% of the participant
have Test Managed approach (TMap) certification while 12%
have Certified Associate in Software Testing (CAST). 9% each
have foundation level certificate tester (ISTQB) and Certified
Software Quality Engineer (CSQE).
3.1.7 Type of Methodology is being use
In this question we asked for the methodology is being used in
participant’s organization. As this is the era of Agile and most of
the participants belongs to medium level organizations while large
organization still use traditional methodology like waterfall and
iterative. Due to complexities of agile in large scale software.
Figure-8 shows the following results
Figure. 8. Current Development Team
47% of the respondents respond that they use agile methodology
while 28% of the participants are still using traditional
methodology. A new trend could be seen in small and medium
size organizations is 25% do not explicitly distinguish which
means there could be mix of both methodologies.
3.1.8 Organization Size
In this question we asked for company size of respondents. There
are five categories of companies
Very small (1-09)
Small (10-49)
Medium (50-249)
Large (250-999)
Very large (1000+)
Figure. 9. Number of Employees
45% of the participants belong to medium size organizations
while 27% belongs to small companies. 9% each from large and
very large organizations. It showed that participants from all size
organization took part in this survey which is very satisfactory.
3.1.9 Programming Language
In this question we asked for the programming language use in
participant’s organization for development. This question is very
important as asp.net and java provides very strong automate
testing framework (JUNIT and NUNIT). The result is shown in
Figure-10
Figure. 10. Programming Languages
28% of the organizations use java because of latest trend of
mobile application development and android development. 26%
of the organization use C/C++ are being used in embedded
software, while 18% are uses .NET family (C#, VB) and at last
(28%) of the organization are using other languages like PHP,
Ruby on Rails etc.
3.2 Test Type/Levels
3.2.1 Test Case and Test Suits
In this question we asked about different types of test to the
participants. The results are shown in Figure-11 which are of very
changing as 30% of the participants use User acceptance testing.
27%-unit testing, 23% of the respondents use functional/system
testing. Only 5% of the participants use performance testing while
just 1% say they use all or mix of these testing types.
Figure. 11. Test Cases and Test Suites
3.2.2 Effort Spent on software Testing
In this question we asked about effort estimation spent on
different software testing types (Unit testing, System testing). A
participant could choose 40% on Unit testing, 40% on Functional
Testing and 20% on system Testing. The results are shown in
Figure-12
Figure 14 Generate Test Cases
Figure. 12. Effort spent on testing
Most of the respondents spent 21-40% on Unit testing while 41-
60% is spent on user acceptance testing. Those results are same
for System testing, GUI testing, Performance testing, Stress
testing and most of the participants 0-20% on other type of
testing.
3.2.3 Which Testing Activities
In this question we asked for the type/phase of SDLC. The results
are shown in Figure-13
Figure. 13. Test Activities
The results are varying as 36% of the respondents respond that
they use Test-Last development (TLD) while 35% of the
respondents use Test-Driven development (TDD). 27% of the
respondents uses Behavior-Driven development (BDD). While
1% using smoke or sanity testing.
3.3 Testing Techniques
3.3.1 Generate Test Case
In this question we ask for the technique participants use for
testcase generation. Figure-14 shows the results. These results are
mix as 29% of the respondents uses model-based technique. 23%
of the respondents uses boundary value analysis to generate test-
cases. 17% and 19% use code-coverage (white-box testing) and
exploratory testing respectively while 7% use category
partitioning (i.e. equivalence classing). 15% of the participants do
not use any technique explicitly.
3.3.2 Mutation Testing
In this question we asked about mutation testing (in which faults
are injected). We asked do they use mutation testing in their all
projects or in some projects not using it at all. The Figure-15
shown the results
Figure. 14. Mutation Testing
The results are very satisfying. As 26% of the respondents
respond that they use mutation testing in their all projects while
42% use it in some of their projects. 32% of the participants
mostly belong to small organizations do not use mutation testing.
3.4 Automatic Testing and Tools
3.4.1 Automatic Testing and Manual Testing
Figure. 15. Automate vs Manual Testing
In this question we asked to respondents about automated testing
and manual testing. Figure-16 shown the results. The results are
unexpected as the use of manual testing 21-40% while use of
automatic testing is 41-60% This is due to high use of Java and
.NET framework and web (selenium) and android development
which is help-full in automated testing.
3.4.2 Testing Tools
Figure. 16. Testing Tools
We asked about which software testing tools and testing
framework used by most of the participants. Figure-17 shown that
38% of the participant use family of commercial functional test
tools (for non-website) e.g. IBM. 26% each use testing tools for
web application and XUNIT framework e.g. JUNIT, NUNIT.
10% of the participants use tools and framework.
3.5 Software Testing Measurements
3.5.1 What type of Testing Measurement
Measurement in software testing is very important instrument. It
is very important to determine that what type of measurement are
done for software testing in Pakistan. As we measure these
measurements in white-box testing and in black-box testing we
use code coverage to measure how much of the system is tested.
The choice given to participants was
Line(statement) coverage
Decision(branch) coverage
Condition coverage
MC/DC coverage
Figure. 17. Code (Test) Coverage
Figure-18 showed that 99% of the participants use one of the
above approaches. 43% of the respondents use decision (Branch)
coverage while 37% of the respondents use condition coverage.
15% uses Line (Statement) coverage and 45 of MC/DC coverage.
3.5.2 Test and Quality Metrics used in projects
We also asked the participants about test and quality metrics they
use in current or previous projects. Figure-19 shown the results
Figure. 18. Test and Quality Metrics
39% of the respondents say they use number of test-cases
execution defined by time while 20% use total number of defects
detected per day (week, month). 16% and 15% respondents
respond that they use testers defect (bug) detection productivity
(bugs found per day by each tester) respectively. 10% uses
number of passing user acceptance tests.
3.6 Termination and Prevention
3.6.1 Criteria to Terminate Testing Phase
In this question we asked the participants that what criteria they
use to terminate the testing phase. This question is very important
as it tells how to terminate a testing phase. It directly depends
upon the quality of product and size of company. Figure-20
presents the results
Figure. 19. Termination Criteria
35% of the respondents responds that they execute all test cases
without finding more defects(bugs). 22% and 21% of the
respondents say they use fixed time duration and no bugs are
found anymore (reliability growth/saturation models). Other
participants use some other criteria.
3.6.2 Hurdles in Adopting Systematic Software Testing
In this question we asked the respondents about the barriers that
prevent them adopting systematic software testing methodology.
Figure-21 shows that 62% of the participants respond that time is
the most effective barrier while 22% responds that no support of
higher-level management and 9% say cost is the barrier while
adopting the systematic testing methodologies.
Figure. 20. What prevents from adopting Testing
3.6.3 Effort spent on overall Testing
In modern software development, testing phase is taking more and
more effort in shape of budget and time. It is very important to
know how much Effort is being spent on software testing activity
in IT industry of Pakistan. Figure-22 presents following results
Figure. 21. Effort spent on overall testing
46% of the participants spent 41-60% effort on software testing
phase. 19% and 17% of participants spent 21-40% and 61-80% of
budget and time on testing phase respectively. Just 12% of the
participants spend below 20% of their budget and time on testing.
3.7 Training and Challenges
3.7.1 How much Formal Training
The formal training about testing provides a solid foundation for
software testers. It is very important to have formal training for
software testing. This training includes on-site training and
training in university. This question excludes the self-training and
study in this survey. This training is in hours. Figure-23 shown the
results
Figure. 22. Formal Training
32% of the participants have 41-60 hours of training while 21%
have 21-40 hours training. 17% of the participants have 61-80 and
0-20 hours of formal training each. 6% and 7% of participants
have 81-100 or 0 hours of training respectively.
3.7.2 Testing Challenges
In this question we asked the participants for their three top
challenges they want academia to solve. These are the challenges
faced by the industry professionals. Figure-24 shown the results
Figure. 24. Testing Challenges
46% of the participants faces time, cost and effort and staffing.
46% of the respondents respond test-code engineering as the
biggest challenges they are facing. While 42% of the participants
think training is the problem. While 22% think metrics is the
problem they are facing, and they want academia to solve these
problems.
4 Discussion
Figure. 23. Overall Results
This research paper shows that most of the testing is performed by
developers. Most of the developers has BS Software Engineering
as academic degree. Mostly software organizations are software
contracting companies. The IT organizations has most employee
with average experience of 02-05 years with 0-2 years of
experience in Software testing. The software companies are using
agile methodology. The average size of companies in Pakistan is
50-249 employees. C++ and Java still dominate the Pakistani IT
Industry. User acceptance is the most used metric for product
quality. 41-60% of effort is spent on different testing techniques.
Most of the organizations use test-last development and use
model-based technique for generating Test-cases. A new trend is
being seen is use of Mutation testing in IT industry of Pakistan.
Some are using it in all projects but mostly use it in some projects.
Design (branch) coverage is used for code coverage and number
of test cases in defined period is the quality metrics. The
termination criteria of Testing Phase are all test cases executed
within a time. This paper shows that time is barriers in choosing
systematic testing methodology. This paper presents a new trend
in IT industry of Pakistan which is the formal training of
employee on software testing and the top three challenges faced
by IT industry which are Triple constraints and, Test-code
Engineering and Training. This study showed the overall growth
of IT industry of Pakistan especially in Software Testing.
There are many research papers about software testing and quality
but very few related to software testing in IT industry of Pakistan.
Some research is being done on topic of software quality in
Pakistan which covered software testing to some extent. We try to
cover every aspect of testing in IT industry of Pakistan. Due to
nature of our survey we could not compare our set of results with
other previous researches on the topic of Software testing related
to IT industry of Pakistan but could compare some question with
similar studies in other countries like Canada, Australia and
Norway.
There is some Common trend in IT industry of Pakistan like other
countries:
Pakistan IT industry is dominated by Black-box Testing
which is like the finding in Canada reported in Vahid
Garousia and et at. (2012).
Still decision and condition coverage are being use for
code coverage in Pakistan similar to finding of Vahid
Garousia and et at. (2012).
User Acceptance testing is the most used testing
technique than unit and funxti0onal testing in Pakistan
which is similar to finding of Anca Deak and et al.
(2013) in Norway.
A new trend of spending of 41-60% of effort on testing
is similar to finding of Taipale and et al. (2005)
.A similar trend could be seen in Pakistan is the formal
training of testers which is 93% more than the findings
of S.P. Ng et al[23] which are 72%.
Some of our results differ then the finding in other countries
and are as follows:
Automated testing is new trend in Pakistan’s IT industry
due to high development of Mobile and Web
development.
IT industry of Pakistan is paying more and more
intention to mutation testing which varies then all three
countries mentioned in this paper.
Junit is still dominate the IT industry with an increase of
use of Web testing tool (Selenium) with shows a
difference with the findings of Taipale et al. (2005) and
Vahid Garousia and et at. (2012).
Testing certification is now become a core requirement
for newcomers who chose software testing as their
career which is similar to other countries.
A new trend of test-driven development inn Pakistan but
still Test-last development dominate the industry. These
finding are similar finding of Vahid Garousia and et at.
(2012) [14] and T. Hynninen et al. [3]
Time is the main barrier while choosing the systematic
testing methodologies which clearly differs from S.P.
Ng et al[23]
5 Conclusion and Future Work
In this paper, we have conducted a survey among organizations of
Pakistan which are paying more attention to the software testing
activity. Software testing is the core activity of software SDLC. In
this survey we witnessed some encouraging trends in Pakistan.
Most of the testers start their career as a developer and when they
got experience, they move to the job of software testing. Most of
the testers are well educated and have received relative degree
with IT. Industry is paying attention to Test-Certification. Effort
spending on testing is better than some developed countries.
Diversity of IT industry is the key to success like in Canada most
of the IT industry use TLD but in Pakistani industry is using TLD,
TDD and BDLL. organizations are using systematic testing
methodology. Selenium is tool use for automating testing of web
development that’s why Automate testing is getting more and
more attention in Pakistan. Mutation testing is relatively new and
not many developed IT Industry is still using it but in Pakistan we
witnessed a different trend as most of the organizations are using
Mutation testing in some part of testing. There were times when
formal training was conducted on Testers end but now most of the
organization are conducted Formal training for their testers but
still the training time is very low. Testing phase termination
criteria is still time, budget or manager. In large organization there
are formal methods for termination but in medium and small
organization still the manager’s orders are the criteria.
This paper showed the growth and trends in IT industry of
Pakistan with respect to some developed countries like USA,
Canada etc. but also alarm us on the growth speed. Pakistan still
lagging behind where it could be at this time but with passage of
time and more and more investment in IT sector and shifting of
multinational companies in Pakistan, we hope the IT industry of
Pakistan will flourish in near future and will challenge not just
other developing industries but also many developed IT
industries. There is need to do more surveys in Pakistan as well as
other developing countries and compare these results with them,
so we could assess where IT industry of Pakistan stands now and
where it should be in future. In future we will provide the
relationship between different questions to assess more accurate
and related results.
REFERENCES
1. Kit, E., Software testing in the real world: improving the process. 1995:
Addison-wesley.
2. Planning, S., The economic impacts of inadequate infrastructure for software
testing. National Institute of Standards and Technology, 2002.
3. Hynninen, T., et al. Software testing: Survey of the industry practices. in 2018
41st International Convention on Information and Communication Technology,
Electronics and Microelectronics (MIPRO). 2018. IEEE.
4. Bertolino, A. Software testing research: Achievements, challenges, dreams. in
2007 Future of Software Engineering. 2007. IEEE Computer Society.
5. Rooksby, J., M. Rouncefield, and I. Sommerville, Testing in the wild: The
social and organisational dimensions of real world practice. Computer
Supported Cooperative Work (CSCW), 2009. 18(5-6): p. 559.
6. Engström, E. and P. Runeson. A qualitative survey of regression testing
practices. in International Conference on Product Focused Software Process
Improvement. 2010. Springer.
7. Ahonen, J.J., T. Junttila, and M. Sakkinen, Impacts of the organizational model
on testing: Three industrial cases. Empirical So ftware Engineering, 2004. 9(4):
p. 275-296.
8. Batra, D., Modified agile practices for outsourced software projects.
Communications of the ACM, 2009. 52(9): p. 143-148.
9. Naqvi, N., A. Khan, and A. Khan, Survey to Improve Software Quality
Assurance in Developing Countries. International Journal of Technology and
Research, 2015. 3(1): p. 1.
10. Torkar, R. and S. Mankefors. A survey on testing and reuse. in Proceedings
2003 Symposium on Security and Privacy. 2003. IEEE.
11. Geras, A.M., M. Smith, and J. Miller, A survey of software testing practices in
Alberta. Canadian Journal of Electrical and Computer Engineering, 2004.
29(3): p. 183-191.
12. Kasurinen, J., O. Taipale, and K. Smolander, Software test automation in
practice: empirical observations. Advances in Software Engineering, 2010.
2010.
13. Deak, A. and T. Stålhane. Organization of testing activities in Norwegian
Software Companies. in 2013 IEEE Sixth International Conference on Software
Testing, Verification and Validation Workshops. 2013. IEEE.
14. Garousi, V. and J. Zhi, A survey of software testing practices in Canada.
Journal of Systems and Software, 2013. 86(5): p. 1354-1376.
15. Garousi, V., et al., Testing embedded software: A survey of the literature.
Information and Software Technology, 2018.
16. Garousi, V. and T. Varma, A replicated survey of software testing practices in
the Canadian province of Alberta: What has changed from 2004 to 2009?
Journal of Systems and Software, 2010. 83(11): p. 2251-2262.
17. Garousi, V., M. Felderer, and F.N. Kılıçaslan, A survey on software testability.
Information and Software Technology, 2018.
18. Mailewa, A., J. Herath, and S. Herath. A Survey of Effective and Efficient
Software Testing. in The Midwest Instruction and Computing Symposium.
Retrieved from http://www. micsymposium.
org/mics2015/ProceedingsMICS_2015/Mailewa_2D1_41. pdf. 2015.
19. Belt, P., et al., Improving the efficiency of verification and validation.
International Journal of Services and Standards, 2008. 4(2): p. 150.
20. Ferreira, C. and J. Cohen. Agile systems development and stakeholder
satisfaction: a South African empirical study. in Proceedings of the 2008
annual research conference of the South African Institute of Computer
Scientists and Information Technologists on IT research in developing
countries: riding the wave of technology. 2008. ACM.
21. Runeson, P., A survey of unit testing practices. IEEE software, 2006. 23(4): p.
22-29.
22. Ng, S., et al. A preliminary survey on software testing practices in Australia. in
2004 Australian Software Engineering Conference. Proceedings. 2004. IEEE.
23. Sneha, K. and G.M. Malle. Research on software testing techniques and
software automation testing tools. in 2017 International Conference on Energy,
Communication, Data Analytics and Soft Computing (ICECDS). 2017. IEEE.
24. Agarwal, A., N. Garg, and A. Jain. Quality assurance for Product development
using Agile. in 2014 International Conference on Reliability Optimization and
Information Technology (ICROIT). 2014. IEEE.
25. Doherty, B., et al. Defect Analysis in Large Scale Agile Development: Quality
in the Agile Factory Model. in 2016 Joint Conference of the International
Workshop on Software Measurement and the International Conference on
Software Process and Product Measurement (IWSM-MENSURA). 2016. IEEE.
26. Sultania, A.K. Developing software product and test automation software using
Agile methodology. in Proceedings of the 2015 Third International Conference
on Computer, Communication, Control and Information Technology (C3IT).
2015. IEEE.
27. Mandić, V. and N. Gvozdenović, An extension of the GQM+ Strategies
approach with formal causal reasoning. Information and Software Technology,
2017. 88: p. 127-147.
... We explored several survey studies on testing conducted in the context of non-MSAbased systems (e.g., [163][164][165][166][167][168][169][170][171][172] ), and we compared our key findings on testing strategies, tools, and challenges with these surveys. Regarding testing strategies, the results of our survey and the surveys related to non-MSA-based systems (e.g., [163][164][165][166][167] ) show that unit testing is commonly used as a de facto strategy for testing all types of systems. ...
... To our surprise, we did not find evidence regarding the use of scalability, integration contract, and consumer-driven contract testing strategies in the context of non-MSA-based systems. About testing tools, JUnit, JMeter, and Selenium are commonly used by practitioners of both MSA-based and non-MSA-based systems (e.g., [165][166][167][168] ), for example, Selenium is used as a tool to perform testing on MSA-based systems (see Figure 3. 19), and is also employed for testing non-MSA-based systems due to its test automation capabilities. Regarding testing challenges, the results of our survey and other surveys (e.g., [167,[171][172] ) ...
Thesis
Full-text available
This thesis explored software architecture design of microservices systems in the context of DevOps and make the following novel aspects: (1) This thesis proposes a set of taxonomies of the research themes, problems, solutions, description methods, design patterns, quality attributes as well as the challenges of microservices architecture in DevOps that contributes to the software engineering body of knowledge by conducting state of the art (i.e., systematic mapping) and practice (i.e., mixed-method) studies on architecture design of microservices systems. These studies mainly identify, analyze, and classify the challenges and the solutions for microservices architecture in DevOps, design of microservices systems, as well as monitoring and testing of microservices systems. The findings of these studies can help practitioners to improve the design of microservices systems. (2) This thesis proposes a taxonomy of issues occurring in microservices systems by analyzing 1,345 issue discussions extracted from five open source microservices systems. The proposed taxonomy of issues consisting of 17 categories, 46 subcategories, and 138 types. This thesis also identified a comprehensive list of causes and mapped them to the identified issues. The identified causes consist of 7 categories, 26 subcategories, and 109 types. The proposed taxonomy and identified causes can help practitioners to avoid and address various types of issues in the architecture design of microservices systems. (3) This thesis proposes a set of decision models for selecting patterns and strategies in four MSA design areas: application decomposition into microservices, microservices security, microservices communication, and service discovery. The proposed decision models are based on the knowledge gained from the systematic mapping study, mixed-method study, exploratory study, and grey literature. The correctness and usefulness of the proposed decision models have been evaluated through semi-structured interviews with microservices practitioners. The proposed decision models can assist practitioners in selecting appropriate patterns and strategies for addressing the challenges related to the architecture design of microservices systems.
... We explored several survey studies on testing conducted in the context of non-MSA-based systems (e.g., [123,124,125,126,127,128,129,130,131,132]), and we compared our key findings on testing strategies, tools, and challenges with these surveys. Regarding testing strategies, the results of our survey and the surveys related to non-MSA-based systems (e.g., [123,124,125,126,127]) show that unit testing is commonly used as a de facto strategy for testing all types of systems. ...
... To our surprise, we did not find evidence regarding the use of scalability, integration contract, and consumer-driven contract testing strategies in the context of non-MSA-based systems. About testing tools, JUnit, JMeter, and Selenium are commonly used by practitioners of both MSAbased and non-MSA-based systems (e.g., [125,126,127,128]), for example, Selenium is used as a tool to perform testing on MSA-based systems (see Figure 19), and is also employed for testing non-MSA-based systems due to its test automation capabilities. Regarding testing challenges, the results of our survey and other surveys (e.g., [127,131,132]) show that two testing challenges (i.e., integration testing, creating and implementing automated tests) are commonly faced by practitioners of both MSA-based and non-MSA-based systems. ...
Preprint
Full-text available
Context: Microservices Architecture (MSA) has received significant attention in the software industry. However, little empirical evidence exists on design, monitoring, and testing of microservices systems. Objective: This research aims to gain a deep understanding of how microservices systems are designed, monitored, and tested in the industry. Method: A mixed-methods study was conducted with 106 survey responses and 6 interviews from microservices practitioners. Results: The main findings are: (1) a combination of domain-driven design and business capability is the most used strategy to decompose an application into microservices, (2) over half of the participants used architecture evaluation and architecture implementation when designing microservices systems, (3) API gateway and Backend for frontend patterns are the most used MSA patterns, (4) resource usage and load balancing as monitoring metrics, log management and exception tracking as monitoring practices are widely used, (5) unit and end-to-end testing are the most used testing strategies, and (6) the complexity of microservices systems poses challenges for their design, monitoring, and testing, for which there are no dedicated solutions. Conclusions: Our findings reveal that more research is needed to (1) deal with microservices complexity at the design level, (2) handle security in microservices systems, and (3) address the monitoring and testing challenges through dedicated solutions.
... We explored several survey studies on testing conducted in the context of non-MSA-based systems (e.g., [123,124,125,126,127,128,129,130,131,132]), and we compared our key findings on testing strategies, tools, and challenges with these surveys. Regarding testing strategies, the results of our survey and the surveys related to non-MSA-based systems (e.g., [123,124,125,126,127]) show that unit testing is commonly used as a de facto strategy for testing all types of systems. ...
... To our surprise, we did not find evidence regarding the use of scalability, integration contract, and consumer-driven contract testing strategies in the context of non-MSA-based systems. About testing tools, JUnit, JMeter, and Selenium are commonly used by practitioners of both MSAbased and non-MSA-based systems (e.g., [125,126,127,128]), for example, Selenium is used as a tool to perform testing on MSA-based systems (see Figure 19), and is also employed for testing non-MSA-based systems due to its test automation capabilities. Regarding testing challenges, the results of our survey and other surveys (e.g., [127,131,132]) show that two testing challenges (i.e., integration testing, creating and implementing automated tests) are commonly faced by practitioners of both MSA-based and non-MSA-based systems. ...
Article
Full-text available
Context: Microservices Architecture (MSA) has received significant attention in the software industry. However, little empirical evidence exists on design, monitoring, and testing of microservices systems. Objective: This research aims to gain a deep understanding of how microservices systems are designed, monitored, and tested in the industry. Method: A mixed-methods study was conducted with 106 survey responses and 6 interviews from microservices practitioners. Results: The main findings are: (1) a combination of domain-driven design and business capability is the most used strategy to decompose an application into microservices, (2) over half of the participants used architecture evaluation and architecture implementation when designing microservices systems, (3) API gateway and Backend for frontend patterns are the most used MSA patterns, (4) resource usage and load balancing as monitoring metrics, log management and exception tracking as monitoring practices are widely used, (5) unit and end-to-end testing are the most used testing strategies, and (6) the complexity of microservices systems poses challenges for their design, monitoring, and testing, for which there are no dedicated solutions. Conclusions: Our findings reveal that more research is needed to (1) deal with microservices complexity at the design level, (2) handle security in microservices systems, and (3) address the monitoring and testing challenges through dedicated solutions.
... With expanding competitors, software houses understood the significance of drawing in the customers and points of interest. Customer is one of the key factor and the state of achievement of the software business (Jahan et al., 2019;Kilu et al., 2019). CKMC preferences ought to be taken in the software firms so information and experience are intentionally applied, with the end goal that creation, ability, effectiveness and responsibility of the association are enhanced (Esterhuizen et al., 2012;Akhtar et al., 2018). ...
... According to the Pakistan Software Export Board (PSEB), the total size of software industry is approximately US$6.5bn, by continuing both exports and domestics turnover is expected to grow at least 3.5% in the next five years. The chief executive officer of Pakistan National Technology declares the software industry growth will be twice in year 2020 (Talib et al., 2017;Jahan et al., 2019). ...
Article
Full-text available
Purpose The purpose of this study is to examine the relationship between the customer knowledge management capability (CKMC) on project performance through strategic agility in the context project based software companies of Pakistan. The aim of the paper is to find out whether and how is customer knowledge beneficial for project performance and recognized as the important source of advancement of the knowledge management (KM) theory and the essential subject in practical ground. Design/methodology/approach In this study, non-probability, simple random sampling method was used to collect the data because it excludes bias from the data collection process. Although, the population of this research includes 307 employees working in different 30 public and private projects based software firms, operating in twin cities Rawalpindi, Islamabad. The respondents are project supervisors, team members, customers working on these different projects. Because of time limitation data has been collected within four months (i.e. November 2018 to February 2019) for this study, it is not time-lagged study and the data were collected at one time, so the design is cross-sectional in nature. The analysis was established using partial least squares-structural equation modeling (Smart PLS-SEM v.3.2.8) software to test hypotheses. Findings The results revealed the structural equation modeling that the components creating, transferring, integrate and influence ensure the most significant job in clarifying the customer knowledge and enhancing the capability to understand the customer needs and want which lead to decrease project delay, over consumption of the budget and directly lead to increase the project performance. The analyzed results also successfully justified the gap of this research study by showing the significant relationship between CKMC and project performance, also the indirect effect of CKMC through strategic agility on project performance more than its direct effect. So, the strategic agility plays positive and significant mediating role between CKMC and project performance, therefore the all sub-hypothesis and primary hypothesis were accepted. Originality/value This study sets the context with a brief summary of the key characteristics of the CKMC to improve the new product performance, enhance product/service quality, also reduce costs and enhance the competitiveness of organizations. Organization ought to acknowledge how to use KM to generate their revenues and achieve their goals. However, available techniques and methodology to measure the sufficiency are dissatisfying and consistent need for assessments and evaluations of this issue are felt.
... We note that emphasis is made on functional tests over non-functional tests. Our findings contradict the findings by [11] conducted in Pakistan and established that the User acceptance test was the most performed. Most projects handled are Web applications, yet the least type of testing conducted is Security Testing raises significant concerns. ...
Article
Full-text available
ABSTRACT–Given the increased reliance on technology, Software Quality Assurance(SQA) has become a vital area in Software Engineering (SE). SQA practices require training, cost and often take more time than actual code writing. Owing to these requirements, software developers often ignore or partly implement SQA practices, leading to potentially poor quality software development. The goal of the study is to characterise SQA practices of software developers in Kenya. As such, quantitative empirical research was conducted. Seventy-seven (N=77) completed questionnaires were received and analysed to yield the required insights. The analysis of the findings indicates compliance with SQA practices. However, the research unearths concerns such as failure to comply with Software Development Life Cycle (SDLC) models as having the potential to lower the quality of software products. The assessment found that Unit testing was the most common type of software test. Based on the findings and literature, recommendations are made. The need to improve software engineering education and invest in software testing is underscored. The results can be generalised to most developing countries and used by software developers and trainers to identify areas in SQA that need strengthening.
Chapter
Software testing, which is a knowledge-intensive and collaborative activity, is a sub-area of software engineering. Software testing knowledge can be applied to different testing tasks and purposes. Since software development is an error-prone task, in order to achieve quality software products, validation and verification should be carried throughout the development. This study, using qualitative methods, investigates the current practice of software testing practices in two software companies on the basis that they both claimed to apply software testing practices in their software development work. Interview results revealed some interesting latest trends in software testing from both companies.
Article
Full-text available
Context: Embedded systems have overwhelming penetration around the world. Innovations are increasingly triggered by software embedded in automotive, transportation, medical-equipment, communication, energy, and many other types of systems. To test embedded software in an effective and efficient manner, a large number of test techniques, approaches, tools and frameworks have been proposed by both practitioners and researchers in the last several decades. Objective: However, reviewing and getting an overview of the entire state-of-the-art and the –practice in this area is challenging for a practitioner or a (new) researcher. Also unfortunately, as a result, we often see that many companies reinvent the wheel (by designing a test approach new to them, but existing in the domain) due to not having an adequate overview of what already exists in this area. Method: To address the above need, we conducted and report in this paper a systematic literature review (SLR) in the form of a systematic literature mapping (SLM) in this area. After compiling an initial pool of 588 papers, a systematic voting about inclusion/exclusion of the papers was conducted among the authors, and our final pool included 312 technical papers. Results: Among the various aspects that we aim at covering, our review covers the types of testing topics studied, types of testing activity, types of test artifacts generated (e.g., test inputs or test code), and the types of industries in which studies have focused on, e.g., automotive and home appliances. Furthermore, we assess the benefits of this review by asking several active test engineers in the Turkish embedded software industry to review its findings and provide feedbacks as to how this review has benefitted them. Conclusion: The results of this review paper have already benefitted several of our industry partners in choosing the right test techniques / approaches for their embedded software testing challenges. We believe that it will also be useful for the large world-wide community of software engineers and testers in the embedded software industry, by serving as an “index” to the vast body of knowledge in this important area. Our results will also benefit researchers in observing the latest trends in this area and for identifying the topics which need further investigations.
Conference Paper
Full-text available
The development of large software systems is a complex and error prone process. Errors may occur at any stage of software development. These errors, sometimes referred to as bugs, can cause great losses in terms of both time and money if not identified and removed as early as possible. Software testing plays a major role in software development life cycle (SDLC). It ceases the propagation of errors and thus decreases costs. But software testing itself can be costly. Research point out more than 50% of the total cost of development is devoted to software testing. However, testing is essential to ensure that the software is capable of doing what it was designed to do. Testing becomes expensive since it takes much time. This is because of the need to test many combinations of its functions, integrity, performance and so on, which can be called as test cases. A software company's goal would be to reduce the testing time, so that they can save money and deliver the product much faster to the customer. Testing time can be reduced in two main ways, first by reducing number of test cases and second by automating repeatedly tested areas. This paper will discuss fundamentals of testing such as importance and difference of verification and validation, recent advances in agile software testing, testing throughout the SDLC and testing methods, levels and types. Finally as a conclusion it will also shed some light on combinatorial testing to reduce number of test cases and test automation to reduce manual testing time as the solutions to long testing time problem.
Conference Paper
Full-text available
Despite the widespread proliferation of agile practices, implementation often fails due to lack of adequate project management support, decreased motivation or fear of increased interaction. Consequently, few organizations effectively adopt agile processes with tailoring often required to integrate agile methodology. This presentation provides an overview of an innovative large-scale tailored implementation of agile development termed the Agile Factory Model (AFM), with the aim of highlighting defect issues of specific importance to organizations undertaking large scale agile development. Drawing upon a substantial internal repository of defects, this study profiles defect severity, cause and lifecycle. Our results indicate that during development and testing the majority of defects which occur in the AFM are of severity 3, with requirements defects making up only 0.7%. Acceptance, Integration and Performance (AIP) testing captures the largest quantity of defects at 26.14% whilst a further 17.28% show up in Factory Acceptance Testing (FAT), where FAT and AIP are sequential stages in the AFM respectively. The results demonstrate that agile practices can be effectively translated to a globally distributed development environment.
Chapter
The objective of this industry study is to shed light on the current situation and improvement needs in software test automation. To this end, 55 industry specialists from 31 organizational units were interviewed. In parallel with the survey, a qualitative study was conducted in 12 selected software development organizations. The results indicated that the software testing processes usually follow systematic methods to a large degree, and have only little immediate or critical requirements for resources. Based on the results, the testing processes have approximately three fourths of the resources they need, and have access to a limited, but usually sufficient, group of testing tools. As for the test automation, the situation is not as straightforward: based on our study, the applicability of test automation is still limited and its adaptation to testing contains practical difficulties in usability. In this study, we analyze and discuss these limitations and difficulties.
Article
Context: Successful organizations need to manage and synchronize their strategic objectives with daily operations and activities. In general, achieving that requires a continuous process of organizational alignment. GQM⁺Strategies is an approach that helps software organizations with documenting and aligning organizational goals and strategies, and developing measurement programs. Objective: In this paper, the GQM⁺Strategies approach is further evolved and extended to include capabilities to evaluate the relationships of organizational goals and strategies through causal analysis. Method: We used an analytical paradigm to develop a formal causal model over the GQM⁺Strategies structure. In addition, an empirical pre-experimental study was designed to test practitioners’ abilities to provide necessary input for the formal causal model. Results: A developed formal causal model over the GQM⁺Strategies structure allows the use of causal reasoning for the purpose of analyzing dependencies among chosen sets of goals. We illustrate this by showing how to analyze the impact of risky goals on other goals in the grid. The results of the empirical study showed that the practitioners had no difficulties providing their predictions, i.e. inputs into the causal model. Conclusion: The proposed solution extends the existing GQM⁺Strategies knowledge base by further elaborating and clarifying the process of creating grids by introducing causality theory. The use of causality theory allows experts to quantify their knowledge and beliefs regarding the effectiveness of organizational strategies. As a part of future work, formal causal models and causal reasoning can be implemented as a supporting software tool for the GQM⁺Strategies approach.
Article
Frustration with the bureaucratic nature of the disciplined approach has led to the call for agile development. The new approach is defined by the Agile Manifesto (http://agilemanifesto.org/), which values individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, and agility in responding to change over following a prescribed plan. Agile development does not focus on process improvement; instead it focuses on customer satisfaction and employee empowerment. This is evident from reading the stated values and principles of the Agile Manifesto, which include fairly extreme positions such as "welcome changing requirements, even late in development" and "the best architectures, requirements, and designs emerge from self-organizing teams." An interesting issue arising from the call for agile development is its role in distributed development, which usually translates to offshore development. A recent study indicates that agile practices can reduce temporal, geographical, and socio-cultural distances in distributed development projects. The study researched agile development between teams located in the U.S. and Ireland, and while it reported that overall communication was improved, it also noted problems related to geographical, temporal, and even language distances. Although there are other reported successes of distributed agile development, the projects are generally small, the team members are likely familiar with each other, and the participants are largely experts or high caliber developers. This raises a research, as well as a practical, question: can we extend the use of agile practices from small projects to medium and large projects that involve a significant outsourcing component? To address this question, we must drop constraints such as small size projects, and expert developers belonging to the same company, and examine problems arising from geographical, temporal, and cultural distances. Accordingly, agile practices may need to be modified. In this article, the key issues of software projects with an outsourced component are first identified. These issues are then used as a background to evaluate how standard agile practices stand up when applied to larger projects. This evaluation is followed by recommendations for modified agile practices for outsourced software projects.
Conference Paper
Achieving the highest level of quality for the software product being delivered is the goal of any IT organization. Every organization wants to implement processes and practices that would help achieving this goal of increasing the quality of a software product. There are so many models available today that an organization can use for developing a software but considering the dynamics of today s world where technology is changing at a fast pace and innovative products are hitting the market at great speed, these organizations do not want a software development model that consume lot of time and efforts, and hence most of the business houses are moving towards Agile approach for software development. With this change in approach for software development, the big question is that how to ensure Quality of product developed using Agile model. The traditional approach has a separate phase for testing a software product which ensures that an independent team has validated the product per specified requirements. However, with Agile way of software development, this leverage of involvement of independent test teams and test levels has taken a back. This paper, will highlight broadly the role of QA within Agile development model, with focus on fresh thoughts and approaches to improve the overall quality of product developed using Agile methodology. The use and importance of Metrics for accessing the Quality within Agile model will also be discussed.