No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Crowd in Requirements Engineering: The Landscape and Challenges
... In the context of crowd requirements engineering (CrowdRE) [8], requirements researchers have started to investigate the potential benefits of performing feedback mining to gather new requirements. This does not only enable us to validate, verify, or identify requirements for a product, but such continuous mining also allows us to identify potential bugs in a product. ...
... On the other side, it is mentioned that the way feedback is requested, as well as the simplicity and usability of the feedback process, are important in motivating users to provide more feedback [16]. Moreover, it is noted that users' motivation is heavily dependent on them recognizing that the system is improved because of their feedback [8] [16] [17]. ...
... Using both initiation types within an app will allow reaching all app users. For example, pull initiation can be used to request feedback from "passive and stingy" app users who provide little to no feedback [8]. Push initiation, on the other hand, can allow "perfectionists and complainers" who are motivated to find improvement areas within apps and provide frequent feedback to submit their feedback directly from within the app [8]. ...
Mobile apps are becoming increasingly important in everyone’s daily life. The success of an app is linked to high user acceptance. Therefore, it is necessary to capture users’ expectations, needs, and problems regarding an app in any situation. For this reason, we conducted a systematic mapping study to obtain an overview of the current state of the art of in-app feedback collection approaches. We analyzed 36 publications and derived requirements for in-app feedback tools. We categorized the requirements into these groups: initiation of data collection, data collection enhancements, integrated communication regarding feedback provided, and transparency.
... Recent trends promote the direct involvement of stakeholders, most notably users. Some techniques promote the construction of a crowd of stakeholders that actively participate in the RE process (Groen et al., 2017;Snijders et al., 2015). Others focus on the topic of data-driven requirements engineering, the science of eliciting requirements from data provided by users of software systems (Maalej, Nayebi, Johann, & Ruhe, 2015), where the emphasis is on the data that a user produces and the way requirements analysts can benefit from it. ...
... Crowd participation is different from crowdsourcing requirements because it does not delegate micro-tasks to an anonymous crowd but rather engages user to share their opinions in order to deliver better software products to them. This approach is called CrowdRE (Groen et al., 2017) and the principal objective is to derive requirements for software from a large user base that might not be able to influence software development otherwise. This large base of users can be incentivized to participate through gamification, where they are invited to contribute and rate contributions of others. ...
... THE RELATIONSHIPS AMONG THE ASPECTS OF CROWD-BASED REQUIREMENTS ENGINEERING(GROEN ET AL., 2017) ...
... With the emergence of new data sources, this changes: Researchers have shown different approaches to extract requirements from feedback channels such as tweets or app store reviews [2], [3]. Another approach to gather a broad range of feedback is crowdsourcing [4]. In 2016, Murukannaiah et al. elicited 2,966 requirements for smart home applications from crowd workers [5]. ...
... Such forms of user feedback can be used to identify, prioritize, and manage requirements [6] for software products and to increase user satisfaction [7]. However, automatic techniques are necessary to derive useful insights the from large amounts of raw data the crowd can produce [6], [8], [4]. This becomes even more apparent as the decision-making process in requirements engineering shifts towards a more data-driven approach [9]. ...
... Acquiring requirements and requirements-related information from crowd workers facilitates a user-centered RE process and enables engineers to consider requirements form a broad and heterogeneous set of potential users [4]. However, crowdsourced information or information from other user feedback platforms are raw and unstructured. ...
... The consequent uncertainty during the stakeholder analysis process can negatively impact the quality of system requirements. To deal with this challenge, researchers have begun to develop techniques, known as crowd-based requirements processes (CrowdRE), that improve requirement quality by gathering feedback from crowds of stakeholders [20]. While CrowdRE has traditionally focused on eliciting stakeholder feedback via polls and surveys prior to the formation of requirements and marketing-driven mechanisms for prioritizing existing requirements, Glinz [19] highlights the direct solicitation of requirements from crowds of highly-engaged stakeholders as an opportunity for the field. ...
... Crowdsourcing, however, also has drawbacks. A high volume of requirements can overwhelm engineers and project managers [20]. Some experts [55] have also questioned the quality of crowdsourced requirements. ...
... In this case, OSS projects outsource requirements generation to crowds of developers. CrowdRE automates the analysis of stakeholder feedback [20]. This research leverages CrowdRE to study the effect of crowd participation on OSS projects and suggests when OSS project managers should consider employing CrowdRE techniques. ...
Crowdsourcing system requirements enables project managers to elicit feedback from a broader range of stakeholders. The advantages of crowdsourcing include a higher volume of requirements reflecting a more comprehensive array of use cases and a more engaged and committed user base. Researchers cite the inability of project teams to effectively manage an increasing volume of system requirements as a possible drawback. This paper analyzes a data set consisting of project management artifacts from 562 open-source software (OSS) projects to determine how OSS project performance varies as the share of crowdsourced requirements increases using six measures of effectiveness: requirement close-out time, requirement response time, average comment activity, the average number of requirements per crowd member, the average retention time for crowd members, and the total volume of requirements. Additionally, the models measure how the impact of increasing the share of crowdsourced requirements changes with stakeholder network structure. The analysis shows that stakeholder network structure impacts OSS performance outcomes and that the effect changes with the share of crowdsourced requirements. OSS projects with more concentrated stakeholder networks perform the best. The results indicate that requirements crowdsourcing faces diminishing marginal returns. OSS projects that crowdsource more than 70% of their requirements benefit more from implementing processes to organize and prioritize existing requirements than from incentivizing the crowd to generate additional requirements. Analysis in the paper also suggests that OSS projects could benefit from employing CrowdRE techniques and assigning dedicated community managers to more effectively channel input from the crowd.
... First, dynamic data-driven requirements engineering facilitates secondary use of data, which eliminates the need for collecting data specifically for requirements engineering, in turn enhancing scalability. Second, unintended digital sources can include data relevant for new system requirements that otherwise would not be discovered since utilizing such data sources allows for the collection of data from larger and global stakeholders who are beyond the reach of an organization relying on traditional elicitation methods [19]. Including such requirements, which a current software system is not supporting, can bring business values in the form of improved customer satisfaction, cost and time reduction, and optimized operations [20]. ...
... There is still a lack of methods to infer requirements as well as evidence regarding the applicability of the proposed approach to more diverse types of process-mediated and machine-generated data. Process-mediated and machinegenerated data enable run-time requirements elicitation [19]. They also help system developers to understand usage data and the corresponding context, which allows elicitation of performance-related as well as context-dependent requirements [19]. ...
... Process-mediated and machinegenerated data enable run-time requirements elicitation [19]. They also help system developers to understand usage data and the corresponding context, which allows elicitation of performance-related as well as context-dependent requirements [19]. In addition, almost all of the studies have focused on using only a single type of dynamic data and typically also a single data source. ...
Requirements engineering has traditionally been stakeholder-driven. In addition to domain knowledge, widespread digitalization has led to the generation of vast amounts of data (Big Data) from heterogeneous digital sources such as the Internet of Things (IoT), mobile devices, and social networks. The digital transformation has spawned new opportunities to consider such data as potentially valuable sources of requirements, although they are not intentionally created for requirements elicitation. A challenge to data-driven requirements engineering concerns the lack of methods to facilitate seamless and autonomous requirements elicitation from such dynamic and unintended digital sources. There are numerous challenges in processing the data effectively to be fully exploited in organizations. This article, thus, reviews the current state-of-the-art approaches to data-driven requirements elicitation from dynamic data sources and identifies research gaps. We obtained 1848 hits when searching six electronic databases. Through a two-level screening and a complementary forward and backward reference search, 68 papers were selected for final analysis. The results reveal that the existing automated requirements elicitation primarily focuses on utilizing human-sourced data, especially online reviews, as requirements sources, and supervised machine learning for data processing. The outcomes of automated requirements elicitation often result in mere identification and classification of requirements-related information or identification of features, without eliciting requirements in a ready-to-use form. This article highlights the need for developing methods to leverage process-mediated and machine-generated data for requirements elicitation and addressing the issues related to variety, velocity, and volume of Big Data for the efficient and effective software development and evolution.
... Regarding RQ1, we take the perspective of CrowdRE researchers. We describe the current state and features of the ORKG as a crowdsourcing platform that involves its crowd members in the development along the four key activities of CrowdRE (motivating crowd members, eliciting feedback, analyzing feedback, monitoring context & usage data) [16]. This overview shows to what extent and how the ORKG already addresses CrowdRE, and highlights improvement potential. ...
... The digital transformation has compelled the domain of requirements engineering (RE) to respond in various ways, including placing greater emphasis on automated RE [24]. A particular kind of data-driven RE [25], [26] relies on the involvement of a crowd as a source of data, and is typically referred to as CrowdRE [16]. CrowdRE describes an iterative cycle of eliciting feedback from the crowd and monitoring context and usage data to derive the needs and requirements of the crowd, which, once validated, are implemented into the product [16]. ...
... A particular kind of data-driven RE [25], [26] relies on the involvement of a crowd as a source of data, and is typically referred to as CrowdRE [16]. CrowdRE describes an iterative cycle of eliciting feedback from the crowd and monitoring context and usage data to derive the needs and requirements of the crowd, which, once validated, are implemented into the product [16]. Current CrowdRE research focuses on the use of various existing channels with a general purpose, aimed at accessing the crowd, such as social networks and mobile application marketplaces [18]. ...
In recent decades, there has been a major shift towards improved digital access to scholarly works. However, even now that these works are available in digital form, they remain document-based, making it difficult to communicate the knowledge they contain. The next logical step is to extend these works with more flexible, fine-grained, semantic, and context-sensitive representations of scholarly knowledge. The Open Research Knowledge Graph (ORKG) is a platform that structures and interlinks scholarly knowledge, relying on crowdsourced contributions from researchers (as a crowd) to acquire, curate, publish, and process this knowledge. In this experience report, we consider the ORKG in the context of Crowd-based Requirements Engineering (CrowdRE) from two perspectives: (1) As CrowdRE researchers, we investigate how the ORKG practically applies CrowdRE techniques to involve scholars in its development to make it align better with their academic work. We determined that the ORKG readily provides social and financial incentives, feedback elicitation channels, and support for context and usage monitoring, but that there is improvement potential regarding automated user feedback analyses and a holistic CrowdRE approach. (2) As crowd members, we explore how the ORKG can be used to communicate scholarly knowledge about CrowdRE research. For this purpose, we curated qualitative and quantitative scholarly knowledge in the ORKG based on papers contained in two previously published systematic literature reviews (SLRs) on CrowdRE. This knowledge can be explored and compared interactively, and with more data than what the SLRs originally contained. Therefore, the ORKG improves access and communication of the scholarly knowledge about CrowdRE research. For both perspectives, we found the ORKG to be a useful multi-tool for CrowdRE research.
... In the pre-development stage, customer feedback is commonly captured through traditional methods involving interviews and questionnaires or via prototyping and A/B testing, and in post-development through reviews, usage data and support tickets (Fabijan et al. 2015). To further engage users, crowd-based approaches (Groen et al. 2017) introduce automated ways of deriving requirements through collecting and analyzing user feedback from large user groups on various channels such as app stores, forums or social media. Customer feedback serves as an input to plan further incremental releases where an additional set of requirements is implemented. ...
... In the predevelopment stage, customer feedback is commonly captured through traditional methods involving interviews and questionnaires or via prototyping and A/B testing, and in postdevelopment through reviews, usage data and support tickets (Fabijan et al. 2015). To further engage users, crowd-based approaches (Groen et al. 2017) introduce automated ways of deriving 143 requirements through collecting and analyzing user feedback from large user groups on various channels such as app stores, forums or social media. ...
With cloud and mobile computing, a new category of software products emerges as mass-market information systems (IS) that addresses distributed and heterogeneous end-users. Understanding user requirements and the factors that drive user adoption are crucial for successful design of such systems. IS research has suggested several theories and models to explain user adoption and intentions to use, among them the IS Success Model and the Technology Acceptance Model (TAM). Although these approaches contribute to theoretical understanding of the adoption and use of IS in mass-markets, they are criticized for not being able to drive actionable insights on IS design as they consider the IT artifact as a black-box (i.e., they do not sufficiently address the system internal characteristics). We argue that IS needs to embrace market research techniques to understand and empirically assess user preferences and perceptions in order to integrate the "voice of the customer" in a mass-market scenario. More specifically, conjoint analysis (CA), from market research, can add user preference measurements for designing high-utility IS. CA has gained popularity in IS research, however little guidance is provided for its application in the domain. We aim at supporting the design of mass-market IS by establishing a reliable understanding of consumer’s preferences for multiple factors combing functional, non-functional and economic aspects. The results include a “Framework for Conjoint Analysis Studies in IS” and methodological guidance for applying CA. We apply our findings to the privacy-aware design of mass-market IS and evaluate their implications on user adoption. We contribute to both academia and practice. For academia, we contribute to a more nuanced conceptualization of the IT artifact (i.e., system) through a feature-oriented lens and a preference-based approach. We provide methodological guidelines that support researchers in studying user perceptions and preferences for design variations and extending that to adoption. Moreover, the empirical studies for privacyaware design contribute to a better understanding of the domain specific applications of CA for IS design and evaluation with a nuanced assessment of user preferences for privacy-preserving features. For practice, we propose guidelines for integrating the voice of the customer for successful IS design.
... Involvement of the users early in the project is essential to ensure the software success [5]. To ensure collection of diverse perspectives about the software (in terms of requirements representing user needs), the crowd of users are involved during requirements engineering using crowdsourcing platforms like social media platforms (also termed as crowd based requirements engineering) [6]. The rationale is that crowd involvement helps the requirement analyst to get access to diverse user needs and improve accuracy of requirement ranking by lowering the impacts of "bad" inputs from few users, which are marginal amongst a large population of crowd (if any). ...
... The main feature of crowds is that they don't know each other as crowdsourcing is an open call for participation [6][7]. Even though, if some users invite other users to participate in requirements engineering (recommendation) then bias is automatically minimized as such recommendations will be minimal with respect to the crowd. ...
The author comment on the paper “A Social Network Based Process to Minimize In-Group Biasedness During Requirement Engineering” by Mughal et al., and point out that the technique can be subsequently improved by focusing on three major areas (a) differential treatment of crowd of users (b) Ranking of requirements (i.e. determining the importance of the requirements for different stakeholders) based on analysis of social interactions among stakeholders on Facebook and (c) extension as well as improvement in result validations by improving effectiveness calculation and including scalability as one of the efficiency metrics. Scalability will influence both effectiveness as well as efficiency of the proposed technique.
... With the recent widespread availability of NL content relevant to RE, such as feedback from users in app stores and social media, and developers' comments in discussion forums and bug tracking systems, we have observed a rising interest in using ML techniques to support data-driven RE [38] and crowd-based RE [39]. These areas aim to leverage information available from stakeholders' implicit and explicit feedback, including diverse sources as app reviews [40], issue tracking systems [41], Twitter [42] or user fora [43], to improve RE activities such as requirements elicitation and prioritization. ...
... 38 Stemming A crude heuristic process that chops off the ends of words in the hope of achieving this goal correctly most of the time, and often includes the removal of derivational affixes. 39 Lemmatization Use a vocabulary and morphological analysis of words, normally aiming to remove inflectional endings only and to return the base or dictionary form of a word, which is known as the lemma. 40 Stop-Word Removal Words which are filtered out before or after processing of natural language data (text). ...
Research in applying natural language processing (NLP) techniques to requirements engineering (RE) tasks spans more than 40 years, from initial efforts carried out in the 1980s to more recent attempts with machine learning (ML) and deep learning (DL) techniques. However, in spite of the progress, our recent survey shows that there is still a lack of systematic understanding and organization of commonly used NLP techniques in RE. We believe one hurdle facing the industry is lack of shared knowledge of NLP techniques and their usage in RE tasks. In this paper, we present our effort to synthesize and organize 57 most frequently used NLP techniques in RE. We classify these NLP techniques in two ways: first, by their NLP tasks in typical pipelines and second, by their linguist analysis levels. We believe these two ways of classification are complementary, contributing to a better understanding of the NLP techniques in RE and such understanding is crucial to the development of better NLP tools for RE.
... The literature reviews conducted by Dieste and Juristo [9] and Ambreen et al. [3] also report on empirical studies (primarily case studies and experiments) that assess the effectiveness of some of these elicitation techniques and models. Other recent work has taken into consideration a number of current challenges and recent opportunities in this area, even addressing crowd-based RE [11] and datadriven RE [20][21]. ...
... We distinguished 3 broad categories of role in the context of a holistic perspective of each organization. This approach is adopted either because some subjects did not know all of the details of the elicitation process, especially Internal Strategic [8] Technical [15] Organization (not specified) [8] Developer / Development team / Scrum team [4] Analyst (Business analyst) [7] Customer Proxy (invented customer) [1] Market unit [4] Generic (Technicians/Technical team) [4] Technical/System manager [2] Consultant (internal) [3] Software architect [1] System / Lead engineers [2] Requirements-related [11] Project manager [1] Business team [1] Interaction designers [1] Web editors [1] Function owner [1] Product owner [1] Owner [2] External Customer (requester) [15] Third-party [3] End-user [4] Real [2] Citizen (potential user) [2] Service providers (carrier) [2] Customers of customers [1] Consultant (specialist) [2] Fig. 2 Roles that participate in requirements elicitation: A classification 4 Note that both interview subjects work for the same company, a public transport administration. in big companies (e.g., S7(D) who indicated that: "The business requirements are gathered from the customers by another part of the organization", or because no clear roles had been defined for conducting the elicitation process. ...
Context. Requirements engineering remains a discipline that is faced with a large number of challenges, including the implementation of a requirements elicitation process in industry. Although several proposals have been suggested by researchers and academics, little is known of the practices that are actually followed in industry. Objective. We investigate the SoTA with respect to requirements elicitation, examining practitioners' practices. We focus on the techniques, the roles involved, and the challenges associated to the process. Method. We conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies. Results. We found that group interaction techniques, including meetings and workshops, are the most popular type of elicitation techniques that are employed by the practitioners, except in the case of small projects. We noted that customers are frequently involved in the elicitation process, except in the case of market-driven organizations. Technical staff (for example, developers and architects) are more frequently involved in the elicitation process compared to the involvement of business- or strategic staff. Finally, we identified a number of challenges with respect to stakeholders. These challenges include difficulties in understanding and prioritizing their needs. Further, it was noted that requirements instability (i.e., caused by changing needs or priorities) was a predominant challenge. These observations need to be interpreted in the context of the study. Conclusion. The relevant observations regarding the survey participants' experiences should be of interest to the industry; experiences that should be analyzed in the practitioners' context. Researchers may find evidence for the use of academic results in practice, thereby inspiring future theoretical work, as well as further empirical studies in the same area.
... The literature reviews conducted by Dieste and Juristo [9] and Ambreen et al. [3] also report on empirical studies (primarily case studies and experiments) that assess the effectiveness of some of these elicitation techniques and models. Other recent work has taken into consideration a number of current challenges and recent opportunities in this area, even addressing crowd-based RE [11] and datadriven RE [20][21]. ...
... We distinguished 3 broad categories of role in the context of a holistic perspective of each organization. This approach is adopted either because some subjects did not know all of the details of the elicitation process, especially Internal Strategic [8] Technical [15] Organization (not specified) [8] Developer / Development team / Scrum team [4] Analyst (Business analyst) [7] Customer Proxy (invented customer) [1] Market unit [4] Generic (Technicians/Technical team) [4] Technical/System manager [2] Consultant (internal) [3] Software architect [1] System / Lead engineers [2] Requirements-related [11] Project manager [1] Business team [1] Interaction designers [1] Web editors [1] Function owner [1] Product owner [1] Owner [2] External Customer (requester) [15] Third-party [3] End-user [4] Real [2] Citizen (potential user) [2] Service providers (carrier) [2] Customers of customers [1] Consultant (specialist) [2] Fig. 2 Roles that participate in requirements elicitation: A classification in big companies (e.g., S7(D) who indicated that: "The business requirements are gathered from the customers by another part of the organization", or because no clear roles had been defined for conducting the elicitation process. Consider, for example, S4(B)'s observation: "The workshops involve different people in the organization". ...
Requirements engineering remains a discipline that is faced with a large number of challenges, including the implementation of a requirements elicitation process in industry. Although several proposals have been suggested by researchers and academics, little is known of the practices that are actually followed in industry. Our objective is to investigate the state-of-practice with respect to requirements elicitation, by closely examining practitioners’ current practices. To this aim, we focus on the techniques that are used in industry, the roles that requirements elicitation involves, and the challenges that the requirements elicitation process is faced with. As method, we conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies, and we recorded the interviews and analyzed these recordings by using quantitative and qualitative methods. Several results emerged from the studies. Group interaction techniques, including meetings and workshops, are the most popular type of elicitation techniques that are employed by the practitioners, except in the case of small projects. Additionally, practitioners tend to use a variety of elicitation techniques in each project. We noted that customers are frequently involved in the elicitation process, except in the case of market-driven organizations. Technical staff (for example, developers and architects) are more frequently involved in the elicitation process compared to the involvement of business or strategic staff. Finally, we identified a number of challenges with respect to stakeholders. These challenges include difficulties in understanding and prioritizing their needs. Further, it was noted that requirements instability (i.e., caused by changing needs or priorities) was a predominant challenge. These observations need to be interpreted in the context of the study. We conclude that the relevant observations regarding the survey participants’ experiences should be of interest to the industry; experiences that should be analyzed in the practitioners’ context. Researchers may find evidence for the use of academic results in practice, thereby inspiring future theoretical work, as well as further empirical studies in the same area.
... User feedback can take implicit and explicit forms, and different means are available to collect this information. In particular, data analytics applied to user opinions and to usage data has seen an increasing interest in the last years, leading to the birth of RE sub-fields such as crowd RE (Murukannaiah et al., 2016;Groen et al., 2017) and data-driven RE Williams and Mahmoud, 2017). In the case of bespoken development (i.e., when customer-or domain-specific products' requirements need to be engineered), it is still common to resort to traditional RE practices, such as prototyping, observations, usability testing, and focus groups (Zowghi and Coulin, 2005). ...
Capturing users engagement is crucial for gathering feedback about the features of a software product. In a market-driven context, current approaches to collect and analyze users feedback are based on techniques leveraging information extracted from product reviews and social media. These approaches are hardly applicable in bespoke software development, or in contexts in which one needs to gather information from specific users. In such cases, companies need to resort to face-to-face interviews to get feedback on their products. In this paper, we propose to utilize biometric data, in terms of physiological and voice features, to complement interviews with information about the engagement of the user on the discussed product-relevant topics. We evaluate our approach by interviewing users while gathering their physiological data (i.e., biofeedback) using an Empatica E4 wristband, and capturing their voice through the default audio-recorder of a common laptop. Our results show that we can predict users' engagement by training supervised machine learning algorithms on biometric data, and that voice features alone can be sufficiently effective. The performance of the prediction algorithms is maximised when pre-processing the training data with the synthetic minority oversampling technique (SMOTE). The results of our work suggest that biofeedback and voice analysis can be used to facilitate prioritization of requirements oriented to product improvement, and to steer the interview based on users' engagement. Furthermore, the usage of voice features can be particularly helpful for emotion-aware requirements elicitation in remote communication, either performed by human analysts or voice-based chatbots.
... With the help of Crow-dRE, requirements engineers are no longer limited to performing only traditional requirements engineering to create the requirements for a productthey are now also supported in paying attention to the information provided by its potential user base. In contrast to giving tasks to the crowd, CrowdRE gives the crowd its own voice [7]. ...
Mobile apps have penetrated the market and are being used everywhere. Companies developing apps face increasing challenges such as short time to market or demand for high quality. Furthermore, the success of an app also depends on how users perceive its quality. Feedback provided by users influences other potential users and provides new opportunities for identifying features. Consequently, it is a valuable source of input for app developers with respect to product improvements. One form is textual feedback. This kind of feedback is usually distributed across various data sources. Therefore, it must be captured from these sources and put into one single pool of data before it can be analyzed. The analysis must take into account the peculiarities of the short release cycles and high change rate of features for mobile apps. In this article, we present User Echo Service (UES), which was built to address heterogeneous data sources. The aim of UES is to allow product managers to be able to be always up to date with the latest feedback data. Therefore, we have created an ex-tensible architecture aimed at supporting different data sources and present the feedback collection scheduling system. This forms the prerequisite for subsequent analyses of the collected data. We discuss our solution and provide ideas for future development.
... Manually eliciting software requirements from online feedback can be extremely time intensive due to the large volumes and varying quality of text language that comes from highly distributed user-bases [11]. Requirement elicitation can be further complicated when it is part of a software ecosystem, a growing trend in the software landscape [12]. ...
Many software users give feedback online about the applications they use. This feedback often contains valuable requirements information that can be used to guide the effective maintenance and evolution of a software product. Yet, not all software users give online feedback. If the demographics of a user-base aren’t fairly represented, there is a danger that the needs of less vocal users won’t be considered in development. This work investigates feedback on three prominent online channels: app stores, product forums, and social media. We directly survey software users about their feedback habits, as well as what motivates and dissuades them from providing feedback online. In an initial survey of 1040 software users, we identify statistically significant differences in the demographics of users who give feedback (gender, age, etc.), and key differences in what motivates them to engage with each of the three studied channels. In a second survey of 936 software users, we identify the top reasons users don’t give feedback, including significant differences between demographic groups. We also present a detailed list of user-rated methods to encourage their feedback. This work provides meaningful context for requirements sourced from online feedback, identifying demographic groups who are underrepresented. Findings on what motivates and discourages user feedback give insight into how feedback channels and developers can increase engagement with their user-base.
... Once the startup scales up in the market (with growing market share), the face to face interaction becomes difficult. Thus, in later stages of the startup, the product is evolved by getting access to the customer feedback through a variety of implicit and explicit feedback channels (for instance, crowdsourcing platforms, social media, Electronic emails, App store reviews and telephone calls etc.) [7]. ...
Pandemic situations impact the ability of the startups to identify the product features that have match with market needs; the activity that requires direct interaction with the customers at the same physical space. Online tools can overcome this limitation, but early stage startups have too limited resources and lack of access to the potential customers, which make their online interactions quite limited. The divergent creativity is required to identify the Requirement Elicitation methods and tools that could help startups to identify product/market fit with limited same physical space interaction with customers. The open innovation involving academia, experts and researchers could help startups to get access to the market needs. This paper reports one such consulting experience of the author with the Madrid (Spain) based startup which successfully identified its market in pandemic time through market research driven by secondary studies, primary research involving potential clients (or users) through online means and limited interactions at the same physical space. Daily brainstorming with a team of researchers, experts and professors helped to generate divergent ideas about identifying markets amid pandemic and testing them in real context that proved to be successful for the startup.
... With the help of CrowdRE, requirements engineers are no longer limited to performing only traditional requirements engineering to create the requirements for a product -they are now also supported in paying attention to the information provided by its potential user base. In contrast to giving tasks to the crowd, CrowdRE gives the crowd its own voice [7]. ...
Mobile apps have penetrated the market and are being used everywhere. Companies developing apps face increasing challenges such as short time to market or demand for high quality. Furthermore, the success of an app also depends on how users perceive its quality. Feedback provided by users influences other potential users and provides new opportunities for identifying features. Consequently, it is a valuable source of input for app developers with respect to product improvements. One form is textual feedback. This kind of feedback is usually distributed across various data sources. Therefore, it must be captured from these sources and put into one single pool of data before it can be analyzed. The analysis must take into account the peculiarities of the short release cycles and high change rate of features for mobile apps. In this article, we present User Echo Service (UES), which was built to address heterogeneous data sources. The aim of UES is to allow product managers to be able to be always up to date with the latest feedback data. Therefore, we have created an extensible architecture aimed at supporting different data sources and present the feedback collection scheduling system. This forms the prerequisite for subsequent analyses of the collected data. We discuss our solution and provide ideas for future development.
... Consequently, new paradigms such as agile requirements engineering (agileRE) and crowd requirements engineering (crowdRE) have emerged. These paradigms involve multiple stakeholders, advocate face-toface communication, require minimal documentation, and favor iterative requirements evolution [13,17,31]. However, such paradigms also bring several new challenges. ...
... The large number of stakeholders makes the traditional activities of requirements gathering and analyzing extremely costly and timeconsuming, and thus, these approaches miss the opportunity to continuously involve large groups of users who express their feedback or feature requests through a variety of media. The new trend is now shifting toward the CrowdRE, which focuses on automation or semi-automation of the requirements gathering process so that validated user requirements can be derived from a crowd [22]. ...
Mailing list is widely used as an important channel for communications between developers and stakeholders. It consists of emails that are posted for various purposes, such as reporting problems, seeking help in usage, managing projects, and discussing new features. Due to the intensive amount of new incoming emails every day, some valuable emails that intend to describe new features may get overlooked by developers. However, identifying these feature requests from development emails is a labor-intensive and challenging task. In this paper, we propose an automated solution to discover feature requests from development emails by leveraging semantic sequence patterns. First, we tag sentences in emails by using 81 fuzzy rules proposed in our previous study. Then we represent the semantic sequence with the contextual information of an email in a 2-g model. After applying sequence pattern mining techniques, we generate 10 semantic sequence patterns from 317 tagged emails that are randomly sampled from the Ubuntu community. We also conduct an empirical evaluation of their capability to discover feature requests from massive emails in Ubuntu and other four open source communities. The results show that our approach can effectively identify feature requests from these emails. Compared to existing baselines, our approach can achieve a better performance in terms of precision, recall, F1-score, AUC, and positive, with the average precision and recall for discovering feature requests from emails being 76% and 86%, respectively.
... In 2015, the term Crowd-Based Requirements Engineering, also known as CrowdRE, was coined [10]. After that, in [9], E.C. Groen et al. defined CrowdRE as an umbrella term for automated or semiautomated RE approach for gathering and analyzing information from a crowd to derive validated user requirements. ...
... Users proactively use social media, forums, and marketplaces to provide feedback on their software usage experience. On the other hand, developers need to extract information from a crowd of users to make strategic decisions about software evolution [10]. Popular software receives hundreds of thousands or even millions of reviews, and manual analysis of this large volume of information becomes an impractical task [11,12]. ...
... 1 The product value innovation is a tool to sustain competitive advantage; the task for which the involvement of customers is of utmost importance throughout the product lifecycle to get continuous product related feedback. 2,3 The software team deployed the computationally advanced infrastructures (for instance, crowd-based RE), [4][5][6][7] for feedback management, i.e., acquisition and its analysis, to drive future software market releases as the product gained market share. The need for such a complex feedback management infrastructure in the context of early-stage startups (or startups hereafter), which are in their initial business operations seeking early adopters with too limited resources, is bare minimum. ...
The microenterprises called Startups perform series of interactions with potential customers using prototypes to validate the business model related assumptions, especially value proposition. Value proposition refers to the value the product is going to offer to the customers by providing them benefits and addressing their difficulties in getting their tasks done. The importance of value proposition innovation is evident from the fact that most of the software Startups fail due to their inability to reach product/market fit, i.e., building the product for which there is no market demand; attributable to requirement engineering related issues.1 The product value innovation is a tool to sustain competitive advantage; the task for which the involvement of customers is of utmost importance throughout the product lifecycle to get continuous product related feedback.2,3
... To further engage users, crowd-based approaches [26] introduce automated ways of deriving requirements through collecting and analyzing user feedback from large user groups on various channels such as app stores, forums or social media. ...
With mobile and cloud computing, the software industry has witnessed a shift from customer-specific systems to a mass-market scenario. Integrating the "voice of the customer" is a critical success factor for mass-market products, but remains a challenge given the heterogeneity and distribution of their users. Classical requirements engineering methods rely on close interactions to gather user requirements and feedback regarding product features. However, they are hardly applicable in mass market scenarios, and new methods are required to assist product managers in designing their products while taking into consideration the opinion of the mass. In this paper, we argue for using market research techniques as preference-based approach in software product management. Following method engineering guidelines, we develop a method component ¹ that refines conjoint analysis for the use in software product management. We present the meta-model and procedure and demonstrate it in a study on secure cloud storage services. We evaluate the feasibility, usefulness and ease of use of method component with experts in the RE domain. We contribute to mass-market software product management domain through a method component that builds on detailed understanding of user preferences for successful product design.
... The success of software can be seen from the percentage of achievement of the desired goal. One way to ensure the achievement of the desired goal is with Requirements Engineering (RE) [4]. RE is done by identifying the stakeholders involved and their needs, documenting in a form that can be analyzed, negotiated, and then implemented. ...
... As such, NFRs were largely left to the developers to self-manage in an ad-hoc manner. Despite the numerous frameworks and models developed through research [72], [73], [74], [75], there exists a gap between industry and practice on whether they can actually be used to solve this issue of NFR prioritization. This is a significant empirical finding adding to the scarce evidence on how (the lack of) NFR prioritization is handled in industrial versus research settings. ...
Non-functional requirements (NFR), which include performance, availability, and maintainability, are vitally important to overall software quality. However, research has shown NFRs are, in practice, poorly defined and difficult to verify. Continuous software engineering practices, which extend agile practices, emphasize fast paced, automated, and rapid release of software that poses additional challenges to handling NFRs. In this multi-case study we empirically investigated how three organizations, for which NFRs are paramount to their business survival, manage NFRs in their continuous practices. We describe four practices these companies use to manage NFRs, such as offloading NFRs to cloud providers or the use of metrics and continuous monitoring, both of which enable almost real-time feedback on managing the NFRs. However, managing NFRs comes at a cost as we also identified a number of challenges these organizations face while managing NFRs in their continuous software engineering practices. For example, the organizations in our study were able to realize an NFR by strategically and heavily investing in configuration management and infrastructure as code, in order to offload the responsibility of NFRs; however, this offloading implied potential loss of control. Our discussion and key research implications show the opportunities, trade-offs, and importance of the unique give-and-take relationship between continuous software engineering and NFRs. Research artifacts may be found at https://doi.org/10.5281/zenodo.3376342.
A large volume of highly diverse and real-time data provided by social network sites
has revolutionized several data analytical sciences such as stock exchange fluctuation
prediction, sentiment analysis, and political affiliations prediction. From a software
engineering viewpoint, social network sites have produced an unprecedented opportunity
for software development organizations to monitor the opinion of the large
population of users of their systems. The extracted data from social network sites
can be used to extract information about newly launched systems, and it enable software
development organizations to obtain rapid technical and social feedback about
their systems. This paper presents on a systematic literature review of the published
research articles on how requirements engineering (RE) activities are conducted using
social network sites, elaborated in studies published between 2011 and 2019. From
the resulting studies, we extracted data to answer our four questions related to social
network site (SNS)-based RE approaches focusing on requirements elicitation, prioritization,
and negotiation activities. Thirty-one relevant studies were identified and
analyzed to answer our research questions. Among 31 studies, 62% of the studies
reportedly support requirements elicitation activity, and 14% of the studies reportedly
support requirements elicitation, prioritization, and negotiation activities. Ten
percent studies were about requirements elicitation and prioritization activities, 7%
of the studies support both requirements elicitation and negotiation activities, and
prioritization activity was supported by 7% of the studies. It was also observed that
58% of the selected studies adopted semi-automatic approaches to conduct RE
activities while 21% of the selected studies proposed automatic tools, and 21% of
the selected studies adopted manual approaches to conduct RE activities. The outcomes
of this systematic literature review indicate that social network sites can be a
major source that can be used successfully to extract and identify user requirements.
Moreover, this systematic literature review also presents implications of this study
for practitioners and researchers including challenges, unresolved issues, and future
research directions.
Data-driven approaches are becoming dominant in almost every single software engineering activity, and requirements engineering is not the exception. The analysis of data coming from several sources may indeed become an extremely useful input to requirements elicitation and management. However, benefits do not come for free. Techniques such as natural language processing and machine learning are difficult to master and require high-quality data and specific competences from different fields, whilst their generalization remains as a challenge. This paper introduces the main concepts behind data-driven requirements engineering, provides an overview of the state of the art in the field and identifies the main challenges to be addressed.
Batch processing reduces processing time in a business process at the expense of increasing waiting time. If this trade-off between processing and waiting time is not analyzed, batch processing can, over time, evolve into a source of waste in a business process. Therefore, it is valuable to analyze batch processing activities to identify waiting time wastes. Identifying and analyzing such wastes present the analyst with improvement opportunities that, if addressed, can improve the cycle time efficiency (CTE) of a business process. In this paper, we propose an approach that, given a process execution event log, (1) identifies batch processing activities, (2) analyzes their inefficiencies caused by different types of waiting times to provide analysts with information on how to improve batch processing activities. More specifically, we conceptualize different waiting times caused by batch processing patterns and identify improvement opportunities based on the impact of each waiting time type on the CTE. Finally, we demonstrate the applicability of our approach to a real-life event log.
[Context and motivation] According to Data-Driven Requirements Engineering (RE), explicit and implicit user feedback can be considered a relevant source of requirements, thus supporting requirements elicitation. [Question/problem] Less attention has been paid so far to the role of implicit feedback in RE tasks, such as requirements validation, and on how to specify what implicit feedback to collect and analyse. [Principal idea/results] We propose an approach that leverages on goal-oriented requirements modelling combined with Goal-Question-Metric. We explore the applicability of the approach on an industrial project in which a platform for online training has been adapted to realise a citizen information service that has been used by hundreds of people during the COVID-19 pandemic. [Contributions] Our contribution is twofold: (i) we present our approach towards a systematic definition of requirements for data collection and analysis, at support of software requirements validation and evolution; (ii) we discuss our ideas using concrete examples from an industrial case study and formulate a research question that will be addressed by conducting experiments as part of our research.
Requirements engineering (RE) is gaining acceptance by industries and practitioners as a significant process in systems development. Similarly, the publications on the research related to the RE topic progresses. However, there is a lack of study on the progressions of this research based on a bibliographic portfolio. This paper presents the emergence of the RE field and mapping the state of the art of scientific production using a bibliometric survey approach. This study identifies the research trends based on the observations of scientific production over the past two decades. Quantitative and qualitative information in requirements engineering research through various bibliographic attributes are presented, such as the leading journals, top-cited articles, top authors, and top countries. Research trends in RE can be observed based on the number of research themes, and the involvement of various disciplines from RE begin to get acknowledged until this study was conducted. The results provide evidence and an overview of future directions in this research field.
Diversity is a great challenge for software engineers in the social sector context. The objective of this paper is to contribute to the identification of the RE processes and associated challenges in releasing the software in the social sector markets for which an exploratory case study is conducted. The outcome of the case study indicates that the diversity limits the ability to involve the representative samples of user populations using the same set of RE tools and techniques as one size fits all solution for all segments. The diverse user base must be partitioned into different segments, with each segment triggered using a suitable set of RE techniques i.e., traditional and crowd-based RE. The diverse perspectives learned as a result of the interaction with each segment, must be merged together into a single perspective about the software meant to be used in the social sector. There is a need for a new RE process specially designed for handling the complexities of the social sector, which this paper terms as Social Sector Requirement Engineering (SSRE). There is an increased need for collaboration between government social sector institutions and software engineers to get access to diverse customers to improve software quality.
Many applications need to meet diverse requirements of a large-scale distributed user group. That challenges the current requirements engineering techniques. Crowd-based requirements engineering was proposed as an umbrella term for dealing with the requirements development in the context of the large-scale user group. However, there are still many issues. Among others, a key issue is how to merge these requirements to produce the synthesized requirements description when a set of requirements descriptions from different participants are received. Appropriate techniques are needed for supporting the requirements synthesis. Diagrams are widely used in industry to represent requirements. This paper chooses the activity diagrams and proposes a novel approach for the activity diagram synthesis which adopts the genetic algorithm to repeatedly modify a population of individual solutions toward an optimal solution. As a result, it can automatically generate a resulting diagram which combines the commonalities as many as possible while leveraging the variabilities of a set of input diagrams. The approach is featured by: 1) the labelled graph proposed as the representation of the candidate solutions during the iterative evolution; 2) the generalized entropy proposed and defined as the measurement of the solutions; 3) the genetic algorithm designed for sorting out the high-quality solution. Four cases of different scales are used to evaluate the effectiveness of the approach. The experimental results show that not only the approach gets high precision and recall but also the resulting diagram satisfies the properties of minimization and information preservation and can support the requirements traceability.
Context
Feedback from end users of software applications is a valuable resource in understanding what users request, what they value, and what they dislike. Information derived from user-feedback can support software evolution activities, such as requirements prioritization. User-feedback analysis is still mostly performed manually by practitioners, despite growing research in automated analysis.
Objective
We address two issues in automated user-feedback analysis: (i) most of the existing automated analysis approaches that exploit linguistic analysis assume that the vocabulary adopted by users (when expressing feedback) and developers (when formulating requirements) are the same; and (ii) user-feedback analysis techniques are usually experimentally evaluated only on some user-feedback dataset, not involving assessment by potential software developers.
Method
We propose an approach, ReFeed, that computes, for each requirement, the set of related user-feedback, and from such user-feedback extracts quantifiable properties which are relevant for prioritizing the requirement. The extracted properties are propagated to the related requirements, based on which ranks are computed for each requirement. ReFeed relies on domain knowledge, in the form of an ontology, helping mitigate the gap in the vocabulary of end users and developers. The effectiveness of ReFeed is evaluated on a realistic requirements prioritization scenario in two experiments involving graduate students from two different universities.
Results
ReFeed is able to synthesize reasonable priorities for a given set of requirements based on properties derived from user-feedback. The implementation of ReFeed and related resources are publicly available.
Conclusion
The results from our studies are encouraging in that using only three properties of user-feedback, ReFeed is able to prioritize requirements with reasonable accuracy. Such automatically determined prioritization could serve as a good starting point for requirements experts involved in the task of prioritizing requirements Future studies could explore additional user-feedback properties to improve the effectiveness of computed priorities.
Requirements Engineering (RE) is a co-creation process that requires continuous involvement of the users. Diversity in the user’s segments helps to provide holistic view of the expected solution from multiple perspectives to the software engineers but this diversity is a great challenge for software engineers in social sector context. The objective of this paper is to contribute to the identification of the RE processes and associated challenges in releasing the software in the social sector markets. To meet this objective, an exploratory case study is conducted with two software companies to identify the RE processes and the challenges in executing such processes with the diverse user segments. The outcome of the case study indicates that the diversity limits the ability to involve the representative samples of user populations using the same set of RE tools and techniques as one size fits all solution for all segments. The diverse user base must be partitioned into different segments, with each segment triggered using a suitable set of RE techniques, i.e. traditional and crowd based RE. The diverse perspectives learned as a result of the interaction with each segment, must be merged together into a single perspective about the software meant to be used in the social sector. There is a need for new RE process specially designed for handling the complexities of social sector, which this paper terms as Social Sector Requirements Engineering (SSRE).
The research chapter proposes a crowdsourcing and gamification based Requirements Engineering (RE) process that help software engineers to identify the features for the social sector applications and rank them. Requirements engineering activities are conducted by converging the diverse perspectives of the heterogenous crowds (i.e., citizens), which are actual users of the social sector software application. The algorithm elicits the customer journeys from the users (i.e., the citizens), which are then subjected to further elaboration and prioritisation by initiating the social interactions between the users. The proposed solution is evaluated using the real case of the social sector application which was executed for social welfare by one of the leading Indian NGO. The NGO activists were involved to provide software features along with their perspectives about their priorities which were compared with the benchmark results; the results represented by the 1-year execution history of the live social sector application.
This paper focuses on the collaborative aspects of requirements elicitation, in the context of software, systems and service development. The aim is to identify and understand challenges of requirements elicitation in general and in distributed environments. We focus on human, social, and cultural factors that have an impact on communication in the requirements elicitation process. More specifically we aim to i) unfold potential cultural impediments that hamper the requirements elicitation process; ii) highlight cultural factors that should be taken into account in the requirements elicitation process in order to avoid incomplete and inconsistent requirements; and iii) make recommendations for alleviating the problems. In this paper our first step is to report on the findings of a review of the literature regarding culture in RE. The results suggest that the cultural studies in the field of RE are insufficient and thus more empirical studies are required. Secondly, we look at current solutions that are being adopted to assist in improving the cultural aspect of the requirements elicitation process. In the following step we map the identified communication gaps to the SPI Manifesto Values revealing the manifestations of the problems and finally we prescribe a set of recommendations that could be exercised and fulfilled by actors in the requirements elicitation process in order for them to improve cultural considerations in the RE process. These recommendations address the shortcomings that were identified in the literature review and mapped the Values of the SPI Manifesto. The proposals regard technologies, platforms, methods, and frameworks that are readily available. A requirements elicitation process that adopts one or a number of these proposals can help alleviate the challenges invoked by stakeholders’ cultural diversity in the RE process, thus leading to systems development and deployment that much better reflects the requirements/needs of diverse stakeholders and users.
Supervised learning using shallow machine learning methods is still a popular method in processing text, despite the rapidly advancing sector of unsupervised methodologies using deep learning. Supervised text classification for application user feedback sentiments in Indonesian Language is one of the applications which is quite popular in both the research community and industry. However, due to the nature of shallow machine learning approaches, various text preprocessing techniques are required to clean the input data. This research aims to implement and evaluate the role of Levenshtein distance algorithm in detecting and preprocessing misspelled words in Indonesian language, before the text data is then used to train a user feedback sentiment classification model using multinomial Naïve Bayes. This research experimented with various evaluation scenarios, and found that preprocessing misspelled words in Indonesian language using the Levenshtein distance algorithm could be useful and showed a promising 8.2% increase on the accuracy of the model's ability to classify user feedback text according to their sentiments.
Mobile applications usually fail to deliver the right set of features for its users, either by not offering necessary functionalities or by offering unnecessary ones. In the last few years, some modern Requirements Engineering methods have been created to better design mobile applications. One of these methodologies is the Crowdsourcing Requirements Engineering, based on short cycles of implementations and feedbacks by a large group of actual users. This work aims to validate the requirements of a mobile application for the management of domestic animals, through the use of crowndsourcing questionnaires. Two questionnaires were designed and implemented to assess the needs of the users. The first questionnaire verified functionalities provided by similar softwares, while the second one verified requirements established by the authors. From the analysis of the data gathered, all requirements, except for one, were validated. Finally, after the final functionalities of the software were defined, an alpha version of the software could be created.
Users of the Twitter microblogging platform share a vast amount of information about various topics through short messages on a daily basis. Some of these so called tweets include information that is relevant for software companies and could, for example, help requirements engineers to identify user needs. Therefore, tweets have the potential to aid in the continuous evolution of software applications. Despite the existence of such relevant tweets, little is known about their number and content. In this paper we report on the results of an exploratory study in which we analyzed the usage characteristics, content and automatic classification potential of tweets about software applications by using descriptive statistics, content analysis and machine learning techniques. Although the manual search of relevant information within the vast stream of tweets can be compared to looking for a needle in a haystack, our analysis shows that tweets provide a valuable input for software companies. Furthermore, our results demonstrate that machine learning techniques have the capacity to identify and harvest relevant information automatically.
Open-Source Software (OSS) community members report bugs, request features or clarifications by writing messages (in unstructured natural language) to mailing lists. Analysts examine them dealing with an effort demanding and error prone task, which requires reading huge threads of emails. Automated support for retrieving relevant information and particularly for recognizing discussants’ intentions (e.g., suggesting, complaining) can support analysts, and allow them to increase the performance of this task. Online discussions are almost synchronous written conversations that can be analyzed applying computational linguistic techniques that build on the speech act theory. Our approach builds on this observation. We propose to analyze OSS mailing-list discussions in terms of the linguistic and non-linguistic acts expressed by the participants, and provide a tool-supported speech-act analysis method. In this paper we describe this method and discuss how to empirically evaluate it. We discuss the results of the first execution of an empirical study that involved 20 subjects.
Users' feedback is vital to improve software quality and it provides developers with a rich knowledge on how software meets users' requirements in practice. Feedback informs how software should adapt, or be adapted, at runtime and what evolutionary actions to take in the next release. However, studies have noted that accommodating the different preferences of users on how feedback should be requested is a complex task and requires a careful engineering process. This calls for an adaptive feedback acquisition mechanisms to cater for such variability. In this paper, we tackle this problem by employing the concept of Persona to aid software engineers understand the various users' behaviours and improve their ability to design feedback acquisition techniques more efficiently. We create a set of personas based on a mixture of qualitative and quantitative studies and propose PAFA, a Persona-based method for Adaptive Feedback Acquisition.
App Stores, such as Google Play or the Apple Store, allow users to provide feedback on apps by posting review comments and giving star ratings. These platforms constitute a useful electronic mean in which application developers and users can productively exchange information about apps. Previous research showed that users feedback contains usage scenarios, bug reports and feature requests, that can help app developers to accomplish software maintenance and evolution tasks. However, in the case of the most popular apps, the large amount of received feedback, its unstructured nature and varying quality can make the identification of useful user feedback a very challenging task. In this paper we present a taxonomy to classify app reviews into categories relevant to software maintenance and evolution, as well as an approach that merges three techniques: (1) Natural Language Processing, (2) Text Analysis and (3) Sentiment Analysis to automatically classify app reviews into the proposed categories. We show that the combined use of these techniques allows to achieve better results (a precision of 75% and a recall of 74%) than results obtained using each technique individually (precision of 70% and a recall of 67%).
Social networks have changed our daily life and they have the potential to significantly influence and support Requirements Engineering (RE) activities. Social network-based RE approaches will allow us to overcome limitations of traditional approaches and allow end users to play a more prominent role in RE. They are key stakeholders in many software projects. However, involving end users is challenging, particularly when they are not within organizational reach. The goal of our work is to increase end user involvement in RE. In this paper we present an approach where we harness a social network to perform RE activities such as elicitation, prioritization and negotiation. Our approach was applied in three studies where students used Facebook to actively participate in RE activities of different projects. Although there are limitations, the results show that a popular social network site can support distributed RE.
Crowdsourcing is an emerging business model where tasks are accomplished by the general public; the crowd. Crowdsourcing has been used in a variety of disciplines, including information systems development, marketing and operationalization. It has been shown to be a successful model in recommendation systems, multimedia design and evaluation, database design, and search engine evaluation. Despite the increasing academic and industrial interest in crowdsourcing, there is still a high degree of diversity in the interpretation and the application of the concept. This paper analyses the literature and deduces a taxonomy of crowdsourcing. The taxonomy is meant to represent the different configurations of crowdsourcing in its main four pillars: the crowdsourcer, the crowd, the crowdsourced task and the crowdsourcing platform. Our outcome will help researchers and developers as a reference model to concretely and precisely state their particular interpretation and configuration of crowdsourcing.
Adaptive systems are characterized by the ability to monitor changes in their volatile world and react to monitored changes when needed. The ultimate goal of adaptation is that users' requirements are always met correctly and efficiently. Adaptation is traditionally driven by the changing state of the system internal and its surrounding environment. Such state should be monitored and analyzed to decide upon a suitable alternative behaviour to adopt. In this paper, we introduce another driver for adaptation which is the users' collective judgement on the alternative behaviors of a system. This judgmenet should be infered from the individual users' feedback given iteratviely during the lifetime of a system. Users' feedback reflects their main interest which is the validity and the quality of a system behaviour as a means to meet their requirements. We propose social adaptation which is a specific kind of adaptation that treats users' feedback, obtained during the software lifetime, as a primary driver in planning and guiding adaptation. We propose a novel requirements engineering modelling and analysis approach meant for systems adopting social adaptation. We evaluate our approach by applying it in practice and report on the results.
Post-deployment user feedback such as feature requests and bug reports have the potential to improve software quality and reveal missing functionality. Nevertheless, in practice software companies often ignore user feedback and simply stick with predefined roadmaps and development plans, because of two main problems. First, in order to benefit from user feedback, developers have to analyze, consolidate, and structure it, which becomes time-consuming when feedback volume is high. Second, in order to prioritize their work, developers need to assess how many users are affected by a particular feedback, by identifying duplicate and similar feedback in a manual way. This thesis describes Portneuf, a framework which tackles both problems. It consolidates user feedback by employing a context-aware, hybrid recommender system. Moreover, it introduces FeedbackRank, an algorithm which allows developers to assess the importance of collected user feedback to the user community. We demonstrate the feasibility and applicability of our approach in two real-world applications. In a quasi-experiment with a large number of users, we showed that Portneuf increases developers' efficiency when working with user feedback. The framework significantly reduces the amount of user feedback duplicates by over 67%, with the recommender system showing a hit-rate of over 82%. Moreover, we obtained an average precision of around 76% for FeedbackRank, which suggests that it provides a valuable estimation of what is important to the user community.
Crowdsourcing is an emerging and promising approach which involves delegating a variety of tasks to an unknown workforce—the crowd. Crowdsourcing has been applied quite successfully in various contexts from basic tasks on Amazon Mechanical Turk to solving complex industry problems, e.g. InnoCentive. Companies are increasingly using crowdsourcing to accomplish specific software development tasks. However, very little research exists on this specific topic. This paper presents an in-depth industry case study of crowdsourcing software development at a multinational corporation. Our case study highlights a number of challenges that arise when crowdsourcing software development. For example, the crowdsourcing development process is essentially a waterfall model and this must eventually be integrated with the agile approach used by the company. Crowdsourcing works better for specific software development tasks that are less complex and stand-alone without interdependencies. The development cost was much greater than originally expected, overhead in terms of company effort to prepare specifications and answer crowdsourcing community queries was much greater, and the time-scale to complete contests, review submissions and resolve quality issues was significant. Finally, quality issues were pushed later in the lifecycle given the lengthy process necessary to identify and resolve quality issues. Given the emphasis in software engineering on
identifying bugs as early as possible, this is quite problematic.
MyERP is a fictional developer of an Enterprise Resource Planning (ERP) system. Driven by the competition, they face the challenge of losing market share if they fail to de-ploy a Software as a Service (SaaS) ERP system to the European market quickly, but with high quality product. This also means that the requirements engineering (RE) activities will have to be performed efficiently and provide solid results. An additional problem they face is that their (potential) stakeholders are phys-ically distributed, it makes sense to consider them a "crowd". This competition paper suggests a Crowd-based RE approach that first identifies the crowd, then collects and analyzes their feedback to derive wishes and needs, and validate the results through prototyping. For this, techniques are introduced that have so far been rarely employed within RE, but more "traditional" RE techniques, will also be integrated and/or adapted to attain the best possible result in the case of MyERP.
Online user feedback is principally used as an information source for evaluating customers' satisfaction for a given goods, service or software application. The increasing attitude of people towards sharing comments through the social media is making online user feedback a resource containing different types of valuable information. The huge amount of available user feedback has drawn the attention of researchers from different fields. For instance, data mining techniques have been developed to enable information extraction for different purposes, or the use of social techniques for involving users in the innovation of services and processes. Specifically, current research and technological efforts are put into the definition of platforms to gather and/or analyze multi-modal feedback. But we believe that the understanding of the type of concepts instantiated as information contained in user feedback would be beneficial to define new methods for its better exploitation. In our research, we focus on online explicit user feedback that can be considered as a powerful means for user-driven evolution of software services and applications. Up to our knowledge, a conceptualization of user feedback is still missing. With the purpose of contributing to fill up this gap we propose an ontology, for explicit online user feedback that is founded on a foundational ontology and has been proposed to describe artifacts and processes in software engineering. Our contribution in this paper concerns a novel user feedback ontology founded on a Unified Foundational Ontology (UFO) that supports the description of analysis processes of user feedback in software engineering. We describe the ontology together with an evaluation of its quality, and discuss some application scenarios.
MyERP is a fictional developer of an Enterprise Resource Planning (ERP) system. Driven by the competition, they face the challenge of losing market share if they fail to deploy a Software as a Service (SaaS) ERP system to the European market quickly, but with high quality product. This also means that the requirements engineering (RE) activities will have to be performed efficiently and provide solid results. An additional problem they face is that their (potential) stakeholders are physically distributed, it makes sense to consider them a “crowd”. This competition paper suggests a Crowd-based RE approach that first identifies the crowd, then collects and analyzes their feedback to derive wishes and needs, and validate the results through prototyping. For this, techniques are introduced that have so far been rarely employed within RE, but more “traditional” RE techniques, will also be integrated and/or adapted to attain the best possible result in the case of MyERP.
Stakeholders who are highly distributed form a large, heterogeneous online group, the so-called “crowd”. The rise of mobile, social and cloud apps has led to a stark increase in crowd-based settings. Traditional requirements engineering (RE) techniques face scalability issues and require the co-presence of stakeholders and engineers, which cannot be realized in a crowd setting. While different approaches have recently been introduced to partially automate RE in this context, a multi-method approach to (semi-)automate all RE activities is still needed. We propose “Crowd-based Requirements Engineering” as an approach that integrates existing elicitation and analysis techniques and fills existing gaps by introducing new concepts. It collects feedback through direct interactions and social collaboration, and by deploying mining techniques. This paper describes the initial state of the art of our approach, and previews our plans for further research.
Mobile platforms and applications are an exciting and important phenomenon in today's software and business world. They are being woven into the fabric of daily life faster than expected. Continuous collection of user feedback enabling the improvement of platforms and applications becomes critical to support the continuous evolution of mobile systems. Particularly user feedback is needed to provide systems that best fit user needs. We have designed a mobile feedback approach, which enables users to document individual feedback on mobile systems in situ. This information can then be evaluated and used as new requirements by developers. Based on this solution we have developed a feedback app for two different mobile platforms. Furthermore, we have conducted a study with smartphone users applying this approach and communicating feedback on a mobile platform and pre-installed apps. The study revealed that users were able to give individual feedback and that a large amount of this feedback was considered to be useful for mobile system improvement by a platform developer.
The World Wide Web and the services it provides are continually evolving. Even for a single time instant, it is a complex task to methodologically determine the infrastructure over which these services are provided and the corresponding effect on user perceived performance. For such tasks, researchers typically rely on active measurements or large numbers of volunteer users. In this paper, we consider an alternative approach, which we refer to as passive crowd-based monitoring. More specifically, we use passively collected proxy logs from a global enterprise to observe differences in the quality of service (QoS) experienced by users on different continents. We also show how this technique can measure properties of the underlying infrastructures of different Web content providers. While some of these properties have been observed using active measurements, we are the first to show that many of these properties (such as location of servers) can be obtained using passive measurements of actual user activity. Passive crowd-based monitoring has the advantages that it does not add any overhead on Web infrastructure, it does not require any specific software on the clients, but still captures the performance and infrastructure observed by actual Web usage.
This review article discusses various techniques of environmental monitoring (EM) systems and what is required for the variations in hardware implementation and/or algorithmic logic. This review presents an overview of the existing state-of-the-art practices of environmental monitoring systems and is mainly focused on energy-efficient and low-cost environment monitoring systems. The following are some of the major factors that usually rule the development of EM systems, namely, energy efficiency, cost of the overall system, response time of the sensor module, good accuracy of the system, adequate signal-to-noise ratio, radio frequency interference/electromagnetic interference (RFI/EMI) rejection during varying atmospheric conditions and in inhomogeneous environments, a user friendly interface with the computer, and complexity of computation. The above concerns are also recognized by reference to research articles on environmental monitoring systems. Emphasis is on the necessity of robust systems that address all or most of the above mentioned criteria.
An increasing part of software development is devoted to products that are offered to an open market with many customers. Market-driven development imposes special challenges for the requirements engineering process. This chapter provides an overview of the special characteristics of market-driven requirements engineering and describes the most important challenges of the area. Key elements of market-driven requirements engineering processes are presented together with a definition of process quality. Requirements state models and requirements repositories are also described and examples of typical solutions to progress tracking and data management are provided. The difficult problem of release planning is also discussed and an industrial example of a release planning process is given.
Current requirements engineering practices for gathering user input are characterized by a number of communication gaps between users and engineers, which might lead to wrong requirements. The problem situations and context which underlie user input are either gathered back in time, or submitted with wrong a level of details. We think that making user input a first order concern of both software processes and software systems harbours many innovation opportunities. We propose and discuss a continuous and context-aware approach for communicating user input to engineering teams and other users, by a) instrumenting the problem domain, b) proactively recommending to share feedback and c) annotating graphical interfaces.
In market-driven software development there is a strong need for support to handle congestion in the requirements engineering
process, which may occur as the demand for short time-to-market is combined with a rapid arrival of new requirements from
many different sources. Automated analysis of the continuous flow of incoming requirements provides an opportunity to increase
the efficiency of the requirements engineering process. This paper presents empirical evaluations of the benefit of automated
similarity analysis of textual requirements, where existing information retrieval techniques are used to statistically measure
requirements similarity. The results show that automated analysis of similarity among textual requirements is a promising
technique that may provide effective support in identifying relationships between requirements.
A Novel Method for Large Scale Requirement Elicitation
- V Dheepa
- D J Aravindhar
- C Vijayalakshmi
V. Dheepa, D.J. Aravindhar, and C. Vijayalakshmi, "A Novel Method for Large Scale Requirement Elicitation,"
Int'l J. Eng. and Innovative Technology, vol. 2, no. 7, 2013, pp. 375-379.