Petra Heck

Fontys University of Applied Sciences · Institute of ICT

Certification of software artifacts offers organizations more certainty and confidence about software. Certification of software helps software sales, acquisition, and can be used to certify legislative compliance or to achieve acceptable deliverables in outsourcing. In this article, we present a software product certification model. This model has...

Until now, quality assessment of requirements has focused on traditional up-front requirements. Contrasting these traditional requirements are just-in-time (JIT) requirements, which are by definition incomplete, not specific and might be ambiguous when initially specified, indicating a different notion of "correctness." We analyze how the assessmen...

Just-in-time (JIT) requirements drive agile teams in planning and implementing software systems. In this paper, we start with the hypothesis that performing informal verification of JIT requirements is useful. For this purpose we propose a framework for quality criteria for JIT requirements. This framework can be used by JIT teams to define 'just-e...

Agile projects typically employ just-in-time requirements engineering and record their requirements (so-called feature requests) in an issue tracker. In open source projects, we observed large networks of feature requests that are linked to each other. Both when trying to understand the current state of the system and to understand how a new featur...

While requirements for open source projects originate from a variety of sources like e.g. mailing lists or blogs, typically, they eventually end up as feature requests in an issue tracking system. When analyzing how these issue trackers are used for requirements evolution, we witnessed a high percentage of duplicates in a number of high-prole proje...

This paper presents a mapping study on "How to do data quality engineering for AI systems?". The paper analyzes data quality engineering solutions from 11 papers on AI engineering. Implications for researchers and practitioners are also discussed.

[Available online: https://fontysblogt.nl/llmops-engineering-trustworthy-llm-systems/] In this post, I analyze the quality characteristics of LLM systems and discuss the challenges for engineering LLM systems. I also present the solutions I have found untill now to address the quality characteristics and the challenges. In future work we will engi...

AI systems cannot exist without data. Now that AI models (data science and AI) have matured and are readily available to apply in practice, most organizations struggle with the data infrastructure to do so. There is a growing need for data engineers that know how to prepare data for AI systems or that can setup enterprise-wide data architectures fo...

Background:Measuring and monitoring stress has potential benefits for the care and self-management of stressors for people with dementia. Early identification of stressors may help to cope with challenging behaviours (CB), occurring in up to 80% of nursing home residents with dementia. The identification of stressors causing CB is difficult (as oft...

List of retrieved papers in the mapping study; includes categorization of selected papers

Introduction: The high prevalence of Behavioural and Psychological Symptoms of Dementia (BPSD) is a challenge for (in)formal caregivers. Stress is a common problem for both people with dementia and their caregivers. The development of wearable technologies to measure stress in people with dementia has the potential to help manage BPSD. Wearable tec...

Full-text see https://fontysblogt.nl/a-quality-model-for-trustworthy-ai-systems/ Although ISO25000 claimed in 2011 that “the characteristics defined … are relevant to all software products and computer systems”, by now ISO is working on an specific SQuaRE version for AI systems. While waiting for that update I took (grey) literature on quality of...

The previous chapters provided gentle introductions to various important topics in the area of data analytics. In this chapter, we present three real-life case studies that illustrate how the methods and approaches outlined in the previous chapters can be put into practice. The first case study shows how the Dutch company BagsID uses data analytics...

For an AI solution to evolve from a trained machine learning model into a production-ready AI system, many more things need to be considered than just the performance of the machine learning model. A production-ready AI system needs to be trustworthy, i.e. of high quality. But how to determine this in practice? For traditional software, ISO25000 an...

Based on Zhang et al. [3] and Kuwajima & Ishikawa [2] we have formulated nine quality characteristics for AI systems that are missing in the ISO25010 System and software quality models [1]. The proposed additions naturally relate to the difference between AI systems and traditional rule-based software systems: • For AI systems we do not program the...

Recently, the job market for Artificial Intelligence (AI) engineers has exploded. Since the role of AI engineer is relatively new, limited research has been done on the requirements as set by the industry. Moreover, the definition of an AI engineer is less established than for a data scientist or a software engineer. In this study we explore, based...

Langdurige of intense stress kan tot lichamelijke en psychische gezondheidsklachten leiden. Goed stressmanagement helpt bij het terugdringen van deze klachten. Door het stressniveau met draagbare sensoren (wearables) te meten, kan iemand inzage krijgen in zijn eigen stressniveau - of het stressniveau van iemand die dat zelf niet goed kan communic...

AI-enabled systems are software systems that contain a machine learning (ML) component. Developing these AI systems requires a specific engineering approach, different from traditional rule-based software, and thus a specific approach to quality. In this presentation, we discuss quality criteria for AI systems, based on the ISO/IEC 25000 series of...

(available online at https://fontysblogt.nl/a-toolbox-for-the-applied-ai-engineer/) In my previous post on AI engineering I defined the concepts involved in this new discipline and explained that with the current state of the practice, AI engineers could also be named machine learning (ML) engineers. In this post I would like to 1) define our view...

This chapter discusses how to build production-ready machine learning systems. There are several challenges involved in accomplishing this, each with its specific solutions regarding practices and tool support. The chapter presents those solutions and introduces MLOps (machine learning operations, also called machine learning engineering) as an ove...

(see online: https://fontysblogt.nl/ai-engineering-and-mlops/) The past two years I have conducted an extensive literature and tool review to answer the question: “What should software engineers learn about building production-ready machine learning systems?”. During my research I noted that because the discipline of building production-ready machi...

Over the past three years we have built a practice-oriented, bachelor level, educational programme for software engineers to specialize as AI engineers. The experience with this programme and the practical assignments our students execute in industry has given us valuable insights on the profession of AI engineer. In this paper we discuss our progr...

The development of artificial intelligence (AI) has made various industries eager to explore the benefits of AI. There is an increasing amount of research surrounding AI, most of which is centred on the development of new AI algorithms and techniques. However, the advent of AI is bringing an increasing set of practical problems related to AI model...

More and more software applications contain machine learning (ML) modules. Developing ML applications requires a specific engineering approach, different from traditional rule-based software. In this presentation, we identify engineering challenges for ML applications and present some of the solutions we collected until now.

In industry as well as education as well as academics we see a growing need for knowledge on how to apply machine learning in software applications. With the educational programme ICT & AI at Fontys UAS we had to find an answer to the question: "How should we educate software engineers to become AI engineers?" This paper describes our educational p...

In this post I give an overview of the theory, tools, frameworks and best practices I have found until now around the testing (and debugging) of machine learning applications. I will start by giving an overview of the specificities of testing machine learning applications. Available online at: https://fontysblogt.nl/testing-machine-learning-applica...

Application of machine learning is within reach for every software developer because of the availability of computing power (cloud systems), open source tools/libraries and even pretrained models or APIs. We see a strong parallel with what happened in software development. Only now machine learning applications are widespread in production-like env...

The quality of requirements is typically considered as an important factor for the quality of the end product. For traditional up-front requirements specifications, a number of standards have been defined on what constitutes good quality : Requirements should be complete, unambiguous, specific, time-bounded, consistent, etc. For agile requirements...

The goal of this thesis was to obtain a deeper understanding of the notion of quality for Just-in-Time (JIT) Requirements. JIT requirements are the opposite of up-front requirements. JIT requirements are not analyzed or defined until they are needed meaning that development is allowed to begin with incomplete requirements. We started our analysis b...

Verification activities are necessary to ensure that the requirements are specified in a correct way. However, until now requirements verification research has focused on traditional up-front requirements. Agile or just-in-time requirements are by definition incomplete, not specific and might be ambiguous when initially specified, indicating a diff...

An intuitive definition of a dependable system is a system that does not fail in unexpected or catastrophic ways. A software product without faults is the best guarantee for dependability. Our model indicates the checks that need to be done to retrieve as many faults as possible. We believe that provable quality of dependable systems is based on th...

The quality of any product depends on the quality of the basis of making it, i.e., the quality of the requirements has strong effect on the quality of the end products. In practice, however, the quality of requirement specifications is poor, in fact a primary reason why so many projects continue to fail. Thus, the current approaches as applied in p...

verification and validation of software systems and their intermediate artifacts. If an organization wants certainty about or confidence in a software artifact a LaQuSo certificate can be requested. Being part of universities, LaQuSo is able to perform the independent evaluator role in many certification projects. Certification is a check that the...

The paper describes the preparation and execution of the site acceptance testing of a large scale industrial system. A use case based approach to testing is developed. The approach introduces three-level test artifact specifications. At the highest level test scenarios are used to validate system use cases. The lower levels are presented by test sc...