No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
HCD3A: An HCD Model to Design Data-Driven Apps
Abstract
This contribution introduces HCD3A, a process model to guide and support the development of data-driven applications. HCD3A is a specialized human-centered design (HCD) process model derived from and based on the ISO 9241-210 standard. In order to test the suitability of the HCD3A process model a prototype of a machine learning (ML) application is developed along this process. This application is integrated in a learning management system and tailored to the needs of computer science students in an online learning context. The learning application uses an ML approach to support students in their learning behavior by helping them to avoid procrastination and motivating them for assignments and final exams. This is e.g. done by predicting the students exam success probability. The most important claim in regard to the ML components was explainability. Although the evaluation of the prototype in regard to the suitability of HCD3A has not been completed the first results show that it is promising in particular to make ML applications more transparent for the users.
... In this case, HCD might be a supporting tool for evaluating the clustering outcomes together with domain experts. We recommend bringing ML and domain experts together to (iteratively) review the clustering algorithms' results, as was proposed by [21]. Adapting the whole clustering strategy can also be part of these reviews if necessary. ...
In many areas, only small data sets are available and big data does not play a significant role, e.g., in Human-Centered Design research. In the context of machine learning analysis, results of small data sets can be biased due to single variables or missing values. Nevertheless, reliable and interpretable results are essential for determining further actions, such as, e.g., treatments in a health-related use case. In this paper, we explore machine learning clustering algorithms on the basis of a small, health-related (variance) data set about early dyslexia screening. Therefore, we selected three different clustering algorithms from different clustering methods: K-Means, HAC and DBSCAN. In our case, K-Means and HAC showed promising results, while DBSCAN did not deliver distinct results. Based on our experiences, we provide first proposals on how to handle small data set clustering and describe situations in which using Human-Centered Design methods can increase interpretability of machine learning clustering results. Our work represents a starting point for discussing the topic of clustering small data sets.
In the last few years, Artificial Intelligence (AI) has achieved a notable momentum that, if harnessed appropriately, may deliver the best of expectations over many application sectors across the field. For this to occur shortly in Machine Learning, the entire community stands in front of the barrier of explainability, an inherent problem of the latest techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI (namely, expert systems and rule based models). Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is widely acknowledged as a crucial feature for the practical deployment of AI models. The overview presented in this article examines the existing literature and contributions already done in the field of XAI, including a prospect toward what is yet to be reached. For this purpose we summarize previous efforts made to define explainability in Machine Learning, establishing a novel definition of explainable Machine Learning that covers such prior conceptual propositions with a major focus on the audience for which the explainability is sought. Departing from this definition, we propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at explaining Deep Learning methods for which a second dedicated taxonomy is built and examined in detail. This critical literature analysis serves as the motivating background for a series of challenges faced by XAI, such as the interesting crossroads of data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence , namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to the field of XAI with a thorough taxonomy that can serve as reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability.
Academic procrastination is a fact related to the delay or postpone of academic work until last minute. This phenomenon is evident in a vast majority of university students, and its occurrence is increasing. In order to analyse possible causes and/or solutions, we studied if longer time for accomplishing an assignment incentives or avoids procrastination among university students. Results showed that both short and long time-frame groups tended to procrastinate in the same way. Additionally, academic grades did not revealed differences between groups, as the procrastination was the same between groups. Thus, this study shows that even with longer period of time to accomplish a task, university students tend to procrastinate, and thus seem to have a negative effect on their assignment grades. Therefore, it seems a current problem and measures should be developed in order to solve it.
Machine learning (ML) plays an increasingly important role in improving a user’s experience. However, most UX practitioners face challenges in understanding ML’s capabilities or envisioning what it might be. We interviewed 13 designers who had many years of experience designing the UX of ML-enhanced products and services. We probed them to characterize their practices. They shared they do not view themselves as ML experts, nor do they think learning more about ML would make them better designers. Instead, our participants appeared to be the most successful when they engaged in ongoing collaboration with data scientists to help envision what to make and when they embraced a data-centric culture. We discuss the implications of these findings in terms of UX education and as opportunities for additional design research in support of UX designers working with ML.
Learning Analytics focuses on the collection and analysis of learners’ data to improve their learning experience by providing informed guidance and to optimise learning materials. To support the research in this area we have developed a dataset, containing data from courses presented at the Open University (OU). What makes the dataset unique is the fact that it contains demographic data together with aggregated clickstream data of students’ interactions in the Virtual Learning Environment (VLE). This enables the analysis of student behaviour, represented by their actions. The dataset contains the information about 22 courses, 32,593 students, their assessment results, and logs of their interactions with the VLE represented by daily summaries of student clicks (10,655,280 entries). The dataset is freely available at https://analyse.kmi.open.ac.uk/open_dataset under a CC-BY 4.0 license.
As AI is increasingly being adopted into application solutions, the challenge of supporting interaction with humans is becoming more apparent. Partly this is to support integrated working styles, in which humans and intelligent systems cooperate in problem-solving, but also it is a necessary step in the process of building trust as humans migrate greater responsibility to such systems. The challenge is to find effective ways to communicate the foundations of AI-driven behaviour, when the algorithms that drive it are far from transparent to humans. In this paper we consider the opportunities that arise in AI planning, exploiting the model-based representations that form a familiar and common basis for communication with users, while acknowledging the gap between planning algorithms and human problem-solving.
Universities gather large volumes of data with reference to their students in electronic form. The advances in the data mining field make it possible to mine these educational data and find information that allow for innovative ways of supporting both teachers and students. This paper presents a case study on predicting performance of students at the end of a university degree at an early stage of the degree program, in order to help universities not only to focus more on bright students but also to initially identify students with low academic achievement and find ways to support them. The data of four academic cohorts comprising 347 undergraduate students have been mined with different classifiers. The results show that it is possible to predict the graduation performance in 4th year at university using only pre-university marks and marks of 1st and 2nd year courses, no socio-economic or demographic features, with a reasonable accuracy. Furthermore courses that are indicators of particularly good or poor performance have been identified.
The 1st international conference on "Educational Data Mining" (EDM) took place in Montreal in 2008 while the 1st international conference on "Learning Analytics and Knowledge" (LAK) took place in Banff in 2011. Since then the fields have grown and established themselves with an annual international conference, a journal and an association each, and gradually increase their overlapping. This paper begins with some considerations on big data in education. Then the principal analysis methods used with educational data are reviewed and are illustrated with some of the tasks they solve. Current emerging trends are presented. Analysis of educational data on a routine basis to understand learning and teaching better and to improve them is not a reality yet. The paper concludes with challenges on the way.
This paper examines the effects and significance of various controllable marketing factors likely to influence the online consumers' behaviour and buying decisions. The study is based on a proposed taxonomy of these factors, classified as elements of what is commonly called Online or Web Experience. The paper analyses the results of a consumer survey held in a realistic online shopping environment and ascertains the influence of the various elements of the Online Experience on the choice of a virtual vendor. Next to evaluating and validating the various elements of the proposed classification the study identifies the relative importance of these elements and their effects on the online consumers' decisions when acting in combination. The statistical analysis of the survey results leads to the conclusion that the usability and the trust-building elements of web sites are the most significant influencing factors of online customer behaviour.
Addresses one of the fundamental issues of e-marketing: how to attract and win over the consumer in the highly competitive Internet marketplace. Analyses the factors affecting the online consumer's behavior and examines how e-marketers can influence the outcome of the virtual interaction and buying process by focusing their marketing efforts on elements shaping the customer's virtual experience, the Web experience. Identifying the Web experience components and understanding their role as inputs in the online customer's decision-making process are the first step in developing and delivering an attractive online presence likely to have the maximum impact on Internet users. Click-and-mortar firms delivering superior Web experience influence their physical clients’ perceptions and attitudes, driving additional traffic to traditional sales outlets. Provides a contribution to the theoretical debate around the factors influencing the online consumer's behavior and outlines some noticeable similarities and differences between the traditional and virtual consumers.
Explainability is essential for users to effectively understand, trust, and manage powerful artificial intelligence applications.
Machine learning (ML) is now a fairly established technology, and user experience (UX) designers appear regularly to integrate ML services in new apps, devices, and systems. Interestingly, this technology has not experienced a wealth of design innovation that other technologies have, and this might be because it is a new and difficult design material. To better understand why we have witnessed little design innovation, we conducted a survey of current UX practitioners with regards to how new ML services are envisioned and developed in UX practice. Our survey probed on how ML may or may not have been a part of their UX design education, on how they work to create new things with developers, and on the challenges they have faced working with this material. We use the findings from this survey and our review of related literature to present a series of challenges for UX and interaction design research and education. Finally, we discuss areas where new research and new curriculum might help our community unlock the power of design thinking to re-imagine what ML might be and might do.
With the rapid growth in online programs come concerns about how best to support student learning in this segment of the university population. The purpose of this study was to investigate the impact of effort regulation, a self-regulatory skill, and intrinsic motivation on online graduate students' levels of academic procrastination, behavior that can adversely affect both the quality and quantity of student work. This research was guided by one primary question: Are online graduate students' intrinsic motivation and use of effort regulation strategies predictive of procrastination? Results indicated that as intrinsic motivation to learn and effort regulation decrease, procrastination increases. Specific strategies for encouraging effort regulation and intrinsic motivation in online graduate students are presented.
Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, ***, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
When user experience designers partner with data scientists
- F Girardin
- N Lathia
Girardin, F., Lathia, N.: When user experience designers partner with data scientists. In:
Designing the User Experience of Machine Learning Systems. 2017 AAAI Spring Symposium
Series, pp. 376-381 (2017)
UX design guide for data scientists and AI products
- S S Nazrul
Nazrul, S.S.: UX design guide for data scientists and AI products (2018). https://towardsdatas
cience.com/ux-design-guide-for-data-scientists-and-ai-products-465d32d939b0. Accessed
19 May 2021
- D Pásztor
Pásztor, D.: AI UX: 7 Principles of designing good AI products (2018). https://uxstudioteam.
com/ux-blog/ai-ux/. Accessed 19 May 2021
UX for AI: trust as a design challenge
- V Shapiro
Shapiro, V.: UX for AI: trust as a design challenge (2018). https://medium.com/sap-design/
ux-for-ai-trust-as-a-design-challenge-62044e22c4ec. Accessed 19 May 2021
- M Fox
- D Long
- D Magazzeni
Fox, M., Long, D., Magazzeni, D.: Explainable Planning. arXiv preprint arXiv:1709.10256
(2017)