Johannes Kunkel’s research while affiliated with University of Duisburg-Essen and other places

How users interact with an intelligent system is determined by their subjective mental model of the system’s inner working. In this paper, we present a novel method based on card sorting to identify such mental models of recommender systems quantitatively. Using this method, we conducted an online study ($N \!=\!170$). Applying hierarchical clustering to the results revealed distinct user groups and their respective mental models. Independent of the recommender system used, some participants held a strict procedural-based, others a concept-based mental model. Additionally, mental models can be characterized as either technical or humanized. While procedural-based mental models were positively related to transparency perception, humanized models might influence the perception of system trust. Based on these findings, we derive three implications for the consideration of user-specific mental models in the design of transparent intelligent systems.

While online content is personalized to an increasing degree, eg. using recommender systems (RS), the rationale behind personalization and how users can adjust it typically remains opaque. This was often observed to have negative effects on the user experience and perceived quality of RS. As a result, research increasingly has taken user-centric aspects such as transparency and control of a RS into account, when assessing its quality. However, we argue that too little of this research has investigated the users' perception and understanding of RS in their entirety. In this paper, we explore the users' mental models of RS. More specifically, we followed the qualitative grounded theory methodology and conducted 10 semi-structured face-to-face interviews with typical and regular Netflix users. During interviews participants expressed high levels of uncertainty and confusion about the RS in Netflix. Consequently, we found a broad range of different mental models. Nevertheless, we also identified a general structure underlying all of these models, consisting of four steps: data acquisition, inference of user profile, comparison of user profiles or items, and generation of recommendations. Based on our findings, we discuss implications to design more transparent, controllable, and user friendly RS in the future.

Recommender systems (RS) are very common tools designed to help users choose items from a large number of alternatives. While their algorithms are already quite mature in terms of precision, RS cannot unfold their full potential due to a lack of transparency and missing means of control. In this paper we introduce a method aiming at creating recommendations that are comprehensible and controllable by their users while granting an overview over the item domain. To achieve this, the entire item space of a domain is visualized using a map-like interface. Inside, users can express their preferences on which the RS reacts with matching recommendations. To change recommendations, users can alter their preferences expressed, which creates a continuous feedback loop between user and RS. We demonstrate our general method using two prototype applications, located in different item domains and utilizing different forms of visualization and interaction modalities. Empirical user studies with both prototypes show a great potential of our method to increase overview, transparency and control in RS.

Empfehlungssysteme, die mit Hilfe latenter Faktormodelle Empfehlungen generieren, arbeiten äußerst genau und sind entsprechend weit verbreitet. Da die Berechnung der Empfehlungen jedoch auf der statistischen Auswertung von Benutzerbewertungen basiert, gestaltet es sich schwierig, die Empfehlungen dem Nutzer gegenüber zu erklären. Daher werden die Systeme häufig als intransparent wahrgenommen und können oft ihr volles Potential nicht entfalten. Erste Ansätze zeigen allerdings, dass die latenten Faktoren solcher Modelle semantische Eigenschaften der Produkte widerspiegeln. Dabei ist bislang unklar, ob die zum Teil sehr komplexe Parametrisierung, die z.B. die Anzahl der Faktoren festlegt, Auswirkungen auf die semantische Verständlichkeit hat. Da dies sehr von der subjektiven Wahrnehmung abhängt, präsentieren wir mit LittleMissFits ein Online-Spiel, das es erlaubt, mittels Crowd-Sourcing die Konsistenz der latenten Faktoren zu untersuchen. Die Ergebnisse einer Nutzerstudie mit diesem Spiel zeigen, dass eine höhere Anzahl von Faktoren das Modell weniger verständlich erscheinen lässt. Darüber hinaus fanden sich Unterschiede innerhalb der Faktormodelle bezüglich der Verständlichkeit der einzelnen Faktoren. Zusammengenommen stellen die Ergebnisse eine wertvolle Grundlage dar, um künftig die Transparenz entsprechender Empfehlungssysteme zu steigern.

Trust in a Recommender System (RS) is crucial for its overall success. However, it remains underexplored whether users trust personal recommendation sources (i.e. other humans) more than impersonal sources (i.e. conventional RS), and, if they do, whether the perceived quality of explanation provided account for the difference. We conducted an empirical study in which we compared these two sources of recommendations and explanations. Human advisors were asked to explain movies they recommended in short texts while the RS created explanations based on item similarity. Our experiment comprised two rounds of recommending. Over both rounds the quality of explanations provided by users was assessed higher than the quality of the system's explanations. Moreover, explanation quality significantly influenced perceived recommendation quality as well as trust in the recommendation source. Consequently, we suggest that RS should provide richer explanations in order to increase their perceived recommendation quality and trustworthiness.

Recommender systems relying on latent factor models often appear as black boxes to their users. Semantic descriptions for the factors might help to mitigate this problem. Achieving this automatically is, however, a non-straightforward task due to the models' statistical nature. We present an output-agreement game that represents factors by means of sample items and motivates players to create such descriptions. A user study shows that the collected output actually reflects real-world characteristics of the factors.

Recommender systems relying on latent factor models often appear as black boxes to their users. Semantic descriptions for the factors might help to mitigate this problem. Achieving this automatically is, however, a non-straightforward task due to the models' statistical nature. We present an output-agreement game that represents factors by means of sample items and motivates players to create such descriptions. A user study shows that the collected output actually reflects real-world characteristics of the factors.

Empfehlungssysteme, die auf latenten Faktormodellen basieren, sind dafür bekannt sehr genaue Vorschläge zu generieren. Häufig werden diese Systeme jedoch von Nutzern als intransparent wahrgenommen. Semantische Beschreibungen der latenten Faktoren könnten helfen, dieses Problem zu lindern. Solche Beschreibungen automatisch zu ermitteln gestaltet sich allerdings aufgrund der statistischen Herleitung der Faktoren aus numerischen Bewertungsdaten als schwierig. In diesem Beitrag stellen wir ein Output-Agreement-Spiel vor, das Spieler dazu motiviert, anhand repräsentativer Produkte Beschreibungen zu den Faktoren zu erstellen. Eine durchgeführte Nutzerstudie zeigt, dass das Spiel viel Spaß bereitet und die erhobenen Beschreibungen realweltliche Eigenschaften der Faktoren widerspiegeln.