Content uploaded by Max Sauer
Author content
All content in this area was uploaded by Max Sauer on Apr 15, 2024
Content may be subject to copyright.
— User-friendly Integration of
Identity Wallets and Mobility
Platforms
A User Experience Study Conducted in the
SDIKA Project
FZI Forschungszentrum Informatik: Max Sauer, Simon Pfeifer, Jan Sürmeli
raumobil GmbH: Elias Siebert, Isabelle Woytal
User-friendly Integration of Identity Wallets and Mobility Platforms
2
List of Contents
Abstract ........................................................................................................................................................................... 6
Preface ............................................................................................................................................................................. 7
1. Showcase Secure Digital Identities Karlsruhe (SDIKA) ........................................................................................... 9
2. Preliminaries ............................................................................................................................................................. 10
2.1. Problem definition .......................................................................................................................................... 10
2.2. Goals and Approach ........................................................................................................................................ 10
2.3. Structure of this Report .................................................................................................................................. 11
3. Methodology ............................................................................................................................................................. 12
3.1. Evaluation object and setup........................................................................................................................... 12
3.2. Study procedure and evaluation methods ................................................................................................... 14
3.3. Test subjects .................................................................................................................................................... 16
4. Results and discussion ............................................................................................................................................. 19
4.1. Thinking aloud ................................................................................................................................................. 19
4.2. Eye tracking ...................................................................................................................................................... 21
4.3. Questionnaires: SUS and UEQ-S .................................................................................................................... 24
4.4. Synthesis of the evaluation results................................................................................................................ 27
5. Limitations ................................................................................................................................................................ 29
5.1. Laboratory setting ........................................................................................................................................... 29
5.2. Testing device and wallet prototype ............................................................................................................. 29
5.3. Fictitious personal data .................................................................................................................................. 30
5.4. Synthesis of the limitations ............................................................................................................................ 30
6. Conclusions and future work .................................................................................................................................. 31
7. Acknowledgments .................................................................................................................................................... 32
8. References ................................................................................................................................................................. 33
9. Appendix .................................................................................................................................................................... 34
Authors .......................................................................................................................................................................... 39
Imprint ........................................................................................................................................................................... 40
User-friendly Integration of Identity Wallets and Mobility Platforms
3
List of Figures
Figure 1: Project overview of SDIKA.............................................................................................................................. 9
Figure 2: User journey .................................................................................................................................................. 13
Figure 3: Evaluation setup ........................................................................................................................................... 13
Figure 4: Evaluation recordings .................................................................................................................................. 14
Figure 5: User test process ........................................................................................................................................... 15
Figure 6: IT-affinity of test subjects ............................................................................................................................ 16
Figure 7: Experience of test subjects with regiomove and wallets ......................................................................... 16
Figure 8: Age of test subjects ....................................................................................................................................... 17
Figure 9: Test subjects age sorted by method ........................................................................................................... 17
Figure 10: Test subject IT-affinity, sorted by authentication method .................................................................... 18
Figure 11: SUS score per test subject ......................................................................................................................... 24
Figure 12: SUS scores by age cohorts ......................................................................................................................... 25
Figure 13: UEQ-S scores by test subject ..................................................................................................................... 25
Figure 14: UEQ-S scores by age cohort....................................................................................................................... 26
Figure 15: SUS scores by authentication method ..................................................................................................... 26
Figure 16: UEQ-S scores by authentication method ................................................................................................. 27
Figure 17: regiomove, home screen ........................................................................................................................... 34
Figure 18: regiomove, account .................................................................................................................................... 34
Figure 19: regiomove, account settings ..................................................................................................................... 34
Figure 20: regiomove, activate stadtmobil ................................................................................................................ 34
Figure 21: regiomove, ID card details ......................................................................................................................... 34
Figure 22: regiomove, driving license ID number...................................................................................................... 34
Figure 23: regiomove, request videocall .................................................................................................................... 35
Figure 24: regiomove, accept videocall...................................................................................................................... 35
Figure 25: regiomove, activation dialogue ................................................................................................................ 35
Figure 26: regiomove, push notification .................................................................................................................... 35
Figure 27: regiomove, stadtmobil account ................................................................................................................ 35
Figure 28: regiomove, open with wallet ..................................................................................................................... 35
Figure 29: Wallet, credential details ........................................................................................................................... 36
Figure 30: Wallet, add to wallet ................................................................................................................................... 36
Figure 31: Wallet, stored credential ............................................................................................................................ 36
User-friendly Integration of Identity Wallets and Mobility Platforms
4
Figure 32: Authentication with 4-digit-Pin ................................................................................................................. 36
Figure 33: Authentication with 6-digit-Pin ................................................................................................................. 36
Figure 34: Authentication with password .................................................................................................................. 36
Figure 35: Authentication with fingerscan ................................................................................................................. 37
Figure 36: Scan path of stadtmobil account screen ................................................................................................. 37
Figure 37: Scan path of stadtmobil account screen, age 50-60 ............................................................................... 37
Figure 38: Heatmap of stadtmobil account screen, age > 60................................................................................... 37
Figure 39: Heatmap of import credential................................................................................................................... 37
Figure 40: Scan path of import credential ................................................................................................................. 37
User-friendly Integration of Identity Wallets and Mobility Platforms
5
List of Tables
Table 1: Eye Tracking results ....................................................................................................................................... 38
Table 2: Login times ..................................................................................................................................................... 38
User-friendly Integration of Identity Wallets and Mobility Platforms
6
Abstract
Digital identities function as electronic representations of persons, organizations or other entities in virtual
space. Their main purpose is to verify the identity of a person or – more generally, an entity – online and to
regulate their access to digital resources, systems or services. Currently, digital identities require the creation
of numerous different accounts. The long-term plan is to solve this problem by introducing digital identity
wallets, such as the EUDI wallet introduced in EIDAS 2. These wallets allow users to manage and use their
digital identities as well as verification documents such as ID cards or driving licenses by storing all relevant
data in one place on their devices. This allows users to retain control over their data and decide for themselves
what information they wish to disclose. A good user experience (UX) is crucial for the successful use of wallets,
as this reduces the susceptibility to errors and makes it easier to store, manage and share sensitive identity
data. However, current research shows that the wallets developed to date have room for improvement
regarding UX. For example, users have difficulties understanding the concept of these wallets. It is thus
required to identify UX issues and improve the UX of wallets. In order to gain suggestions for improving, a UX
evaluation was conducted in the project ‘Showcase Secure Digital Identities Karlsruhe’ (short: SDIKA) funded
by the German Federal Ministry of Economics and Climate Action (BMWK). To this end, 24 different test
subjects were involved in the UX evaluation in order to test an internal project prototype involving the wallet
used in the project for demonstration purposes. The prototype includes a mobility platform called
‘regiomove’, in which a car-sharing provider is to be activated. The activation is done inside the mobility app
with a driving license using a fictitious video identification procedure. The test subjects were then asked to
export their driving licenses from regiomove to their wallet, so it could be used in further applications. Eye
tracking, thinking aloud and two questionnaires (SUS and UEQ-S) were used for the UX evaluation.
User-friendly Integration of Identity Wallets and Mobility Platforms
7
Preface
Secure digital identities: A driver for digital innovations
Connecting people, organizations and objects via digital technologies is a cornerstone of the digital
transformation. As such, ensuring trustworthy interaction and secure data exchange between different
entities is crucial for the success and the adoption of new technologies and processes. In particular, it gives
rise to solutions that allow entities to prove their identity and present verifiable data towards third parties.
The emerging concept of digital iden tity wallets – applications to manage one’s digital identity and credentials
– promises to change the way of implementing trustworthy interaction. Instead of centralized identity
providers and registries, entities carry their own verifiable identities and credentials. Trust into the validity of
identity and credentials is then established by the so-called trust triangle between the three roles of holder,
issuer and verifier of entities: The Holder can present a credential to a Verifier, and prove that a specific Issuer
created the credential for the Holder. In turn, if the Verifier trusts the Issuer, the Verifier can trust the contents
of the credential.
User Experience: essential for acceptance
While there exist standards for file formats and protocols, and EU legislation actively follows a wallet-based
approach in the upcoming EIDAS 2 regulation, the acceptance of such technologies by users or industry is not
ensured yet. As with most emerging technologies, it is reasonable to assume that user experience (UX) will have
a strong influence on the acceptance of wallet technologies and their integration into everyday use cases. Ev en
more so, as ‘identity’ is currently rather conceptualized as a door opener for use cases, and not as a provider
of immediate value.
Showcase Secure Digital Identities Karlsruhe: A project involving the UX study of wallets
Given the role as an expected key factor for acceptance, studying the UX of processes involving wallets is an
important research task in the project Showcase Secure Digital Identities Karlsruhe (SDIKA), funded by the
Federal German Ministry of Economics and Climate Action (BMWK). This whitepaper reports on a UX study
from the mobility domain conducted as part of the SDIKA project.
The process under study: Service activation and credential export in mobility
The SDIKA project studies use cases from different domains, among them mobility apps. As a resident of the
Karlsruhe region, one can use the regiomove mobility app to access different mobility services. One of such
services is the car sharing service stadtmobil. Activating and using the stadtmobil service in regiomove requires
the presentation of a valid driver’s license. As such a check involves user interaction and their physical
documents, a promising direction to improve user acceptance and reduce costs, is to make this check only
once, and provide proof of previous validations as a credential to be stored in the user’s wallet, as an easy-to-
validate derivative of the physical document. The use case is well-suited for UX study, because the user journey
spans over different applications, that is, the regiomove app and the wallet application, and also involves
interaction with personnel for video identification.
User-friendly Integration of Identity Wallets and Mobility Platforms
8
The system under study: A prototype and its variants
To discover potential UX issues and providing best practices for its implementation, the process described
above has been implemented as a prototype in figma. It simulates all essential steps of the user journey and
allows trials with human test subjects. As wallets are an important research focus of the SDIKA project, the
prototype has been developed in different variants, showcasing different means of authentication of users
towards the wallet.
The methodology: User-based evaluation
There exists a plethora of methods to evaluate the UX of a system, to discover issues and to make suggestions
on fixing them. The design of the study at hand focuses on user-based evaluation. As the system is in the
prototype stage, the study is conducted with human test subjects, acting as users of the simulated system.
The different variants of the prototype are mirrored in the study design: For each variant, a group of users was
put together from people with different age and level of experience.
User-friendly Integration of Identity Wallets and Mobility Platforms
9
1. Showcase Secure Digital Identities Karlsruhe (SDIKA)
The project ‘Showcase Secure Digital Identities Karlsruhe’ (short SDIKA) is nationwide one of four projects
funded by the German Federal Ministry of Economics and Climate Action (BMWK) as part of the showcase
program ‘Secure Digital Identities’ [1]. The SDIKA project pursues the goal of realizing far-reaching, cross-
application identities in open ecosystems. Technical, semantic and economic interoperability are intended to
promote the use of solutions that ensure high functionality, information security and digital sovereignty. The
building blocks o f the project build a triad of (1) development of the so called ‘SDI-X system’, (2) demonstration
and evaluation in the regional showcase, and (3) the establishment of a cross-regional ecosystem.
Individuals and organizations should be able to choose for themselves between identity solutions of different
types (in a cloud or locally on a mobile phone). Application-specific and self-sovereign electronic identities [2]
(for example, the electronic ID card) are to be supported. The purpose of the SDI-X system is to enable
organizations that need to handle digital identities to easily connect their respective systems to all identity
solutions available in the ecosystem. Technically, this is achieved via the so called ‘SDI-X adapter’ – a software
component that wraps important functions to interact with wallets and to issue new and verify presented
credentials.
The benefits of the approach are demonstrated to citizens and organizations in Karlsruhe and the
Metropolitan Region Rhine-Neckar (MRN) based on use cases from the areas of health, mobility, digital
planning and construction, digital urban society and e-government. The goal is to show a municipal ecosystem
of acceptance points and trust services [3], and make the use of digital identities tangible for residents in their
everyday lives [4].
Figure 1 shows the interconnected task areas of the SDIKA project.
Figure 1: Project overview of SDIKA
(Illustration from the project consortium)
User-friendly Integration of Identity Wallets and Mobility Platforms
10
2. Preliminaries
Chapter 2 describes the initial situation of this report and its underlying experiment. It begins with a problem
definition in Chapter 2.1. The goals are explained in Chapter 2.2. The structure of this contribution is clarified
in Chapter 2.3.
2.1. Problem definition
Current research [5], [6], [7], shows that existing digital identity wallets (hereafter called ‘wallets’ for short)
need to be further improved regarding UX. The test subjects involved in the UX study by Sartor et al. [5]
expressed that the wallet is too difficult to understand, e.g., no intuitive wording is used and no help options
are offered. They also criticized that there is no option to postpone credential offer to some later time and that
a short demo is missing when the wallet is started for the first time. Khayretdinova et al. [6] explored in their
UX study that users' mental models may not always correspond with those of developers well-versed in
technologies such as public key infrastructures and electronic signatures. Users frequently develop a distinct
understanding influenced by the hierarchical solutions they are accustomed to, leading to difficulties in
utilizing and managing credentials within a decentralized architecture. This challenge becomes particularly
evident when extending beyond the basic issuance and verification of credentials. Furthermore, important
aspects of the identity life-cycle, such as backup and recovery, as well as deleting credentials or entire
accounts, were a major challenge for the test subjects. The study of Kostic and Poikela [7] shows that test
subjects had problems with overly technical terms and QR codes. In addition, the test subjects expected the
wallet to make automatic backups and not have to perform them manually.
To summarize, UX needs to be improved for wallets – and as a consequence – UX improvement suggestions
and standardized design patterns need to be developed for different use cases.
2.2. Goals and Approach
In order to obtain UX improvement suggestions, a UX study of the SDIKA project's internal wallet1 prototype
was conducted based on a use case from the mobility domain. The prototype shows the interaction of a wallet
with a mobility platform, called regiomove2. In regiomove, a car-sharing provider called stadtmobil3 is to be
activated with the help of a verified driving license using a fictitious video identification procedure. The test
subjects were then asked to export their driving licenses from regiomove to their wallet. The aim of the UX
study is to include different user groups, e.g., people who are not IT-savvy or older people. The UX evaluation
results were to be obtained with the help of different UX evaluation methods, i.e. eye tracking [8], thinking
aloud [9] and two questionnaires (SUS [10] and UEQ-S [11]). In this way, both qualitative and quantitative
results can be obtained.
Finally, it was envisioned that UX evaluation results and improvement suggestions could be used for other
wallets and use cases.
1 developed by CAS Software AG, https://www.cas.de
2 regiomove is a mobility platform for seamless travel planning and access to transportation services within
the German city Karlsruhe, https://www.kvv.de/mobilitaet/regiomove.html
3 stadtmobil is a car sharing provider, https://www.karlsruhe.stadtmobil.de
User-friendly Integration of Identity Wallets and Mobility Platforms
11
2.3. Structure of this Report
The following Chapter 3 describes the methodology of the UX study. Firstly, Chapter 3.1 describes the
evaluation object and setup, in particular the evaluated prototype and the four different versions of it. In
Chapter 3.2, the study procedure and the UX evaluation methods called eye tracking, thinking aloud and two
questionnaires (SUS and UEQ-S) are then described. Chapter 3.3, explains the test subjects and their
properties. The evaluation results are then presented in Chapter 4. Firstly, the results obtained from thinking
aloud are described in Chapter 4.1, followed by those from eye tracking in Chapter 4.2 and from the
questionnaires (SUS and UEQ-S) in Chapter 4.3. The results of the evaluation methods are then synthesized in
Chapter 4.4. Chapter 5 describes the limitations that arose during the UX study. It begins with limitations due
to the laboratory setting in Chapter 5.1. The limitations of the testing device and the wallet prototype are
described in Chapter 5.2. The limitations of the fictitious data are explained in Chapter 5.3. The limitations of
the results explained in Chapter 5.1 to 5.3 are put together in Chapter 5.4. Finally, conclusions and future work
are described in Chapter 6. The individual prototype screens can be found in the appendix in Chapter 9.
User-friendly Integration of Identity Wallets and Mobility Platforms
12
3. Methodology
Chapter 3 describes the methodology of the UX evaluation. It begins with a description of the evaluation object
and setup in Chapter 3.1. The study procedure and evaluation methods are then described in Chapter 3.2,
followed by test subjects in Chapter 3.3.
3.1. Evaluation object and setup
The basis of the study was formed by a high-fidelity figma4 prototype consisting of two parts: (1) the activation
procedure and (2) the export procedure.
Part 1: Activation of the car-sharing provider stadtmobil
In part 1, the prototype encompassed to activate the car-sharing provider stadtmobil within the regiomove
app through a simulated video identification process. To do so, the test subjects (see Chapter 3.3) must open
the regiomove app and navigate to the account settings via the account icon in the right-hand corner of the
bottom navigation bar (see Figure 17). In the account settings, test subjects are required to select ‘Edit
account’ (see Figure 18). The lower part of the screens shows an overview of mobility providers available in
regiomove. At this point, test subjects are required to click on stadtmobil (see Figure 19). The following screen
(see Figure 20) explains the pathways through which the provider stadtmobil can be activated: Via video
identification process, import of a driver's license stored in a digital wallet, or physical presentation of the
driver's license at the stadtmobil office. Following the comprehension of the information, test subjects are
required to click on the ‘Continue’ button. Subsequently, personal identification (see Figure 21) and driving
license data (see Figure 22) must be entered, and consent to the privacy policy provided. According to the
actual situation, the test subjects are unable to initiate the video call required for driving license verification
themselves — they can only request it. This action is to be performed by clicking the ‘Request video call’ button
(see Figure 23). Subsequently, it is simulated that a person from the service provider's side is available to
accept the call. The fictitious call can be accepted by clicking the ‘Accept video call’ button (see Figure 24).
Following the video call, a dialogue box appears, notifying that the provider stadtmobil will be activated in a
few minutes (Figure 25) and that once activated, a further notification will be received. Upon completion of
the video identification process, part 1 of the prototype is finished.
Part 2: Export of the driving license credential to the wallet
In Part 2, a credential created by the video identification procedure within regiomove is to be exported to the
wallet, stating that the driver’s license has been checked and found valid. While the credential is not an actual
but a derivative of the driver’s license, it can be handled as such in different use cases. To this end, after a brief
simulated waiting period, a push notification will be received within the regiomove app, indicating that
stadtmobil is now activated (see Figure 26). This notification includes a text button prompting test subjects to
press it in order to access their newly created stadtmobil account. The second option to access the stadtmobil
account would be through the account settings, as described in part 1. The derived driving license is now
stored in the stadtmobil settings. Within the account settings, the test subjects must now click on the button
labeled ‘Export driving license’, which will open a selection dialogue in a style native to the Android mobile
operating system. From this dialogue, test subjects must select the wallet associated with the SDIKA
4 Figma is a collaborative interface design tool to create, prototype, and collaborate on digital designs,
https://www.figma.com
User-friendly Integration of Identity Wallets and Mobility Platforms
13
project(see Figure 27). A simulation of the wallet app opens and requires authentication in order to access its
functions. For this purpose, four variants of the wallet section of the prototype were created, each with a
different authentication method: 4-digit pin (see Figure 32), 6-digit pin (see Figure 33), password (see Figure
34) and fingerscan (see Figure 35). After authentication, the test subjects see a detailed overview of the driving
license credential within the wallet (see Figure 29). Below the credential details, there is an ‘Add’ button, which
the test subjects must click to import the driving license credential into their wallet. Clicking this button
triggers a pop-up, within which the test subjects must reconfirm the import of this credential (see Figure 30).
The prototype concludes with the imported driving license credential being visibly stored within the wallet
(see Figure 31).
The entire journey is illustrated in Figure 2.
Figure 2: User journey
As test setup, the wallet prototype operated on a Samsung Galaxy S22 running the Android operating system.
Throughout the study, the mobile device was mounted to the tobii Mobile Testing Accessory (MTA)5 to ensure
consistent eye tracking results. The tobii Pro Spark6 was used as eye tracker. Moreover, the test setup
described in Figure 3 included a camera and a microphone to capture facial expressions, gestures, and spoken
interactions.
Figure 3: Evaluation setup
5 https://www.tobii.com/products/accessories/mobile-testing-accessory
6 https://www.tobii.com/products/eye-trackers/screen-based/tobii-pro-spark
User-friendly Integration of Identity Wallets and Mobility Platforms
14
The mobile phone, camera, and eye tracker were connected to the PC, where the entire eye tracking process
could be monitored on the screen in the software of the eye tracker, called tobii Pro Lab7.
Figure 4 demonstrates the moderators’ view of the testing session.
Figure 4: Evaluation recordings
3.2. Study procedure and evaluation methods
For each group, their 6 members among the 24 test subjects described in Chapter 3.3 evaluated a prototype
variant described in Chapter 3.1. The variants differed only in the authentication method. As stated in Chapter
3.3, care was taken to ensure that the 6 test subjects in each group had similar demographic data.
At the start of the user test, each test subject was asked to sign a declaration of consent for data collection and
complete a questionnaire on their demographic data. The test subject was then given instructions on a paper
containing relevant information of the user test: The test subject was asked to put themselves in the shoes of
a fictitious person called Robert Glaser. Instead of a real video identification procedure, a fictitious still image
of a person appeared, which could also be skipped. There was no show or validation of a driving license. No
personal, re al data were supposed to be entered by the test subjects. Instead, the data was automatically filled
in the input fields with the data of the fictitious Robert Glaser. The individual tasks to be performed by the test
subject were also described.
Now the test subject should carry out part 1 of the prototype described in Chapter 3.1 using the instructions
on the paper and the UX evaluation method thinking aloud [9]. Before starting, thinking aloud was explained
to the test subject. Thinking Aloud involves test subjects evaluating the application and giving their feedback
verbally. The feedback shows how the test subjects understand the software and its functions. So, thinking
aloud can be used to collect positive and negative UX aspects of the prototype. The separate camera and
microphone record the thinking aloud.
After that, the test subject should carry out part 2 of the prototype described in Chapter 3.1. In part 2, eye
tracking [8] is used to evaluate the gaze detection of the test subject, in particular to answer predefined
research questions (RQ1-7) and test predefined hypotheses (H1-7):
7 https://www.tobii.com/products/software/behavior-research-software/tobii-pro-lab
User-friendly Integration of Identity Wallets and Mobility Platforms
15
RQ1: Will the test subject read the information text for exporting the driving license to the wallet? (see Figure
27)
H1: The test subject will read the information text.
RQ2: Will the test subject notice the info button for exporting the driving license? (see Figure 27)
H2: The test subject will notice the info button.
RQ3: In which order does the test subject view the information on the screen of the stadtmobil account? (see
Figure 27)
H3: The test subject will read the text linearly in the reading direction.
RQ4: Does the test subject notice that the issuer is not verified? (see Figure 29)
H4: The test subject will notice that the issuer is not verified.
RQ5: Does the test subject completely read the text about the issuer? (see Figure 29)
H5: The test subject will completely read the info text about the issuer.
RQ6: Does the test subject read the detail information of the credential? (see Figure 29)
H6: The test subject will read the detail information of the credential.
RQ7: Does the test subject directly see the red button to add the credential and click on it without reading the
rest of the information? (see Figure 29)
H7: The test subject will first look at all the information on the screen and will then press the button.
To calibrate the eye tracker, the test subject should first look at 6 different symbols on the mobile phone. Then,
the test subject starts with the second part 2 and carry out the tasks on the paper. As soon as the driving license
has been transferred to the wallet, eye tracking is stopped. The test subject is then asked for general feedback
on part 1 and 2.
Finally, the test subject completes two questionnaires System Usability Scale (SUS) [10] and UX Questionnaire
Short (UEQ-S) [11]. The SUS questionnaire evaluates specifically usability as an attribute of UX. The UEQ-S
questionnaire is not limited to just one UX attribute, but takes a broader view of UX.
At the end of the user test, the test subject receives 20 euros for participating and is bid farewell.
The results will then be compared and discussed for different user groups and for the various prototypes with
the different authentication methods.
The following Figure 5 shows the described study procedure with the evaluation methods.
Figure 5: User test process
User-friendly Integration of Identity Wallets and Mobility Platforms
16
3.3. Test subjects
A total of 24 persons took part as test subjects in this study, who were divided between the 4 different
authentication methods. When selecting and distributing the test subjects, a balanced age distribution
between the groups was considered. Further characteristics were collected in a demographic questionnaire,
aiming at conclusions regarding correlations between such characteristics and study results.
Of the 24 test subjects, 14 stated ‘male’ and 10 ‘female’ as their respective genders. As Figure 6 shows, 3 test
subjects described themselves as not IT-savvy, 9 as neutral, 8 as IT-savvy and 4 with strong IT-affinity.
Figure 6: IT-affinity of test subjects
Figure 7 shows statements regarding experience with regiomove and wallets: 10 test subjects stated that they
had no previous experiences with either, 5 had previous experience with regiomove (but not wallets), 3 with
wallets (but not regiomove), and 6 with both regiomove and wallets.
Figure 7: Experience of test subjects with regiomove and wallets
User-friendly Integration of Identity Wallets and Mobility Platforms
17
Figure 8 shows the age distribution of the test subjects. There were 3 test subjects under 20, 9 test subjects
between 20 and 29, 7 test subjects between 30 and 49, 1 test subject between 50 and 60 and 4 test subjects
over 60. In particular, one 80-year-old took part in the study.
Figure 8: Age of test subjects
Figure 9 shows how the test subjects were distributed among the authentication methods according to their
age. The distribution is relatively even and each group had one member aged 60 or older.
Figure 9: Test subjects age sorted by method
User-friendly Integration of Identity Wallets and Mobility Platforms
19
4. Results and discussion
Chapter 4 describes and discusses the results of the UX study. The results of the thinking aloud experiment
and the eye tracking experiment are explained in Chapter 4.1. The results of the two UX questionnaires are
explained in Chapter 4.3. Chapter 4.4 concludes with a synthesis of chapters 4.1 to 4.3.
4.1. Thinking aloud
As described in Chapter 3.2, the test subjects were asked to verbally express their thoughts and feelings
(“thinking aloud”) while carrying out the first part of the car sharing registration process shown in Chapter 3.1.
Using this open method of user feedback allows for gaining insights into the UX of the tested process that
might not have been thought about by the testers and thus not be part of other evaluation methods. In this
chapter, the most relevant and significant findings from this first evaluation method are described and
discussed, following the order of appearance of the according screen in the car sharing registration process.
Finally, a set of particularly relevant individual statements will be discussed.
In order to register for the car sharing provider stadtmobil in regiomove, the test subjects need to navigate to
the account menu (see Figure 18) and from there to the sub menu (see Figure 19) for editing personal account
data. Inside this sub menu, the buttons for adding a specific mobility provider – such as stadtmobil – can be
found, starting the registration process. In the user test, while a majority of 18 out of 24 test subjects expected
the start of the registration process in the account menu, only 7 out of 24 test subjects expected it to be located
in the described sub menu. The big difference between test subjects reaching the account menu and test
subjects reaching the correct sub menu to change personal account data suggests that the user flow would
benefit strongly from relocating the service to the first level of the account menu.
After the test subjects have chosen to add the car sharing provider, they are presented an information screen
(see Figure 20) for the car sharing registration process. On this screen, large icons are placed to indicate the
available registration methods. A vast majority of 21 out of 24 test subjects interpreted these icons as buttons
to start the registration process via the chosen method, where the options were (1) import driving license from
wallet, (2) verify driving license via video call, (3) verify driving license at the provider’s office. This strongly
indicates a problematic design of the screen element and the need to reconsider the arrangement of the
screen as a whole, as the screens interaction pattern did not meet the test subject’s expectations based on the
impression of the visual elements.
In the following screens (see Figure 21 and Figure 22), the test subjects were asked to fill in data from their ID
and driving license, or to import the demanded data, requiring the according documents - ID and driving
license - deposited in their wallet. For the user test, the test subjects were instructed to not use the import
feature, as no documents would be available in their wallet in this stage. However, 6 test subjects tried to
import documents from the wallet, while 4 reactions could not be interpreted. As the registration via wallet
import was explicitly not part of the tested process, the attempts to do so were counted as error. The rate of
at least 6 of the subjects trying to use the import button indicates potential to improve the wording and/or
styling of the according element, giving higher emphasis on the necessity of credentials already being saved
in the user’s wallet for this feature to work.
After the form data is sent, the next step is to choose the video call feature and perform the video call. Due to
technical constraints, test subjects could not instantly start a call, but were required to request a call (see
Figure 23) and then wait to be called (see Figure 24), while staying on the same screen. Despite this technical
User-friendly Integration of Identity Wallets and Mobility Platforms
20
limitation and the unusual user flow that comes along with it, the task of requesting and accepting the video
call (see Figure 24) was successfully performed by 19 test subjects. On the other hand, 8 test subjects were
irritated and 5 did not proceed at all without help by the team conducting the user test. The latter 5 test
subjects did not notice the incoming call, because they assumed choosing the call in the first step would
immediately start a call. As the user flow in this part differs from the usual process of actively starting a call,
the user guidance in this step needs to be laid out with special care, transparently communicating each event
of requesting and accepting an incoming call.
The dialog following the video call (see Figure 25) informs the test subjects that their car sharing account will
be activated in a few minutes and a push message will be sent to them when the activation is finished. Though
the dialog was interpreted correctly by a majority of 15 test subjects, 7 test subjects skipped the dialog without
reading and 2 read the dialog text misinterpreting its meaning. The resulting measure could be to shorten the
dialog text in order to allow for a quicker comprehension.
The aforementioned measure of reducing text load would also consider the findings that (1) longer texts were
often skipped by a significant share of test subjects and (2) five test subjects explicitly mentioned some kind
of problem with too much text throughout the process. Two test subjects expressed a general preference for
a visual interface characterized by fewer textual elements and a greater emphasis on intuitive icons. This result
underscores the significance of balancing textual information with visual cues to improve user satisfaction
within a complex wallet interaction process, at the same time ensuring transparency regarding the handling
of sensitive user data.
Regarding the style of the studied wallet app (see Figure 29), one test subject expressed concerns regarding
the connotations associated with red, typically interpreted as a signal or warning color. This feedback
highlights the importance of considering cultural and psychological associations when selecting color
schemes within digital interfaces in general, even more crucial when applied in a delicate use case dealing
with security matters. The importance of this conclusion is further supported by the finding that a majority of
the test subjects did not notice a warning hint placed in the driving license detail screen of the wallet (see
Figure 29). The hint indicates that the issuer for the credential is not verified, and as such, the credential itself
is potentially insecure. This warning is colored in a red hue, so it matches the primary color heavily used for
interaction elements on the screen, favoring a possible neglect.
There were also points regarding user feedback in the wallet: When importing the driving license, the
additional confirmation of the import by the pop-up is generally considered unnecessary (see Figure 30).
According to some test subjects, this is only necessary if the opposite site is classified as not trustworthy. In
addition, a success message was desired when the credential of the driver's license has been successfully
saved in the wallet (see Figure 31).
Additionally, test subject feedback highlighted the wish for a search bar within the account menu (see Figure
18) of regiomove. This observation emphasizes the user’s desire for enhanced navigation efficiency and
streamlined access to the car sharing registration feature. On the other hand, another test subject voiced
concerns about overlooking text prompts during the car sharing registration process due to the smooth user
flow. This observation suggests that while a smooth user flow without hurdles in the process is desirable, there
exists a delicate balance between fluidity and the visibility of informative content. One main task for designing
user flows dealing with identity processes is to prioritize both ease of use and effective communication,
necessitating careful consideration of user preferences and cognitive processes to optimize overall usability
and subjective as well as effective security matters.
User-friendly Integration of Identity Wallets and Mobility Platforms
21
4.2. Eye tracking
This Chapter explains the results of the eye tracking experiment and is structured by the research questions
from Chapter 2. For each research question, the results shown in Table 1 are explained and an answer is given
for each hypothesis. The test subjects describing themselves as ‘not IT-savvy’ or ‘neutral’ are combined into
‘not IT-savvy’, and test subjects who picked ‘IT-savvy’ or ‘strong IT-affinity’ are grouped into ‘IT-savvy’. At the
end, the results for the login times are discussed as given by Table 2.
RQ1: Will the test subject read the information text for exporting the driving license to the wallet?
H1: The information text is read by the test subject.
Overall: 7 read the information text, 10 did not, 2 saw or skimmed it and 5 did not show sufficient eye
tracking data to answer the research question. This shows that the hypothesis is incorrect for more
than half of the test subjects and is therefore falsified. This scan path (see Figure 37) is an example for
a test subject that read the information text, because it shows some fixations after each other on the
information text. When looking at the raw eye tracking data, it can be seen how the gaze jumps along
the lines.
IT-savvy: 3 read the information text, 4 did not, 1 skimmed it and 4 times not enough data.
Not IT-savvy: 4 read the information text, 6 did not, 1 saw it, 1 not enough data.
By age:
• Out of the 3 test subjects younger than 20 no one read the information text (see Figure 36).
• In the age range of 20-29, 4 out of 8 read the information text, 4 did not.
• In the age range of 30-49, 1 of 4 read the information text.
• In the age range above 50, 2 of 3 read the information text.
RQ2: Will the test subject notice the info button for exporting the driving license?
H2: The test subject will notice the info button.
Overall: 6 test subjects noticed the info button, 13 did not and for 5 test subjects there was not enough
data to answer the question.
IT-savvy: 3 notice the info button, 8 did not and 1 times not enough data.
Not IT-Savvy: 3 notice the info button, 5 did not and 4 times not enough data.
By age:
• No test subjects of less than 20 years noticed the info button, which can be seen on the
corresponding heatmap (see Figure 36) where there is no color on the info button.
• In comparison the heatmap for test subjects above 60 (Figure 38) shows a colored dot at the
info button that is clearly separate from the colored mass on the export button.
• In From the 20-29 only 2 noticed the info button, while 7 did not see it. For the other age
groups the number of test subjects noticing the info button is balanced with the ones not
noticing the test button.
User-friendly Integration of Identity Wallets and Mobility Platforms
22
RQ3: In which order does the test subject view the information on the screen of the stadtmobil account?
H3: The test subject reads the text linearly in the reading direction.
Overall: 12 test subjects started in the middle of the screen, 8 at the top and 1 at the bottom. For 3 test
subjects there was not enough data collected. Out of the ones starting in the middle 6 found the
targeted button directly without scanning the rest of the screen and going on with the flow. The ones
starting on the top of the screen, moving on to the middle, where 2 clicked the button without ever
looking at the bottom of the screen. The others moved on to the bottom of the screen with 4 test
subjects going back to the middle to move on while the last 2 test subjects went back to the top of the
screen and clicked the button in the middle afterwards. The test subjects starting on the top of the
screen more likely looked at everything on the screen before clicking on the button they are searching
for. One test subject started on the bottom of the screen and moved on to the middle part where they
clicked the button to export the credential.
IT-savvy and not IT-savvy: The scan paths of IT-savvy and not IT-savvy test subjects look similar
without special behavior.
By age:
• 20-29-year olds tend to directly find and click on the button they are looking for, without
scanning the rest of the screen.
• 30-49-year olds started on the top of the screen.
RQ4: Does the test subject notice that the issuer is not verified?
H4: The test subject notices that the issuer is not verified.
Overall: 5 test subjects read that the issuer is not verified while 14 did not. 3 test subjects saw the
notification without processing it and for 2 test subjects there was no eye tracking data answering the
research question. Therefore, the hypothesis is wrong because there were nearly 3 times the test
subjects that missed the notification than test subjects that read it, even though the heatmap of the
screen (see Figure 39) suggests that the notification is read by most test subjects. This shows that
heatmaps are not reliable for telling if something is really processed, just looked at or skimmed.
IT-savvy and not IT-savvy: For IT-savvy and non-IT-savvy test subjects, the distribution is the same as
described in the overall results.
By age:
• 20-49-year olds were more likely to miss the notification.
• Test subjects below 20 and over 50 more read the notification than there were test subjects
not reading it.
RQ5: Does the test subject completely read the text about the issuer?
H5: The test subject completely reads the info text about the issuer.
Overall: The text about the issuer was read by 11 test subjects. 9 test subjects did not read the issuer
text at all while 2 skimmed it. For 2 test subjects there was not enough data to file an answer to the
research question.
User-friendly Integration of Identity Wallets and Mobility Platforms
23
IT-savvy: The IT-savvy test subjects seemed to be more prone to reading the text because 6 read it, 2
hovered over it and 4 did not read it at all.
Not IT-savvy: 5 test subjects read the issuer text and 5 did not read it.
By age:
• 2 test subjects below 20 read the issuer text and 1 skimmed it.
• 4 test subjects of the 20-29-year olds read the text and 4 did not.
• 2 test subjects of the 30-49 read the text while 4 did not and 1 skimmed it.
• Test subjects over 50 read the text except for 1.
RQ6: Does the test subject read the detail information of the credential?
H6: The test subject reads the detail information of the credential.
Overall: The detailed information of the credential was read by 9 test subjects and 5 skimmed it,
whereas 7 did not read it and for 3 test subjects there was no eye tracking data to answer this question.
IT-savvy: 4 read the detailed information, 3 skimmed it and 5 did not read it at all.
Not IT-savvy: Only 2 did not read the information at all. 5 read the credential information and 2
skimmed it.
By age:
• Every test subject over 50 read the detailed information of the credential
• 4 test subjects of the 20-29-year olds read the text and 4 did not.
• Test subjects below 20 skimmed it at least
• For the other test subjects its balanced between not reading at all and read plus skimmed.
RQ7: Does the test subject directly see the red button to add the credential and click on it without reading
the rest of the information?
H7: The test subject first looks at all the information on the screen and then presses the button.
Overall: Of all the test subjects, only 2 looked directly at the red button and no test subject pressed
the red button without first looking at other content on the screen. Therefore, the hypothesis is not
completely wrong, even if the test subjects tend to look at the parts that seem interesting to them and
not necessarily at the complete information on the screen. This statement can be demonstrated by
this scan path (see Figure 40), which shows that many of the first fixations on the screen are in the
center of the display and not on the red button, and the heatmap of the screen(see Figure 39) shows
that all important information is looked at on the screen. It also became clear that people remain on
the screen where they left off on the last screen and tend to move upwards if no relevant information
is found.
For the login times, the time spent by the test person on the screen of the authentication method was used
(see Table 2). The very long time required for the 4 pin is striking and almost twice as long as the longest time
required for the password. This value was therefore classified as an outlier and a separate evaluation was
created for the 4 Pin without this outlier.
User-friendly Integration of Identity Wallets and Mobility Platforms
24
The comparison clearly shows that the password and 6-pin take significantly longer than the 4-pin and finger
scan. The password takes almost twice as long as the 6-pin. This provides the UX with a ranking of the
authentication methods based on the time required. Finger scan and 4-pin are clearly preferable and a 6 pin
is still preferable to a password
4.3. Questionnaires: SUS and UEQ-S
The third UX evaluation method applied are two standard questionnaires commonly used in UX research: SUS
[10] and UEQ-S [11], as described in Chapter 3.2.
The overall score of SUS is 64 of 100 with single results, with a large deviation range from 2,5/100 up to
97,5/100, shown in Figure 11.
Figure 11: SUS score per test subject
An analysis based on age cohorts shows a tendency for the score to lower with growing age, with a robust
plateau in the age cohorts from 20 to 60 years, with a score between approximately 60 and 70. As pictured in
Figure 12, the youngest age cohort shows an exceptionally positive median score of 79, while the oldest age
cohort (>60 years) yields results far below the others with a score of 41.
User-friendly Integration of Identity Wallets and Mobility Platforms
25
Figure 12: SUS scores by age cohorts
As for the UEQ-S, the overall score median of all test subjects is 0,39, with the pragmatic score being 0,71 and
a hedonic score of 0,06. Figure 13 reveals an again large deviation range in the results, reaching from -1,88 as
the lowest up to +1,88 as the highest score and most positive user feedback. These results indicate a sub-par
overall Usability [13], especially in terms of joy and engagement while performing the task. The higher
pragmatic score shows that despite the lack of joy in performing the task, the underlying functionality is
appreciated by the test subject group.
Figure 13: UEQ-S scores by test subject
Similar to the SUS score, the results of the UEQ-S questionnaire tend to decrease with growing age of the test
subject, as shown in Figure 14. This validates the overall tendency that older users have a less than ideal
experience using the application, indicating the necessity to take care of the specific needs of this user group
when designing digital processes.
User-friendly Integration of Identity Wallets and Mobility Platforms
26
Figure 14: UEQ-S scores by age cohort
As Figure 15, analyzing UEQ-S and SUS results by authentication method reveals a slight disadvantage of the
UX when authenticating with a password. While this conclusion may seem connected to the difficulty
performing the authentication methods – ty ping in an alphanumeric, case sens itive, multi-character password
with special characters is more complicated than, for example, putting a thumb onto the screen for a
fingerprint authentication – a closer look at the UEQ-S results in Figure 16 questions this conclusion: The
fingerprint scan is rated best in pragmatic quality but rated worst in terms of hedonic quality. This kind of
anomaly suggests a statistic distortion, caused by the relatively small sample size and the cross influence of
factors outside the authentication method test part, like differences in the experience of test subjects
regarding the whole registration process.
Figure 15: SUS scores by authent ication method
User-friendly Integration of Identity Wallets and Mobility Platforms
27
Figure 16: UEQ-S scores by authentication method
The large deviation range in the results of both questionnaires indicates a very heterogeneous UX amongst the
test subject group, hence the results should be used with caution. Conclusions regarding the authentication
methods assigned to different user groups can hardly be drawn from the questionnaire results, because (1)
the questionnaires refer to the whole prototype, of which the authentication method is only a small part, and
(2) the respective user groups were too small for a significant resolution.
4.4. Synthesis of the evaluation results
The following list is a synthesis of all evaluation results as presented in Chapter 4.1to 4.3.
– The stadtmobil activation is not easy to find and is not assumed to be in the sub menu ‘edit account’
(see Figure 18), but rather one level higher in the account menu (see Figure 19).
– On the information screen of the car sharing registration process (see Figure 20), the icons of the
activation options were interpreted as buttons instead of icons.
– The test subjects were initially told to start the user test without credentials in the wallet. The
activation options of stadtmobil are (1) import driving license from wallet, (2) verify driving license via
video call and (3) verify driving license at the provider’s office. The test subjects were supposed to
select option (2) and subsequently enter the data of their ID card (see Figure 21) and driving license
(see Figure 22) manually. However, several attempts were made to select option (1), which indicates
that the design and wording should be adapted and perhaps the concept of the wallet was not
properly understood.
– The concept of the wallet was not properly understood several times, especially by non-IT-savvy test
subjects. More detailed explanations and help are needed before and during use. However, it should
be noted that overly detailed explanations and help can be annoying for IT-savvy users. It would be
conceivable to ask users to what extent they understand the concept of the wallet before using it for
the first time and then offer more or less help accordingly. This would also be conceivable for older
users in particular, which could improve the poor SUS and UEQ-S scores for over 60-year-olds.
User-friendly Integration of Identity Wallets and Mobility Platforms
28
– Due to technical limitations, a video call must first be requested by pressing a button (see Figure 23)
and then accepted again by pressing a button (see Figure 24). Pressing the button twice sometimes
led to confusion.
– After the video call, stadtmobil is not activated immediately, but a message appears stating that the
video call was successful (see Figure 25) and that users will be notified when stadtmobil has been
activated (see Figure 26). A few test subjects closed this message immediately. The reason could be
the large amount of text, which should be summarized and displayed more compact. In general,
longer texts were often skipped by a significant number of users and some explicitly mentioned a
problem with too much text throughout the process. The results underscore the significance of
balancing textual information with visual cues to improve UX within a complex wallet interaction
process, at the same time ensuring transparency regarding the handling of sensitive user data.
– The wallet uses a red color design (see Figure 29) which raises concerns about the connotations
associated with the color red, as it is usually interpreted as a signal or warning color. This feedback
underscores the significance of considering cultural and psychological connotations when choosing
color palettes for digital interfaces , particularly in sensitive contexts. Althou gh red can signify u rgency
and capture users' attention, excessive or inappropriate usage could desensitize users to certain color
ranges or unintentionally cause alarm.
– The security notice ‘contact is not verified’ displayed during the credential import into the wallet, was
often ignored (see Figure 29). This was also measured again objectively during eye tracking. Security-
critical information should be clearly displayed and the design should prevent secur ity-critical errors.
– The additional confirmation of a credential import by the pop-up is generally considered unnecessary
(see Figure 30). It only makes sense if the opposite site has been classified as untrustworthy. In
general, there were also comments that user feedback was missing, such as when a credential was
successfully saved in the wallet (see Figure 31).
– Desire for a search function was expressed, as well as for shortcuts to simplify finding of and
navigating to functions.
– The SUS and UEQ-S scores are worst for the prototype variant with the password as the
authentication method, suggesting that other authentication methods should be preferred.
User-friendly Integration of Identity Wallets and Mobility Platforms
29
5. Limitations
Chapter 5 describes the limitations of the UX study. Starting with limitations due to the laboratory setting
(Chapter 5.1), it continues with limitations of the testing device and the wallet prototype (Chapter 5.2) as well
as the limitations of the fictitious data (Chapter 5.3. Finally, the limitations of Chapter 5.1 to 5.3 are synthesized
in Chapter 5.4.
5.1. Laboratory setting
An artificial testing environment creates a certain distortion concerning how individuals would otherwise
behave in a real-world setting. Environmental factors such as lighting or noise levels can influence the
behavior of test subjects, for example in terms of concentration.
Besides, social influencing factors , such as the ability to ask questions at any time or receive prompts at certain
points, arise from the presence of moderators during a testing session. Moderators also impact test subjects
in this respect as test subjects are aware of being observed, which can generate uncertainties on one hand,
while it simultaneously provides a sense of security on the other hand, knowing that no serious mistakes could
be made.
One test subject explicitly stated that the examination would have been conducted with more care and
attention to detail at home. Furthermore, it was noted by another individual that the presence of a moderator
contributed to an overall perception of safety throughout the process.
5.2. Testing device and wallet prototype
Furthermore, all testing sessions were conducted using the Samsung Galaxy S22 and a prototype tailored to
the Android operating system. The use of a potentially unfamiliar mobile phone and operating system can lead
to deviations in the results.
No actual wallet was installed on the test device, instead, a figma prototype of the wallet was used. Using a
prototype instead of an actual product introduces limitations related to the explorative scope of the tested
apps. The prototype was limited to the functions of the regiomove app pertinent to this study, namely, the
activation of stadtmobil and the exporting of the driving license credential to a wallet. Furthermore, the
section involving interaction with the wallet only included the requisite screens.
Additionally, only the necessary clickable areas for the process were outlined with hotspots in the prototype,
preventing test subjects from becoming disoriented. To still introduce a degree of realism into the process,
the automatic graphical indication of clickable areas by figma was deactivated.
While prototypes can be designed to such a level that users may not realize they are not interacting with a real
application, some aspects cannot be replicated with a figma prototype. These include real-time interactions
with individual technical components of the process, such as in this case, the video call and interactions with
the wallet.
In addition to that, the prototype mirrored ideal scenarios, meaning that there were no technical issues
encountered during the video call or when exporting the driving license to the wallet.
User-friendly Integration of Identity Wallets and Mobility Platforms
30
5.3. Fictitious personal data
To navigate the prototype, test subjects required a set of personal data, including:
– first name
– last name
– place of birth
– ID card number
– driving license class
– driving license ID number
– expiration date of the driving license
The utilization of fictitious data necessitated test subjects to identify themselves as ‘Robert Glaser’ during the
testing session and manage their fictitious personal data. Both factors could foster a casual attitude toward
managing personal data, which might have been more prudent under real circumstances when using their real
personal data.
Moreover, the prototype was configured so that the personal data was not required to be entered manually.
Consequently, test subjects were unable to make typographical errors and were also relieved of the obligation
to verify data inputs.
Furthermore, in those parts of the prototype where data transmission occurs, test subjects could not perform
a realistic data verification, as they were not dealing with their own data.
According to assertions made by certain test subjects, their awareness of the inability to induce significant
harm may conceivably impact their extent of exploration throughout the prototype's execution.
5.4. Synthesis of the limitations
The mentioned limitations collectively underscore reasons why the results of this study should be approached
with care. However, the challenges inherent in laboratory settings and the use of a prototype might be
disregarded when considering both the efficiency of laboratory tests and the benefits of user testing
conducted before development.
The utilization of fictitious personal data and the presence of moderators represent the most significant
limitations. It is noteworthy, however, that conducting user tests with genuine personal data raises critical
privacy concerns.
Regarding the moderators, efforts were made to minimize their influence as much as possible. Nevertheless,
residual influence stemming from a moderator's presence remains.
User-friendly Integration of Identity Wallets and Mobility Platforms
31
6. Conclusions and future work
Existing wallets currently have UX problems that need to be solved. In order to find suggestions for
improvement, a UX evaluation was carried out with the existing wallet in the SDIKA project. To this end, 24 test
subjects were invited to take part in the UX study. Test subjects had different demographic data, such as
different IT affinity, age and gender. The test subjects were asked to test a prototype on a mobile phone. In
Part 1, the test subjects were asked to activate a car-sharing provider called stadtmobil in the mobility app
regiomove. For this purpose, they had to complete a fictitious video identification procedure with a fictitious
driving license. They were asked to use thinking aloud to collect positive and negative UX aspects. In Part 2,
the test subjects were asked to export the driving license from regiomove to their wallet. Eye tracking was used
to evaluate the gaze detection of the test subjects. Finally, the test subjects were asked to complete two
questionnaires (SUS and UEQ-S).
In the future, the results of this study will be used to improve the regiomove app and influence the wallet
implementation in the SDIKA project. The results promise to be also applicable to improve other existing
wallets. In addition, various UX guidelines, so-called heuristics (e.g., heuristics of Nielsen [12]), will be
developed on the basis of the results. Furthermore, a method developed in the SDIKA project, will be used to
evaluate the interactions between UX and information security and to develop suggestions for improvement,
with the results of this UX study serving as input.
User-friendly Integration of Identity Wallets and Mobility Platforms
32
7. Acknowledgments
This report is a result of the project ‘SDIKA - Showcase Secure Digital Identities Karlsruhe’. The goal of the
project is to use digital identities to connect people, organizations, and processes. The values of digital
sovereignty, fairness, and interoperability are guiding principles of the project and for the regional showcase.
This Project is supported by the Federal Ministry for Economic Affairs and Climate Action (BMWK) on the basis
of a decision by the German Bundestag.
We would also like to thank CAS Software AG for kindly providing us with wallet screens for the prototype.
User-friendly Integration of Identity Wallets and Mobility Platforms
33
8. References
[1] “Showcase programme ‘Secure Digital Identities.’” Accessed: May 02, 2023. [Online]. Available:
https://www.digitale-
technologien.de/DT/Navigation/EN/ProgrammeProjekte/AktuelleTechnologieprogramme/Sichere_Dig
itale_Identitaeten/sichere_digitale_ident.html
[2] M. Kubach, C. H. Schunck, R. Sellung, and H. Roßnagel, “Self-sovereign and Decentralized identity as the
future of identity management?,” 2020, doi: 10.18420/OIS2020_03.
[3] S. Schwalm, D. Albrecht, and I. Alamillo, “eIDAS 2.0: Challenges, perspectives and proposals to avoid
contradictions between eIDAS 2.0 and SSI,” 2022, doi: 10.18420/OID2022_05.
[4] “SDIKA,” SDIKA. Accessed: Nov. 02, 2022. [Online]. Available: https://www.sdika.de/
[5] S. Sartor, J. Sedlmeir, A. Rieger, and T. Roth, “Love at First Sight? A User Experience Study of Self-
Sovereign Identity Wallets,” in Proceedings of the 30th European Conference on Information Systems -
New Horizons in Digitally United Societies, Timisoara, Rumänien, 2022.
[6] A. Khayretdinova, M. Kubach, R. Sellung, and H. Roßnagel, “Conducting a Usability Evaluation of
Decentralized Identity Management Solutions,” in Selbstbestimmung, Privatheit und Datenschutz, M.
Friedewald, M. Kreutzer, and M. Hansen, Eds., in DuD-Fachbeiträge. , Wiesbaden: Springer Fachmedien
Wiesbaden, 2022, pp. 389–406. doi: 10.1007/978-3-658-33306-5_19.
[7] S. Kostic and S. Poikela, “Do Users Want To Use Digital Identities? A Study Of A Concept Of An Identity
Wallet,” in Proceedings of the 8th Symposium on Usable Privacy and Security (SOUPS 2022), Boston, 2022.
[8] A. Bojko, “Eye Tracking in User Experience Testing: How to Make the Most of It,” Proceedings of the 14th
Annual Conference of the Usability Professionals’ Association (UPA). Montréal, Canada, 2005.
[9] T. Boren and J. Ramey, “Thinking aloud: Reconciling theory and practice,” Professional Communication,
IEEE Transactions on, vol. 43, pp. 261–278, Oct. 2000, doi: 10.1109/47.867942.
[10] J. Brooke, “SUS: A quick and dirty usability scale,” Usability Evaluation In Industry, 1995.
[11] M. Schrepp, A. Hinderks, and J. Thomaschewski, “Design and Evaluation of a Short Version of the User
Experience Questionnaire (UEQ-S),” International Journal of Interactive Multimedia and Artificial
Intelligence, vol. 4, 2017.
[12] J. Nielsen, “Enhancing the explanatory power of usability heuristics,” in Proceedings of the SIGCHI
conference on Human factors in computing systems celebrating interdependence - CHI ’94, Boston,
Massachusetts, United States: ACM Press, 1994, pp. 152–158. doi: 10.1145/191666.191729.
User-friendly Integration of Identity Wallets and Mobility Platforms
34
9. Appendix
Figure 17: regiomove, home screen
Figure 18: regiomove, account
Figure 19: regiomove, account
settings
Figure 20: regiomove, activate
stadtmobil
Figure 21: regiomove , ID card
details
Figure 22: regiomove, driving
license ID number
User-friendly Integration of Identity Wallets and Mobility Platforms
35
Figure 23: regiomove, request
videocall
Figure 24: regiomove, accept
videocall
Figure 25: regiomove, activation
dialogue
Figure 26: regiomove, push
notification
Figure 27: regiom ove, stadtmobil
account
Figure 28: regiomove, open with
wallet
User-friendly Integration of Identity Wallets and Mobility Platforms
36
Figure 29: Wallet, credential
details
Figure 30: Wallet, add to wallet
Figure 31: Wallet, stored credential
Figure 32: Authentication with 4-
digit-Pin
Figure 33: Authentication with 6-
digit-Pin
Figure 34: Authentication with
password
User-friendly Integration of Identity Wallets and Mobility Platforms
37
Figure 35: Authentication with
fingerscan
Figure 36: Scan path of stadtmobil
account screen
Figure 37: Scan path of stadtmobil
account screen, age 50-60
Figure 38: Heatmap of stadtmobil
account screen, age > 60
Figure 39: Heatmap of import
credential
Figure 40: Scan path of import
credential
User-friendly Integration of Identity Wallets and Mobility Platforms
38
Table 1: Eye Tracking results
Table 2: Login times
User-friendly Integration of Identity Wallets and Mobility Platforms
39
Authors
Max Sauer is research scientist at the FZI Forschungszentrum Informatik in
Karlsruhe in the research division “Software Engineering”. In addition, he is PhD
student at the Karlsruhe Institute of Technology (KIT), in the research group
Information Systems. His present research field relates to the conflict area of UX
and information security.
email: sauer@fzi.de
Simon Pfeifer is research assistant at the FZI Forschungszentrum Informatik in
Karlsruhe in the research division “Software Engineering”. He is a computer
science master student at the Karlsruhe Institute of Technology (KIT) and
specializes in software engineering.
email: pfeifer@fzi.de
Jan Sürmeli is Senior Expert “Digital Identity and Trust” at the FZI
Forschungszentrum Informatik in the research division “Software Engineering”,
and local project lead for the “Showcase Secure Digital Identities Karlsruhe”
(SDIKA). His interests as a post-doctoral researcher evolve around the modeling
and analysis of distributed systems, where he focuses on topics such as identity,
privacy and interoperability.
email: suermeli@fzi.de
Elias Siebert is a UX/UI designer at raumobil GmbH. With a diploma in
communication design and a versatile design background, he specializes in
finding solutions at the intersection of UX, strategic product development
and technical requirements.
email: e.siebert@raumobil.com
Isabelle Woytal is a UX/UI designer at raumobil GmbH, holding a master's
degree in Media and Communication. Her work encompasses the entirety of
user-centered design practices, from conducting user research and usability
testing to conceptualizing and designing user interfaces, with a strong emphasis
on accessibility.
email: i.woytal@raumobil.com
User-friendly Integration of Identity Wallets and Mobility Platforms
40
Imprint
Editor
FZI Forschungszentrum Informatik
Haid-und-Neu-Str. 10-14
76131 Karlsruhe
+49 721 9654-0
fzi@fzi.de
www.fzi.de
raumobil GmbH
Auer Straße 19
76227 Karlsruhe
+49 721-6607245
info@raumobil.com
www.raumobil.com
The editor's work is licensed under the Creative
Commons Attribution 4.0 International license (CC BY
4.0). You can read the license term
s here:
http://creativecommons.org/licenses/by/4.0
