PreprintPDF Available

se-atlas: Mapping of Health Care Providers for People With Rare Diseases - a Cross-sectional Survey From the User Perspective

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

Background In rare diseases, only a low number of regionally distributed experts are available in medical care. The health service platform for rare diseases (www.se-atlas.de) provides a search for health care providers and patient organisations in Germany for specific rare diseases and presents the results to patients or physicians. The objective of this study was to examine the background and purpose, user satisfaction and usability when using se-atlas and to receive suggestions on improvements for implementation in the next release of se-atlas. Methods We conducted an online survey over a total period of five weeks between December 2020 and January 2021. Participants were members of patient organisations of rare diseases and experts of rare diseases centres in Germany. The questionnaire addressed the objectives of this study in 10 questions. We used Likert scales (4 to 6 points), as well as the System Usability Scale to measure usability (range: 0 to 100). The data obtained from the survey was analysed descriptively. Results In total, 55 participants completed the survey (16 experts and 39 members of patient organisations). The results demonstrate that users know se-atlas mainly through patient organisations and the German National Action League for People with Rare Diseases. Furthermore, the experts use se-atlas more frequently than members of patient organisations do. Regarding to user satisfaction, participants were satisfied when using se-atlas (scale 1-6, mean = 4.31, SD = 1.18). They rated se-atlas functions with an average between 3.82 and 4.4 (scale 1-6). Additionally, se-atlas functions were considered as important with an average between 3.11 and 3.75 (scale 1-4). With regard to usability, the website was rated with an overall SUS score of 67.1, whereas the results differ between the participants group (experts = 76.1, patient organisations = 63.1). Moreover, participants made suggestions, e.g. that more disease entries should be available and usability can be improved. Conclusions This study involved experts and members of patient organisations to assess the background and purpose, user satisfaction and usability when using se-atlas. Despite the promising results and first new implementations, further optimisations of the platform in terms of usability and various functionalities are necessary.
Content may be subject to copyright.
Page 1/17
se-atlas: Mapping of Health Care Providers for People
With Rare Diseases - a Cross-sectional Survey From the
User Perspective
Jannik Schaaf ( jannik.schaaf@kgu.de )
Medical Informatics Group, University Hospital Frankfurt, Goethe University Frankfurt https://orcid.org/0000-
0002-0058-155X
Michaela Neff
Goethe University Frankfurt: Goethe-Universitat Frankfurt am Main
Manuela Till
Goethe University Frankfurt: Goethe-Universitat Frankfurt am Main
Niels Tegtbauer
Goethe University Frankfurt: Goethe-Universitat Frankfurt am Main
Holger Storf
Goethe University Frankfurt: Goethe-Universitat Frankfurt am Main
Research Article
Keywords: Rare Diseases, Health Personnel, Usability, Survey
DOI: https://doi.org/10.21203/rs.3.rs-829482/v1
License: This work is licensed under a Creative Commons Attribution 4.0 International License.  Read Full
License
Page 2/17
Abstract
Background
In rare diseases, only a low number of regionally distributed experts are available in medical care. The health service
platform for rare diseases (www.se-atlas.de) provides a search for health care providers and patient organisations
in Germany for specic rare diseases and presents the results to patients or physicians. The objective of this study
was to examine the background and purpose, user satisfaction and usability when using se-atlas and to receive
suggestions on improvements for implementation in the next release of se-atlas.
Methods
We conducted an online survey over a total period of ve weeks between December 2020 and January 2021.
Participants were members of patient organisations of rare diseases and experts of rare diseases centres in
Germany. The questionnaire addressed the objectives of this study in 10 questions. We used Likert scales (4 to 6
points), as well as the System Usability Scale to measure usability (range: 0 to 100). The data obtained from the
survey was analysed descriptively.
Results
In total, 55 participants completed the survey (16 experts and 39 members of patient organisations). The results
demonstrate that users know se-atlas mainly through patient organisations and the German National Action League
for People with Rare Diseases. Furthermore, the experts use se-atlas more frequently than members of patient
organisations do. Regarding to user satisfaction, participants were satised when using se-atlas (scale 1-6,
mean=4.31, SD=1.18). They rated se-atlas functions with an average between 3.82 and 4.4 (scale 1-6).
Additionally, se-atlas functions were considered as important with an average between 3.11 and 3.75 (scale 1-4).
With regard to usability, the website was rated with an overall SUS score of 67.1, whereas the results differ between
the participants group (experts=76.1, patient organisations=63.1). Moreover, participants made suggestions, e.g.
that more disease entries should be available and usability can be improved.
Conclusions
This study involved experts and members of patient organisations to assess the background and purpose, user
satisfaction and usability when using se-atlas. Despite the promising results and rst new implementations, further
optimisations of the platform in terms of usability and various functionalities are necessary.
1. Background
Rare diseases (RDs) are diseases that affect only a small proportion of the population. In the European Union (EU),
a disease is dened as rare if it affects less than one in 2.000 people. This corresponds to about 0.05 % of the
population. These diseases often lead to a chronic manifestation and considerable restriction of quality of life [1–
3]. In the literature, different causes for RDs are discussed. Most of the disorders are of genetic origin (71.9 %),
resulting in frequent affected children (69.9 %) [4]. However, other causes like infectious, immunological or
environmental factors are observed [5–7].
Besides the low prevalence of RDs, specialised experts are also rare and regionally distributed. Thus, affected
persons are depended on health care providers (HCPs), like university hospitals or medical-centres, that are often far
Page 3/17
away from their own residence. In addition, many affected persons and physicians have insucient information
about treatment options or specialised care facilities [8, 9]. Therefore, the platform „se-atlas“ was developed for the
purpose of combining existing collections of HCPs that diagnose and treat RDs and present them in a user-friendly
and transparent way. The project was part of the German National Action League for People with Rare Diseases
(NAMSE) within the national strategy for RDs in Germany. NAMSE was funded by the German Federal Ministry of
Health and included 52 different projects [8, 10].
Since se-atlas was published in February 2015, patients with RDs and their relatives, physicians, non-medical staff
and people with a general interest are able to get an overview of HCPs and patient organisations regarding to RDs in
Germany on the website www.se-atlas.de. For instance, a person affected by a RD can enter the name of the disease
in a exible search and subsequently receive all results on existing HCPs, including specialized rare diseases centres
(RDCs) and patient organisations in Germany. Furthermore, se-atlas displays information on research networks
such as the European Reference Networks (ERNs) and medical societies [8, 11]. The search for HCPs and patient
organisations is the central function of se-atlas. The search results for a disease are displayed geographically on a
map as well as in a list. By clicking on a HCP, the user is derived to its detailed presentation, which provides a
description of the HCP, as well as further contact information like address or consultation hours [8]. Furthermore, se-
atlas supports users in their individual research of RDs.
The information about all institutions that are provided in se-atlas are collected in different ways. Generally,
information is integrated from various existing data sources. For example, medical societies or patient
organisations can recommend a HCP who is an expert for certain RDs. In addition, experts can also enter
themselves into se-atlas. In any case, the data is only released after a comprehensive review by the se-atlas editorial
team, which has a medical background and is responsible for the quality assurance of the data. Entries will only be
releases after the editorial team has validated the entries with other source [8, 11].
However, not only the data quality is an important factor for the success of health information websites like se-
atlas. User interfaces and the presentation of the respective information contributes to the long-term success of
such a website. Outdated user interface designs and poor presentation can decrease the success and the website is
not used or recommended to others [12, 13]. To describe the effectiveness, eciency and user satisfaction of a
software-system, the term usability is often used. To increase usability, it is necessary to understand the background
and purpose of the user, whether the user receives the needed information and satisfaction rate with the system
[12]. Furthermore, the participation of the user during the implementation is important to develop a meaningful and
useful system [14]. However, there are only few studies available that assess the usability of health information
websites [15–17].
Since se-atlas was launched 6 years ago, it was visited more than 300.000 times. Therefore, it is necessary and
essential to continuously develop the system and to involve users in this process. We performed an evaluation
study of se-atlas to get feedback and requirements for further development of the website. The objective of this
study was to investigate the background and purpose, as well as user satisfaction when using se-atlas.
Furthermore, we evaluated how high the users assess the usability of se-atlas and derived recommendations for
optimization. Finally, concrete implementation changes for the new release of se-atlas were created and
implemented.
2. Methods
Page 4/17
2.1 Design
To address our objectives, we performed a cross-sectional survey. This quantitative design was chosen to gather
relevant information, opinions and needs of all stakeholders. We applied the STROBE reporting guideline
(Strengthening the Reporting of Observational Studies in Epidemiology) to report this study [18]. We considered 20
of 22 STROBE items. A checklist is provided in additional le 1.
2.2 Setting and sampling
The target group was selected according to their representation of the se-atlas user group: (a) experts, working in
RDCs, and (b) members of patient organisations, which includes patients and their relatives. In Germany, RDCs are
part of medical centres (e.g. university hospitals) and responsible for diagnosis, treatment and research for RDs
[10].
According to Meuser and Nagel, “experts” have a special knowledge and experience in their working or research area
[19]. We considered the following characteristics: Participants are working in a RDC. They have a completed medical
degree, as well as a specialist training in human medicine. Based on these characteristics, 33 potential study
participants were identied, since 33 RDCs are established in Germany [10].
The group of members of patient organisations were recruited with the help of the German National Alliance for
Chronic Rare Diseases (ACHSE e.V.), which is the head organisation of all patient organisations for RDs in Germany
[20].
The survey was conducted with a questionnaire using the online survey application LimeSurvey over a total period
of ve weeks between December 2020 and January 2021 [21]. For recruitment, the experts group was contacted by
e-mail. If there was no reply to the e-mail or the survey was not successfully lled within two weeks, the participants
were contacted again. Members of patient organisations were invited by ACHSE via e-mail two times. The members
were asked to complete the survey, if they have already used se-atlas in the past. A study information letter was sent
along to all invited study participants within the e-mail invitation. Before the survey was distributed to the
participants, a short internal pre-test was performed, followed by a minimal adaption of the questionnaire.
2.3 Instrument
For the evaluation of se-atlas, we developed a self-designed questionnaire, which consists of four parts, regarding to
our research questions: background and purpose, user satisfaction, usability and user recommendations. To answer
these questions, we created 10 questions as open-ended, single and multiple choice questions, as well as using the
Likert scale from 4 to 6 points. To avoid the bias in the questionnaire, recommendations by Choi and Pak were
applied, which include e.g. how to deal with problems in wordings or questionnaire structure [22]. The whole
questionnaire is shown in additional le 2.
The questions 1 to 4 of the questionnaire belong to the research question “background and purpose”, questions 5, 7
and 8 addresses “user satisfaction”, whereas question 6 assesses the usability of se-atlas. Furthermore, the
questions 9 to 10 were used to derive concrete suggestions to improve se-atlas.
In question 6, the System Usability Scale (SUS), a common questionnaire for assessing the usability of a software
system, was used to measure the usability of se-atlas. The SUS includes 10 items, with a 5-point Likert scale (from
“1 = strongly disagree” to “5 = strongly agree”) [23].
Page 5/17
2.4 Data analysis and processing
Data collected from the survey were summarized using common descriptive statistics, including frequency,
percentage of response, as well as mean, standard deviation (SD) and range, were applicable. We analysed the
results for all study participants and separate for both user groups.
For the analysis of the SUS of question 6, we used the approach of Bangor et al., which species a range from "0" to
"100". This range should not be interpreted as a percentage value and must be normalised. According to Bangor et
al., different methods for normalisation are available. We used the following normalization range to interpret how
well the usability of the system is to be rated [23]:
84.1–100.0 = “Best Imaginable”
71.1–80.7 = “Good”
51.7–71.0 = “OK”
25.1–51.6 = “Poor”
0.0–25.0 = “Worst Imaginable”
For the evaluation of open-ended question 10, we analysed the responses of the participants and created categories
to present the results. For the further implementation of the new version of se-atlas, the results of this study were
discussed with the se-atlas project team and prioritised. Subsequently, these requirements were implemented in the
programming of the new version.
3. Results
In total, 55 participants successfully completed the survey. We contacted 33 experts, whereas 16 responded to our
invitation and participated in the study. Furthermore, 39 members of patient organisations participated in the survey.
3.1 Results of the questionnaire
In the following section, we present the main results of the survey. The complete and detailed results for all groups
are available in the appendix (additional le 3).
3.1.1 Background and purpose
The results of the rst part of the questionnaire regarding “background and purpose” are presented in this section,
showing question 1 and 2 in Table1 and question 3 and 4 in Table2.
The results of question 1 demonstrate that 34.38 % of participants use se-atlas because of a recommendation from
a patient organisation or through a referral via NAMSE (26.56%). In the experts group, most participants became
aware of se-atlas via NAMSE (41.18%) or via recommendation by medical staff (29.41 %). Furthermore, in the
patient organisation group, the majority of the study participants know se-atlas via one of the patient organisations
(46.81 %) or NAMSE (21.28 %).
The results of question 2 reveal that the majority of participants search on HCPs for a specic disease (24.80 %) or
on information on RDCs (24.00 %) as shown in Table 1. Furthermore, the participants were searching on information
on a specic HCP (16.80 %). The participants of the experts group indicated that they mainly search on information
Page 6/17
on RDCs (33.33 %), HCPs for a specic RD (27.78 %) and information on a specic HCP (22.22 %). The group of
members of patient organisations often use of the functionality to search for HCPs for a specic RD (24.14 %),
information on RDCs (20.69 %) and information on a specic rare disease (18.39 %). The information on other
recommendation channels are available in Table 1.
Table 1
Overview of the results (question 1–2)
Item
no Question Answer N %
1 How did you become aware of
se-atlas? Recommendation by a patient organisation 22 34.38
Recommendation by NAMSE 17 26.56
Search engines (Google, Bing, T-Online, ...) 7 10.94
Link on other websites 7 10.94
Recommendation by medical staff (physician, nurse,
…) 6 9.38
Recommendation by a private contact (friend,
colleague, ...) 5 7.81
2 What kind of information do
you search on the website? Search health care providers for a specic rare
disease 31 24.80
Information on Rare Diseases Centres 30 24.00
Information on a specic health care provider (e.g.
contact details of this health care provider) 21 16.80
Information on a specic rare disease 18 14.40
General search in the eld of rare diseases 11 8.80
Information on the se-atlas project (e.g. aim of the
project, contact person) 10 8.00
Others 7 3.20
Regarding the regular use of se-atlas (question 3), the participants indicated that they use the website rarely (36.36
%) or several times a year (25.45 %), see Table 2. Slightly more than a quarter of the participants (29.09 %) reported
using the website once a month or once a week. Concerning the experts group, it can be observed that many
participants use the website weekly (43.75 %), whereas all results were in a range between several times a week and
several times a year. The members of patient organisations mainly use the system rarely (51.28 %) with a range
between several times a week and rarely, whereas only one participant use se-atlas weekly.
The results of question 4 note a frequent access of se-atlas via a laptop (41.82 %) or desktop computer (38.18 %). A
smaller proportion of the participants (18.18%) uses the system on mobile devices, e.g. tablet, smartphone or
phablet. The experts group uses se-atlas primarily via a desktop computer (62.50 %) and laptop (31.25 %). The
members of patient organisations indicated a use of laptop (46.15 %), desktop computer (28.21 %) and mobile
devices (23.08 %).
Page 7/17
Table 2
Overview of the results (question 3–4)
Item no Question Answer N %
3 How often do you use se-atlas? Rarely 20 36.36
Several times a year 14 25.45
Once a month 9 16.36
Once a week 7 12.73
Several times a week 5 9.09
Daily 0 0
4 Which device do you use to access se-atlas? Laptop 23 41.82
Desktop computer 21 38.18
Tablet 5 9.09
Smartphone 4 7.27
Phablet 1 1.82
Other device 1 1.82
3.1.2 User satisfaction
The results of the second part of the questionnaire regarding to user satisfaction and usability are presented in this
section. Details are shown in additional le 3.
In question 5, participants could rate their satisfaction to se-atlas from “very dissatised (1)” to “very satised (6)”.
Overall satisfaction reached a mean of 4.31 (SD = 1.18). The results of the experts group indicate a rather
homogeneous result in the upper scale range (more satised to quite satised), with a mean value of 4.94 (SD = 
0.57). The members of patient organisations rated with a mean score of 4.05 (SD = 1.28).
In question 7, the participants were asked to indicate how satised they are with the use of particular se-atlas
functionalities from 1 (“very dissatised”) to 6 (“satised”). The results show an average between 3.82 and 4.44
across all functionalities. The functionality “Display of health care providers on map and list view” was rated as
best (mean = 4.44, SD = 1.3), whereas the function “Creating of health care providers or patient organisations and
assignment of diseases” (mean = 3.82, SD = 1.14) was rated as lowest. In the experts group, se-atlas functions
achieved an average rating of 4.0 to 5.13. The functionality „Display of health care providers on a map and list
view” was rated as best (mean = 5.13, SD = 1.02). The functionality “General search in the eld of rare diseases”
achieved the lowest result in this group on average (mean = 4.0, SD = 0.89). The results of the members of patient
organisations show an average rating between 3.69 and 4.15, whereas the function “Display of health care
providers on a map and list view” (mean = 4.15, SD = 1.31) was rated as best. The function “Creating of health care
providers or patient organisations and assignment of diseases” was rated as low (mean = 3.69, SD = 1.24).
While question 7 asked the participants to indicate their satisfaction with the use of se atlas functionalities,
question 8 asked for the relevance of the functionalities from 1 ("not important") to 4 ("very important"). The results
Page 8/17
show that all se-atlas functionalities are considered as important with an average between 3.11 and 3.75. The
function “Search for health care providers of a specic rare disease” was rated as highest (mean = 3.75, SD = 0.48),
whereas the function “General search in the eld of rare diseases” was rated as lowest (mean = 3.11, SD = 0.85).
Figure 1 shows the combined results based on the mean values of question 7 and 8.
3.1.3 Usability
In question 6 of the questionnaire, the study participants were able to rate the usability of se-atlas. Overall a score of
“67.1” could be achieved which corresponds to the rating “OK”, according to Bangor et al [23]. The experts group
rates the usability of se-atlas with 76.7, which results in the "Good" range. The members of patient organisations
rated the usability with 63.1 ("OK").
3.1.4 User recommendations
The results in the third part of the questionnaire show that 40 participants (72.73 %) stated that all functionalities
they needed for their visit to se-atlas were available, while the remaining participants denied the statement (27.27 %)
(question 9). These participants gave recommendations for improvement of se-atlas (question 10). To illustrate
these recommendations, we created the categories “Usability recommendations”, “Functionality recommendations”
and “Content recommendations” and summarized the responses as shown below. All responses are available in
German in additional le 3.
Usability recommendations
Some study participants stated se-atlas should be more user-friendly and the website should have a better overall
structure. Furthermore, other participants indicated that they could not nd individual functions (e.g. creation and
link of patient organisations).
Functionality recommendations
The study participants made suggestions regarding the functionality of se-atlas, which for example refer to creating
of patient organisations in the system. One study participant suggested integrating a comment function to share his
opinion on whether or not a HCP has the expertise for a particular RD. In addition, more information on the diseases
should be displayed.
Content recommendations
Study participants indicated that they would like to see further increases in entries of diseases as well as regular
updates. One study participant also stated that almost all diseases in a specic disease group (vascular
malformations) cannot be found. The patient organisations would like to identify HCPs who no longer have
competence for an RD.
3.2 Results of the implementation
In this section, we show examples of concrete implementation changes for the new release of se-atlas, which were
implemented after analysing and prioritizing the results of the study.
As the search again proved to be one of the central functions of se-atlas (highest relevance with a mean of 3.75)
and almost 50 % of the respondents cited the search and information on RDCs as the main reason for their visit, it
was further optimized and its speed improved (see Fig. 2). One point of criticism from the survey participants was
Page 9/17
the lack of diseases and outdated data. Improvements have been made on the technical side, e.g. the latest version
of the Orphanet nomenclature and classication, which is the largest and complete classication of rare diseases, is
now used to identify the search results of the diseases [24].
To simplify access se-atlas data, have a clear and simple overall structure and to improve the usability, the design
and the overall structure of se-atlas were redesigned. These wishes were also mentioned in the users'
recommendations. The starting page was redesigned and the search function has been placed in the foreground
(Fig. 2). In addition, the colour design of the website was changed with the aim of improving user-friendliness and
the recommendation to improve the basic structure of se-atlas and its usability. The starting page now also includes
an integrated slider-menu which leads directly to the central functions of se-atlas, such as creating of HCPs or
patient organisations (high relevance: 3.67), as shown in Fig. 2.
Furthermore, to ensure quick access to the overview of RDCs (Fig. 3), a direct access to this subpage has been
implemented in the slider-menu (see Fig. 2). Furthermore, this overview of RDCs was new designed and
implemented (Fig. 3), since it was one of the main reasons for visits according to our survey (24.0 %).
Since the survey showed generally sucient usability, se-atlas was also further developed with a focus on
increasing mobile use in the future. On the technical side, the editorial page was adapted with the new design.
4. Discussion
This quantitative study investigated with a survey the background and purpose, as well as user satisfaction when
using se-atlas. Furthermore, it was assessed how high the users rate the usability. We derived recommendations for
optimizing the website and implemented them in a new version of se-atlas.
4.1 Discussion of results
Background and purpose
The results demonstrate the majority of the study participants became aware of se-atlas through recommendation
by a patient organisation, by NAMSE or through members of the medical care team. There was only a smaller
proportion of participants who know se-atlas through other channels. The participants’ groups in this study differ in
their backgrounds, which is reected in the results. For instance, the recommendation of se-atlas by a patient
organisation was thus more strongly represented among the members of a patient organisation. This can be
caused due to that patients are active members in patient organisations and these actively try to promote se-atlas
among their members [25]. In the experts group, se-atlas was passed on in the professional environment of the
members of the medical care team. Furthermore, the dissemination of se-atlas was part of the NAMSE action plan
[26], which contributes to the representation of NAMSE in the top three answers for both groups of participants in
the survey.
Another difference in the groups of participants can be determined with regard to the question of what information
the participants are searching for on the website. Both groups predominantly use the central function of se-atlas in
the context of searching for HCPs for a specic RD. Likewise, information about RDCs is an important aspect for
both groups. However, the experts group is less likely to use the information on a specic disease and the general
research options on RDs. In the group of members of patient organisations, these functions are used more
frequently. This could be caused due to the experts use se-atlas mainly in the context of patient care, for example to
Page 10/17
refer patients to other HCPs. This assumption is also available in the results of the question about the frequency of
use of se-atlas. The experts group predominantly indicated weekly or monthly use, while the members of patient
organisations uses se-atlas rather rarely or only several times a year.
The results on use of se-atlas show that the system is mainly used via laptop or desktop computers and only a
smaller proportion of the study participants indicated mobile use (18.17 %), especially the group of members of
patient organisations. However, this result is not in line with the increasing trend of mobile device use on the internet
and does not decrease the relevance of mobile use [27–29].
User satisfaction
Most of the study participants are quite satised with the system. In the set of questions on satisfaction and
relevance of specic functionalities (questions 7 and 8), the results show of a satisfaction with a mean value of
4.01 (SD: 0.07). Furthermore, the relevance of all functionalities of se-atlas is high (mean: 3.30, SD: 0.11). The
central function of searching for facilities for a specic rare disease seems to be important for the users, from which
it can be assumed that se-atlas fulls its basic function. The members of the patient organisations show a
willingness to actively participate in se-atlas through their high rating of the relevance of creating entries for HCPs
and patient organisations. In summary, these results are similar to question 5, where users were able to rank their
overall satisfaction to the system. On average, users were quite satised with se-atlas (mean: 4.30, SD: 1.18).
Usability
With an SUS score of "67.1" according to Bangor et al [23], the overall group of participants achieved a usability
rating of "OK". Brooke et al. investigated in a study 500 usability studies of different software-systems and
calculated an average SUS score of 68 across all systems (at or around the 50th percentile) which is close to our
results. Other patient information websites achieved a score of 72.5 [30] and 92.5 [31]. Furthermore, in another
patient portal, the authors noted user-depended results in different user groups with an SUS from 63 to 81 [32]. This
is similar to our results where the experts group rated the usability of se-atlas at 76.7 (“good”) and patient
organisations with 63.1 ("OK"). Tieu et al species that a lower SUS for a subgroup of participants with limited
health literacy is associated with a greater burden of barriers to use [32]. This may also be the case here, but cannot
be reconstructed on the available evidence.
User recommendations and implementation
In the last part of the study, further elds of action for the improvement of se-atlas were gathered by the users.
Some users gave recommendations on usability, other users made suggestions to integrate various new functions.
For instance, a need to provide more information on individual diseases was stated. Furthermore, patient
organisations noted to identify HCPs who no longer have competence for an RD or to make suggestions for
changes. However, this requirement must be linked to a further validation process by the se-atlas editorial team.
Unfortunately, not all comments and results could be implemented in the new version of se-atlas. These are planned
for future releases. Since the design of se-atlas was implemented completely new, further studies need to evaluate
whether the usability was improved.
4.2 Discussion of methods
In this study, we have chosen a quantitative design with a questionnaire since it is easy to implement and allows us
to primarily assess usability weaknesses, user satisfaction and suggestions to improvements for se-atlas. In the
Page 11/17
future, qualitative studies like Thinking-Aloud-Tests (TA-Test) could be performed, which allow to get opinions of the
users by recording their thoughts while they interact with the software [33, 34]. The advantage of such a qualitative
study is a comprehensive insight into the behaviour of the test persons with a software-system. Response and bias
tendencies are largely minimized here. However, the transferability of the results to the real usage situation must
always be investigated, e.g. through combination with standardized questionnaire interviews. In addition, the effort
required for qualitative studies is signicantly higher [35]. In a further step, other approaches like the Technology
Acceptance Model (TAM) could be used. These model is less appropriate to investigate concrete re-design
suggestions, but allows to identify implementation barriers [36].
Limitations
The study has several limitations. Due to the design of the study, not all user could be reached. Therefore, the
generalisability is limited. In addition, the two groups of study participants were of different sizes. Therefore, the
results can only be compared to a limited extent. Furthermore, in the group of members of patient organisations,
unfortunately not all organisations could be reached. The group of patients and those affected, who are not active
in patient organisations and often only visit se-atlas once, were dicult to recruit. To reach these users, a
quantitative survey directly at the se-atlas website could be considered. However, it is dicult to assess whether the
desired user group is reached, as every website visitor could participate. Likewise, a survey on the website itself,
rather raises the rst impression instead of an overall impression. According to Thielsch et al., website users are
attracted by high aesthetics in web design and are more strongly inuenced by perceived aesthetics and usability in
the rst impression [37].
5. Conclusion
This quantitative study involved experts and members of patient organisations of RDs to assess their background,
purpose and user satisfaction when using se atlas. Furthermore, the usability of se-atlas was assessed and we
derived improvements suggested by the user to implement a new version.
The study is as a snapshot and representation of the need for further development of se-atlas from the perspective
of two specic user groups. The results show that the participant groups in this study differ in their backgrounds
and show different user behaviour and purpose of use. Most of the study participants are quite satised with the
system and its functionalities. This is also reected in the usability rating, which is according to the SUS, between
“OK” and “Good”. Furthermore, initial indications and concrete recommendations for se-atlas were expressed in this
study, for instance to constantly increase the entries of diseases.
Further studies and considerations should address the continuous optimisation process of se-atlas and the
validation of the data to further increase the satisfaction of all user groups and to maintain user loyalty. A further
evaluation based on this survey and its adaptations implemented in se-atlas will be targeted.
Abbreviations
ACHSE: Alliance for Chronic Rare Diseases; ERN: European Reference Network; EU: European Union; HCP: Health
care provider; NAMSE: German National Action League for People with Rare Diseases; RDs: Rare Diseases; RDC:
Rare Diseases Centre; SD: Standard Deviation; STROBE: Strengthening the Reporting of Observational Studies in
Epidemiology; SUS: System Usability Scale, TA-Test: Thinking-Aloud-Test, TAM: Technology Acceptance Model
Page 12/17
Declarations
Author's contributions
JS and MN designed the study and formulated the research questions. The study was performed by JS and MN, as
well as the data analysis. Results of the study were discussed between all authors. Translation from German to
English in context of this study was performed by MN and checked by JS. The rst draft of this publication was
written by JS, whereas all authors provided valuable input. The nal manuscript was written by JS and approved by
all authors.
Acknowledgements
We would like to thank all study participants for conducting and supporting this study, and ACHSE for the support
with the survey.
Funding
se-atlas was funded from 2013 to 2017 by the German Federal Ministry of Health.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on
reasonable request.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Not applicable.
References
1. Lopes MT, Koch VH, Sarrubbi-Junior V, Gallo PR, Carneiro-Sampaio M. Diculties in the diagnosis and
treatment of rare diseases according to the perceptions of patients, relatives and health care professionals.
Clinics (Sao Paulo). 2018; 73:e68.
2. Evans WR, Ra I. Rare diseases in general practice: recognising the zebras among the horses. Br J Gen Pract.
2016; 66:550-1.
3. Danese E, Lippi G. Rare diseases: the paradox of an emerging challenge. Ann Transl Med. 2018; 6:329-329.
4. Nguengang Wakap S, Lambert DM, Olry A, Rodwell C, Gueydan C, Lanneau V, Murphy D, Le Cam Y, Rath A.
Estimating cumulative point prevalence of rare diseases: analysis of the Orphanet database. Eur J Hum Genet.
2020; 28:165-173. https://doi.org/10.1038/s41431-019-0508-0
Page 13/17
5. Genetic Alliance UK. What is a Rare Disease. 2018. https://www.raredisease.org.uk/what-is-a-rare-disease.
Accessed 13 August 2021.
. Griffon N, Schuers M, Dhombres F, Merabti T, Kerdelhue G, Rollin L, Darmoni SJ. Searching for rare diseases in
PubMed: a blind comparison of Orphanet expert query and query based on terminological knowledge. BMC
Med Inform Decis Mak. 2016; 16:101.
7. Institute of Medicine (US) Committee on Accelerating Rare Diseases Research and Orphan Product
Development. Rare Diseases and Orphan Products: Accelerating Research and Development. 2nd ed.
Washington (DC): National Academies Press (US); 2010.
. Haase J, Wagner T, Storf H. se-atlas - the health service information platform for people with rare diseases :
Supporting research on medical care institutions and support groups. Bundesgesundheitsblatt
Gesundheitsforschung Gesundheitsschutz. 2017; 60(5):503.509.
9. Storf H, Schaaf J, Kadioglu D, Gobel J, Wagner TOF, Uckert F. Registries for rare diseases: OSSE - An open-
source framework for technical implementation. Bundesgesundheitsblatt Gesundheitsforschung
Gesundheitsschutz. 2017; 60(5):523–31.
10. Geschäftsstelle des Nationalen Aktionsbündnisses für Menschen mit Seltenen Erkrankungen (NAMSE).
National action league for people with rare diseases. 2010.
https://www.namse.de/leadmin/user_upload/downloads/National_Plan_of_Action.pdf. Accessed 13 August
2021.
11. Schaefer J, Tegtbauer N, Pfeiffer W, Wagner TOF, Storf H. Mapping of Health Care Providers for People with
Rare Diseases - From Vision to Implementation. Stud Health Technol Inform. 2018; 247:940-944.
12. Baldwin JL, Singh H, Sittig DF, Giardina TD. Patient portals and health apps: Pitfalls, promises, and what one
might learn from the other. Healthc (Amst). 2017; 5:81-85.
13. Eyasu T, Leung K, Strudwick G. Guiding Improvements in User Experience: Results of a Mental Health Patient
Portal User Interface Assessment. Stud Health Technol Inform. 2019; 257:110-114.
14. Kooij L, Groen WG, van Harten WH. Barriers and Facilitators Affecting Patient Portal Implementation from an
Organizational Perspective: Qualitative Study. J Med Internet Res. 2018; 20:e183-e183.
15. Reen GK, Muirhead L, Langdon DW. Usability of Health Information Websites Designed for Adolescents:
Systematic Review, Neurodevelopmental Model, and Design Brief. J Med Internet Res. 2019; 21:e11584-e11584
1. Nahm E-S, Preece J, Resnick B, Mills ME. Usability of health Web sites for older adults: a preliminary study.
Comput Inform Nurs. 2004; 22:326-34.
17. Bernstam EV, Sagaram S, Walji M, Johnson CW, Meric-Bernstam F. Usability of quality measures for online
health information: Can commonly used technical quality criteria be reliably assessed? Int J Med Inform. 2005;
74:675-683.
1. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting
of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for reporting observational studies.
Int J Surg Open. 2014; 12:1495-1499.
19. Meuser M, Nagel U. Das Experteninterview - konzeptionelle Grundlagen und methodische Anlage. In: Pickel S,
Pickel G, Lauth H-J, Jahn D, editors. Methoden der vergleichenden Politik- und Sozialwissenschaft: Neue
Entwicklungen und Anwendungen. Wiesbaden: VS Verlag für Sozialwissenschaften; 2009. p. 465-79.
20. ACHSE. ACHSE - Allianz Chronischer Seltener Erkrankungen. 2021. https://www.achse-online.de/de/index.php.
Accessed 18 August 2021.
Page 14/17
21. LimeSurvey. Professionelle Online-Umfragen mit LimeSurvey. 2021. https://www.limesurvey.org/. Accessed 18
August 2021.
22. Choi BCK, Pak AWP. A catalog of biases in questionnaires. Prev Chronic Dis. 2005; 2:A13-A13.
23. Bangor A, Kortum PT, Miller JT. An Empirical Evaluation of the System Usability Scale. Int J Hum Comput
Interact. 2008; 24:574-594.
24. Orphanet. Nomenclature and Classication of Rare Diseases. 2021. http://www.orphadata.org/cgi-
bin/rare_free.html . Accessed 18 August 2021.
25. Delisle VC, Gumuchian ST, Rice DB, Levis AW, Kloda LA, Koerner A, Thombs BD. Perceived Benets and Factors
that Inuence the Ability to Establish and Maintain Patient Support Groups in Rare Diseases: A Scoping Review.
Patient. 2017; 10:283-293.
2. Geschäftsstelle des Nationalen Aktionsbündnisses für Menschen mit Seltenen Erkrankungen (NAMSE).
Zwischenbericht zur Umsetzung des Nationalen Aktionsplans für Menschen mit Seltenen Erkrankungen. 2017.
https://www.achse-
online.de/de/was_tut_ACHSE/namse/pdf/nationaler_Aktionsplan/namse_monitoringbericht_oktober_2017.pdf.
Accessed 18 August 2021.
27. Kim SU, Martinović I, Katavić SS. The use of mobile devices and applications for health information: A survey
of Croatian students. J Librariansh Inf Sci. 2019; 52:880-894.
2. Parasuraman S, Sam AT, Yee SWK, Chuon BLC, Ren LY. Smartphone usage and increased risk of mobile phone
addiction: A concurrent study. Int J Pharm Investig. 2017; 7:125-131.
29. Khan NA, Habib MA, Jamal S. Effects of smartphone application usage on mobility choices. Transp Res A.
2020; 132:932-947.
30. Knijnenburg SL, Kremer LC, Versluys AB, Braam KI, Mud MS, van der Pal HJ, Caron HN, Jaspers MW. Evaluation
of a patient information website for childhood cancer survivors. Suppor Care Cancer. 2013; 21:919-926.
31. Tanbeer S, Sykes E. MyHealthPortal - A web-based e-Healthcare web portal for out-of-hospital patient care. Digit
Health. 2021; 7:205520762198919. https://doi.org/10.1177/2055207621989194
32. Tieu L, Schillinger D, Sarkar U, Hoskote M, Hahn KJ, Ratanawongsa N, Ralston JD, Lyles CR. Online patient
websites for electronic health record access among vulnerable populations: portals to nowhere? J Am Med
Inform Assoc. 2017; 24:e47-e54.
33. Konrad K. Lautes Denken. Handbuch Qualitative Forschung in der Psychologie. VS Verlag für
Sozialwissenschaften; 2010.
34. Richardson S, Mishuris R, O’Connell A, Feldstein D, Hess R, Smith P, McCullagh L, McGinn T, Mann D. “Think
aloud” and “Near live” usability testing of two complex clinical decision support tools. Int J Med Inform. 2017;
106:1-8.
35. Thielsch M. Expertise Website-Evaluation: Übersicht über bestehende Evaluationsmethoden und
Entscheidungshilfe für die Evaluation bestehender sowie neu geschaffener Websites. 2018.
https://doi.org/10.17623/BZGA:225-EWE-1.0. Accessed 18 August 2021.
3. Rahimi B, Nadri H, Lotfnezhad Afshar H, Timpka T. A Systematic Review of the Technology Acceptance Model
in Health Informatics. Appl Clin Inform. 2018; 9:604-634.
37. Thielsch MT, Blotenberg I, Jaron R. User Evaluation of Websites: From First Impression to Recommendation.
Interact Comput 2014; 26:89-102.
Page 15/17
Figures
Figure 1
Overview of the results (question 7-8)
Page 16/17
Figure 2
New starting page of se-atlas.de with slider-menu
Page 17/17
Figure 3
Overview of RDCs in se-atlas
Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.
Additionalle1STROBEchecklist.pdf
Additionalle2Questionnaireofthesurvey.pdf
Additionalle3Resultsofthesurvey.pdf
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Current e-Health portal platforms provide support for patients only if they have previously registered and received service from a healthcare facility (e.g., hospital, healthcare clinic, etc.). These portals are usually connected to a central EMR/EHR system linked to a central system. Furthermore, these portals are restrictive in that they are only accessible by these patients at the exclusion of parents, relatives and others that participate in providing care to the patient. Further complications include the increasing demand from our healthcare systems for patients to receive more off-site, non-primary, in-homecare, and/or specialized healthcare services at home (e.g., therapy, nursing, personal support, etc.). Lastly, an increasing number of people would like to have more autonomy over their health in terms of increased access to their own medical records and the services they receive. In this work, we addressed these limitations by creating MyHealthPortal – a patient portal aimed at non-primary care, in-homecare, and/or special healthcare for patients. MyHealthPortal can assist homecare and clinic-based healthcare services along with the benefits of existing portals (e.g., online appointment scheduling, monitoring, and information sharing). MyHealthPortal is secure, robust, flexible and user-friendly. We developed it in partnership with our industry partner, Closing the Gap Healthcare. Closing the Gap is a prominent homecare and clinic-based healthcare service provider that became the first homecare agency to score 100% on standards from accreditation Canada and was awarded the exemplary standing. In this paper we present MyHealthPortal, the architectural framework that we designed and developed to support the system, and the results of a usability study conducted from real field studies. Our system was tested in a variety of conditions and achieved SUS usability scores of 92.5% (high).
Article
Full-text available
Background: Adolescence is a unique developmental period characterized by biological, social, and cognitive changes, as well as an interest in managing one's own health care. Many adolescents use the internet to seek health care information. However, young people face barriers before they can understand and apply the health information that they access on the web. It is essential that usability of adolescent health websites on the internet is improved to help adolescents overcome these barriers and allow them to engage successfully with web-based health care content. Objective: The aim of this review was to synthesize the usability of specific health information websites. These findings were mapped onto the adolescent neurodevelopmental profile, and a design brief based on the findings was developed to tailor future websites for specific adolescent requirements. Methods: A systematic search conducted using PubMed, PsycINFO, and Education Resources Information Center (ERIC) identified 25 studies that assessed the usability of health information websites. Adolescent feedback was collected by a mixture of surveys, focus groups, interviews, and think-aloud procedures. Results: A majority of the information websites were developed for specific health issues that may be relevant to adolescents. The most preferred website features were interactive content such as games and quizzes, as well as videos, images, audio clips, and animations. Participants also preferred communicating with other adolescents with similar conditions or learning about their experience through real stories and testimonials. Adolescents found it difficult to use health information websites if they contained too much text, were too cluttered, or had features that made it difficult to access. The findings are considered in the context of adolescent social processes, low tolerance of delayed gratification, and attraction to novelty and mapped onto a neurodevelopmental model of adolescence. Conclusions: Young people's feedback can determine usability and content that make a health information website easy or informative to use. Neurodevelopmental profiles and the users' specific preferences and skills should be addressed in future development of health information websites for adolescents.
Article
Full-text available
This special issue of Annals of Translational Medicine, entitled “Rare and orphan disorders: an emerging challenge?”, will include a compilation of articles aimed at providing updated evidence on some “rare” human pathologies, thus including biliary tract cancers, cystic fibrosis, sclerosing mesenteritis, insulin autoimmune syndrome, chronic-inflammatory demyelinating polyneuropathies, neurodegenerative dementia, rare neonatal disorders, sleep apnea in childhood, acute intestinal ischemia, cardiac arrhythmias in sarcoidosis, Kounis syndrome, along with rare thrombophilic conditions, computer-related thrombosis, rare bleeding disorders, rare forms of von Willebrand disease (VWD) and hereditary spherocytosis. It is our hope that the current gap of clinical knowledge could be at least partially filled, and that both diagnosis and treatment of patients with rare (orphan) diseases may be improved. We also wish to express our sincere gratitude to the authors of this special issue of the journal, trusting that these articles may be of interest for our readership.
Article
Full-text available
Background: The number of patient portals is rising, and although portals can have positive effects, their implementation has major impacts on the providing health care institutions. However, little is known about the organizational factors affecting successful implementation. Knowledge of the specific barriers to and facilitators of various stakeholders is likely to be useful for future implementations. Objective: The objective of this study was to identify the barriers to and facilitators of patient portal implementation facing various stakeholders within hospital organizations in the Netherlands. Methods: Purposive sampling was used to select hospitals of various types. A total of 2 university medical centers, 3 teaching hospitals, and 2 general hospitals were included. For each, 3 stakeholders were interviewed: (1) medical professionals, (2) managers, and (3) information technology employees. In total, 21 semistructured interviews were conducted using the Grol and Wensing model, which describes barriers to and facilitators of change in health care practice at 6 levels: (1) innovation; (2) individual professional; (3) patient; (4) social context; (5) organizational context; and (6) economic and political context. Two researchers independently selected and coded quotes by applying this model using a (deductive) directed content approach. Additional factors related to technical and portal characteristics were added using the model of McGinn et al, developed for implementation of electronic health records. Results: In total, we identified 376 quotes, 26 barriers, and 28 facilitators. Thirteen barriers and 12 facilitators were common for all stakeholder groups. The facilitators' perceived usefulness (especially less paperwork) was mentioned by all the stakeholders, followed by subjects' positive attitude. The main barriers were lack of resources (namely, lack of staff and materials), financial difficulties (especially complying with high costs, lack of reimbursements), and guaranteeing privacy and security (eg, strict regulations). Both similarities and differences were found between stakeholder groups and hospital types. For example, managers and information technology employees mainly considered guaranteeing privacy and security as a predominant barrier. Financial difficulties were particularly mentioned by medical professionals and managers. Conclusions: Patient portal implementation is a complex process and is not only a technical process but also affects the organization and its staff. Barriers and facilitators occurred at various levels and differed among hospital types (eg, lack of accessibility) and stakeholder groups (eg, sufficient resources) in terms of several factors. Our findings underscore the importance of involving multiple stakeholders in portal implementations. We identified a set of barriers and facilitators that are likely to be useful in making strategic and efficient implementation plans.
Article
Full-text available
OBJECTIVES The aim of this study is to present a survey of vulnerabilities and to suggest approaches for the treatment of rare diseases according to the perceptions of a group of affected individuals, patient association representatives and health care professionals. METHODS The focus group technique was used in interviews with patients and primary caregivers, patient support groups/non-governmental organizations, primary health care professionals and physician specialists. RESULTS The transcript analysis focused on thematic units, which were tailored to each group and allowed comparisons in search of concordant views. Unanimity was observed in relation to the physical, emotional and social damage to the life standards of the affected individuals and their families as a result of illness. The Brazilian health system was unanimously classified as inadequate to respond to the needs of patients with rare diseases, and this inadequacy led to unpleasant experiences, such as the seemingly endless referrals among health services to reach a final diagnosis and develop a treatment plan. CONCLUSIONS The complex set of health system requirements necessary to support the care of patients with rare diseases represents an obstacle to successfully meeting the needs of patients and their families. Therefore, it is important to develop specific public policies to create referral services, guarantee access to appropriate therapeutic modalities and incorporate technologies that promote research for developing new, affordable therapies.
Article
This paper examines the impacts of smartphone application usage on mobility choice dimensions, such as individuals’ visits to new places, individuals’ trips planned in groups, their participation at social gatherings, and vehicle kilometers traveled. It uses data from the Smartphone Use and Travel Choice Survey 2015, which was conducted exclusively on smartphone users of Halifax, Canada. A latent class random parameter logit (LCRPL) modeling technique is applied in this study that provides a better understanding of the effects of smartphone application usage on mobility choices. The model results offer behavioral insights regarding the influence of individuals’ attitudes, travel characteristics, built environment and accessibility measures on the relationship between smartphone application usage and mobility choices. For instance, living in the higher mixed land-use areas, individuals are less likely to increase their vehicle kilometers traveled due to smartphone application usage, especially if they are tech savvy. Such individuals tend to increase their participation in social gatherings. One of the unique features of this study is that it explores the effects of individuals’ smartphone application usage on mobility choices in terms of their attitudes. The study reveals that in case of people with positive attitude towards sustainable travel, smartphone application usage tends to decrease vehicle kilometers traveled, new place visits and planned group trips, however, increase participation in social gatherings. Results of this study provide critical behavioral insights that could be useful for transportation planners and policy makers to develop flexible policy interventions.
Article
This study aims to understand the information behaviours of youth seeking health information on mobile devices and to explore differences by prior knowledge, gender and grade level. A survey was conducted in two high schools in Osijek, Croatia. Results indicate that among the total 408 participants, 84.3% reported having used mobile devices for health information and 54.7% reported having used applications for health information. Students seek health information about physical activity and eating issues/nutrition mostly on mobile devices. Students with more prior medical knowledge tend to have more information needs, perceive mobile devices as more beneficial, search health information and use health-related applications more frequently, and consider the tracking/recording/monitoring feature of applications more useful. Students with less prior medical knowledge tend to consider the video feature of applications more useful. Students rely heavily on searching health topics in search engines on mobile devices, rather than browsing or using applications. This study provides implications to better assist young people in using mobile devices and applications to manage their health issues.
Article
Patient portals provide patients with electronic access to their health records. Since there has been limited use of patient portals in mental health settings, there is a lack of research regarding the usability of the technology amongst this patient population. The purpose of this study was to assess the usability of a mental health patient portal, and to provide the study site with design recommendations. Ten (n=10) participants completed a guided user interface assessment on laptops and tablet devices, along with a structured questionnaire. Findings revealed a number of modifiable aspects of the portal design to improve the usability of the technology for the end user.
Article
Background: One common model utilized to understand clinical staff and patients' technology adoption is the technology acceptance model (TAM). Objective: This article reviews published research on TAM use in health information systems development and implementation with regard to application areas and model extensions after its initial introduction. Method: An electronic literature search supplemented by citation searching was conducted on February 2017 of the Web of Science, PubMed, and Scopus databases, yielding a total of 492 references. Upon eliminating duplicates and applying inclusion and exclusion criteria, 134 articles were retained. These articles were appraised and divided into three categories according to research topic: studies using the original TAM, studies using an extended TAM, and acceptance model comparisons including the TAM. Results: The review identified three main information and communication technology (ICT) application areas for the TAM in health services: telemedicine, electronic health records, and mobile applications. The original TAM was found to have been extended to fit dynamic health service environments by integration of components from theoretical frameworks such as the theory of planned behavior and unified theory of acceptance and use of technology, as well as by adding variables in specific contextual settings. These variables frequently reflected the concepts subjective norm and self-efficacy, but also compatibility, experience, training, anxiety, habit, and facilitators were considered. Conclusion: Telemedicine applications were between 1999 and 2017, the ICT application area most frequently studied using the TAM, implying that acceptance of this technology was a major challenge when exploiting ICT to develop health service organizations during this period. A majority of the reviewed articles reported extensions of the original TAM, suggesting that no optimal TAM version for use in health services has been established. Although the review results indicate a continuous progress, there are still areas that can be expanded and improved to increase the predictive performance of the TAM.
se-atlas - the health service information platform for rare diseases - is part of the German National Action Plan for People with Rare Diseases. The website www.se-atlas.de provides an overview of health care providers and support groups focusing on rare diseases in Germany. Since the start of se-atlas in 2013, several strategies are being developed and evaluated. This paper gives an overview about the expectations and visions for se-atlas at the beginning of the project, the challenges and lessons learned within the project period and how se-atlas is implemented today.