Available via license: CC BY
Content may be subject to copyright.
TYPE Systematic Review
PUBLISHED 04 May 2023
DOI 10.3389/fpubh.2023.999958
OPEN ACCESS
EDITED BY
Chander Prakash Yadav,
National Institute of Malaria Research
(ICMR), India
REVIEWED BY
Sergey Soshnikov,
Bukhara State Medical Institute, Uzbekistan
Syed Shah Areeb Hussain,
National Institute of Malaria Research
(ICMR), India
Anuj Kumar,
ICMR-National Institute of Cancer Prevention
and Research, India
*CORRESPONDENCE
Annett Schulze
annett.schulze@bfr.bund.de
RECEIVED 21 July 2022
ACCEPTED 05 April 2023
PUBLISHED 04 May 2023
CITATION
Schulze A, Brand F, Geppert J and Böl G-F
(2023) Digital dashboards visualizing public
health data: a systematic review.
Front. Public Health 11:999958.
doi: 10.3389/fpubh.2023.999958
COPYRIGHT
©2023 Schulze, Brand, Geppert and Böl. This
is an open-access article distributed under the
terms of the Creative Commons Attribution
License (CC BY). The use, distribution or
reproduction in other forums is permitted,
provided the original author(s) and the
copyright owner(s) are credited and that the
original publication in this journal is cited, in
accordance with accepted academic practice.
No use, distribution or reproduction is
permitted which does not comply with these
terms.
Digital dashboards visualizing
public health data: a systematic
review
Annett Schulze*, Fabian Brand, Johanna Geppert and
Gaby-Fleur Böl
Study Centre for Social Science Research, Department Risk Communication, German Federal Institute
for Risk Assessment, Berlin, Germany
Introduction: Public health is not only threatened by diseases, pandemics, or
epidemics. It is also challenged by deficits in the communication of health
information. The current COVID-19 pandemic demonstrates that impressively.
One way to deliver scientific data such as epidemiological findings and
forecasts on disease spread are dashboards. Considering the current relevance
of dashboards for public risk and crisis communication, this systematic review
examines the state of research on dashboards in the context of public health risks
and diseases.
Method: Nine electronic databases where searched for peer-reviewed journal
articles and conference proceedings. Included articles (n= 65) were screened and
assessed by three independent reviewers. Through a methodological informed
dierentiation between descriptive studies and user studies, the review also
assessed the quality of included user studies (n= 18) by use of the Mixed Methods
Appraisal Tool (MMAT).
Results: 65 articles were assessed in regards to the public health issues
addressed by the respective dashboards, as well as the data sources, functions and
information visualizations employed by the dierent dashboards. Furthermore,
the literature review sheds light on public health challenges and objectives and
analyzes the extent to which user needs play a role in the development and
evaluation of a dashboard. Overall, the literature review shows that studies that
do not only describe the construction of a specific dashboard, but also evaluate
its content in terms of dierent risk communication models or constructs (e.g., risk
perception or health literacy) are comparatively rare. Furthermore, while some of
the studies evaluate usability and corresponding metrics from the perspective of
potential users, many of the studies are limited to a purely functionalistic evaluation
of the dashboard by the respective development teams.
Conclusion: The results suggest that applied research on public health
intervention tools like dashboards would gain in complexity through a theory-
based integration of user-specific risk information needs.
Systematic review registration: https://www.crd.york.ac.uk/prospero/display_
record.php?RecordID=200178, identifier: CRD42020200178.
KEYWORDS
visualization, risk information, health literacy, information needs, representations,
dashboard
1. Introduction: monitoring public health
The current COVID-19 pandemic poses immense challenges for nation-states and
civil society alike. Not only does the current situation severely restrict public and private
life, but also affects governmental agencies which are constantly confronted with dynamic
decision-making situations. Both private individuals and decision-makers are carefully
Frontiers in Public Health 01 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
observing developments and using different types of media
and formats to make sense of the current crisis as well as
finding appropriate ways to communicate data and messages (1).
Quality media such as public service broadcasting in Germany
use figures from universities or from national and international
health organizations such as the Robert Koch-Institute (RKI) or
the World Health Organization (WHO) in their reporting. The
findings and forecasts on the spread of the virus are increasingly
presented in so-called dashboards (2) i.e., through a specific type
of visualization “of a consolidated set of data for a certain purpose”
(3), using a combination of numerical, temporal, geographical, and
diagrammatic forms of presentation.
These dashboards capture the extent of the outbreak by
visualizing cases, hospitalizations, deaths, vaccination rates etc. and
allow to track the outbreak from a regional up to a global scope.
They can be used to gain a quick overview, allow specific analysis
and facilitate decision-making. Thereby, surveillance activities
provide an instrument to prevent diseases, reduce morbidity
and mortality, and promote health—objectives that define public
health (4).
Worldwide, the globalization and the dissolution of national
boundaries for diseases, disease spread, pollution, or environmental
catastrophes foster the emergence of public health surveillance
infrastructures (5) including a wide range of mobile health tools
(6). With the expanding digitization, data-driven developments
become more important for the assessment and surveillance of
public health issues (7,8). In the context of infectious disease
surveillance, for example, dashboards are often the focus of
scientific interest as a tool for visualizing epidemic data (9,10).
The focus of these studies is on increasing the efficiency of
surveillance systems by identifying potential gaps—ranging from
technical improvements over data quality to modeling these data.
Additionally, the COVID-19 pandemic has shown, that not only
epidemiologists, statisticians or data modelers are interested in near
real-time COVID-19 data (11), but also the general public seeks for
information about the spread of the virus (12,13).
Therefore, the evaluation of an online communication
format such as a dashboard is important with regard to many
different aspects. Through a meta literature review, we were
able to crystallize a not necessarily exhaustive but nonetheless
comprehensive list of four different aspects that are important
to consider in dashboard research. Major aspects mentioned in
the literature here were (a) how public health data is visualized
(14,15), (b) the modes of communication used (16), (c) how the
visualized data can be understood, is read and filled with meaning
by various subpopulations (17,18), and (d) how effective different
(communication) formats are (16,17,19). At the same time, the
large amount of data that can be provided via dashboards, as well as
their scientific nature, pose various challenges to users—whether
in understanding, processing or contextualizing the information
(13,20). Accordingly, there is a need for research on the needs
of users.
Until 2020 and to the best of our knowledge, no systematic
review on public health dashboards existed. Only two other
reviews have appeared in this context by now (June 2022).
A literature review provides insights into the technological
advances of dashboards (21). One dashboard review sheds light
on design modes of U.S. COVID-19 State Government Public
Dashboards (15).
Therefore and from a communication science perspective,
we investigate scientific studies on dashboards as a form of
diagrammatic images in science communication covering public
health issues—from non-communicable diseases (e.g., diabetes),
communicable diseases (e.g., Ebola) and natural disasters (e.g.,
floods) to addictive disorders and related health risks such as drug
abuse (22) or obesity (23). These behavioral risk factors have a
public health impact as they can cause non-communicable diseases.
We are particularly interested in whether empirical analysis will
provide indications for a more effective visualization of scientific
data, e.g., by drawing on cognitive and affective factors to process
visual information. Thus, this systematic review aims to assess
the state of research on dashboards, that are utilized in a public
health context and provide information on divergent public health
phenomena such as risks, pandemics, infections or health crises,
with a focus on the methods of gathering and presenting public
health information as well as the methodological approaches used
to develop or evaluate the dashboards.
2. Methods
Our systematic literature review followed the steps,
comprehensively described by Xiao and Watson (24): (1)
formulating the research problem, (2) developing and validating
the review protocol, (3) searching the literature, (4) screening for
inclusion, (5) assessing quality, (6) extracting data, (7) analyzing
and synthesizing data, and (8) reporting the findings.
2.1. Formulating the research problem
Research on the effective visualization of scientific data through
dashboards from a communication science perspective is scarce.
This literature review is therefore devoted to two distinctive
objectives, which in turn are structured by a total of three research
questions (RQs). First, it aims to offer an overview of different
dashboards described in the scientific literature as relevant to the
field of public health, thereby encompassing elements of a scoping
review (RQ 1 & RQ 2). Second, it pursues to gain insights into the
needs and demands of different user groups while engaging with
a public health dashboard (RQ 3). Answers to the last research
question are expected to be gained exclusively from those studies
that have conducted a user study, assessing their specific needs and
demands. Thus, the review needs to further differentiate between
user studies and mere descriptive studies (see Section 2.5). In that
sense, the derived research questions have been defined as follows:
•RQ 1: Which dashboards that are thematically related to the
field of public health have been examined in the scientific,
peer-reviewed literature and what is known about them?
In particular:
◦RQ 1.1: Which areas relevant to public health—such as
diseases, risks or crises—are covered by these dashboards?
◦RQ 1.2: From which sources do these dashboards retrieve
their data?
◦RQ 1.3: What information (data or indicators) is visualized
through these dashboards?
Frontiers in Public Health 02 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
◦RQ 1.4: Which graphical representations are used to
visualize the data or indicators in these dashboards?
◦RQ 1.5: Which functions do these dashboards offer besides
the pure visualization of information?
•RQ 2: Which challenges and objectives are addressed in the
sampled articles (a) in regards to the consolidation of public
health and (b) in regards to the use of dashboards in that
specific context?
◦RQ 2.1a: Which public health challenges do the sampled
articles address?
◦RQ 2.1b: What public health objectives are they pursuing?
◦RQ 2.2a: What specific technological or administrative
challenges are associated with the use of dashboards in
public health?
◦RQ 2.2b: What are the specific technological or
administrative objectives associated with the use of
dashboards in public health?
•RQ 3: Which information needs can be identified in the
assessed user studies regarding the engagement with public
health dashboards?
2.2. Developing and validating the review
protocol
Before the systematic search was carried out, we conducted
a cursory review and pre-review mapping of relevant articles
on the use of dashboards in public health settings. These
articles were identified through quick-scan searches in
various databases such as Scopus or Google Scholar. A loose
combination of search words (such as “public health dashboard”,
“evaluation”, or “perception”) was used in order to obtain
an overview of the body of literature on dashboard research
and to identify possible keywords for the definition of viable
search strings.
2.3. Searching the literature—identifying
relevant articles
After formulating the research questions, validating,
and publishing our research protocol on PROSPERO
(CRD42020200178), two different search strings were
conceptualized in order to retrieve relevant articles. Using
Boolean operators “AND”, “OR”, “NOT”, the first search
string combined different user-centered (e.g., “literacy” or
“knowledge”) as well as visualization-centered (e.g., “graph”
or “multimodal”) keywords with the search term “dashboard”
and different areas of public health (e.g., “epidemiology”).
The focus on these categories is intended to limit the broad
field of dashboard research to those articles that specifically
relate to the field of public health and potentially address the
question of user preferences and design considerations. Due
to the increasing and striking relevance of dashboards in the
context of the current COVID-19 pandemic [for a critical
discussion see Everts (25)], we further defined an additional
search string, covering a spectrum of recently published articles on
COVID-19-relevant dashboards.
To conduct the review, multi-disciplinary databases such
as Scopus, Web of Science, technical-oriented databases like
IEEE Xplore and ACM Digital Library and databases from
different disciplinary fields such as communication sciences
(Communication Abstracts, Communication & Mass Media
Complete) or psychology (PsycArticles, PsycInfo) were selected.
We included Open Gray as an additional database to identify
further relevant papers. Through this range of databases, it is
assumed that a wide range of literature on public health dashboards
is covered, as, for example, Scopus also includes records from the
MEDLINE and EMBASE databases.
2.4. Screening for inclusion
Before running both search strings in the mentioned academic
databases, several inclusion and exclusion criteria were defined
in order to evaluate identified papers for further consideration
in the literature review (eligibility assessment). These criteria
are presented in Appendix A.Figure 1 illustrates the complete
search process.
After retrieving a total of 1,836 papers by running both
search strings in the aforementioned nine academic databases (see
Section 2.3), an automated duplicate removal, supplemented by a
subsequent hand search for duplicates, reduced our sample to a
total of 1,191 papers.
These remaining 1,191 papers went through different selection
stages. To test for interrater reliability two researchers randomly
selected 100 papers from our sample and assessed their titles for
further selection based on the previously defined inclusion and
exclusion criteria (see Appendix A). Belur et al. (26) report several
methods for calculating interrater reliability, including Cohen’s κ,
where a score of 1 indicates perfect agreement and a score of 0
equates agreement totally due to chance. By comparing individual
ratings, we finally calculated a Cohen’s κof 0.78—implying,
according to Landis and Koch (27), substantial agreement.
Our review applies a titles-first then abstracts screening
strategy, which was already recommended by Mateen et al.
(28) based on an empirical comparison of different screening
methods, as a titles-first strategy guarantees an “accurate, less
time-consuming process that does not compromise the quality
of the final review”. In accordance with a previously defined
code book, supplementing our defined inclusion and exclusion
criteria (see Appendix A), all 1,191 identified papers were assessed
for eligibility based on their titles. This procedure left us
with 296 remaining papers of which all titles and abstracts
were read and assessed for eligibility in accordance with the
above mentioned inclusion and exclusion criteria. Critical or
unclear cases were deferred for further review by all researchers
involved. Finally, discrepancies or disagreements concerning the
eligibility assessment were solved by discussion and consensus-
based decision-making. The review of the remaining abstracts
left us with a total of 86 papers. However, nine more papers
Frontiers in Public Health 03 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
FIGURE 1
Research questions and visualization of the literature search process including search strings and number of retrieved and assessed publication.
had to be further excluded from the study either because they
were not available or could not be acquired. After a thorough
reading of identified and potentially relevant full-text articles as
well as a consequent reapplication of the defined inclusion and
exclusion criteria, we finally selected 65 articles for our final
literature review.
Frontiers in Public Health 04 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
2.5. Assessing quality
In order to adequately assess the quality of identified studies,
we developed a scheme to differentiate the selected 65 articles
according to their empirical focus (see Figures 2,3). Studies that
had executed a user study (n=18), meaning an empirical
assessment of a focal dashboard through different user groups, were
considered for further quality assessment by means of the Mixed
Methods Appraisal Tool (MMAT) which was specifically developed
for critically appraising the quality of different study designs in
systematic mixed studies reviews (29).
The MMAT provides the possibility of assigning ratings in
order to record the quality of the included studies by using
descriptors such as (∗) or (%). The final quality rating is determined
by the summarized total number of “yes” items assigned to the
respective study category (e.g., qualitative studies). For mixed-
methods studies, the developers of the MMAT state that “the overall
quality of a combination cannot exceed the quality of its weakest
component” (30). Since there are 15 criteria to rate for mixed-
methods studies (including the five items for the first applied
method as well as five items for the second method employed in
the respective articles), the overall score for these types of studies is
based on the lowest score of all considered study components.
The remaining 47 articles focused either on the development of
dashboards and their respective testing through various IT-related
measures or on the pure description of a respective dashboard
system and were classified as descriptive studies. They were
considered relevant for answering the defined research questions
as well and thus incorporated in the next step.
2.6. Extracting, analyzing, and synthesizing
data
After performing a comprehensive quality assessment, all
65 articles were finally coded with MAXQDA according to the
research questions, defined above. Both inductive and deductive
coding was used. Three researchers were involved in the inter-coder
process to achieve coding consistency (31,32). Disagreements
were debated until consent was reached. After the first tests for
consistency, all papers were coded by two researchers. Whenever
discrepancies arose, a third researcher was consulted. Every time
a new code was added to the coding system, all papers that had
already been coded were revised again. After initial coding and
fine-tuning of respective coding categories, further fine coding was
carried out, which formed the basis for the results reported below.
3. Results: answering the research
questions
3.1. Public health dashboards in the
scientific literature providing information
on public health issues (RQ 1)
3.1.1. Public health issues covered by dashboards
(RQ 1.1)
In total 65 papers were included in our literature review. They
cover topics from infectious diseases like Dengue (33), Ebola (34),
or COVID-19 (35) (n=21), crises caused by emergencies and
disasters, such as floods [e.g., (36)] (n=6) or other health hazards
such as those caused by pollution (e.g., 37) (n=4) (see Appendix B
for raw data, Figure 4 on dashboard topics).
3.1.2. Data sources used by dashboards (RQ 1.2)
Data displayed on the dashboards is derived from different
sources like (a) governmental institutions (37) (n=14), (b) health
organizations like the World Health Organization and health care
facilities (38) (n=25) (c) national or local Research Organizations
like the National Center for Health Statistics (39) (n=6), (d) cities
or communities (40) (n=11), (e) news and journals (41) (n=
8), and (f) social media such as Twitter (42) (n=8). Also, eleven
papers report that (g) the users of the dashboard can be a source
of information (43). Often dashboards derive their information
from more than one source (see Appendix C). For example, Zheng
et al. (44) created a dashboard to exchange critical information
for the private and public sector in case of a crisis situation. The
information is gathered from County Emergency Offices, company
reports and messages as well as the news. Also, users can add further
reports. Another dashboard tracking COVID-19 cases collects and
displays data from a medical community online platform as well as
Twitter and online news (35).
3.1.3. Information (data or indicators) visualized
through dashboards (RQ 1.3)
As stated above, the papers analyzed describe dashboards that
deal with the visualization of data on, for example, diseases, crises
and risks. Key indicators mentioned in different studies are (see
Appendix D):
1. The number of reported cases (e.g., of a disease) or rates (e.g.,
death rates) (n=15).
2. Health data including patient attributes (e.g., weight) and type
of disease (e.g., HIV) (n=43).
3. Social and environmental factors (e.g., education) (n=7).
4. Environmental data (e.g., air pollution, temperature) (n
=15).
5. Demographics (e.g., age, gender) (n=14).
6. Time (e.g., time of an event, variation in time) (n=14).
7. Location (e.g., region or country) (n=38).
3.1.4. Graphical representations used to visualize
data or indicators in dashboards (RQ 1.4)
The visualization of data is one of the main goals of the
dashboards. To do so, the dashboards mainly feature maps (see
Figure 5), charts and tables. Forty dashboards reporting incidences
of health hazards or the magnitude of a crisis caused e.g., by natural
disasters, use maps to visualize the spread or effected areas (45).
These are often further enhanced by symbols (38) (n=5), icons
(46) (n=6) or pop-ups (47) (n=11) that become visible when the
users hover over the map.
Charts and graphs are used in different formats such as bar
charts (48) (n=24), pie charts (33) (n=16), or line graphs (49)
(n=6; see Figure 6). All types of charts and graphs facilitate date
visualization in general but it is not further explained how the
Frontiers in Public Health 05 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
FIGURE 2
Characteristics and associated codes for user studies.
FIGURE 3
Characteristics and associated codes for descriptive studies.
developers of the dashboards decided which type of chart or graph
they were going to use. Tables are used to display rankings, precise
numbers and scores and to list different data on one aspect (50)
(n=21).
Besides the mentioned, common visualizations, four
dashboards incorporate timelines aiming at a more holistic
understanding of the situation and analyze events over a period
of time (41,51). Concannon et al. (47), for example, uses tree
maps as they are preferred by the users of the dashboard and
allow for more precise display of labels. Word clouds are primarily
used to visualize social media data such as keywords from Twitter
posts to give a quick overview of main topics or locations (52) (n
=3). Several papers describe the use of distinct sub-sections of
the page like sidebars (37) or tabs (53) (n=9) which facilitate
the navigation.
Nineteen papers describe the use of color to further enhance
understanding. Some of them explicitly use the traffic light
colors—green, amber and red—to take advantage of the popular
associations regarding these colors (54). In some areas—as
described by Bernard et al. (55) for the medical sector—it is
beneficial to use color codes that are prominent in a certain work
environment (e.g., black for “death of disease”) (see Appendix E).
3.1.5. Functions that dashboards oer besides the
pure visualization of information (RQ 1.5)
Dashboards are not only used for the visualization of data
but offer further functions, features and components depending
on the situation or task at hand. These include, for example, the
possibility to look at data representing longer time scales (56)
(n=13) or to conduct predictive analysis (57) (n=4). The
possibility of data customization is described in almost half of the
papers considered in the review. This includes the possibility of
(a) selecting and filtering datasets (58) (n=27), (b) searching for
datasets of variables (38) (n=8), and (c) sorting or grouping data
(59) (n=4).
In addition, ten papers describe dashboards that offer direct
export e.g., of data files, screenshots (50) or reports (60). These
downloads can be used for in-depth analysis, as illustrative material
in meetings, or they can be uploaded into other tools for further use
Frontiers in Public Health 06 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
FIGURE 4
Subcategories of the four main dashboard topics.
(61). For participatory dashboards that rely on data from sources
such as the public or medical staff (62), the possibility to directly
add data to the dashboard is an important function. Data entry is
provided through web-based report files (63), customized online
forms, via posts or SMS and some dashboards provide direct data
upload (64). To further enhance user experience, data can be copied
and edited (65) (n=22). Eight papers note that an alarm function
is particularly useful for dashboards on crisis management, which
allows users to receive messages about alarming situations or
noteworthy developments via SMS or email (66). Seven dashboards
make use of apps to display alerts or to report data (64).
To facilitate cooperation and communication between
dashboard users, dashboards can offer the possibility to
communicate within the dashboard (67) via discussion forums,
messaging and comments (68) (n=10).
Over one third of the described dashboards offer possibility to
customize the visualization of the dashboard (n=24). Especially
zooming in or out of maps and drilling down to a specific
region, for example, enables the user to explore the data in
detail (47) (n=12). Moreover, modifying templates, charts and
other visual elements enhances user experience (59) (n=3) (see
Appendix F).
3.2. Using dashboards in public health:
challenges and objectives (RQ 2)
In terms of RQ 2, dashboard objectives offered answers
to public health challenges. First, we will sketch these
public health objectives and challenges. Second, the
objectives and challenges of public health dashboards
described in the study sample will be outlined (see
Appendix G).
3.2.1. Public health challenges addressed (RQ
2.1a)
3.2.1.1. Challenge: data collection for developing and
implementing interventions
The first challenge addresses the identification of
health threats by adequate surveillance/monitoring
systems. These health threats can be classified into
three categories. In some of the articles examined, the
disease is explicitly associated with certain risks or vice
versa, leading to counting in several categories (see
Appendix H):
Frontiers in Public Health 07 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
FIGURE 5
Example for a map with symbols, taken from “Disease Monitoring Dashboard” by Lara Savini et al. is licensed under CC BY 4.0 (38).
FIGURE 6
Example for a line graph, taken from “Trend of Number of Families faced with Unhealthy Family” by Puangrat Jinpon et al. is licensed under
CC BY-NC-ND 4.0 (49).
a) Risks such as obesity (69), environmental pollution (70), food
contamination (64), or injuries (51) (n=18);
b) Communicable/infectious diseases like Dengue Fever (33)
or reproductive tract infections (62) (n=29) as well as
Frontiers in Public Health 08 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
non-communicable diseases like cancer (55) or dementia (71)
(n=10);
c) Emergencies such as natural catastrophes (72) or human-
made disasters (73) (n=17).
All three kinds of health threats are a global issue beyond
political borders due to rising cross-border mobility, poverty or
climate change. This requires an alignment of data: So far, missing
or not transferable data makes it difficult to identify new diseases
(58), to track and explore these diseases (74) as well as to develop
strategies eliminating causes for illnesses or death (54).
3.2.1.2. Challenge: communication management and the
use of information and communication technology
Related to the first challenge is the question of how to manage
the vast amount of produced and collected information in public
health. All articles included in the sample deal in one way or
another with time, effort and cost as a key challenge in dealing
with the high volume of data and its digitisation. Articles critically
addressed an insufficient use of health-related ICT solutions in
(1) monitoring social disparities leading to higher mortality and
morbidity rates (62), (2) enabling access to health care as a
marginalized community (64), or (3) dealing adequately with mis-
and disinformation (52,75).
Furthermore, a lack of training for health workers was
stated—leading to an improper use of digital tools (75). These
shortages result in (a) a poor management of scarce resources (72),
(b) missing target group specific evidence-based communication
strategies including the tracking of health issues as an objective (52)
and (c) inefficient decision-making (76) leading to high economic
and social costs.
3.2.2. Public health objectives pursued (RQ 2.1b)
Four main public health objectives could be identified to tackle
these challenges. (A) Threatening situations shall be controlled, for
example, through surveillance or risk prevention (77) (n=40).
(B) Information management has to be improved (n=26) by for
instance enhancing knowledge (75) or addressing target groups
(78). C) Quality of life has to be enhanced (n=17) by improving
health care and services, e.g., through health promotion (39) or risk
reduction (79). D) And in response to threatening situations, public
health policies resp. measures have to be adjusted (n=16): Policy
programs focusing on health promotion, for example, need to be
sustainable and long-term, community protection initiatives need
to be supported, and digital tools for efficient decision-making need
to be implemented as well as their access guaranteed (42).
3.2.3. Specific technological or administrative
challenges related to the use of public health
dashboards (RQ 2.2a)
Besides the distinctive objectives of public health dashboards,
the reviewed literature also helps to extract various challenges (see
Appendix G) that might be of relevance while constructing, using
or deploying dashboards in a public health context. The identified
dashboard challenges refer to (a) the visualization and processing
of the data (n=46), (b) the development of the dashboard (n=7)
and (c) the use of the dashboard (n=9).
3.2.3.1. Challenges regarding the visualization and
processing of data
First and foremost, the identified literature focused on
different challenges associated with the visualization as well as the
complexity, integration, quality and analysis of data. Zhu et al. (53),
for example, underline the challenge that data visualizations need
to be adaptable to different usage patterns as well as scenarios,
while Zheng et al. (80) accentuate the need of accurate, visual
information summarization for an appropriate understanding of
e.g., crises or outbreak events. This last aspect already points to
another challenge, associated with the development and use of
public health dashboards: the complexity of visualized data. Husain
et al. (59) note that the complexity and heterogeneity of (big)
data may ultimately constrain the use of established methods, tools
and services. In this context, challenges regarding the construction
of dashboards may especially involve the need to tackle possible
information overload (76), associated with e.g., data redundancy
or the amount of information, received by a respective dashboard
system (44). Corresponding with this finding, another issue
described in the reviewed literature is the integration and transfer
of data from diverse and heterogeneous sources. Data collected
through different systems such as spreadsheets, via email or non-
interoperable systems could cause serious problems in regards to its
integration in a coherent dashboard system (65). Lack of standards
or unstructured data formats, often coming from different sources
(76), may ultimately inhibit holistic data understanding and
interpretation (59). In addition, the reviewed articles highlighted
that there are challenges in designing dashboards in terms of data
quality, especially in the context of public health. In this context,
Vila et al. (40) note diverse challenges such as data accuracy
(66) and consistency as well as ensuring and fulfilling the legally
required regulations on data protection. Lastly, the literature also
frequently discussed challenges regarding the analysis of data. Rees
et al. (37) accentuate that the type of surveillance method employed
by involved response units (for example in infectious diseases
control) can lead to an under- or overestimation in observed
prevalence. Recently, and especially concerning dashboards that
integrate data from diverse social media platforms, misinformation
has been noted as a major problem, compromising data analysis
(52). In line with this, the time needed to analyze visualized data
may also pose a major challenge in dashboard design (68).
3.2.3.2. Challenges regarding the development of the
system or dashboard
Further challenges discussed in the reviewed literature were
concerned with the development of the system incorporating a
dashboard or the dashboard itself. A concern that was selectively
addressed in the identified literature has been the cost effectiveness
in regards to a specific dashboard and its system architecture (81).
Moreover, the use and design of dashboards in a public health
context also faces legal challenges in particular, as pointed out by
Vila et al. (40). As already mentioned, the design of dashboards and
the use and visualization of specific data needs to be aligned with
and fulfill respective government regulations and laws.
Frontiers in Public Health 09 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
3.2.3.3. Challenges regarding the use of dashboards
Other major challenges that have occasionally been discussed
in the reviewed articles relate to the actual use of a dashboard.
In this context, the articles particularly highlight challenges with
regard to the use of a corresponding dashboard by specific user
groups. Key aspects in this context were that the dashboard itself
is “user-friendly” (44), implying the need to design dashboards
that are easy to understand, appealing and intuitive. Appropriately
designed systems should take the information-seeking behavior of
respective user groups and their respective health literacy skills
into account (82), as these aspects may ultimately affect the
utilization of a dashboard and the interpretation of its visualized
and aggregated data sets. Furthermore, and with special regard to
participatory dashboards, the design of a dashboard system needs
to be concerned with securing the pro-active participation of focal
user groups (68).
3.2.4. Specific technological or administrative
goals related to the use of public health
dashboards (RQ 2.2b)
Besides underlining the challenges associated with the
development or use of public health dashboards, the reviewed
literature also helps to identify objectives that are specifically
linked to the use of dashboards in a public health context (see
Appendix G). Overall, the objectives that are discussed in the
literature can be grouped into four main categories, underlining
the aims that are hoped to be achieved by implementing or using
a dashboard: (a) improving surveillance and monitoring (n=
49), (b) improving (crises) management procedures as well as
inter-agency coordination (n=22), (c) providing (public) access
to information (n=18) and, finally, (d) enabling participation (n
=8).
3.2.4.1. Improve surveillance and monitoring of public
health risks or crises
The literature reviewed primarily highlights the function of
dashboards to improve the monitoring and surveillance of, for
example, infectious disease outbreaks. Benson et al. (83) note that
dashboards might support involved response units in situational
awareness and collaborative decision-making. In this context, the
cross-verification (68) and early warning (34) of outbreak or
other adverse events as well as the possibility to trace back and
rapidly detect respective crises situations (74), were repeatedly
underlined as objectives of data visualization as well as aggregation
via dashboards. However, the discussed dashboards are not just
limited to the immediate surveillance of crises events, but also aim
at the prediction of outbreaks and other adverse events, as was
noted for the dashboard, focused on in Jamil et al. (77). More so,
dashboards aim to present relevant information and thus reduce
time spent searching for information (44).
3.2.4.2. Improve (crises) management procedures and
inter-agency coordination
The above mentioned factors associated with the improvement
of surveillance and monitoring ultimately correspond to another,
frequently discussed objective of public health dashboards: the
improvement of (crises) management procedures. In this context,
public health dashboards support decision-making under high time
pressure and thus reduce the time needed for effectively mitigating
the effects of outbreak events (63). In addition, they improve inter-
agency coordination or cross border surveillance (58) by combining
and aggregating data from agencies with different mandates (37).
Furthermore, dashboards may as well facilitate information sharing
between different actors.
3.2.4.3. Provide (public) access to information
The legitimation of political-administrative decision-making
by means of data visualization through public health dashboards
played a marginal role in the reviewed literature and, even more
so, was not mentioned as a particular objective of information
provision. Nevertheless, the relevance of public access to certain
information was discussed in a fraction of evaluated articles—both
for non-professionals and citizens as well for special user groups,
such as public health experts and professionals (64). Associated
with this, Thomas and Narayan (62), for example, discussed the
relevance of dashboards for supporting the health of citizens by
increasing access to health related information and allowing to
understand crises situations across space and time (37).
3.2.4.4. Enable participation
In addition to the mere access to or the reception of relevant
information, reviewed articles have occasionally also noted the
active involvement and inclusion of user groups in order to support
the surveillance and management of infectious disease outbreaks
or public health in general. Tegtmeyer et al. (74), for example,
cite the general participation of users as a distinctive objective of
their focal dashboard. Moreover, Rees et al. (37) explicitly note
the involvement of users in reporting—in this case: of suspect
animals—as an objective of their dashboard.
3.3. Information needs when engaging with
public health dashboards (RQ 3)
The findings presented in the following are based exclusively on
the assessment of the eighteen identified user studies. We refrain
here from quantifying aspects and thus from stating item numbers
in relation to the various information needs. This particular caution
is mainly due to the fact that relevant terms such as “ease-of-use” or
“usability” were often not operationalised consistently or at all in
the evaluated articles. This in turn has made it difficult to compare
the results of the different articles in a meaningful way. At the same
time, however, specific article numbers are not given here, as a small
ncould imply that a certain aspect was not as relevant as others
were, although this often does not have to correspond to its actual
relevance, but can also be related to the focus of the studies and the
overemphasis on other aspects.
Although the information needs of specific user groups may
vary due to the diversity of dashboards (see Appendix I), a number
of studies have identified similar core criteria.
The ease of use was one aspect frequently mentioned in
the studies. The user must be able to use the dashboard
intuitively. Some applications require technical understanding
or a certain literacy as well as skills and qualifications of the
users, which influences their acceptance of the dashboard and its
Frontiers in Public Health 10 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
implementation into the workflow (47). As described by Hamoy
et al. (75), it is beneficial to train the staff or users of the dashboard,
e.g., through workshops. The provision of a hotline can be another
way to improve acceptance and ease of use (75). Furthermore, the
technical devices should allow for easy handling of the application.
Usability is otherwise limited (e.g., small screen for displaying
complex tables).
Besides the qualifications of the users, the compatibility of the
dashboard with the work environment of the user is crucial for
its successful implementation. Several papers describe the demand
that dashboards have to be compatible with the users’ workflow.
This implies that its use (a) does not entail more work but facilitates
specific work steps like data collection, updates or analysis while
also (b) saving time (42,75). The latter often includes the need to
work with real-time data. Thus, saving time refers to both, finishing
a task in less time but also saving time in the provision of data. The
application should allow the quick update of data (61). There are
also additional delays when data needs to be validated or verified.
Dashboards that can be accessed independent of time and place are
particularly convenient (84).
Rural areas are a particular challenge with regard to the
collection of data, as the infrastructure is not always in place and
developers have to plan with fewer employees, lack of electricity,
poor internet reception, and inadequate availability of technology
(75). In this case, the question of how users can access and enter the
data is a particular challenge.
Several aspects can enhance the engagement with a dashboard
and facilitate the usage. For example, several papers state that
users wish for interactive features such as notifications. These can
be used to inform the user about news on the dashboard or can
pop-up whenever a task, such as a data upload, is completed.
Besides notifications, the possibility of networking is mentioned
to be a helpful and often requested feature of a dashboard
(61,85). Depending on the requirements, networking can include
a messaging tool, the possibility to share data or a way to comment
on or reply to other users’ posts or other forms of input (85).
As described above, a multitude of visual elements is used
in the dashboards. However, the use of different elements and
colors is rarely evaluated in detail. More often, studies describe the
overall success of the dashboard. It can be noted that the use of
colors seems to facilitate understanding and is mostly intuitively
understood [e.g., red for danger or severity, see Bernard et al. (55)].
3.4. Mixed Methods Appraisal Tool
Of the eighteen studies that were explicitly considered as user
studies (and thus considered in the critical appraisal stage via
MMAT), eight articles exclusively applied qualitative methods,
while seven articles were decidedly quantitatively oriented in
their study approach (see Appendix J). Three articles employed
a mixed methods approach by combination of qualitative and
quantitative methods. Our sample included neither randomized
controlled trials nor non-randomized studies. Surveys were the
method most often used in the quantitative studies. However, the
insufficient description of the sample and target groups in some
articles sometimes did not allow for an accurate assessment of the
representativeness of the survey sample for the target population.
Moreover, in most cases, a final assessment concerning the
risk of nonresponse bias as well as the appropriateness of the
studies’ statistical approaches was confounded by the lack of
necessary data or information in the respective papers. In regards
to the qualitative studies, interviews, were the most frequently
used method. However, in some cases, authors simply stated,
that they had received “input” from an unspecified group, which
made it difficult to clearly evaluate the methods being used in
these studies. Other methods used were focus groups as well as
participant observations.
All in all, the quality appraisal of included studies by means
of the MMAT yielded an average overall rating score of 40%,
indicating a rather moderate average methodological quality of the
eighteen studies considered in the quality appraisal step of our
literature review.
However, significant differences in overall quality can be
observed between the different types of studies. With regard to the
qualitative oriented studies considered in this step of our literature
review, a quality range of 20 to 100% can be noticed, whereby the
average score for qualitative studies was 55%, suggesting a score
higher than the overall average score. Assessing the quantitative
studies as well as studies with a mixed-methods design, we see
a considerably lower mean value with regard to the respective
study quality (qualitative studies: 29%; mixed-methods studies:
27%). However, these final assessments should be approached with
caution, since we had to select “Can’t tell” at least once in each
study, except for two qualitative oriented studies. As was discussed
above, this indicates that critical or relevant data, required for
a final assessment on a certain item, is often missing. This
deficit, however, points to a general problem of methodological
reporting in empirical studies, which is why a comprehensive
and accurate appraisal of included studies is often more difficult
than anticipated.
4. Discussion
Assuring public health in a world that is confronted with
ever changing challenges due to globalization, climate change and
various other developments demands for adapted technologies.
The results of this literature review show that dashboards cover
a wide range of public health issues—from foodborne diseases
to environmental hazards (see Appendix B), and provide data
for different target groups such as medical experts, researchers,
or specifically concerned communities. Dashboards have become
an important tool for communicating health risks through
the visualization of data—offering options such as (near) real-
time monitoring or retrieving data from a variety of sources
ranging from health authorities on different levels, healthcare
organizations to research organizations and the media. The
dashboards addressed public health objectives in at least one of
the four dimensions: Controlling threatening situations, improving
information management, enhancing quality of life and adjusting
public health policies and measures (see Appendix H).
This review examined 65 papers that allowed conclusions to
be drawn about the objectives and challenges of public health
Frontiers in Public Health 11 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
communication via dashboards. In total 18 of them also provided
user research and information on the user needs. Most of the
papers emphasized that dashboards enable users to add, enter,
copy or merge data followed by data export opportunities and
data analysis. Involving users and enabling their (continuous)
participation thus not only forms an objective of information
provision via dashboards themselves, but also aims at supporting
and improving the surveillance and management procedures,
thereby improving public health surveillance. Linked to this is
the argument that detection, prediction and the management of
outbreaks will become easier. Dashboards provide a timely and
accurate overview of the situation and automatically notify the user
of alerts. We can conclude that the overall aim is thus to raise
the situational awareness of health professionals, politicians and
citizens in general.
Secondly, communication (management) processes can be
improved through data reporting and sharing as well as specific
data visualizations such as maps or graphs. Here our systematic
review sheds light on the specific challenges faced by dashboard
developers. These range from the integration and transmission
of data from different and heterogeneous sources, to the
alignment of data with legal requirements, data accuracy, as
well as appropriate and comparable surveillance methods (see
Appendix G). Interestingly, dashboards that work with social media
data are particularly challenged when it comes to misinformation.
As for the role of misinformation in crises (51), this is a research
gap that definitely needs to be addressed.
Design is a challenge and essential: Maps showing disease
or risk distribution and diagrams in all their variations play the
most important role—often combined with questions of color
use. Graphics, animations, or audio-visual means such as social
media streams or videos were less frequently reported. Although
a variety of visual elements are used in the dashboards, a detailed
evaluation of these elements is missing, especially an evaluation
of the interdependencies of different modes such as layouts or
color. This is consistent with research gaps identified by Berg et al.
(16). The compositionality of these individual modes can produce
a different meaning compared to analyzing the modes separately
(86). In addition, and given that somewhat more than a third of the
articles included in the review describe how users can customize
the visualization of the data, a related research question for future
studies would have to be: How do dashboard users interpret
the visualized data and make an overall coherence between the
interacting modes? This also refers to the long-held recognition
that users, as recipients, need to be seen as active participants who
contribute content (87), draw their conclusions from the data on
risks and take protective measures if necessary, or may misjudge
risks, for example due to a lack of health literacy.
Another finding of this review also concerns the role of
users in improving access to information through dashboards.
Those studies considering the specific challenges and objectives
from a technological, administrative, as well as a user perspective
made evident how dashboards increase access to health related
information and enable an understanding of critical public health
issues (37,62). Important for understanding the data, however, is
health literacy, which is very rarely addressed in the sample studied.
This also corresponds to existing research gaps identified so far and
demands for future socio-technical research (13,88).
One aim of this literature review was to identify information
needs of dashboard users (see Appendix I). However, most studies
are limited to describing the process of technical construction
and design of a particular dashboard (n=47). A comparatively
small number of publications deal explicitly with the reception
of dashboards by users (n=18). Furthermore, some of these
studies are limited to a purely functional evaluation of the
dashboard by the respective development teams without applying
user-centered design approaches. Identifying information needs
by using risk communication models such as the Health Belief
Model or the Extended Risk Assessment Model is the exception
(58,62). Relevant constructs such as risk perception, perceived
severity and self-efficacy as well as existing concepts such as health
literacy, numerical literacy and data visualization literacy (88) are
not sufficiently taken into account to provide insights for data
visualization and thus increase the comprehensibility of the data.
Thus, the sample did not provide sufficient information on whether
the dashboards meet the requirements of the respective users. This
is consistent with the findings of reviews looking at public health
dashboards (11,89) revealing a relevant research gap, which should
be taken into account for future projects. Accordingly, it can be
concluded that a user-driven development strategy, theory- and
evidence-informed, is key to developing a user-friendly design
by capturing key information through a user-friendly interface
design, for example by collecting data on perceived ease of use and
perceived usefulness.
Precisely because public and scientific institutions also want
to reach the public via an open data policy with the dashboard
they created in connection with the COVID-19 pandemic (35),
these gaps need to be explored. One way to do this is to use
known communication models on information behavior to survey
information needs and to take the corresponding results into
account when designing the user interface.
4.1. Limitations
One limitation of the analysis of the papers was the inconsistent
differentiation of the term “dashboard”. While some papers only
refer to dashboards as the visual representation of data (63),
others describe entire systems that include various functions, as
dashboards (73). We applied the understanding of the term that
was expressed in the respective papers to our analysis.
As already described, the papers report little on their
methodological approach. Accordingly, the educational effect for
other researchers is limited. Even more than a shortcoming of
the respective authors, we see a possible reason in the restrictive
publication requirements of some journals, which make a detailed
description of the methods difficult or even impossible.
Although a systematic approach in retrieving articles on
public health dashboards was followed, we cannot guarantee that
all eligible studies offering answers to the research questions
were found. Firstly, we limited the number of years (2010–
2020) and databases. Since we limited the field to dashboard
solutions that are scientifically covered, the overview (Appendix B)
does not provide information on all existing public health
dashboards. Secondly, we had to differentiate between a user
Frontiers in Public Health 12 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
study and a descriptive one including brief communications
articles as well as developer studies—excluding studies that
only focus on predictive models instead of developing a real
dashboard. There may be studies in which the difference between
modeling and developing is very small. Thirdly, we conducted
a review that explicitly aimed at papers from various scientific
disciplines. The article followed specific rules of writing and
structuring articles resulting in challenges to compare data,
reporting, etc. Finally, we reviewed data reported in included
studies. We did not request any further data by contacting the
first authors.
5. Conclusion: implications for
dashboard research
The aim of our systematic review was firstly to identify the
public health challenges and objectives that were displayed by
dashboards between 2010 and 2020. Analyzing the visualization
of data and included functions, we aimed to outline solutions
that dashboards offer as a specific digital health technology.
Secondly, the review aimed to evaluate the empirical studies
that focused on the needs of the users by applying the MMAT.
Although dashboards have come to play an important role in
data-based visualization of public health issues, particularly
due to their use during the COVID-19 pandemic, the
number of publications explicitly addressing user reception
of dashboards is small. As a specific form of data visualization,
dashboards are of particular importance—especially, when
detecting and monitoring risks and crises and their effects on
public health.
The dashboards studied reflect the challenges identified in
the field of public health in relation to technological progress.
They enable faster data collection, sharing and analysis of
data. However, one identified research gap seems to be very
important with regard to the usefulness of this risk and crisis
communication tool. If the needs of users in the context
of health information behavior are not sufficiently empirically
investigated, the benefits of dashboards for risk reduction or
risk behavior change will remain without evidence. This point
goes hand in hand with the need to examine the information
behavior of specific target groups based on existing and valid
theoretical models and to think about multimodality in meaning-
making.
Applied research would benefit (a) from including risk
communication models and constructs such as scientific
literacy as well as different disciplinary perspectives
(e.g., IT, communication studies, psychology) and (b)
from a more inclusive approach that involves potential
target users throughout the construction and design
process. For this, a pre-design consideration of risk
information needs that potential target groups might have
is essential.
Data availability statement
The original contributions presented in the study are included
in the article/Supplementary material, further inquiries can be
directed to the corresponding author/s.
Author contributions
AS: conceptualization (main idea and theory), project
administration, methodology (design and operationalization),
data collection, data analysis, writing—original draft, and
writing—review and editing. FB: methodology (design and
operationalization), data collection, data analysis, writing—
original draft, and writing—review and editing. JG: data analysis,
writing—original draft, and writing—review and editing. G-FB:
conceptualization (main idea and theory) and writing—review and
editing. All authors contributed to the article and approved the
submitted version.
Acknowledgments
We would like to thank our colleagues Natalie Berger
and Severine Koch for having given advice on the research
objectives as well as Eridy Lukau (Fraunhofer Institute for Open
Communication Systems [FOKUS]) for technical insights into
dashboard architectures as well as to our colleague Till Büser for
having reviewed this paper.
Conflict of interest
The authors declare that the research was conducted in the
absence of any commercial or financial relationships that could be
construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those
of the authors and do not necessarily represent those of
their affiliated organizations, or those of the publisher,
the editors and the reviewers. Any product that may be
evaluated in this article, or claim that may be made by
its manufacturer, is not guaranteed or endorsed by the
publisher.
Supplementary material
The Supplementary Material for this article can be found
online at: https://www.frontiersin.org/articles/10.3389/fpubh.2023.
999958/full#supplementary-material
Frontiers in Public Health 13 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
References
1. Oxman AD, Fretheim A, Lewin S, Flottorp S, Glenton C, Helleve A, et al. Health
communication in and out of public health emergencies: to persuade or to inform?
Health Res Policy Syst. (2022) 20:28. doi: 10.1186/s12961-022-00828-z
2. Dixon BE, Dearth S, Duszynski TJ, Grannis SJ. Dashboards are trendy,
visible components of data management in public health: sustaining their use
after the pandemic requires a broader view. Am J Public Health. (2022) 112:900–
3. doi: 10.2105/AJPH.2022.306849
3. Matheus R, Janssen M, Maheshwari D. Data science empowering the public: data-
driven dashboards for transparent and accountable decision-making in smart cities.
Gov Inf Q. (2020) 37:101284. doi: 10.1016/j.giq.2018.01.006
4. World Health Organization. Surveillance in Emergencies. World Health
Organization (2022). Available online at: https://www.who.int/emergencies/
surveillance (accessed April 13, 2023).
5. Abeysinghe S. Global Health Governance and Pandemics: Uncertainty
and Institutional Decision-Making. In: Pandemics, Publics, and Politics:
Staging Responses to Public Health Crises. Singapore: Springer (2018). p.
11–28. doi: 10.1007/978-981-13-2802-2_2
6. Silenou BC, Nyirenda JLZ, Zaghloul A, Lange B, Doerrbecker J, Schenkel K,
et al. Availability and Suitability of Digital Health Tools in Africa for Pandemic
Control: Scoping Review and Cluster Analysis. JMIR Public Health Surveill. (2021)
7:e30106. doi: 10.2196/30106
7. Wang YC, DeSalvo K. Timely, granular, and actionable: informatics in the public
health 30 era. Am J Public Health. (2018) 108:930–4. doi: 10.2105/AJPH.2018.304406
8. Li L, Novillo-Ortiz D, Azzopardi-Muscat N, Kostkova P. Digital data sources and
their impact on people’s health: a systematic review of systematic reviews. Front Public
Health. (2021) 9:645260. doi: 10.3389/fpubh.2021.645260
9. Lourenço C, Tatem AJ, Atkinson PM, Cohen JM, Pindolia D, Bhavnani
D, et al. Strengthening surveillance systems for malaria elimination: a
global landscaping of system performance, 2015-2017. Malar J. (2019)
18:315. doi: 10.1186/s12936-019-2960-2
10. Madhavan G, Phelps CE, Rouse WB, Rappuoli R. Vision for a systems
architecture to integrate and transform population health. Proc Natl Acad Sci U S A.
(2018) 115:12595–602. doi: 10.1073/pnas.1809919115
11. Budd J, Miller BS, Manning EM, Lampos V, Zhuang M, Edelstein M, et al.
Digital technologies in the public-health response to COVID-19. Nat Med. (2020)
26:1183–92. doi: 10.1038/s41591-020-1011-4
12. Gardner L, Ratcliff J, Dong E, Katz A. A need for open public data
standards and sharing in light of COVID-19. Lancet Infect Dis. (2021)
21:e80. doi: 10.1016/S1473-3099(20)30635-6
13. Dasgupta N, Kapadia F. The future of the public health data
dashboard. Am J Public Health. (2022) 112:886–8. doi: 10.2105/AJPH.2022.
306871
14. Cicalò E, Valentino M. Mapping and Visualisation on of Health
Data. The contribution on of the graphic sciences to medical research
from new york yellow fever to china coronavirus. Disegnarecon. (2019) 12:
12-1-0.9. doi: 10.20365/disegnarecon.23.2019.12
15. Fareed N, Swoboda CM, Chen S, Potter E, Wu DTY, Sieck CJUS. COVID-
19 state government public dashboards: an expert review. Appl Clin Inform. (2021)
12:208–21. doi: 10.1055/s-0041-1723989
16. Berg SH, O’Hara JK, Shortt MT, Thune H, Brønnick KK, Lungu DA, et al.
Health authorities’ health risk communication with the public during pandemics: a
rapid scoping review. BMC Public Health. (2021) 21:1401. doi: 10.1186/s12889-021-
11468-3
17. Eppler MJ, Aeschimann M. A Systematic framework for risk
visualization in risk management and communication. Risk Manage. (2009)
11:67–89. doi: 10.1057/rm.2009.4
18. Harman JL, Weinhardt JM, Beck JW, Mai I. Interpreting Time-Series COVID
data: reasoning biases, risk perception, and support for public health measures. Sci Rep.
(2021) 11:15585. doi: 10.1038/s41598-021-95134-z
19. Chen M, Laramee RS, Ebert D, Hagen H, van Liere R, Ma KL, et al. Data,
information, and knowledge in visualization. IEEE Comput Graphics Appl. (2009)
29:12–9. doi: 10.1109/MCG.2009.6
20. Vázquez-Ingelmo A, Garcia-Peñalvo FJ, Therón R. Information dashboards and
tailoring capabilities - a systematic literature review. IEEE Access. (2019) 7:109673–
88. doi: 10.1109/ACCESS.2019.2933472
21. VahediA, Moghaddasi H, Asadi F, Hosseini AS, Nazemi E. Applications, features
and key indicators for the development of covid-19 dashboards: a systematic review
study. Inform Med Unlocked. (2022) 30:100910. doi: 10.1016/j.imu.2022.100910
22. World Health Organization. The Public Health Dimension of the World Drug
Problem: How Who Works to Prevent Drug Use, Reduce Harm and Improve Safe
Access to Medicines. Geneva: World Health Organization, (2019). Report No.: Contract
No.: WHO/MVP/EMP/2019.02.
23. World Health Organization. Who European Regional Obesity Report 2022.
Copenhagen: World Health Organization. Regional Office for Europe (2022) 2022.
24. Xiao Y, Watson M. Guidance on conducting a systematic literature review. J
Plann Educ Res. (2019) 39:93–112. doi: 10.1177/0739456X17723971
25. Everts J. The dashboard pandemic. Dialogues Hum Geogr. (2020) 10:260–
4. doi: 10.1177/2043820620935355
26. Belur J, Tompson L, Thornton A, Simon M. Interrater reliability in systematic
review methodology: exploring variation in coder decision-making. Sociol Methods Res.
(2021) 50:837–65. doi: 10.1177/0049124118799372
27. Landis JR, Koch GG. The measurement of observer agreement for categorical
data. Biometrics. (1977) 33:159–74. doi: 10.2307/2529310
28. Mateen FJ, Oh J, Tergas AI, Bhayani NH, Kamdar BB. Titles versus titles and
abstracts for initial screening of articles for systematic reviews. Clin Epidemiol. (2013)
5:89–95. doi: 10.2147/CLEP.S43118
29. Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The
mixed methods appraisal tool (MMAT) version 2018 for information professionals and
researchers. Educ Inf. (2018) 34:1–7. doi: 10.3233/EFI-180221
30. Hong QN. Reporting the Results of the MMAT (Version 2018). (2020).
Available online at: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/
fetch/140056890/Reporting%20the%20results%20of%20the%20MMAT.pdf (accessed
July 15, 2022).
31. Mayring P. Qualitative Content Analysis: Theoretical Foundation, Basic
Procedures and Software Solution. Klagenfurt: Klagenfurt. (2014). Available online
at: https://nbn-resolving.org/urn:nbn:de:0168- ssoar-395173 (accessed July 15, 2022).
32. Krippendorff K. Content Analysis: An Introduction to Its Methodology. London:
SAGE Publications (2018). doi: 10.4135/9781071878781
33. Husain W, Abdul Aziz FS, Abdul Rashid NA, Jothi N. M-DENGUE utilizing
crowdsourcing and teleconsultation for location-based dengue monitoring and
reporting system. J Teknol. (2016) 78:89–95. doi: 10.11113/jt.v78.9722
34. Tom-Aba D, Olaleye A, Olayinka AT, Nguku P, Waziri N, Adewuyi P, et al.
Innovative technological approach to ebola virus disease outbreak response in
nigeria using the open data kit and form hub technology. PLoS ONE. (2015)
10:e0131000. doi: 10.1371/journal.pone.0131000
35. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-
19 in real time. Lancet Infect Dis. (2020) 20:533–4. doi: 10.1016/S1473-3099(20)30120-1
36. Saha S, Shekhar S, Sadhukhan S, Das P. An analytics dashboard
visualization for flood decision support system. J Vis. (2018) 21:295–
307. doi: 10.1007/s12650-017-0453-3
37. Rees EE, Gendron B, Lelièvre F, Coté N, Bélanger D. Advancements in
web-database applications for rabies surveillance. Int J Health Geogr. (2011)
10:48. doi: 10.1186/1476-072X-10-48
38. Savini L, Tora S, Di Lorenzo A, Cioci D, Monaco F, Polci A, et al. A web
geographic information system to share data and explorative analysis tools: the
application to west nile disease in the mediterranean basin. PLoS ONE. (2018)
13:e0196429. doi: 10.1371/journal.pone.0196429
39. Gourevitch MN, Athens JK, Levine SE, Kleiman N, Thorpe LE. City-level
measures of health, health determinants, and equity to foster population health
improvement: the city health dashboard. Am J Public Health. (2019) 109:585–
92. doi: 10.2105/AJPH.2018.304903
40. Vila RA, Estevez E, Fillottrani PR. The design and use of dashboards for
driving decision-making in the public sector. In: Kankanhalli A, Ojo A, Soares D,
editors. Proceedings of the 11th International Conference on Theory and Practice
of Electronic Governance. New York: Assoc Computing Machinery (2018). p. 382–
8. doi: 10.1145/3209415.3209467
41. Erraguntla M, May L, Gopal B, Mayer RJ, Benjamin PC. Open source text
based biovigilance. In: Proceedings of the 2012 International Conference on Artificial
Intelligence (ICAI 2012, Vol 1). Las Vegas, NV: ICAI. (2012). p. 417–22.
42. Harris JK, Hinyard L, Beatty K, Hawkins JB, Nsoesie EO, Mansour R, et al.
Evaluating the implementation of a twitter-based foodborne illness reporting tool in
the City of St. Louis Department of Health. Int J Environ Res Public Health. (2018)
15:833. doi: 10.3390/ijerph15050833
43. Kostkova P. A roadmap to integrated digital public health surveillance: the
vision and the challenges. In: WWW ’13 Companion Proceedings of the 22nd
International Conference on World Wide Web. Rio de Janeiro, Brazil (2013). p. 687–
93. doi: 10.1145/2487788.2488024
44. Zheng L, Shen C, Tang L, Li T, Luis S, Chen S-C, et al. Using data mining
techniques to address critical information exchange needs in disaster affected public-
private networks. In: Proceedings of the 16th ACM SIGKDD InternationalConference on
Frontiers in Public Health 14 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
Knowledge Discovery and Data Mining. Washington, DC: Association for Computing
Machinery (2010). p. 125–34. doi: 10.1145/1835804.1835823
45. Kamadjeu R, Gathenji C. Designing and implementing an electronic
dashboard for disease outbreaks response - case study of the 2013-
2014 somalia polio outbreak response dashboard. Pan Afr Med J. (2017)
27:22. doi: 10.11604/pamj.supp.2017.27.3.11062
46. Urosevic V, Paolini P, Tatsiopoulos C. An interactive environment for managing
detected data towards geriatric prevention. In: 2017 IEEE 3rd International Forum on
Research and Technologies for Society and Industry (RTSI). Modena, Italy (2017). p.
1–5. doi: 10.1109/RTSI.2017.8065968
47. Concannon D, Herbst K, Manley E. Developing a data dashboard framework for
population health surveillance: widening access to clinical trial findings. JMIR Format
Res. (2019) 3:e11342. doi: 10.2196/11342
48. Basdere M, Caniglia G, Collar C, Rozolis C, Chiampas G, Nishi M, et al.
Safe: a comprehensive data visualization system. INFORMS J Appl Analyt. (2019)
49:249–61. doi: 10.1287/inte.2019.0989
49. Jinpon P, Jaroensutasinee M, Jaroensutasinee K. Integrated information
visualization to support decision making for health promotion in Chonburi, Thailand.
Walailak J Sci Technol. (2017) 16:551–60. doi: 10.48048/wjst.2019.2181
50. Wissel BD, Van Camp PJ, Kouril M, Weis C, Glauser TA, White PS, et al. An
interactive online dashboard for tracking COVID-19 in US counties, cities, and states
in real time. J Am Med Inform Assoc. (2020) 27:1121–5. doi: 10.1093/jamia/ocaa071
51. Pike I, Smith J, Al-Hajj S, Fuselli P, Macpherson A. The canadian atlas
of child and youth injury: mobilizing injury surveillance data to launch a
national knowledge translation tool. Int J Environ Res Public Health. (2017)
14:982. doi: 10.3390/ijerph14090982
52. Martinez LS, Tsou MH, Spitzberg BH. A case study in belief surveillance,
sentiment analysis, and identification of informational targets for e-cigarettes
interventions. In: SMSociety ’19: Proceedings of the 10th International Conference on
Social Media and Society. Toronto, ON: Association for Computing Machinery (2019).
p. 15–23. doi: 10.1145/3328529.3328540
53. Zhu Z, Heng BH, Teow KL. Interactive data visualization to understand
data better: case studies in healthcare system. Decis Manage. (2017) 4:1–
10. doi: 10.4018/IJKDB.2014070101
54. Bhardwaj S, Barron P, Pillay Y, Treger-Slavin L, Robinson P, Goga A, et al.
Elimination of mother-to-child transmission of HIV in South Africa: rapid scale-up
using quality improvement. S Afr Med J. (2014) 104:239–43. doi: 10.7196/SAMJ.7605
55. Bernard J, Sessler D, Kohlhammer J, Ruddle RA. Using dashboard networks to
visualize multiple patient histories: a design study on post-operative prostate cancer.
IEEE Trans Vis Comput Graph. (2019) 25:1615–28. doi: 10.1109/TVCG.2018.2803829
56. Rees K. Periscopic visualizes symptomatology of pandemic: vast 2010 mini
challenge 2 award: effective visualization of symptoms. In: 2010 IEEE Symposium
on Visual Analytics Science and Technology. Salt Lake City, UT (2010). p. 283–
4. doi: 10.1109/VAST.2010.5652678
57. Federico L, Franco P, Minelli A, Perri A, Caroprese L, Picarelli R, et al. SINSE+:
A software for the acquisition and analysis of open data in health and social area.
In: Bochicchio MA, Mecca G, editors. 24th Italian Symposium on Advanced Database
Systems (SEBD 2016). Ugento, Lecce, Italy: Sistemi Evoluti per Basi di Dati (SEBD)
(2016). p. 310–7.
58. ter Waarbeek H, Hoebe C, Freund H, Bochat V, Kara-Zaïtr C.
Strengthening infectious disease surveillance in a dutch-german crossborder
area using a real-time information exchange system. J Bus Contin Emer Plan.
(2011) 5:173–84.
59. Husain SS, Kalinin A, Truong A, Dinov ID. SOCR data dashboard: an integrated
big data archive mashing medicare, labor, census and econometric information. J Big
Data. (2015) 2:13. doi: 10.1186/s40537-015-0018-z
60. Perez-Gonzalez CJ, Colebrook M, Roda-Garcia JL, Rosa-Remedios
CB. Developing a data analytics platform to support decision making
in emergency and security management. Expert Syst Appl. (2019)
120:167–84. doi: 10.1016/j.eswa.2018.11.023
61. Thorve S, Wilson ML, Lewis BL, Swarup S, Vullikanti AKS, Marathe MV.
Epiviewer: an epidemiological application for exploring time series data. BMC
Bioinform. (2018) 19:449. doi: 10.1186/s12859-018-2439-0
62. Thomas M, Narayan P. The role of participatory communication
in tracking unreported reproductive tract issues in marginalized
communities. Inf Technol Dev. (2016) 22:117–33. doi: 10.1080/02681102.2014.
886549
63. Campbell TC, Mistry ZS, Gorelick-Feldman GN, Hodanics CJ, Babin SM,
Lewis SH. Development of the respiratory disease dashboard for the identification
of new and emerging respiratory pathogens. Johns Hopkins APL Tech Dig.
(2014) 32:726–34.
64. Singh SK. Conceptual framework of a cloud-based decision support
system for arsenic health risk assessment. Enviro Sys Decis. (2017)
37:435–50. doi: 10.1007/s10669-017-9641-x
65. Meng Y, Zhang Y, Wang S, Wu S, Zhou H, Ke C, et al. Lessons
learned in the development of a web-based surveillance reporting system
and dashboard to monitor acute febrile illnesses in Guangdong and Yunnan
Provinces, China, 2017-2019. Health Secur. (2020) 18:14–22. doi: 10.1089/hs.
2019.0079
66. Devi LN, Reddy GK, Rao AN. Live demonstration on smart water quality
monitoring system using wireless sensor networks. In: 2018 IEEE SENSORS. New
Delhi, India (2018). p. 1–4. doi: 10.1109/ICSENS.2018.8589931
67. Brownson RC, Kemner AL, Brennan LK. Applying amixed-methods evaluation
to healthy kids, healthy communities. J Public Health Manage Pract. (2015) 21:16–
26. doi: 10.1097/PHH.0000000000000233
68. Kostkova P, Garbin S, Moser J, Pan W. Integration and Visualization
Public Health Dashboard: The Medi+Board Pilot Project. In: WWW ’14
Companion: Proceedings of the 23rd International Conference on World Wide
Web. Seoul, Korea: Association for Computing Machinery, Inc. (2014). p.
657–62. doi: 10.1145/2567948.2579276
69. Ryan K, Pillai P, Remington PL, Malecki K, Lindberg S. Development of an
obesity prevention dashboard for wisconsin. Wisc Med J. (2016) 115:224–7.
70. Alvarez VC, Huang CY. Visualization of Health Data. In: Lin JCW, Ting IH,
Wang K, Tang T, editors. Multidisciplinary Social Networks Research, 6th International
Conference, MISNC 2019, Wenzhou, China, August 26–28, 2019, Springer. (2019). p.
118–30. doi: 10.1007/978-981-15-1758-7_10
71. Robertson H, Nicholas N, Dhagat A, Travaglia J. A Spatial Dashboard for
Alzheimer’s Disease in New South Wales. In: Ryan A, Schaper LK, Whetton S,
editors. Integrating and Connecting Care.Studies in HealthTechnology and Informatics.
Amsterdam: Ios Press (2017). p. 126–32.
72. Luchetti G, Mancini A, Sturari M, Frontoni E, Zingaretti P. Whistland:
an augmented reality crowd-mapping system for civil protection and
emergency management. ISPRS Int J Geo-Inf. (2017) 6:41. doi: 10.3390/
ijgi6020041
73. Avvenuti M, Cresci S, Vigna FD, Fagni T, Tesconi M. CrisMap: a big data crisis
mapping system based on damage detection and geoparsing. Inf Syst Fronts. (2018)
20:993–1011. doi: 10.1007/s10796-018-9833-z
74. Tegtmeyer R, Potts L, Hart-Davidson W. Tracing and Responding
to Foodborne Illness. In: Proceedings of the 30th ACM International
Conference on Design of Communication. Seattle, Washington, USA:
Association for Computing Machinery (2012). p. 369–70. doi: 10.1145/2379057.
2379131
75. Hamoy GL, Amoranto AJP, Evangelista-Sanchez AMA, Pajarillaga ED, Ongkeko
AM, Sylim PG, et al. Real-Time Regular Routine Reporting for Health (R4health):
lessons from the implementation of a large scale mobile health system for
routine health services in the Philippines. Acta Med Philippina. (2016) 50:280–
94. doi: 10.47895/amp.v50i4.775
76. Nascimento BS, Vivacqua AS, Borges MRS. A flexible architecture for selection
and visualization of information in emergency situations. In: 2016 IEEE International
Conference on Systems, Man, and Cybernetics (SMC 2016). Budapest, Hungary: Institute
of Electrical and Electronics Engineers Inc. (2017). p. 3317–22. doi: 10.1109/SMC.2016.
7844746
77. Jamil JM, Shaharanee INM, Yung VC. An innovative data mining and dashboard
system for monitoring of malaysian dengue trends. J Telecommun Electron Comput
Eng. (2016) 8:9–12. doi: 10.1063/1.4960910
78. Homsuwan P, Soonklang K, Pongpun W, Saeloo S, Roekkasame S, Ungtrakul
T, et al. Visualization development of health data reporting with business intelligence
techniques. J Med Assoc Thailand. (2018) 101:49–54.
79. Mulero R, Urosevic V, Almeida A, Tatsiopoulos C. Towards
ambient assisted cities using linked data and data analysis. J Ambient
Intell Humanized Comput. (2018) 9:1573–91. doi: 10.1007/s12652-018-
0916-y
80. Zheng L, Shen C, Tang L, Zeng CQ Li T, Luis S, et al. Data mining meets the
needs of disaster information management. IEEE T Hum-Mach Syst. (2013) 43:451–
64. doi: 10.1109/THMS.2013.2281762
81. Choudhary V, Teh JH, Beltran V, Lim HB. AirQ: a smart
iot platform for air quality monitoring. In: 2020 IEEE 17th Annual
Consumer Communications & Networking Conference (CCNC).
Las Vegas, NV (2020). p. 1–2. doi: 10.1109/CCNC46108.2020.904
5550
82. Waye KM, Yedinak JL, Koziol J, Marshall BDL. Action-focused, plain language
communication for overdose prevention: a qualitative analysis of rhode island’s
overdose surveillance and information dashboard. Int J Drug Policy. (2018) 62:86–
93. doi: 10.1016/j.drugpo.2018.08.010
83. Benson AL, Biggers K, Wall J, Haselkorn MP. Adaptive development of a
common operating environment for crisis response and management. In: Zobel CF, B.
T. S., editor. ISCRAM 2010 - 7th International Conference on Information Systems for
Crisis Response and Management: Defining Crisis Management 30, Proceedings. Seattle,
WA: Information Systems for Crisis Response and Management, ISCRAM (2010).
84. Pathirannehelage S, Kumarapeli P, Byford R, Yonova I, Ferreira F, de Lusignan
S. Uptake of a Dashboard designed to give realtime feedback to a sentinel network
about key data required for influenza vaccine effectiveness studies. Stud Health Technol
Inform. (2018) 247:161–5. doi: 10.3233/978-1-61499-852-5-161
Frontiers in Public Health 15 frontiersin.org
Schulze et al. 10.3389/fpubh.2023.999958
85. Estuar MRE, Ilagan JO, Victorino JN, Canoy N, Lagmay M, Hechanova
MR. The challenge of continuous user participation in ebayanihan: digitizing
humanitarian action in a nationwide web mobile participatory disaster management
system. In: 2016 3rd International Conference on Information and Communication
Technologies for Disaster Management (ICT-DM). Vienna, Austria (2016). p. 1–
8. doi: 10.1109/ICT-DM.2016.7857215
86. Bucher H-J. Understanding multimodal meaning making: theories of
multimodality in the light of reception studies. In: Seizov O, Wildfeuer J, editors. New
Studies in Multimodality Conceptual and Methodological Elaborations. Bloomsbury
Publishing (2017). p. 91–123.
87. Lupton D. Digital Health: Critical and Cross-Disciplinary
Perspectives. London: Routledge. (2017). doi: 10.4324/97813156
48835
88. Börner K, Bueckle A, Ginda M. Data visualization literacy: definitions,
conceptual frameworks, exercises, and assessments. Proc Nat Acad Sci. (2019)
116:1857–64. doi: 10.1073/pnas.1807180116
89. Murphy DR, Savoy A, Satterly T, Sittig DF, Singh H. Dashboards for visual
display of patient safety data: a systematic review. BMJ Health Care Inform. (2021)
28:e100437. doi: 10.1136/bmjhci-2021-100437
Frontiers in Public Health 16 frontiersin.org
