Citizen science to further precision medicine: From vision to implementation

Abstract

The active involvement of citizen scientists in setting research agendas, partnering with academic investigators to conduct research, analyzing and disseminating results, and implementing learnings from research can improve both processes and outcomes. Adopting a citizen science approach to the practice of precision medicine in clinical care and research will require healthcare providers, researchers, and institutions to address a number of technical, organizational, and citizen scientist collaboration issues. Some changes can be made with relative ease, while others will necessitate cultural shifts, redistribution of power, recommitment to shared goals, and improved communication. This perspective, based on a workshop held at the 2018 AMIA Annual Symposium, identifies current barriers and needed changes to facilitate broad adoption of a citizen science-based approach in healthcare.

Full text

Perspective
Citizen science to further precision medicine: from vision
to implementation
Carolyn Petersen,
1
Robin R. Austin,
2
Uba Backonja,
3,4
Hugo Campos,
5
Arlene E.
Chung,
6
Eric B. Hekler,
7
Pei-Yun S. Hsueh,
8
Katherine K. Kim,
9
Anthony Pho,
10
Liz Salmi,
11
Anthony Solomonides,
12
and Rupa S. Valdez
13
1
Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, Minnesota, USA,
2
School of Nursing, University of
Minnesota, Minneapolis, Minnesota, USA,
3
Nursing & Healthcare Leadership, University of Washington Tacoma, Tacoma, Wash-
ington, USA,
4
Biomedical Informatics & Medical Education, University of Washington School of Med icine, Seatt le, Wa shing-
ton, USA,
5
Kaiser Permanente, Kaiser Permanente Innovation, Oakland, California, USA,
6
Departments of Internal Medicine and
Pediatrics & the Program on Health & Clinical Informatics, UNC School of Medicine, Chapel Hill, North Carolina, USA,
7
Depart-
ment of Family Medicine & Public Health, Center for Wireless & Population Health Systems, Design Lab, Qualcomm Institute, Uni-
versity of California-San Diego, San Diego, California, USA,
8
Center for Computational Health, IBM TJ Watson Research Center,
Yorktown Heights, New York, USA,
9
Betty Irene Moore School of Nursing, Department of Public Health Sciences-School of Medi-
cine, University of California-Davis, Sacramento, California, USA,
10
School of Nursing, Columbia University, New York, New York,
USA,
11
OpenNotes/Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA,
12
Family Medicine, NorthShore Univer-
sity HealthSystem, Evanston, Illinois, USA and
13
Department of Public Health Sciences, University of Virginia School of Medicine,
Charlottesville, Virginia, USA
Corresponding Author: Carolyn Petersen, MS, MBI, FAMIA, Division of Biomedical Statistics and Informatics,
Mayo Clinic, Minnesota BioBusiness Building, Rochester, MN, 55905, USA; petersen.carolyn@mayo.edu
Received 15 January 2019; Revised 4 September 2019; Editorial Decision 7 October 2019; Accepted 25 October 2019
ABSTRACT
The active involvement of citizen scientists in setting research agendas, partnering with academic investigators
to conduct research, analyzing and disseminating results, and implementing learnings from research can im-
prove both processes and outcomes. Adopting a citizen science approach to the practice of precision medicine
in clinical care and research will require healthcare providers, researchers, and institutions to address a number
of technical, organizational, and citizen scientist collaboration issues. Some changes can be made with relative
ease, while others will necessitate cultural shifts, redistribution of power, recommitment to shared goals, and
improved communication. This perspective, based on a workshop held at the 2018 AMIA Annual Symposium,
identifies current barriers and needed changes to facilitate broad adoption of a citizen science-based approach
in healthcare.
Key words: research methodology, healthcare systems, community participation, consumer involvement, patient acceptance of
healthcare, citizen science
INTRODUCTION
Precision medicine has been defined as the development of preven-
tion and treatment strategies that take individual variability into ac-
count.
1
To date, precision medicine research efforts (eg, All of Us
Research Program
2
) have emphasized largely expert-driven efforts
for better identifying different interventions that work for different
people at different times. A central promise of precision medicine is
prevention and treatment approaches that move beyond “on
V
CThe Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/),
which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact
journals.permissions@oup.com 1
JAMIA Open, 0(0), 2019, 1–7
doi: 10.1093/jamiaopen/ooz060
Perspective
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020

average” patient responses
3
to create more personalized, targeted
interventions.
4–6
As evidenced by the extended discussion about var-
ious “precision” efforts,
4–16
how to realize the vision of precision
medicine remains an open question. A key consideration is how to
include patients, citizens, patient innovators, patient informati-
cians, citizen scientists, and scientific citizens in the process and
how informatics infrastructure can incorporate these contributions
meaningfully.
Citizen science has been defined as “the general public engage-
ment in scientific research activities when citizens actively contribute
to science either with their intellectual effort or surrounding knowl-
edge or their tools and resources.”
17
Citizen science initiatives have
contributed to academic science and continue to broaden its scope
and depth.
18
Although much of the work is conducted outside the
walls of academia or other organizational settings via novel meth-
ods, some patients who take on the role of citizen scientists do use
traditional research and/or informatics approaches to answer their
questions.
19
Citizen science can empower individuals to generate scientific
questions and share their data. Citizens can generate answers to
population health questions of interest to both patients and the
healthcare system,
20,21
making citizen science a feasible approach to
healthcare research.
22–25
Citizen science offers an opportunity to
empower marginalized groups, such as sexual and gender minori-
ties, to shape scientific inquiry through participation.
26,27
Engage-
ment of citizen scientists enables studies that aren’t accessible
without them. Technological advances, changing reimbursement
models, innovative informed consent approaches, and other factors
are driving a shift in power dynamics within healthcare, affording
greater integration of citizen scientists’ work into research and clini-
cal care.
28–32
Citizen science has the potential to extend and enhance the prac-
tice of precision medicine if healthcare practitioners and citizen sci-
entists work together to develop scientifically sound
approaches.
33,34
For example, the Personal Genome Project UK uses
a citizen science approach in which participants have consented to
open access release of personal genome, methylome, and transcrip-
tome data and analyses thought to influence gene function.
35
However, the general movement toward citizen science is more
mature in fields other than medicine, and this situation creates an
opportunity for informaticians to think through what citizen sci-
ence means—and could mean—in the informatics context. Within
fields such as biology, conservation, ecology, and astronomy, pre-
viously unimaginable insights into topics such as migration pat-
terns, scope of biodiversity, and asteroid surveillance have been
realized through citizen science-based approaches.
36
In compari-
son, citizen science-based efforts within the field of population
health, clinical medicine, and consumer health are relatively na-
scent.
21,37–40
This viewpoint delineates the minimum technical, or-
ganizational, and citizen engagement requirements needed to
facilitate meaningful integration of citizen science into existing
healthcare systems. The following implementation recommenda-
tions were identified during an expert workshop held at the 2018
AMIA Annual Symposium.
TECHNICAL ISSUES
To support a diverse, open, integrated citizen science ecosystem, sev-
eral technical issues must be addressed. Though some considerations
are specific to citizen science, others relate more broadly to the
healthcare environment’s increasingly person-centric approach.
41
Platforms that facilitate collaboration among patients
and researchers
To accelerate participation, it is critical to have easily accessible, us-
able technology platforms that facilitate idea generation, prioritiza-
tion of research questions, community-building between citizens
and researchers, and results dissemination. Research networks have
demonstrated effective approaches to this challenge although these
projects relied on one-off technical tools.
42–44
Crowdfunding plat-
forms have emerged as examples of how individuals, scientists, and
advocates can reach out to the public for donations and invest-
ments.
45
Although participants have primarily used these platforms
for outreach purposes, these platforms have not enabled collabora-
tive generation of science. Hence, an open-source approach that
citizen-scientist communities could leverage and build upon could
accelerate patient and researcher matching.
A distributed network
Citizen science at scale requires a network facilitating movement of
heterogeneous data sources into a large-scale system in real time.
Such a centralized system, however, further necessitates creating a
governing body which is still able to preserve and promote citizen
scientists’ autonomy within such a framework. In addition, such a
network must facilitate compliance with local and national privacy
regulations, among others. The growth in infrastructure that facili-
tates data sharing will support greater implementation of citizen sci-
ence,
46
so resolution of governance and related infrastructure issues
is paramount. Platforms designed specifically for citizen science
projects support this objective.
47
Advancing ethical structures to
achieve safety, as used in some patient-led efforts such as the Con-
nected and Open Research Exchange,
48
is critical.
49,50
Functionality for tracing provenance of data
contributions from individual data sources
The ability to potentially track the provenance of common types of
data from individuals and organizations is recommended so investi-
gators can explain underlying assumptions and potential sources of
bias in studies. These metadata should be collected only when indi-
viduals wish to have identifiable data elements shared. Ideally, such
functionality would support the remuneration of individuals in ex-
change for sharing their data. Of course, when participants desire to
make personal data open access, as in PGP-UK,
35
system functional-
ity should support this approach as well, provided that the shared
personal data fall under regulation.
Functionality supporting citizen scientist access to their
own health information
Tools that give citizen scientists access to their complete health in-
formation (eg, clinical notes, lab results, and radiology reports) are
central to advancement of the field. This requirement is important
because people who read medical notes report having a better under-
standing of their conditions, are more likely to take medications as
prescribed, and report greater satisfaction and trust of their doc-
tors.
51,52
Similarly, access to diverse types of data from electronic
health records will allow people to provision access and sharing of
their personal data. It is important to acknowledge that individuals
may have a range of motivations that lead to their participation as
citizen scientists. It will be essential for those engaged in supporting
and promoting citizen science efforts in precision medicine to under-
stand and be responsive to this range of motivations, which may
span from an interest in improving individual health to contributing
2JAMIA Open, 2019, Vol. 0, No. 0
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020

to a larger social movement. Technology that provides citizen scien-
tists some control over the management and sharing of data they
contributed will also incentivize research engagement.
53
A standardized, comprehensive approach to
identification and resolution of barriers to data access
Citizen scientists need a standardized process for identifying and
addressing local, state, and federal policies that hinder or obstruct
access to personal health data. This article was formulated in the
context of the US-based healthcare system. Within this context,
patients are increasingly expected to assume responsibilities related
to managing their own health. Proactive management of personal
health benefits from access to data from health providers is currently
difficult to obtain due to restrictive policies. Although we expect cit-
izen scientists to play a key role in advocating for revised policies, it
will be essential to create structures for sharing the burden of
change, particularly in under-resourced settings. With such a process
in place, they could then identify ways to facilitate patients’ access
to their health data. For example, they could negotiate an under-
standing among competing health systems to adopt a data access
policy providing equivalent access for all patients, thereby reducing
the impact of differences in how health systems manage and share
data. Such understandings will be critical in the future as greater
quantities of genomic data become available and both traditional
researchers and citizen scientists seek to use multiple datasets to an-
swer questions of interest.
ORGANIZATIONAL ISSUES
Traditional research institutions must address a number of organiza-
tional issues to work effectively with citizen scientists.
Need to honor agency in individuals, families, and
communities
For citizen science work to thrive, organizations should avoid treat-
ing engaged patients as the exception or as token participants in a
provider-centric system. Acknowledging the contributions of the in-
dividual, family, and community implies respect and an understand-
ing that everyone comes to the table with different abilities and
perspectives. The ideal of honoring agency strives to reduce the au-
thoritative assumption that individuals, families, or communities
need to become “engaged,” when in reality they already are, though
perhaps outside the priorities of healthcare institutions. It includes
recognition of the many forms of expertise that each person (includ-
ing patients) possesses that facilitate progress toward healthier liv-
ing, the need to honor the validity of different perspectives,
54
and
the need to build empathy and acknowledgement of power differen-
tials as a potentially hidden barrier to equitable participation, con-
tribution, and benefit.
55
Commitment to ethical behavior
Organizations must develop an ethical framework rooted in com-
munity values on which to base citizen science efforts.
56
This
framework should inform a broad spectrum of issues, including
but not limited to privacy, security, data ownership, informed con-
sent, consideration of potential unintended consequences of data
sharing, and patient and researcher access. Such a framework
would inform how a local citizen science community might create
a collaborative, less hierarchical culture based on mutual respect
(rather than power) and ethical behavior by all that rewards curi-
osity and fosters trustworthiness. Substantive work has already
been undertaken in this area, as evidenced by the European Citizen
Science Association’s “Ten Principles of Citizen Science,” which
(among other things) emphasize mutual benefit to professional and
citizen scientists, availability of project data and metadata to the
public, and acknowledgement of citizen scientists in results and
publications.
57
Research questions that arise from citizen scientists,
rather than researchers, in a community-based participatory re-
search environment, offer another way to honor the commitment
to ethical behavior.
New and expanded funding models
Because citizen science is participant-driven, it often lacks access to
the traditional sources of funding (eg, government research awards,
foundation grants) that support research in universities and aca-
demic medical centers. Within the United States, funding institutions
(eg, the Robert Wood Johnson Foundation, the National Institutes
of Health) have demonstrated growing interests in models of citizen
science and those that are closely related, such as community-based
participatory research. These funding opportunities, however, fol-
low conventional models of time constraints. As such, while there
are growing opportunities to initiate citizen-science-based research
efforts, questions of sustainability remain unanswered. Funding
structures for citizen science, including financial models that explore
contribution of data where participants have substantial involve-
ment, are needed. Patient-powered research networks may be such a
model, although their sustainability too remains unproven.
New research models
Partnering institutions must embrace a variety of research models.
Health- and self-related citizen science does not necessarily imply a
shared experience, but technology can support both scientific
endeavors and varied forms of sharing and network-building. Many
technology adaptations have been developed as part of the Quanti-
fied Self movement (eg, information retrieval skills for evidence
gathering, measurement device development, provider alliances
based on shared decision-making),
58
and these and other approaches
need to be developed and implemented more broadly.
59
The use of
self-monitoring technologies to create persuasive performance feed-
back that motivates technology users to continue practicing healthy
behaviors for further health improvement is one such new model.
60
Crowdfunding and crowdsourcing approaches have proven success-
ful in microbiome and metagenomics research,
61
and may offer op-
portunities for researcher-citizen scientist partnerships. Actively
engaging patients and caregivers to prevent medical errors has
resulted in new technology designs and improved communication
between clinicians and hospitalized patients.
62,63
Advancing an
“agile scientific” model
64
that focuses first on helping individuals
rather than on producing generalizable knowledge aligns with orga-
nizational structures such as learning healthcare systems, but with
clearer integration of citizen scientists.
65
Authoritative research and engagement guidelines
Patient-researcher partnerships in citizen science range from easily
identifiable relationship models (eg, mentorships, co-mentorships,
and patients as co-investigators) to nontraditional/creative solutions
(eg, patients as principal investigator, patients working indepen-
dently). Citizen science efforts follow a continuum from researcher-
initiated to citizen scientist-initiated, with truly collaborative models
residing in the middle of this spectrum. Each participation model
JAMIA Open, 2019, Vol. 0, No. 0 3
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020

leads to different requirements, roles, and expectations for the indi-
viduals involved in the effort. All parties must seek to understand
how these new models fit within existing academic hierarchies and
promote successful collaboration. The growing community of
researchers with academic affiliations and research-minded patients
and patients doing citizen science requires mutually accepted guide-
lines for engagement and practice.
Broad outreach to allied groups for shared learning and
advancement
The current practice of citizen science, whether consciously de-
scribed as “citizen science,” as patient-clinician engagement,
66
or
otherwise, has evolved in many areas without reference to each
other. Although groups and programs have shared goals, needs, and
challenges, they often lack awareness of and access to each other,
and so miss opportunities for shared learning and development. Citi-
zen scientists who receive healthcare in the United States have a very
different experience than their counterparts in other regions where
universal healthcare, minimal copayment requirements, and rela-
tively smaller health disparities are the norm, and as such, their
practice of citizen science naturally has evolved differently. US-
based citizen scientists in healthcare may learn from colleagues prac-
ticing citizen science elsewhere, wholesale adoption of and align-
ment with other national models is neither likely nor necessarily
beneficial. Citizen science practiced in the United States, in particu-
lar, needs infrastructure that facilitates communication and informa-
tion sharing among organizations, programs, citizen scientists
working in other regions, and potential partners (e.g., funders, gov-
ernment agencies).
COLLABORATION ISSUES
In addition to technical and organizational issues, several collabora-
tion barriers must be broken down.
Failure to see patients as collaborators
Many healthcare institutions lack a culture of working with citizen
scientists and may require deconstruction of a top-down culture that
locates authority in medical professionals. A foundation for culture
change arises through recognition of the value of patient-clinician
partnerships;
67
healthcare organizations must consciously work to-
ward this goal. The historical failure to recognize citizen scientists as
qualified interpreters of human genomes and related material must
be addressed going forward.
68
Targeted approaches for bringing patients into research
efforts
Because patients may have little background in science, they may
need training in research methodology, ethics, and regulation.
Training has already been proven feasible and productive.
69,70
When participants can contribute and prioritize research questions,
and then see the progression from idea to research study to results,
their interest and participation are reinforced.
71
Planning research
efforts of limited duration may facilitate recruitment of patient-
researchers in that minimally ambitious efforts require less time of
patients who may already be juggling health needs, job and family
responsibilities, and other commitments.
72
Similarly, to ensure a
truly reciprocal effort, researchers and clinicians should undertake
training to develop skills in engaging and collaborating with
patients. Movement toward an effective model of citizen science will
require adopting principles of mutual respect and learning, in which
both patients and researchers are trained in ways to engage con-
structively with one another. In addition to efforts initiated by
researchers to engage citizen scientists, it will be essential to create
mechanisms that facilitate citizen scientist’s ability to find and reach
out to research partners.
Improved ability to find and include patient partners
Even when researchers wish to co-manage citizen science projects,
they may experience difficulty finding individuals with the necessary
background, interest, and time. Employing a participatory design
process led by a citizen governance council and the research team
has proven successful for generating participation and engagement
guidelines and for ways to encourage participation throughout the
entire research lifecycle in a patient-powered research network.
69
For example, Citizen Endo project participants self-track endometri-
osis symptoms, an activity that researchers are studying as a way to
phenotype the condition,
73,74
and the “Make the Breast Pump Not
Suck” project recruited nursing mothers to help redesign breast
pumps.
75,76
Other approaches that operate outside traditional foun-
dation and academic structures also are needed, in particular be-
cause although citizen scientists may actively seek clinical partners,
they typically experience difficulty finding them.
Openness to meeting citizen scientists at varying skill
levels
Not all participants come to citizen science with the same set of
skills or background. Each situation presents a unique set of chal-
lenges that motivate individuals to explore new solutions. Academ-
ics learn their trades through a prescribed curriculum, but citizen
scientists in healthcare learn out of the need to survive or improve
quality of life, sheer curiosity, or a combination of the two. Health-
care organizations and researchers need to build capacity for engag-
ing citizen scientists with variable skill sets and to help them to grow
into increasing responsibility within teams as the partnership
matures. Large-scale efforts that engage citizen scientists with mini-
mal formal training, such as the American Gut Project, in which
self-selected citizen scientists mailed samples through the mail at
room temperature, have proven feasible and efficacious.
77,78
Trust-building as an essential target and skillset
Although citizen scientists in health are keen to contribute meaning-
fully to research, they often have had negative and/or unhelpful
experiences in healthcare and research environments, which may be
particularly acute among marginalized populations.
79–87
These
experiences may have left them mistrustful of providers, researchers,
and others working within health systems and academia. Citizen sci-
ence will be advanced more quickly and effectively when those in
healthcare focus on gauging readiness to participate at various
points of the research pipeline and establishing trust with and
among patients doing citizen science.
88,89
Progress will advance
more quickly when researchers use tools and strategies designed to
build trust, such as more comprehensive approaches to informed
consent that both inform and engage citizen scientists.
35,90
CONCLUSION
Collaboration of citizen scientists and professional scientists sup-
ports the goals of knowledge generation and support for action.
Citizens not only assist scientists, but also work across the spectrum
4JAMIA Open, 2019, Vol. 0, No. 0
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020

of informatics: data (fact/observation collection), information (data
analysis), knowledge (information refinement), and wisdom (knowl-
edge generation for the greater good). Such collaboration in preci-
sion medicine is necessary to fully capture the meaningful
differences that indicate which intervention to use, for whom, and in
what context. Healthcare organizations can, and should, integrate
citizen scientists into infrastructures that support evidence genera-
tion both for and with citizens. Citizen science has extraordinary po-
tential to extend and enhance the practice of precision medicine if
certain technical, organizational, and citizen engagement issues are
addressed appropriately.
FUNDING
The authors have no funding sources to declare.
AUTHOR CONTRIBUTIONS
CP, RA, UB, HC, AEC, EBH, P-YSH, KKK, AP, LS, AS, and RSV
developed the ideas in this manuscript, wrote and edited the text,
and revised the manuscript.
ETHICS STATEMENT
No humans or human materials were used in the production of this
work, and no Institutional Review Board approvals were required.
ACKNOWLEDGMENT
The authors gratefully acknowledge Dr. Jennifer Couch for her presentation
during the workshop. The authors would like to acknowledge and thank Dr.
Don E. Detmer for championing the importance of citizen science within AMIA
and for convening many of us interested in this novel area of informatics.
CONFLICT OF INTEREST
The authors have no competing interests.
REFERENCES
1. Collins FS, Varmus H. A new initiative on precision medicine. N Engl J
Med 2015; 372 (9): 793–5.
2. National Institutes of Health. About the All of us Research Program.
Washington, DC: National Institutes of Health; 2017.
3. Fradkin JE, Hanlon MC, Rodgers GP. NIH precision medicine initiative:
implications for diabetes research. Dia Care 2016; 39 (7): 1080–4.
4. Arnett DK, Claas SA. Precision medicine, genomics, and public health.
Dia Care 2016; 39 (11): 1870–3.
5. Vaithinathan AG, Asokan V. Public health and precision medicine share a
goal. J Evid Based Med 2017; 10 (2): 76–80.
6. Yanovski SZ, Yanovski JA. Toward precision approaches for the preven-
tion and treatment of obesity. JAMA 2018; 319 (3): 223–4.
7. Akdis CA, Ballas ZK. Precision medicine and precision health: building
blocks to foster a revolutionary health care model. J Allergy Clin Immunol
2016; 137 (5): 1359–61.
8. Chambers DA, Feero WG, Khoury MJ. Convergence of implementation
science, precision medicine, and the learning health care system: a new
model for biomedical research. JAMA 2016; 315 (18): 1941–2.
9. Dolley S. Big data’s role in precision public health. Front Public Health
2018; 6: 68.
10. Dorsey SG, Resnick BM, Renn CL. Precision health: use of omics to opti-
mize self-management of chronic pain in aging. Res Gerontol Nurs 2018;
11 (1): 7–13.
11. Gambhir SS, Ge TJ, Vermesh O, et al. Toward achieving precision health.
Sci Transl Med 2018; 10 (430). pii: eaao3612.
12. Gillman MW, Hammond RA. Precision treatment and precision preven-
tion: integrating below and above the skin. JAMA Pediatr 2016; 170 (1):
9–10.
13. Kellogg RA, Dunn J, Snyder MP. Personal omics for precision health. Circ
Res 2018; 122 (9): 1169–71.
14. Khoury MJ, Galea S. Will precision medicine improve population health?
JAMA 2016; 316 (13): 1357–8.
15. Khoury MJ, Iademarco MF, Riley WT. Precision public health for the era
of precision medicine. Am J Prev Med 2016; 50 (3): 398–401.
16. Riveroll A, Thompson K, Robertson K, et al. Precision health: a personal-
ized approach to active health management. Can Med Biol Eng Sci 2018;
41: 1–5.
17. European Commission. Green Paper on Citizen Science for Europe: To-
wards a Society of Empowered Citizens and Enhanced Research. Brussels,
Belgium: European Commission; 2014.
18. Dick DM. Rethinking the way we do research: the benefits of community-
engaged, citizen science approaches and nontraditional collaborators. Al-
cohol Clin Exp Res 2017; 41 (11): 1849–56.
[19] Petersen C. Patient informaticians: turning patient voice into patient ac-
tion. JAMIA Open 2018; 1 (2): 130–5.
20. Buyx A, Del Savio L, Prainsack B, et al. Every participant is a PI. Citizen
science and participatory governance in population studies. Int J Epide-
miol 2017; 46 (2): 377–84.
21. Den Broeder L, Devilee J, Van Oers H, et al. Citizen science for public
health. Health Promot Int 2018; 33 (3): 505–14.
22. Armstrong AW, Cheeney S, Wu J, et al. Harnessing the power of crowds.
Am J Clin Dermatol 2012; 13 (6): 405–6.
23. de Weger LA, Hiemstra PS, Op den Buysch E, et al. Spatiotemporal moni-
toring of allergic rhinitis symptoms in The Netherlands using citizen sci-
ence. Allergy 2014; 69 (8): 1085–91.
24. Follett R, Strezov V. An analysis of citizen science based research: usage
and publication patterns. PLoS One 2015; 10 (11): e0143687.
25. Ranard BL, Ha YP, Meisel ZF, et al. Crowdsourcing–harnessing the
masses to advance health and medicine, a systematic review. J Gen Intern
Med 2014; 29 (1): 187–203.
26. University of California San Francisco. The Pride Study. San Francisco,
CA: University of California San Francisco; 2015.
27. PCORnet PPRN Consortium, Daugherty SE, Wahba S, Fleurence R. Pa-
tient-powered research networks: building capacity for conducting
patient-centered clinical outcomes research. J Am Med Inform Assoc
2014; 21 (4): 583–6.
28. King AC, Winter SJ, Sheats JL, et al. Leveraging citizen science and infor-
mation technology for population physical activity promotion. Transl J
Am Coll Sports Med 2016; 1 (4): 30–4.
29. Garrison LP Jr, Towse A. Value-based pricing and reimbursement in per-
sonalised healthcare: introduction to the basic health economics. J Pers
Med 2017; 7 (3). pii: E10.
30. Welch BM, Marshall E, Qanungo S, et al. Teleconsent: a novel approach
to obtain informed consent for research. Contemp Clin Trials Commun
2016; 3: 74–9.
31. Kondylakis H, Koumakis L, H€
anold S, et al. Donor’s support tool: en-
abling informed secondary use of patient’s biomaterial and personal data.
Int J Med Inform 2017; 97: 282–92.
32. Kaye J, Whitley EA, Lund D, et al. Dynamic consent: a patient interface
for twenty-first century research networks. Eur J Hum Genet 2015; 23
(2): 141–6.
33. €
Ozdemir V, Badr KF, Dove ES, et al. Crowd-funded micro-grants for ge-
nomics and “big data”: an actionable idea connecting small (artisan) sci-
ence, infrastructure science, and citizen philanthropy. OMICS 2013; 17
(4): 161–72.
34. Haga SB, Rosanbalm KD, Boles L, et al. Promoting public awareness and
engagement in genome sciences. J Genet Counsel 2013; 22 (4): 508–16.
35. PGP-UK Consortium. Personal Genome Project UK (PGP-UK): a research
and citizen science hybrid project in support of personalized medicine.
BMC Med Genomics 2018; 11: 108.
JAMIA Open, 2019, Vol. 0, No. 0 5
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020

36. National Aeronautics and Space Administration. Target Asteroids! Na-
tional Aeronautics and Space Administration. Washington, DC; 2019.
http://www.asteroidmission.org/get-involved/target-asteroids/ Accessed
May 11, 2019.
37. Bonney R, Phillips TB, Ballard HL, et al. Can citizen science enhance pub-
lic understanding of science? Public Underst Sci 2016; 25 (1): 2–16.
38. Bonney R, Shirk JL, Phillips TB, et al. Citizen science. Next steps for citi-
zen science. Science 2014; 343 (6178): 1436–7.
39. Riesch H, Potter C. Citizen science as seen by scientists: methodological,
epistemological and ethical dimensions. Public Underst Sci 2014; 23 (1):
107–20.
40. Sullivan BL, Aycrigg JL, Barry JH, et al. The eBird enterprise: an inte-
grated approach to development and application of citizen science. Biol
Conserv 2014; 169: 31–40.
41. American Medical Informatics Association. Redefining our Picture of
Health: Towards a Person-Centered Integrated Care, Research, Wellness,
and Community Ecosystem: A White Paper of the 2017 AMIA Policy Invi-
tational. Washington, DC: American Medical Informatics Association.
42. Khodyakov D, Grant S, Meeker D, et al. Comparative analysis of stake-
holder experiences with an online approach to prioritizing patient-
centered research topics. J Am Med Inform Assoc 2017; 24 (3): 537–43.
43. Kim KK, Khodyakov D, Marie K, et al. A novel stakeholder engagement
approach for patient-centered outcomes research. Med Care 2018; 56
(Suppl) (10 Suppl 1): S41–S47.
44. Sylvia LG, Hearing CM, Montana RE, et al. MoodNetwork: an innova-
tive approach to patient-centered research. Med Care 2018; 56 (Suppl)
(10 Suppl 1): S48–S52.
45. Bassani G, Marinelli N, Vismara S. Crowdfunding in healthcare. J Tech-
nol Transf 2019; 44 (4): 1290–310.
46. Stanford Medicine. Stanford Medicine 2018 Health Trends Report: The
Democratization of Health Care. https://med.stanford.edu/content/dam/
sm/school/documents/Health-Trends-Report/HTR-Designed-Version-12.
12.18-930am-PT.pdf Accessed December 13, 2018.
47. Wang Y, Kaplan N, Newman G, et al. CitSci.org: a new model for manag-
ing, documenting, and sharing citizen science data. PLoS Biol 2015; 13
(10): e1002280.
48. Nebeker C, Bloss C, Weibel N. 371. Connected and open research ethics
(CORE) initiative: engaging stakeholders to shape ethics in the digital age.
Biol Psychiatry 2017; 81(10) (Suppl): S151–S152.
49. Aungst H, Fishman JR, McGowan ML. Participatory genomic research:
ethical issues from the bottom up to the top down. Annu Rev Genom
Hum Genet 2017; 18 (1): 357–67.
50. Hekler E. Is it Safe? Opening Pathways. San Diego, CA. http://opening-
pathways.org/is-it-safe Accessed December 31, 2018.
51. Delbanco T, Walker J, Bell SK, et al. Inviting patients to read their doc-
tors’ notes: a quasi-experimental study and a look ahead. Ann Intern Med
2012; 157 (7): 461–70.
52. Bell SK, Gerard M, Fossa A, et al. A patient feedback reporting tool for
OpenNotes: implications for patient-clinician safety and quality partner-
ships. BMJ Qual Saf 2017; 26 (4): 312–22.
53. Petersen C. The future of patient engagement in the governance of shared
data. EGEMS 2016; 4 (2): 1214.
54. Lewis D. Spectrums of Perspective. San Diego, CA: Opening Pathways.
http://openingpathways.org/spectrums-of-perspective.
55. Sullivan M, Kone A, Senturia KD, et al. Researcher and researched—
community perspectives: toward bridging the gap. Health Educ Behav
2001; 28 (2): 130–49.
56. Khodyakov D, Mikesell L, Schraiber R, et al. On using ethical principles
of community-engaged research in translational science. Transl Res 2016;
171: 52–62.
57. European Citizen Science Association. Ten Principles of Citizen Science.
Berlin, Germany: European Citizen Science Association; 2018.
58. Almalki M, Gray K, Martin-Sanchez FJ. Refining the concepts of self-
quantification needed for health self-management: a thematic literature re-
view. Methods Inf Med 2017; 56 (1): 46–54.
59. Hinckson E, Schneider M, Winter SJ, et al. Citizen science applied to
building healthier community environments: advancing the field through
shared construct and measurement development. Int J Behav Nutr Phys
Act 2017; 14 (1): 133.
60. Choe EK, Lee B, Munson S, et al. Persuasive performance feedback: the ef-
fect of framing on self-efficacy. AMIA Annu Symp Proc 2013; 2013:
825–33.
61. Afshinnekoo E, Ahsanuddin S, Mason CA. Globalizing and crowdsourc-
ing biomedical research. Br Med Bull 2016; 120 (1): 27–33.
62. Haldar S, Mishra SR, Khelifi M, et al. Opportunities and design considera-
tions for peer support in a hospital setting. Proc SIGCHI Conf Hum Fac-
tor Comput Syst 2017; 2017: 867–79.
63. University of Washington Information School. Patients as Safeguards. Se-
attle, WA: University of Washington Information School. https://patient-
sassafeguards.ischool.uw.edu/ Accessed May 11, 2019.
64. Agile Science. What is Agile Science? Tempe, AZ: Agile Science; 2015.
65. Hekler EB, Klasnja P, Riley WT, et al. Agile science: creating useful prod-
ucts for behavior change in the real world. Behav Med Pract Policy Res
2016; 6 (2): 317–28.
66. Patient-Centered Research Outcomes Institute. NAPCRG’s Patient and
Clinician Engagement (PaCE) Program 2.0.; 2016. https://www.pcori.
org/research-results/2016/napcrgs-patient-and-clinician-engagement-
pace-program-20 Accessed December 13, 2018.
67. Nickel WK, Weinberger SE, Guze PA; for the Patient Partnership in
Healthcare Committee of the American College of Physicians. Principles
for patient and family partnership in care: an American College of Physi-
cians position paper. Ann Intern Med 2018; 169 (11): 796–9.
68. Reardon J. The ‘persons’ and ‘genomics’ of personal genomics. Per Med
2011; 8 (1): 95–107.
69. Chung AE, Sandler RS, Long MD, et al. Harnessing person-generated
health data to accelerate patient-centered outcomes research: the Crohn’s
and Colitis Foundation of America PCORnet patient powered research
network (CCFA Partners). J Am Med Inform Assoc 2016; 23 (3): 485–90.
70. Chung AE, Vu MB, Myers K, et al. Crohn’s and Colitis Foundation of
America partners patient-powered research network: patient perspectives
on facilitators and barriers to building an impactful patient-powered re-
search network. Med Care 2018; 56 (Suppl 10) (Suppl 1): S33–S40.
71. Kim KK, Helfand M. Engagement in PCORnet research networks. Med
Care 2018; 56 (Suppl 10) (Suppl 1): S1–S3.
72. Leese J, Macdonald G, Kerr S, et al. Adding another spinning plate to an
already busy life’. Benefits and risks in patient-partner-researcher relation-
ships: a qualitative study of patient partners’ experiences in a Canadian
health research setting. BMJ Open 2018; 8 (8): e022154.
73. McKillop M, Voigt N, Schnall R, et al. Exploring self-tracking as a
participatory research activity among women with endometriosis. J
Participatory Med 2016; 8:e17. https://participatorymedicine.org/jour-
nal/evidence/research/2016/12/29/exploring-self-tracking-as-a-participa-
tory-research-activity-among-women-with-endometriosis/ Accessed
April 18, 2019.
74. McKillop M, Mamykina L, Elhadad N. Designing in the dark: eliciting
self-tracking dimensions for understanding enigmatic disease. In: Pro-
ceedings of the 36th Annual Association for Computing Machinery
Conference on Human Factors in Computing Systems; 2018. http://
people.dbmi.columbia.edu/noemie/papers/18chi.pdf Accessed April 18,
2019.
75. Make the Breast Pump Not Suck. Make the Breast Pump Not Suck Hacka-
thon. Cambridge, MA: Make the Breast Pump Not Suck; 2018.
76. Hope A, D’Ignazio C, Hoy J, et al. Hackathons as participatory design: it-
erating feminist utopias. In: CHI Conference on Human Factors in Com-
puting Systems Proceedings (CHI 2019); May 4–9, 2019; Glasgow,
Scotland, UK. ACM, New York, NY. 2019. https://dl.acm.org/citation.
cfm? id¼3300291 Accessed May 11, 2019.
77. McDonald D, Hornig M, Lozupone C, et al. Towards large-cohort com-
parative studies to define the factors influencing the gut microbial com-
munity structure of ASD patients. Microb Ecol Health Dis 2015; 26:
26555.
78. McDonald D, Hyde E, Debelius JW, et al. American Gut: an open plat-
form for citizen science microbiome research. mSystems 2018; 3 (3). pii:
e00031–18.
6JAMIA Open, 2019, Vol. 0, No. 0
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020

79. Whitehead J, Shaver J, Stephenson R. Outness, stigma, and primary health
care utilization among rural LGBT populations. PLoS One 2016; 11 (1):
e0146139.
80. LaVaccare S, Diamant AL, Friedman J, et al. Healthcare experiences of
underrepresented lesbian and bisexual women: a focus group qualitative
study. Health Equity 2018; 2 (1): 131–8.
81. Phelan SM, Burgess DJ, Yeazel MW, et al. Impact of weight bias and
stigma on quality of care and outcomes for patients with obesity. Obes
Rev 2015; 16 (4): 319–26.
82. Crenshaw K. Demarginalizing the intersection of race and sex: a black
feminist critique of antidiscrimination doctrine, feminist theory and anti-
racist politics. U Chi Legal F 1989; 1989: Article 8.
83. L
opez N, Gadsden VL. Health inequities, social determinants, and inter-
sectionality. In National Academy of Sciences Discussion Paper Perspec-
tives: Expert Voices in Health and Health Care, Washington, DC, USA;
2016: 1–15.
84. Bauer GR. Incorporating intersectionality theory into population health
research methodology: challenges and the potential to advance health eq-
uity. Soc Sci Med 2014; 110: 10–7.
85. McCann E, Lee R, Brown M. The experiences and support needs of people
with intellectual disabilities who identify as LGBT: a review of the litera-
ture. Res Dev Disabil 2016; 57: 39–53.
86. Ramasamy V, Rillotta F, Alexander J. Experiences of adults with in-
tellectual disability who identify as lesbian, gay, bisexual, transgen-
der, queer or questioning, intersex or asexual: a systematic review
protocol. JBI Database System Rev Implement Rep 2017; 15 (9):
2234–41.
87. Mattocks KM, Sullivan JC, Bertrand C, et al. Perceived stigma, discrimi-
nation, and disclosure of sexual orientation among a sample of lesbian vet-
erans receiving care in the Department of Veterans Affairs. LGBT Health
2015; 2 (2): 147–53.
88. Khodyakov D, Mikesell L, Bromley E. Trust and the ethical conduct of
community-engaged research. Eur J Pers Cent Healthc 2017; 5 (4): 522–6.
89. Wilkins CH. Effective engagement requires trust and being trustworthy.
Med Care 2018; 56 (Suppl 1): S6–S8.
90. Angrist M. Eyes wide open: the personal genome project, citizen science
and veracity in informed consent. Per Med 2009; 6 (6): 691–9.
JAMIA Open, 2019, Vol. 0, No. 0 7
Downloaded from https://academic.oup.com/jamiaopen/advance-article-abstract/doi/10.1093/jamiaopen/ooz060/5651081 by guest on 15 January 2020