ArticlePDF Available

A Case Study of Engaging Hard-to-Reach Participants in the Research Process: Community Advisors on Research Design and Strategies (CARDS)®



Lack of diversity among study participants in clinical research limits progress in eliminating health disparities. The engagement of lay stakeholders, such as patient or community advisory boards (CABs), has the potential to increase recruitment and retention of underrepresented groups by providing a structure for gathering feedback on research plans and materials from this target population. However, many CABs intentionally recruit prominent stakeholders who are connected to or comfortable with research and academia and thus may not accurately represent the perspectives of underrepresented groups who have been labeled hard-to-reach, including racial minorities and low-income or low-literacy populations. We developed a partnership between the University of Wisconsin-Madison School of Nursing and two community centers to deliberately engage hard-to-reach people in two lay advisory groups, the Community Advisors on Research Design and Strategies (CARDS)®. Community center staff recruited the CARDS from center programs, including parenting and childcare programs, women's support groups, food pantries, and senior meal programs. The CARDS model differs from other CABs in its participants, processes, and outcomes. Since 2010, the CARDS have met monthly with nurses and other researchers, helping them understand how research processes and the language, tone, appearance, and organization of research materials can discourage people from enrolling in clinical studies. We have successfully used the CARDS model to bring hard-to-reach populations into the research process and have sustained their participation. The model represents a promising strategy for increasing the diversity of participants in clinical research. © 2016 Wiley Periodicals, Inc.
A case study of engaging hard-to-reach participants in the
research process: Community Advisors on Research Design and
Strategies (CARDS)®
Betty L. Kaiser [Administrative Program Specialist],
University of Wisconsin-Madison School of Nursing,, 701 Highland Ave., Rm
5138, Madison, WI 53705, 608-263-5292 (office), 608-263-5323 (fax)
Gay R. Thomas [Senior Administrative Program Specialist], and
University of Wisconsin-Madison School of Nursing
Barbara J. Bowers [Professor]
University of Wisconsin-Madison School of Nursing
Lack of diversity among study participants in clinical research limits progress in eliminating health
disparities. The engagement of lay stakeholders, such as patient or community advisory boards
(CABs), has the potential to increase recruitment and retention of underrepresented groups by
providing a structure for gathering feedback on research plans and materials from this target
population. However, many CABs intentionally recruit prominent stakeholders who are connected
to or comfortable with research and academia and thus may not accurately represent the
perspectives of underrepresented groups who have been labeled hard-to-reach, including racial
minorities and low-income or low-literacy populations. We developed a partnership between the
University of Wisconsin-Madison School of Nursing and two community centers to deliberately
engage hard-to-reach people in two lay advisory groups, the Community Advisors on Research
Design and Strategies (CARDS)®. Community center staff recruited the CARDS® from center
programs, including parenting and childcare programs, women’s support groups, food pantries,
and senior meal programs. The CARDS® model differs from other CABs in its participants,
processes, and outcomes. Since 2010, the CARDS® have met monthly with nurses and other
researchers, helping them understand how research processes and the language, tone, appearance,
and organization of research materials can discourage people from enrolling in clinical studies. We
have successfully used the CARDS® model to bring hard-to-reach populations into the research
process and have sustained their participation. The model represents a promising strategy for
increasing the diversity of participants in clinical research.
subject recruitment; healthcare disparities; vulnerable populations; advisory committees;
stakeholder engagement
The authors declare no conflicts of interest.
HHS Public Access
Author manuscript
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Published in final edited form as:
Res Nurs Health
. 2017 February ; 40(1): 70–79. doi:10.1002/nur.21753.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
The elimination of health disparities is a national priority (National Institute on Minority
Health and Health Disparities, n.d.; U.S. Department of Health and Human Services, n.d.).
Clinical research can play an important role in eliminating health disparities by making
novel treatments available to underserved populations and identifying the effectiveness of
particular treatments for particular populations. However, progress toward eliminating health
disparities has been hindered by lack of participant diversity in clinical research studies
(Kitterman, Cheng, Dilts, & Orwoll, 2011; Schroen et al., 2010). Researchers often do not
reach recruitment goals for hard-to-reach participants such as racial and ethnic minorities
and people with low income. Reasons for low recruitment include mistrust of research,
perceived risks, and lack of culturally appropriate information about opportunities to
participate in research (Ford et al., 2008), as well as burdens such as time commitment and
lengthy questionnaires (Paskett et al., 2008).
A recommended strategy for overcoming barriers to recruitment is patient and community
stakeholder involvement in the design and conduct of clinical trials (Institute of Medicine,
2011). One popular form of stakeholder involvement is community advisory boards (CABs).
CABs serve several functions, depending on the project mission and requirements. They
provide input into research agendas; serve as gatekeepers for researcher entry into
communities; formally approve projects; and influence the design, conduct, and
implementation of studies (Fernandez-Pena et al., 2008; Silvestre, Quinn, & Rinaldo, 2010;
Strauss et al., 2001).
Recommendations for CAB membership often emphasize inclusion of influential
stakeholders and opinion leaders from the community (D'Alonzo, 2010). Typical CAB
members include service providers, community leaders, and representatives of local
agencies and organizations (Newman et al., 2011); in this paper, high-level stakeholders are
referred to as “prominent community representatives”. Prominent community
representatives bring valuable skills, perspectives, and resources to CABs due to their
experience with group and meeting processes, influence and connections within the
community, and decision-making power within their organizations. CABs also sometimes
include lay stakeholders who are not prominent community representatives, but can bring an
important viewpoint to a research project, such as residents of a particular neighborhood or
patients or caregivers with experience related to a particular health issue (James et al., 2011;
Pinto, Spector, Rahman, & Gastolomendo, 2013). Lay stakeholders can help researchers
improve the cultural sensitivity and appropriateness of recruitment materials and methods;
recommend study implementation strategies (Joosten et al., 2015); and design relevant,
meaningful interventions (GreenMills, Davison, Gordon, Kaigang, & Jurkowski, 2013). The
perspectives of lay stakeholders who are disconnected from academia and local power
structures may offer clinical researchers unique insights and perspectives on recruitment
barriers and strategies for improving recruitment and retention of hard-to-reach populations.
This paper describes the origins, participants, processes, and outcomes of a unique CAB
comprised solely of lay stakeholders that brings voices of rarely heard groups into the
research enterprise.
Kaiser et al. Page 2
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Project Origins and Community-Academic Partnership
Our team at the University of Wisconsin-Madison (UW-Madison) School of Nursing
partnered with two local community centers to develop lay advisory boards of community
members who are not prominent community representatives. Throughout this paper, “we”
refers to the authors. The Lussier Community Education Center and Goodman Community
Center have served their neighborhoods for over 30 years, offering programs and services
that reflect the diversity of their communities. In 2010, we worked with staff at the
community centers to submit a proposal for a program grant to the National Institutes of
Health (NIH). We received a three-year grant to establish a sustainable infrastructure to link
community members with researchers to improve the quality of health sciences research.
The UW-Madison Health Sciences Institutional Review Board (IRB) designated our grant
activities as exempt from review. For one of our grant activities, we worked with the
community centers to develop and pilot two lay advisory boards, the Community Advisors
on Research Design and Strategies (CARDS)®. We established a CARDS® group at each
partnering community center. For the past five years, each group has met monthly with guest
researchers to provide feedback on the appeal, clarity, and accessibility of materials and
processes used in clinical research. Our NIH grant ended in 2013, and since then we have
sustained the CARDS® as a program within the Wisconsin Network for Research Support
(WINRS), a fee-for-service community and patient engagement center supported by the
School of Nursing ( The UW-Madison Institute for Clinical and
Translational Research (ICTR), a Clinical and Translational Sciences Award site, provides
additional support for the program.
The CARDS® Program
The CARDS® are based in the community, so researchers often assume that CARDS® input
is relevant only for researchers interested in community-based participatory research
(CBPR) or community-engaged research (CER). In fact, since the program’s inception, the
CARDS® have worked extensively with researchers conducting clinic or hospital-based
studies. The CARDS® provide lay, patient-centered feedback on materials for any type of
human subjects research, regardless of the specific research topic, setting, methodology, or
study population. Researchers and their project staff typically attend one or two meetings for
each project that they bring to the CARDS®, although we place no limit on the number of
CARDS® meetings that a researcher may attend. The CARDS® program makes it feasible
for researchers who are not conducting CBPR or CER to get timely, meaningful lay input on
their research materials, without substantial investment of resources.
Program Participants
Members—For the CARDS®, we wanted not prominent community representatives, but
people who have been under-represented in healthcare research (Hasnain-Wynia & Beal,
2014). We intentionally sought to recruit people who are not connected with academia,
research, healthcare, or local power structures. Unlike recruitment for many CABs, we did
not establish specific inclusion criteria, such as residency in a certain neighborhood or
diagnosis of a particular health condition (Stewart et al., 2015). We asked community center
Kaiser et al. Page 3
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
staff (CARDS® liaisons) to recruit typical users of services at their centers. Liaisons shared
an informational flyer with community members who used center services such as parenting
and childcare programs, women’s support groups, job clinics, food pantries, and senior meal
programs. People who were interested followed up with the CARDS® liaison to complete a
membership application that asked applicants to briefly explain why they wanted to be part
of the CARDS®. Sometimes, people who worked or volunteered at the centers heard about
the program and completed applications. Currently, 15 CARDS® participate in the program.
Nine participants (60%) are women and 10 (67%) are African-American. Ages range from
early-20s to mid-70s, and educational attainment ranges from incomplete high school
preparation to post-high school coursework.
Many CABs have expansive roles for their members that may encompass defining research
agendas, providing entrée into a community (gatekeeping), engaging in the conduct of
research, and providing feedback on scientific papers (Israel et al., 2005). The CARDS®
sole function is to meet monthly with researchers to review, discuss, and provide feedback
on research plans and materials, including recruitment materials, data collection procedures
and instruments, web-based materials, and smartphone apps. They are never required to
complete work between meetings. For each 90-minute meeting, the CARDS® receive a cash
stipend of $35. Although it is convenient for institutions to pay board members with checks,
many people in hard-to-reach populations do not have checking accounts. As one of the
CARDS® explained, “$35 isn’t $35 if I have to pay a fee to cash the check.”
WINRS and community center staff—WINRS staff provides overall management of
the CARDS® program. The School of Nursing employs two WINRS staff members at a total
of 1.7 full-time equivalents. WINRS staff work on a variety of projects, but together devote
approximately 0.5 full-time equivalents to management of the CARDS®. Both employees
have advanced degrees (MA; PhD) and had previous experience with community
engagement. One employee has primary responsibility for the program, with the other
providing support. The School of Nursing and ICTR provide salary support for the full-time
employee, and revenues from WINRS support the salary of the part-time employee. In
addition, each community center provides a staff member who serves as the center’s
CARDS® liaison. Table 1 lists the respective responsibilities of WINRS, community center
staff, and the CARDS®. We pay the centers a monthly facility fee to compensate them for
liaison time, room rental, childcare during meetings, and transportation for members as
needed. Monthly facility fees range from $205 – $290, depending on whether childcare is
Guest researchers—Since the CARDS® program began in 2010, we have conducted
meetings with 40 research teams representing 21 disciplines, including Cardiology, Family
Medicine and Community Health, Kinesiology, Nursing, Pharmacy, Rheumatology, and
Surgery. We have also worked with researchers from other institutions, sometimes
conducting CARDS® meetings via teleconference. Nurses constitute the largest proportion
of researcher teams (58%,
= 23) who have used the CARDS® service, and our guests have
included nursing faculty, doctoral students, and nurses working in community settings.
Nurse researchers have brought a variety of materials and plans to the CARDS® for their
Kaiser et al. Page 4
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
review and feedback, including recruitment materials (brochures, flyers, letters, scripts);
study processes (recruitment plans, interview processes, website development); focus group
protocols; interview and survey questions; study information sheets; and consent forms.
We sometimes field inquiries from researchers who are interested in lay review of their
materials but are skeptical that lay advisors who are not members of their particular target
population can provide useful feedback. Materials used in human subjects research—
recruitment notices, information sheets, questionnaires and other instruments, consent forms
—share a common set of challenges for members of the public due to their technical
language, dense presentation of information, embedded assumptions, counter-intuitive
organization, and academic tone. As members of the general public with specific training on
how to give feedback to researchers, the CARDS® can offer a uniquely fresh perspective on
research materials, no matter the specific content of the documents. The CARDS® have
provided useful feedback on materials for a wide range of health studies, despite having
limited or no personal experience with many of the topics that researchers present. At one of
our most memorable meetings, no women in the CARDS® were able to attend, so five male
CARDS® provided feedback on a doctoral student’s survey about dysmenorrhea. We
introduced the survey by reminding the CARDS® that we all have experience with pain and
discomfort, and we asked them to think about the survey questions in the context of their
personal pain experiences. Based on CARDS® feedback, the student made substantial
revisions to her survey.
Program Processes
Member orientation—One of the key features of the CARDS® is our orientation
program. Many CABs provide minimal orientation or training for board members (Albert
Einstein College of Medicine, The Bronx Health Link, & Community-Campus Partnerships
for Health, 2012). In contrast, the focus of the CARDS® orientation program is hands-on
practice with research materials. We designed an interactive group orientation to help the
CARDS® develop and practice the skills needed to participate effectively at meetings. Our
orientation program emphasizes content that is directly relevant to the work that board
members will do and gives members experience reviewing and providing feedback on
recruitment flyers and letters, consent forms, and focus group questions. Completion of the
2.5-hour orientation is a requirement for membership in the CARDS® and provides
meaningful preparation for respectful, productive interactions with researchers.
Preparation of guest researchers—Each CARDS® group meets monthly at the
community centers with guest researchers. Several weeks before the meeting, WINRS staff
sends the scheduled researcher a timeline of steps to prepare for the meeting, along with a
link to a short web-based survey requesting key information about the researcher’s desired
outcomes from the meeting. WINRS staff then engages in a series of email messages and a
telephone call or face-to-face meeting with the researcher to plan and prepare for the
During the planning meeting, WINRS staff and guest researchers review answers to the web-
based survey and discuss possible materials for the CARDS® meeting. Researchers often
Kaiser et al. Page 5
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
bring their staff members and students to planning meetings and CARDS® meetings. We
select the materials that can be thoroughly discussed during a 90-minute CARDS® meeting
and develop a detailed working agenda. Without this preparation, researchers may have
unrealistic expectations of how many documents can be reviewed during a meeting. During
the planning meeting, WINRS staff share advice on working with lay advisors. For example,
we remind researchers to use plain language, and we help them prepare brief,
straightforward explanations of their research interests. We also encourage researchers to
gracefully acknowledge all suggestions from the CARDS®. Realistically, researchers may
not want to or be able to follow certain suggestions, but thanking the CARDS® for all
suggestions helps them feel that their voices are heard and welcomed. Decisions about how
to use CARDS® feedback can be made by researchers after they attend meetings, when they
can decide whether to incorporate specific suggestions fully, partially, or not at all.
As part of meeting preparation, we discuss fees and payment arrangements with the guest
researcher. We bundle our costs for WINRS and community center staff time, CARDS®
stipends, and other costs for the CARDS® service into a fee of $1964 for a 90-minute
meeting, preparatory work with the guest, and post-meeting summary documents. As
required by the UW-Madison, our fee structure is based on a cost-recovery model; the fee
for the CARDS® service covers our costs for providing the service, but does not generate
profit. Researchers typically pay the fee with grant funds. For doctoral students or junior
faculty without funding, we offer a reduced fee. After we meet with guest researchers, we
send a brief meeting announcement to the CARDS® to provide information on the guest
researchers, including their focus areas, materials they will present at the meeting, and type
of help they would like from the CARDS®. The announcement also includes a personal
statement from the guest researcher about why she or he cares about the research topic. We
work closely with researchers to help them craft personal statements. The CARDS® have
told us that learning about a researcher’s personal motivations for doing research has helped
them overcome their stereotypes of researchers as cold academics who use research
participants to serve their own ends.
Meeting structure and practices—We follow a structured sequence of activities at each
meeting. We start each meeting with an opening question. Everyone at the meeting
participates in a round-robin sharing of names and brief responses to the opening questions.
Our opening questions give everyone at the meeting a chance to share something about
themselves. The activity helps to break stereotypes that researchers and lay community
members may hold about each other. Over time, the sharing that occurs with the opening
question helps group members establish personal connections and build a sense of
community. We often craft opening questions that are related to the research topic that we
will discuss at the meeting. For example, our opening question for a meeting with a
researcher studying nasal irrigation was “Please say your name, and tell us one favorite
home remedy for dealing with a stuffy nose or other sinus problems.”
After the opening question, guest researchers introduce themselves briefly, using plain
language to explain the goal of their research and why it is important. We use the remaining
75 minutes of the 90-minute meeting to discuss the researcher’s materials. The structure of
the discussion is standardized and uses 3 steps to elicit feedback from the CARDS®:
Kaiser et al. Page 6
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
1. The researcher describes how and where a prospective study participant
would encounter the research materials, e.g. mailed letter to a home
address; flyer posted in a primary care clinic; consent form presented in a
community setting; website that the participant will access from a home
2. We ask guest researchers to read their materials aloud, several lines at a
time, to facilitate full participation of everyone present, regardless of
literacy level.
3. WINRS staff facilitate a section-by-section review of the materials. After a
section is read, the CARDS® offer comments. We take detailed notes and
write CARDS® feedback on a flip chart, a practice that helps to affirm the
value of everyone’s contribution to the discussion. After finishing review
of one section, we move to the next section and repeat the process.
Post-meeting products and survey—During the first two years of CARDS® meetings,
we provided a summary report to researchers who attended meetings. The 1–3 page report
summarized the overall feedback from the CARDS® on the research materials reviewed at
the meeting and highlighted specific issues related to the content, language, organization,
and format of the materials. However, when researchers later shared their revised materials
with us, we saw that they often did not translate CARDS® feedback into concrete changes to
their materials, even though they rated the value of the meetings very highly. Consequently,
we now deliver revised materials to all guest researchers, to make it easy for them to
implement recommendations from the CARDS®. Researchers consider and balance
numerous factors when designing their study materials, and we encourage researchers to use
the CARDS®-revised materials in whatever way makes the most sense for them, based on
their experience and expertise. To complete our consultation, we send a post-meeting online
survey to all researchers who attend CARDS® meetings to ask them to evaluate their
Program Outcomes
CARDS® Insights and Recommendations
The value of the CARDS® lies in their ability to provide researchers with fresh insights and
feedback that academic or professional colleagues immersed in research may not be able to
offer. As the concept of health literacy has permeated health sciences research, researchers
have developed more awareness of how using technical jargon can undermine subject
recruitment. Use of readability tools such as SMOG (National Cancer Institute, 1989) or the
Flesch-Kincaid tool in word processing programs can be helpful for improving readability.
Researchers also sometimes rely on colleagues, graduate students, high-level stakeholders,
or IRBs to identify glaring problems in research materials. While these resources can be
helpful, they may not be sufficient for ensuring that materials are acceptable and inviting to
the general public. Over five years of monthly meetings, the CARDS® have highlighted
several key characteristics of research materials that may turn people away from
participating in research.
Kaiser et al. Page 7
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Passive language—Plain language guidelines highlight the importance of using active
language instead of passive language (“staff on my research team will ask you several
questions” instead of “you will be asked several questions”). Passive sentences lack the
clarity of active sentences because they do not clearly identify the actor. The CARDS®
highlight other problems with passive language, describing it as confusing, impersonal, and
evasive. For them, passive language in a research document raises questions and reservations
about who is behind the research project, the true motives for the project, and what
researchers will do with participants and their information. Passive language pervades
consent forms, and it particularly troubles the CARDS® when it appears, as it does routinely,
in material related to privacy and confidentiality. Vague assurances that “your information
will be stored in a secure location” arouse their suspicions; they want to know exactly who
will take responsibility for protecting their information.
Specialized use of everyday language—The CARDS® have provided feedback of a
nuanced nature that highlights issues not measured by readability tools. The CARDS®
repeatedly point out things that are confusing, patronizing, or even offensive or frightening
to potential research participants. For example, even though researchers may scrupulously
avoid using technical jargon, they often use everyday language in a highly specialized
manner. The CARDS say that words such as “data”, “procedures”, “health outcomes”,
“technique”, and “investigate” sound ominous or threatening. Even the word “study”,
universally used by researchers in recruitment materials, can evoke images of being
subjected to dangerous tests or used like a guinea pig. The CARDS® strongly prefer
“project” as a non-threatening alternative. When referring to research subjects, the CARDS®
strongly prefer the phrase “project participants”, which sounds inviting, inclusive, and less
scary than the off-putting, objectifying “subjects”. Table 2 lists additional examples of how
the CARDS® interpret common research language.
When we meet with researchers to prepare for CARDS® meetings, we explain that the
CARDS® may occasionally suggest alternative language that an IRB will reject. To our
knowledge, these instances are infrequent and have not limited the value of the service.
Currently, we are collaborating with an IRB workgroup that is charged with improving
templates for informed consent. The working group has attended three CARDS® meetings
to learn what kinds of information about research are important to the non-academic
community and to help craft clearer language for consent documents. In the coming year, the
working group expects to pilot new templates that incorporate CARDS® feedback.
Perceived tone of documents—The CARDS® have noted several other key
considerations for making documents accessible and inviting. The overall tone of a
document can serve to engage or disengage potential participants. For example, although
best practices in health education often recommend repeating key messages in documents,
the CARDS® say that redundant language feels insulting, by implying that they are not
smart enough to understand something the first time that it appears in a document. In
addition, materials that sound non-judgmental may be more likely to engage potential
participants, especially for studies that address health behaviors. The CARDS® often use the
term “gentler” when recommending alternative language. For example, when researchers ask
Kaiser et al. Page 8
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
about tobacco use, the CARDS® note that the simple question “Do you want to quit
smoking?” is a complex, sensitive question and that asking a “yes” or “no” question may not
be appropriate. The CARDS® suggest “Are you thinking about cutting back or quitting?” as
a gentler alternative that raises the smoking issue without provoking defensiveness by
implying that the person is doing something wrong.
Requests for demographic information—Demographic questions are standard items
in many research instruments, and members of the public also frequently encounter them in
the context of consumer research or participation in various programs. Despite the ubiquity
of demographic questions, they may provoke negative reactions due to their highly personal
nature. The CARDS® have told us repeatedly that they expect researchers to use
demographic information to stereotype people and present them in a bad light. The
CARDS® have shared personal stories about skipping demographic questions, providing
false answers, or declining to complete an instrument altogether if it includes demographic
questions. They recommend that researchers ask only those questions that are critical for
answering the study questions. In addition, the CARDS® recommend that researchers
provide a short, straightforward introduction to demographic questions to explain why the
researcher needs the requested information. Explanations that simply invoke grant
requirements for demographic information may be insufficient and potentially offensive;
rather, a thoughtful explanation for asking demographic questions can explicitly
acknowledge that demographic questions are personal and offer a straightforward rationale
for asking them.
Lessons Learned on Engaging Hard-to-reach Groups
When we initiated the CARDS® program, we deliberately sought participants outside of the
academic environment and not situated in positions of power. Our community center liaisons
intentionally recruited people not typical of advisory boards. Many of them have had
challenging life experiences, including poverty, homelessness, long-term underemployment,
and chronic health problems. We have structured the CARDS® program to minimize barriers
to participation and to demonstrate our respect and appreciation for all members. Our use of
conscious practices that engender trust and help people feel valued have helped us to achieve
outstanding retention of our lay advisors. Six of the current members (40%) have
participated since 2010, and the remaining nine members have participated since 2012. Half
of the former members who dropped out of the groups from 2010 – 2012 (
= 6) left because
they moved out of the area, changed their work schedules, or experienced re-incarceration.
Attendance has been very consistent. Since 2013, when we standardized many of our
meeting processes, aggregated attendance for the program is 81% (486 instances of member
present for meeting/601 total opportunities to attend). Attendance for individual CARDS®
over this period has ranged from 58% of meetings attended to 97%.
Consistent staffing at both WINRS and the community centers has been a critical component
of developing trusting relationships with our members and retaining them in the group over
time. The same WINRS staff have facilitated every CARDS® meeting since 2010, and the
CARDS® liaisons have typically stayed in their positions for at least two years. Consistent
staffing has helped us to develop trusting, long-term relationships with our members, which
Kaiser et al. Page 9
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
in turn support honest dialogue and problem-solving with members when challenges arise.
On several occasions, we have met with individual CARDS® to address problems related to
inappropriate behavior toward a program participant or insufficient participation in meeting
discussions. The orientation that all CARDS® complete serves as a useful point of reference
for constructive discussions about performance and expectations.
During our first few years, we often wrote short individualized notes to send to the CARDS®
after meetings, to thank them for their contributions. We call these notes “affirmations”;
because they affirm something unique and positive that the person brings to the group. We
also hold annual celebrations with the CARDS®. Our celebrations have included meals,
holiday treats, and small tokens such as gift cards.
For some of our members, applying for jobs is a frequent and difficult task. We provide
tangible support with letters of reference and assistance with resume-writing. We write
letters of reference for members based on their work with the CARDS® and provide resume
templates that highlight the unique skills and tasks of the CARDS®. With the advent of web-
based systems for providing references, we have also completed on-line surveys to support
job applications for some of the CARDS®.
Program Evaluation by the CARDS®
In 2014, we completed a program evaluation with the CARDS®. We conducted separate
focus groups at each community center to explore members’ opinions and attitudes in four
areas: their orientation to the program; their work at CARDS® meetings; factors that
contribute to their retention in the program; and changes in how they view research and
researchers. The CARDS® valued the orientation and particularly emphasized the impact of
group training activities on communicating respectfully and giving effective feedback. They
also appreciated the orientation as a chance to experience what they would do at meetings,
and several members credited the orientation with giving them confidence that they could do
the job. The CARDS® described multiple characteristics of the monthly meetings that make
the work meaningful and enjoyable. These included learning about research topics that were
relevant to them and their community and having a chance to give advice directly to
researchers. The most prominent theme throughout the focus groups was the sense of
community that motivates the CARDS® to stay in the program. They used words such as
respect, good will, support, camaraderie, and fellowship to explain why they have stayed in
the program. They also described how our standard meeting practices such as the opening
question, use of the flip chart, meeting snacks, and non-hierarchical facilitation contribute to
a sense of community. Another major factor related to retention was the sense of service to
others. As one member said, “I like being able to help others. We may be helping folks who
will never even know we helped them.” Finally, the CARDS® discussed how their attitudes
toward research and researchers have shifted. They said that the opportunity to work
regularly with researchers has helped them respect researchers, rather than mistrust their
motives. Some CARDS® also indicated that they are more interested in participating in
research themselves: “Now I feel it’s important to participate in research. Researchers are
doing it for the community; it’s bigger than just one person.”
Kaiser et al. Page 10
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Researchers’ Evaluation of CARDS® Service
Since 2012, all guests who attend CARDS® meetings receive an email one week after the
meeting inviting them to complete a brief web-based evaluation survey. To minimize the
response burden on researchers, the survey includes just a few key questions to assess
guests’ satisfaction with the overall CARDS® service, including pre-meeting preparation
and post-meeting reports. Guests at CARDS® meetings have rated the service very highly.
Ninety-seven percent (
= 70) of survey respondents have indicated that they felt sufficiently
prepared by WINRS to meet with the CARDS®. Guests typically brought one or two items
for review, and they most frequently brought data collection materials such as survey or
focus group questions (51% of researchers,
= 37) and recruitment materials (37%,
= 27).
A large majority of guests (90%,
= 65) planned to change their materials based on
CARDS® feedback. (Several respondents self-identified as project staff that did not have
authority to make changes to research materials.) Ninety-nine percent of survey respondents
= 71; 1 missing response) have indicated that they would recommend the service to
colleagues. Table 3 displays findings related to guests’ perceptions of the utility of the
CARDS® program.
When we initiated the program, we wondered whether regular contact with researchers
would acclimate the CARDS® to research language and over time diminish their value to
researchers. To preserve the lay perspective of the CARDS®, we have maintained a schedule
of meeting only once monthly at each center. Researchers’ evaluations of the CARDS®
service have been favorable across the lifespan of the program, suggesting that the CARDS®
have continued to offer valuable feedback. Additional findings related to the effectiveness of
the CARDS® are published elsewhere (Bowers, Krupp, & Jacobson, 2016).
Health sciences research should serve everyone. Everyone should have opportunities to
participate in research and benefit from research. When researchers have study participants
who reflect the rich diversity in our communities, our society has a real chance to develop
policies and practices to eliminate health disparities. But current research suggests that
equitable representation of diverse groups in research has yet to be achieved (Bonevski et al.,
Social justice is a central concern of the nursing profession, and nursing research literature
offers numerous examples of studies using approaches such as CBPR to engage
disenfranchised or hard-to-reach stakeholders in the research process (e.g. Perry & Hoffman,
2010; Stacciarini et al., 2011). In addition, many nurse researchers have designed or
implemented interventions with input from community health workers or promotores (e.g.
Nies, Troutman-Jordan, Branche, Moore-Harrison, & Hohensee, 2013; Whittemore,
Rosenberg, Gilmore, Withey, & Breault, 2014). The CARDS® represent a new, unique
model for engagement of people who are not prominent community representatives in the
design of research, and the model offers distinct advantages over other forms of public
engagement. Although lay stakeholder engagement can take diverse forms, sometimes it can
seem as if the options are all or nothing—either researchers conduct CBPR or they do
Kaiser et al. Page 11
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
nothing to seek community input. The CARDS® model makes reaching hard-to-reach
stakeholders feasible and convenient for researchers.
The CARDS® are a truly innovative stakeholder engagement model, one that brings hidden
voices into the research enterprise and makes these voices easily accessible to researchers.
With our partners at the community centers, we have built a program that provides a
valuable service to researchers and a meaningful experience for lay stakeholders. The
CARDS® represent an innovative model for nurse researchers to support sustainable,
meaningful engagement of hard-to-reach populations in research planning and activities;
increase the diversity of groups represented in research studies; and ultimately help to
reduce health disparities.
WINRS and the CARDS® are funded by the University of Wisconsin-Madison School of Nursing and the Clinical
and Translational Science Award (CTSA) program, through the NIH National Center for Advancing Translational
Sciences (NCATS), grant UL1TR000427. The content is solely the responsibility of the authors and does not
necessarily represent the official views of the NIH. The authors thank current and former CARDS® and staff at the
Lussier Community Education Center and Goodman Community Center for their contributions to the CARDS®
Albert Einstein College of Medicine, The Bronx Health Link, & Community-Campus Partnerships for
Health. Community IRBs & research review boards: Shaping the future of community-engaged
research. 2012 Retrieved from
Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, et al. Reaching the hard-to-reach:
A systematic review of strategies for improving health and medical research with socially
disadvantaged groups. BMC Med Res Methodol. 2014; 14:42. [PubMed: 24669751]
Bowers BJ, Krupp A, Jacobson N. Can lay community advisors improve recruitment of
underrepresented populations?. 2016 Manuscript submitted for publication.
D'Alonzo KT. Getting started in CBPR: Lessons in building community partnerships for new
researchers. Nursing Inquiry. 2010; 17(4):282–288. [PubMed: 21059145]
Fernandez-Pena JR, Moore L, Goldstein E, Decarlo P, Grinstead O, Hunt C, et al. Making sure
research is used: Community-generated recommendations for disseminating research. Progress in
Community Health Partnerships: Research, Education, and Action. 2008; 2(2):171–176.
Ford JG, Howerton MW, Lai GY, Gary TL, Bolen S, Gibbons MC, et al. Barriers to recruiting
underrepresented populations to cancer clinical trials: A systematic review. Cancer. 2008; 112(2):
228–242. [PubMed: 18008363]
GreenMills LL, Davison KK, Gordon KE, Kaigang L, Jurkowski JM. Evaluation of a childhood
obesity awareness campaign targeting head start families: Designed by parents for parents. Journal
of Health Care for the Poor & Underserved. 2013; 24(2, Supp):25–33. 29. doi: [PubMed: 23727962]
Hasnain-Wynia R, Beal AC. Role of the patient-centered outcomes research institute in addressing
disparities and engaging patients in clinical research. Clinical Therapeutics. 2014; 36(5):619–623.
[PubMed: 24852595]
Institute of Medicine. Public engagement and clinical trials: New models and disruptive technologies:
Workshop summary. Washington, DC: 2011.
Israel BA, Parker EA, Rowe Z, Salvatore A, Minkler M, Lopez J, et al. Community-based participatory
research: Lessons learned from the centers for children's environmental health and disease
prevention research. Environmental Health Perspectives. 2005; 113(10):1463–1471. [PubMed:
Kaiser et al. Page 12
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
James S, Arniella G, Bickell NA, Walker W, Robinson V, Taylor B, et al. Community action boards:
An innovative model for effective community-academic research partnerships. Progress in
Community Health Partnerships: Research, Education, and Action. 2011; 5(4):399–404.
Joosten YA, Israel TL, Williams NA, Boone LR, Schlundt DG, Mouton CP, et al. Community
engagement studios: A structured approach to obtaining meaningful input from stakeholders to
inform research. Academic Medicine. 2015
Kitterman DR, Cheng SK, Dilts DM, Orwoll ES. The prevalence and economic impact of low-
enrolling clinical studies at an academic medical center. Academic Medicine. 2011; 86(11):1360–
1366. [PubMed: 21952064]
National Cancer Institute. Making health communication programs work. Bethesda, MD: National
Institutes of Health, US Department of Health and Human Services; 1989.
National Institute on Minority Health and Health Disparities. [Retrieved July 23, 2015] About
NIMHD: Vision and mission. (n.d.). from
Newman SD, Andrews JO, Magwood GS, Jenkins C, Cox MJ, Williamson DC. Community advisory
boards in community-based participatory research: A synthesis of best processes. Preventing
Chronic Disease: Public Health Research, Practice, and Policy. 2011; 8(3):A70. doi:
Nies MA, Troutman-Jordan M, Branche D, Moore-Harrison T, Hohensee C. Physical activity
preferences for low-income sedentary urban African American older adults. Journal of
Gerontological Nursing. 2013; 39(6):20–29. 10.
Paskett ED, Reeves KW, McLaughlin JM, Katz ML, McAlearney AS, Ruffin MT, et al. Recruitment of
minority and underserved populations in the United States: The centers for population health and
health disparities experience. Contemporary Clinical Trials. 2008; 29(6):847–861. [PubMed:
Perry C, Hoffman B. Assessing tribal youth physical activity and programming using a community-
based participatory research approach. Public Health Nursing. 2010; 27(2):104–114. 111.
[PubMed: 20433664]
Pinto RM, Spector A, Rahman R, Gastolomendo JD. Research advisory board members' contributions
and expectations in the USA. Health Promot Int. 2013
Schroen AT, Petroni GR, Wang H, Gray R, Wang XF, Cronin W, et al. Preliminary evaluation of
factors associated with premature trial closure and feasibility of accrual benchmarks in phase iii
oncology trials. Clinical Trials. 2010; 7(4):312–321. [PubMed: 20595245]
Silvestre AJ, Quinn SJ, Rinaldo CR. A twenty-two-year-old community advisory board: Health
research as an opportunity for social change. Journal of Community Practice. 2010; 18(1):58–75.
[PubMed: 20523763]
Stacciarini J-MR, Wiens B, Coady M, Schwait AB, Péérez A, Locke B, et al. CBPR: Building
partnerships with Latinos in a rural area for a wellness approach to mental health. Issues in Mental
Health Nursing. 2011; 32(8):486–492. 487. [PubMed: 21767250]
Stewart MK, Felix HC, Olson M, Cottoms N, Bachelder A, Smith J, et al. Community engagement in
health-related research: A case study of a community-linked research infrastructure, jefferson
county, arkansas, 2011–2013. Preventing Chronic Disease: Public Health Research, Practice, and
Policy. 2015; 12:E115.
Strauss RP, Sengupta S, Quinn SC, Goeppinger J, Spaulding C, Kegeles SM, et al. The role of
community advisory boards: Involving communities in the informed consent process. American
Journal of Public Health. 2001; 91(12):1938–1943. doi: [PubMed: 11726369]
U.S. Department of Health and Human Services. [Retrieved July 23, 2015] About Healthy People
2020. (n.d.). from
Whittemore R, Rosenberg A, Gilmore L, Withey M, Breault A. Implementation of a diabetes
prevention program in public housing communities. Public Health Nursing. 2014; 31(4):317–326.
310. [PubMed: 24251677]
Kaiser et al. Page 13
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Kaiser et al. Page 14
Table 1
Responsibilities of Community-Academic Partners for CARDS® Program
Community Center Staff WINRS Staff CARDS®
Recruit people who use center
services to participate in CARDS®
Design, deliver orientation program
for new CARDS®Complete orientation to develop
skills for giving feedback to
Schedule community center facilities
for meetings, orientations Conduct outreach to identify guest
researchers for CARDS® meetings NA
Arrange child care, transportation for
monthly meetings; serve as contact
person for members to confirm
Meet with researchers to prepare
agenda, materials for meetings; send
agenda, meeting reminder to
RSVP for monthly meetings; read
meeting announcement with
description of research topic and
Participate in CARDS® meetings Facilitate CARDS® meetings; take
notes; disburse member stipends Provide feedback to guest researchers
at meetings
Invoice UW-Madison School of
Nursing for facility fee Write summary reports, revised
materials for researchers; complete
post-meeting evaluation surveys with
guest researchers
Problem-solve with members to
address barriers to their participation Work with liaisons, members to
support participation and sustain
As needed, meet privately with
program staff to address issues or
problems related to participation
Participate in program evaluation Design and lead program evaluation Participate in individual interviews
and group discussions to evaluate
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Kaiser et al. Page 15
Table 2
What Researchers Say, What CARDS® Hear, and What CARDS® Recommend
What Researchers Say What CARDS® Hear What CARDS® Recommend
My colleague is the PI on
this study. I work with a private
investigator who will poke into
your personal business.
I work with Xxx Xxxx. (S)he is
the lead researcher on this
Data will be collected about
your lifestyle. We will violate your privacy and
make judgements about your
personal life.
My research team will ask you
some questions about a typical
day for you.
Participants will take part in
several procedures. Participants will have scary,
invasive medical acts performed
on them.
Participants will be involved in
several activities, including…
We will monitor your
progress during this study. We will track you with
something like an electronic
ankle bracelet during this study.
We will keep in touch with you
during the project.
This study will test an
experimental technique for
treating sinus infections.
You will be a guinea pig for
something dangerous, untried,
and invasive.
This project will test a new way
of treating sinus infections.
We would like to include
your name in our study
We want to put your name on a
registry, like a sex offender
We would like to add your name
to a list of people who might
like to be part of future projects.
Your input will contribute
to the design of an
intervention to support
couples in parenting their
You will help the research team
confront couples who are
having problems with parenting.
Your input will help us develop
a new program to support
parents as they care for their
This study aims to improve
how we provide care to
people with type 2 diabetes.
Being in this study will improve
your diabetes care. We’d like to hear about your
experiences to help us figure out
how to improve care for people
with type 2 diabetes.
We will consider your
individual home
environment and search for
assessable risks for falls.
We will judge you and your
home and snoop into your
personal belongings.
We will use a checklist to look
for risks that you can change to
prevent falls.
To manage your blood
pressure, be more active
and drink less alcohol.
We assume that you’re not
active and that you drink too
To manage your blood pressure,
be active and limit your intake
of alcohol.
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Kaiser et al. Page 16
Table 3
Guests’ Perception of Utility of CARDS® Services, 2012 – 2016
N = 72
Which services of the CARDS® program were particularly useful?
Completing web-based survey to initiate planning 28 39
Meeting with program staff
44 61
Presenting ideas and materials to CARDS®56 78
Getting feedback from CARDS®70 97
Receiving summary of CARDS® feedback 65 90
Receiving revised study materials 60 83
. Percentages do not sum to 100% because respondents could select multiple options.
New question added to survey in 2103.
Res Nurs Health
. Author manuscript; available in PMC 2018 February 01.
... This study represents one component of the Patient-Centered Outcomes Research Institute (PCORI)-sponsored Project SOAR (Sharing Opinions and Advice about Research in the Deep South, PCORI Contract #1097). Project SOAR was a community engagement project designed to empower community residents from communities labeled hard to reach in research [15] to work with scientists (i.e., faculty of local, regional, and national universities in the US) and trainees on topics arising from community needs and matching scientific and community research interests. Specifically, the Holt residents, trained as Project Advisory Council members of Project SOAR, identified environmental issues and the need for community gardens as research questions of community interest. ...
... Selection criteria for these individuals included: (1) residence within the designated community or status of having lived in Holt during childhood/K-12, (2) aged 30 or older, (3) ability to read and speak English, (4) absence of a primary diagnosis of severe mental illness, substance-use disorder, suicidal or homicidal ideation, (5) intent to continue to live in the community for at least one year, and (6) stable physical health with no intention to enter long-term care within the next year. Members of these PAC groups were trained to work collaboratively with local, regional, and national research partners on research questions of mutual interest [15]. Two four-hour training sessions included (1) human subjects' ethical research training; (2) the basic elements of community-based participatory research; (3) procedures to evaluate research materials, including recruitment flyers, surveys, and intervention manuals; and (4) communication skills for interacting effectively with scientific research partners in delivering feedback regarding their research materials. ...
... Each community worked with members of the scientific team to identify research questions of interest to their specific community. Examples of topics included materials for PAC training in the Deep South in comparison with CARDS© training conducted in the Midwest [15], surveys regarding the preparation for future healthcare needs, Head Start program implementation, and cognitive functioning and community-based dementia care. As such, the current analysis focuses on Holt, the urban site, because residents identified environmental concerns as their primary area of interest (the rural site focused on diabetes prevention, assessment, and treatment). ...
Full-text available
Through each era, the southeastern United States was and continues to be an epicenter for industrial companies to establish factories and plants. Though this development attracts economic gain for the companies and surrounding areas, low-income and predominantly Black communities bear the brunt of the environmental consequences while frequently remaining stagnant economically. This qualitative, community-based participatory research study grew out of a larger study designed to recruit lay community advisors from communities labeled as hard to reach in research. We focus on Holt, Alabama, an unincorporated community in the southeastern United States region. The primary goal of this research inquiry is to thematically analyze community interviews stemming from a topic of research, practice, and policy interest to community members: the effects of industrial pollution on Holt citizens’ daily lives. Content analysis of focus-group transcripts revealed four emergent themes, including: (1) how the pollution affects their water, soil, and air quality; (2) illness related to pollution; (3) community engagement and empowerment; and (4) suggestions regarding what government officials could do to address this area of need. Building upon the prior research regarding environmental justice, human flourishing, and the definition of nurturing environments, suggestions are made regarding the creation, implementation, and maintenance of project advisory councils focused on issues of environmental justice. Community advocacy and empowerment as well as community and scientific partnerships are imperative to alleviate problems associated with environmental justice.
... For >9 years, WINRS staff have been deeply involved in stakeholder engagement, as well as trained, planned and facilitated >250 lay advisory board meetings. 52,53 This meeting was an orientation and prepared the ambassadors to implement specific elements of Peers EXCEL such as the phone calls. Similar to prior studies, the training focused on the ambassador's role as peer support persons during group sessions and phone calls, using role playing as a training approach. ...
Full-text available
Introduction: Current diabetes self-management programs are often insufficient to improve outcomes for African Americans because of a limited focus on medication adherence and addressing culturally influenced beliefs about diabetes and medicines. This study evaluated the feasibility and acceptability of a novel culturally tailored diabetes self-management intervention that addressed key psychosocial and sociocultural barriers to medication adherence for African Americans. Methods: The intervention consisted of group education and race-congruent peer-based phone support. Three African Americans who were engaged in taking their diabetes medicines (ambassadors), were matched with 8 African Americans who were not engaged in taking medicines (buddies). We conducted a single group, pre/post study design with African Americans with type 2 diabetes. Wilcoxon signed rank tests assessed mean score differences in outcomes at baseline compared with 6-months follow-up. Semi-structured interviews explored buddies' acceptability of the intervention. Results: Buddies and ambassadors were similar in age and mostly female. Recruitment rates were 80% for buddies and 100% for ambassadors. Retention rate for primary outcomes was 75%. Buddies had a mean completion of 13.4/17 of sessions and phone calls. Ambassadors completed 84% of intervention calls with buddies. Although there were no statistically significant differences in mean A1C and medication adherence, we found a clinically meaningful decrease (-0.7) in mean A1C at the 6-month follow up compared to baseline. Secondary outcomes showed signal of changes. Themes showed buddies perceived an improvement in provider communication, learned goal setting strategies, and developed motivation, and confidence for self-management. Buddies perceived the program as acceptable and culturally appropriate. Conclusion: This culturally tailored diabetes self-management intervention that addresses diabetes self-management, psychosocial and behavioral barriers to medication adherence, and incorporates race-congruent peer support from African Americans engaged in taking medicines seemed feasible and acceptable. The results provide support for a fully powered randomized trial to test the intervention's efficacy. Trial registration: Date of registration: April 23, 2021.
... Third, as we have done in the past [15], we collaborated with WINRS, which is a patient and community engagement consultant group housed within the School of Nursing at the PI's institution. Since 2010, WINRS staff have been consulting with researchers and facilitating stakeholder engagement, as well as training, planning, and facilitating more than 250 community-based advisory board meetings [18,19]. ...
Full-text available
African Americans are more likely than non-Hispanic whites to be diagnosed with and die from diabetes. A contributing factor to these health disparities is African Americans’ poor diabetes medication adherence that is due in part to sociocultural barriers (e.g., medicine and illness misperceptions), which negatively affect diabetes management. In our prior work, we engaged with community stakeholders to develop and test a brief version of a culturally adapted intervention to address these barriers to medication adherence. The objective of this study was to elicit feedback to inform the refinement of the full 8-week intervention. We utilized a community-engaged study design to conduct a series of meetings with two cohorts of patient advisory boards of African Americans with type 2 diabetes who were adherent to their diabetes medicines (i.e., peer ambassadors). In total, 15 peer ambassadors were paired with 21 African American participants (i.e., peer buddies) to provide specific intervention support as peers and serve in an advisory role as a board member. Data were collected during nine board meetings with the patient stakeholders. A qualitative thematic analysis of the data was conducted to synthesize the findings. Feedback from the patient advisory board contributed to refining the intervention in the immediate-term, short-term, and long-term. The inclusion of African American community members living with type 2 diabetes on the advisory board contributed to further tailoring the intervention to the specific needs of African Americans with type 2 diabetes in the community.
... Prior research suggests barriers to participation include mistrust of medical research, language barriers, and demands and inconveniences of participation [38,39]. Community-based, tailored, and personalized recruitment efforts may facilitate continued engagement with underrepresented populations [39][40][41]. More research is needed to identify specific strategies that ensure demographic representation in health Table 2 The effect of incentive timing on six-month follow-up survey completion outcomes in a prospective cohort study 1 P-value associated with incentive assignment obtained from logistic regression models predicting each individual variable, with incentive assignment and study site as predictors 2 Not included in p-value calculation outcomes research and ensure the generalizability of the findings of this research [42]. ...
Full-text available
Background: Retaining participants over time is a frequent challenge in research studies evaluating long-term health outcomes. This study's objective was to compare the impact of prepaid and postpaid incentives on response to a six-month follow-up survey. Methods: We conducted an experiment to compare response between participants randomized to receive either prepaid or postpaid cash card incentives within a multisite study of children under 15 years in age who were hospitalized for a serious, severe, or critical injury. Participants were parents or guardians of enrolled children. The primary outcome was survey response. We also examined whether demographic characteristics were associated with response and if incentive timing influenced the relationship between demographic characteristics and response. We evaluated whether incentive timing was associated with the number of calls needed for contact. Results: The study enrolled 427 children, and parents of 420 children were included in this analysis. Follow-up survey response did not differ according to the assigned treatment arm, with the percentage of parents responding to the survey being 68.1% for the prepaid incentive and 66.7% with the postpaid incentive. Likelihood of response varied by demographics. Spanish-speaking parents and parents with lower income and lower educational attainment were less likely to respond. Parents of Hispanic/Latino children and children with Medicaid insurance were also less likely to respond. We found no relationship between the assigned incentive treatment and the demographics of respondents compared to non-respondents. Conclusions: Prepaid and postpaid incentives can obtain similar participation in longitudinal pediatric critical care outcomes research. Incentives alone do not ensure retention of all demographic subgroups. Strategies for improving representation of hard-to-reach populations are needed to address health disparities and ensure the generalizability of studies using these results.
... NIH-NCI has invested in cancer registries that are key for assessing incidence and mortality, 3,4 but underrepresentation of racial and ethnic minorities and rural communities continue to be a problem in cancer registries and research. 5,6 In response to this, a handful of community focused research registries have been established [7][8][9][10][11] to truly reach and engage traditionally underrepresented community members who are less likely to participate in research efforts that utilize more traditional, arms-length and random sampling recruitment procedures. 7-11 COE activities have long been recognized as a critical complement to cancer disparities and health equity research. ...
Full-text available
Abstract Objective To report on the development and preliminary findings of a community‐based cancer registry, including the community‐engaged approach to recruitment, participant profile, and distribution of cancer risk factors by race/ethnicity and geography. Methods Community outreach and engagement best practices were used to recruit a diverse convenience sample of Virginia residents (≥18 years) that oversampled residents living in rural areas, defined as Rural‐Urban Continuum Codes (RUCC) 4–9 and African American (AA)/Black residents. Multiple survey administration methods included electronic (e‐survey) and in‐person survey by community‐based staff. Results At the time of this analysis, 595 participants are enrolled; 73% are rural, 46% are AA/Black. AA/Black participants reported similar education but lower income (p
... A CAB 48 was convened at the start of the project to ensure broad representation of the community and our study population, 49 including stakeholders from the three recruitment/implementation sites as well as representatives from key organizations such as the RI Healthy Schools Coalition, the RI Department of Health, the RI Community Food Bank, and PCMH-Kids in RI. This high-level CAB provides input regarding strategies for reaching prospective families and effectively promoting the intervention in the target communities as well as understanding complexities related to implementation challenges within each setting. ...
Background: Overweight and obesity in children is a public health crisis in the United States. Although evidence-based interventions have been developed, such programs are difficult to access. Dissemination of evidence-based pediatric weight management interventions (PWMIs) to families from diverse low-income communities is the primary objective of the CDC Childhood Obesity Research Demonstration (CORD) projects. Methods: The goal of the Rhode Island CORD 3.0 project is to adapt the evidence-based PWMI, JOIN for ME, for delivery among diverse families from low-income backgrounds and to test it in a hybrid effectiveness-implementation trial design in which the aims are to examine implementation and patient-centered outcomes. Children between the ages of 6 and 12 years with BMI ≥85th percentile and a caregiver will be recruited through two settings, a federally qualified health center, which serves as a patient-centered medical home, or low-income housing. Dyads will receive a remotely delivered group-based intervention that is 10 months in duration and includes 16 weekly sessions, followed by 4 biweekly and 4 monthly meetings. Assessments of child and caregiver weight status and child health-related quality of life will be conducted at baseline, and at 4 and 10 months after the start of intervention. Implementation outcomes assessing intervention acceptability, adoption, feasibility, fidelity, and penetration/reach will be collected to inform subsequent dissemination. Conclusions: If the adapted version of the JOIN for ME intervention can be successfully implemented and is shown to be effective, this project will provide a model for a scalable PWMI for families from low-income backgrounds. no. NCT04647760.
Objectives Health education may improve health in geriatric patients. To evaluate differences between remote and in-person education, the DREAMS ( Developing a Research Participation Enhancement and Advocacy Training Program for Diverse Seniors) health seminar series compared in-person and remote learning groups to assess feasibility, satisfaction, adherence, health literacy, and cognitive outcomes. Research Design Nonrandomized two-arm interventions occurred remotely or in-person. About 130 diverse, older adults ( M age: 70.8 ± 9.2 years; in-person n = 95; remote, n = 35) enrolled. Data from 115 completers (In-person n = 80; Remote n = 35) were analyzed for performance outcomes. Feasibility, adherence, and satisfaction benchmarks were evaluated at baseline, immediately post intervention, and 8 weeks post intervention. Adjusting for baseline performances, outcomes on health literacy and cognitive measures were compared between groups after intervention (at posttest and at 8-week follow-up) using adjusted mean differences (β coefficients). Results Eighty in-person and all remote participants completed at least six modules. Both programs had high satisfaction, feasibility, and strong adherence. After adjusting for demographic covariates and baseline values, cognitive and motor cognitive measures between groups were domain specific (e.g., global cognition, executive function, spatial memory, mental tracking capacity, and cognitive integration). Discussion and Implications This work explores feasible measures of knowledge acquisition and its link to health literacy and cognitive outcomes. Identifying effective delivery methods may increase involvement in clinical research. Future studies may encourage remote learning for increased accessibility.
Full-text available
This article explores the lived experiences and working time complexities of low‐paid workers in legitimate multiple employment and zero hours work. Based on detailed qualitative research, these workers have 2, 3, 4, 5 and even 7 different jobs out of necessity due to low‐pay, unpredictable working hours and employment precarity. The research reveals that workers need to be available for (potential) work at any point but may not actually be offered any hours, which we argue constitutes unremunerated labour time. The findings highlight a densification of working time with zero hours work as employers maximize productive effort into specifically numbered, demarcated and minimized working hours, which tightens the porosity of labour. There is a dual fragmentation and individualization of employment, as these workers traverse multiple, expansive, complex and dynamic temporalities of work. This study identifies new economic and temporal indeterminacies of labour, which fundamentally transform the employment relationship and wage‐effort bargain.
Full-text available
To mitigate the opioid epidemic, a concerted effort to educate, prevent, diagnose, treat, and engage residents is required. In this study, a digitally distributed method to form a large network of organizations was tested with 99 counties in regions with high vulnerability to hepatitis C virus (HCV). The method involved a cascade of contacts going from email to phone calls, to videoconferencing and measuring the number of contacts required, amount of time taken, and the proportion of success at recruiting at least one community organization per county. A recruitment period of 5 months and 2118 contact attempts led to the recruitment of organizations from 73 out of our 99 target counties. Organizations belonging to health departments required more attempts and time to recruit but ultimately enrolled at higher rates than did other organizations such as coalitions and agencies. Organizations from counties more (vs. less) vulnerable to HCV outbreaks required more attempts to recruit and, using multiple recruitment methods (e.g., emails, phone calls, and Zoom meetings), improved enrollment success. Overall, this method proved to be successful at remotely engaging a large‐scale network of communities with different levels of risk within a large geographic region.
Full-text available
Background and Objectives: This paper reports on an interpretive evaluation conducted to identify successes and weaknesses of an 8-week educational intervention co-taught by medical students and faculty that was designed to foster communication between clinicians and researchers and ultimately increase participation in clinical research by older adults, including underrepresented groups. Weekly topics focused on age-related changes and health conditions, socio-contextual factors impacting aging populations, and wellness strategies. Research Design and Methods: A post-intervention focus group was conducted with a representative group of eight older adults (mean age = 76 ± 11 years) from 51 total participants who completed the educational intervention. Participants were diverse in race, socio-economic status, education level, sex, and marital status. Results: Thematic analyses show several participants were motivated by their personal experiences to join the study and continue coming to the seminars throughout the study. While participants viewed most aspects of the study as a success and stated that it was a productive learning experience, most participants had suggestions for improvements in the program content and implementation. Specifically, the composition of and direction to small breakout groups should be carefully considered and planned in this population, and attention should be paid to delivery of sensitive topics, e.g., death, dementia. A clear main benefit of this programmatic approach is the development of rapport among participants and between participants and clinical researchers. Discussion and Implications: Results provide useful insights regarding improving participation among hard-to-reach and historically under-represented groups of older adults in clinical research. Future iterations of the DREAMS program and similar educational interventions can use these findings to improve the program and better achieve the programmatic objectives.
Full-text available
Community-based participatory research (CBPR) is a paradigm to study and reduce disparities in health outcomes related to chronic disease. Community advisory boards (CABs) commonly formalize the academic-community partnerships that guide CBPR by providing a mechanism for community members to have representation in research activities. Researchers and funding agencies increasingly recognize the value of the community's contribution to research and acknowledge that community advisory boards are a key component of successful CBPR projects. In this article, we describe the best processes for forming, operating, and maintaining CABs for CBPR. We synthesize the literature and offer our professional experiences to guide formation, operation, and maintenance of CABs.
Full-text available
Background: Underrepresentation of racial minorities in research contributes to health inequities. Important factors contributing to low levels of research participation include limited access to health care and research opportunities, lack of perceived relevance, power differences, participant burden, and absence of trust. We describe an enhanced model of community engagement in which we developed a community-linked research infrastructure to involve minorities in research both as participants and as partners engaged in issue selection, study design, and implementation. Community context: We implemented this effort in Jefferson County, Arkansas, which has a predominantly black population, bears a disproportionate burden of chronic disease, and has death rates above state and national averages. Methods: Building on existing community-academic partnerships, we engaged new partners and adapted a successful community health worker model to connect community residents to services and relevant research. We formed a community advisory board, a research collaborative, a health registry, and a resource directory. Outcome: Newly formed community-academic partnerships resulted in many joint grant submissions and new projects. Community health workers contacted 2,665 black and 913 white community residents from December 2011 through April 2013. Eighty-five percent of blacks and 88% of whites were willing to be re-contacted about research of potential interest. Implementation challenges were addressed by balancing the needs of science with community needs and priorities. Interpretation: Our experience indicates investments in community-linked research infrastructure can be fruitful and should be considered by academic health centers when assessing institutional research infrastructure needs.
Full-text available
Problem: Engaging communities in research increases its relevance and may speed the translation of discoveries into improved health outcomes. Many researchers lack training to effectively engage stakeholders, whereas academic institutions lack infrastructure to support community engagement. Approach: In 2009, the Meharry-Vanderbilt Community-Engaged Research Core began testing new approaches for community engagement, which led to the development of the Community Engagement Studio (CE Studio). This structured program facilitates project-specific input from community and patient stakeholders to enhance research design, implementation, and dissemination. Developers used a team approach to recruit and train stakeholders, prepare researchers to engage with stakeholders, and facilitate an in-person meeting with both. Outcomes: The research core has implemented 28 CE Studios that engaged 152 community stakeholders. Participating researchers, representing a broad range of faculty ranks and disciplines, reported that input from stakeholders was valuable and that the CE Studio helped determine project feasibility and enhanced research design and implementation. Stakeholders found the CE Studio to be an acceptable method of engagement and reported a better understanding of research in general. A tool kit was developed to replicate this model and to disseminate this approach. Next steps: The research core will collect data to better understand the impact of CE Studios on research proposal submissions, funding, research outcomes, patient and stakeholder engagement in projects, and dissemination of results. They will also collect data to determine whether CE Studios increase patient-centered approaches in research and whether stakeholders who participate have more trust and willingness to participate in research.
Full-text available
This study aims to review the literature regarding the barriers to sampling, recruitment, participation, and retention of members of socioeconomically disadvantaged groups in health research and strategies for increasing the amount of health research conducted with socially disadvantaged groups. A systematic review with narrative synthesis was conducted. Searches of electronic databases Medline, PsychInfo, EMBASE, Social Science Index via Web of Knowledge and CINHAL were conducted for English language articles published up to May 2013. Qualitative and quantitative studies as well as literature reviews were included. Articles were included if they reported attempts to increase disadvantaged group participation in research, or the barriers to research with disadvantaged groups. Groups of interest were those described as socially, culturally or financially disadvantaged compared to the majority of society. Eligible articles were categorised according to five phases of research: 1) sampling, 2) recruitment and gaining consent, 3) data collection and measurement, 4) intervention delivery and uptake, and 5) retention and attrition. In total, 116 papers from 115 studies met inclusion criteria and 31 previous literature reviews were included. A comprehensive summation of the major barriers to working with various disadvantaged groups is provided, along with proposed strategies for addressing each of the identified types of barriers. Most studies of strategies to address the barriers were of a descriptive nature and only nine studies reported the results of randomised trials. To tackle the challenges of research with socially disadvantaged groups, and increase their representation in health and medical research, researchers and research institutions need to acknowledge extended timeframes, plan for higher resourcing costs and operate via community partnerships.
Full-text available
This study focuses on the Implementation Community Collaborative Board (I-CCB) to identify members' anticipated contributions to and returns from the I-CCB; examine whether or not members achieved these contributions and returns over time; and explore barriers and facilitators that influenced accomplishments. Longitudinal study with repeated semi-structured in-depth interviews; baseline captured anticipated contributions and returns; 6- and 18-month follow-ups short- and longer-term achievements. We used content analysis to code/reduce text into variables, describe, count and compare categories. Participants anticipated involvement in I-CCB dynamics/governance and in research tasks/procedures. Anticipated returns included social support. Participants exerting influence on I-CCB's research agenda stayed the same over time. Participants conducting research doubled between follow-ups; those writing grant proposals increased by 50%. Participants receiving emotional support remained the same. Challenges: meetings steered by researchers; lack of time; use of jargon. Facilitators: outreaching to community; being affected by HIV; having overlapping identities/roles as researcher, service consumer and/or practitioner. Research partners can maximize facilitators, redress barriers and improve advisory board members' retention. Findings may help optimize the functioning of advisory boards worldwide.
Full-text available
The Communities for Healthy Living program used a community-based participatory research (CBPR) approach to empower Head Start parents in designing and pilot testing a multi-component family-centered obesity prevention program. One program component was a childhood obesity awareness campaign addressing common parental misconceptions about obesity. The campaign was designed by a community advisory board of parents to target specific issues identified within their own community. Results from pre-post intervention surveys (N=108) showed that campaign exposure was high; 92% of responding parents reported noticing the campaign. Parents also demonstrated significant increases in awareness of childhood obesity, along with decreases in obesity-related misconceptions. Findings, supported by growing literature on CBPR, suggest a CBPR approach to campaign development is an effective strategy to promote parent awareness of childhood obesity.
Full-text available
The objective of this study was to determine (a) activity preferences for low-income sedentary urban African American older adults and (b) information needed to deliver a lay physical activity intervention in the community for this population. This descriptive qualitative study used six focus groups. Participants were African American, 55 and older, had low incomes, and had sedentary behavior. Physical activity themes included excitement/emotion for physical activity, group physical activity, and location of physical activity. Themes regarding aspects of being a lay community health worker included beneficial service, uncertainty, logistics concerns, and delivery method preferences. The findings from this study will provide the basis for an intervention for low-income sedentary African American older adults. Preferences for physical activity, concerns about and supports needed for individuals to serve as lay community health workers, various types of training materials, and preferred technology for physical activity participation are identified and discussed.
The purpose of this study was to describe the process of implementing a diabetes prevention program provided by homecare nurses to residents of public housing communities. A cluster randomization pilot study was conducted comparing enhanced standard care (2 interactive classes on diabetes prevention) to a diabetes prevention program (7 interactive classes and behavioral support). The sample (n = 67) was primarily female (79%), nonwhite (76%), unpartnered (83%), with a mean age of 40 years, and an average of 3 children. Mixed methods were used to evaluate the implementation process. Data were collected on attendance, attrition, and protocol implementation. Interviews were conducted with nurses and community health workers who assisted with program implementation. Homecare nurses were able to implement a diabetes prevention program in public housing communities, with a protocol implementation of 83% across classes and groups. Attendance was suboptimal with 60% for the enhanced standard care group and 54% for the diabetes prevention group. Nurses and community health workers were resourceful and positive about program implementation. Linking existing resources, such as a homecare agency with a public housing community, is one approach to disseminate diabetes prevention programs.
Racial and ethnic minorities, older adults, rural residents, and individuals of low socioeconomic status are underrepresented among participants in cancer-related trials. The authors conducted a systematic review to determine the barriers to participation of underrepresented populations in cancer-related trials. Their search included English-language publications that reported original data on the recruitment of underrepresented groups to cancer treatment or prevention trials between 1966 and December 2005 in multiple electronic databases. They also hand-searched titles in 34 journals from January 2003 to December 2005 and they examined reference lists for eligible articles. Titles and abstracts were reviewed to identify relevant studies. Data on barriers to participation were synthesized both qualitatively and based on statistically significant associations with trial enrollment. Of 5257 studies that were cited, 65 studies were eligible for inclusion in the current analysis, including 46 studies on recruitment into cancer therapeutic trials, 15 studies on recruitment into prevention trials, and 4 studies on recruitment into both prevention and treatment trials. Numerous factors were reported as barriers to participation in cancer-related trials. However, only 20 of the studies reported statistically significant associations between hypothesized barriers and enrollment. The available evidence had limitations in quality regarding representativeness, justification of study methods, the reliability and validity of data-collection methods, potential for bias, and data analysis. The results indicated that underrepresented populations face numerous barriers to participation in cancer-related trials. The current systematic review highlighting the literature on recruitment of underrepresented populations to cancer trials and may be used as the evidence base toward developing an agenda for etiologic and intervention research to reduce the disparities in participation in cancer-related trials. Cancer 2008. © 2007 American Cancer Society.