No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Challenges in Applying Indigenous Evaluation Practices in Mainstream Grant Programs to Indigenous Communities
Abstract
How can indigenous evaluators implement culturally competent models in First Nations communities while ensuring that government grant evaluation requirements are met? Through describing the challenges in one tribal community in the United States, this article will discuss how American Indian/Alaska Native substance abuse prevention programs are evaluating the implementation and outcomes of Strategic Prevention Framework grants from the federal government's Center for Substance Abuse Prevention. Requirements for implementing evidence-based programs normed on other populations and for evaluating data based on quantitative methods add to the challenge. Throughout the process, much is being learned that it is hoped will strengthen indigenous grantees and increase the cultural competence of government evaluation requirements.
... As a result, discussions should not be limited to the cultural competence of the evaluators but must also recognize the culture of evaluation itself (Chouinard & Cram, 2019). Evaluators need to be held accountable for providing culturally safe evaluations, as defined by Indigenous peoples and communities (Freeman, Edwards, et al., 2014;Grover, 2010;Tipene-Leach et al., 2013). ...
... Many authors indicated that evaluation methods built on collaboration, participation, and empowerment are more effective and sustainable than those that position the researcher and members of the Indigenous community as the helper and helpee, respectively. This collaborative approach should be built into the entire evaluation process from initiation through to the implementation and evaluation of recommendations (Broughton et al., 2014;Grover, 2010;Potvin et al., 1995;Tipene-Leach et al., 2013). Collaborative approaches can also help to ensure that the evaluation is practical and improve the trustworthiness throughout the duration of the program or service (Jan, 1998;McShane et al., 2013;Tipene-Leach et al., 2013). ...
... Evaluators must not treat Indigenous people as a homogenous group. There are many diverse Indigenous nations, languages, and cultural practices worldwide, both within countries (Baldwin, 1998;Tipene-Leach et al., 2013;Voyle & Simmons, 1999) and within local Indigenous communities (Baldwin, 1998;Barnett & Kendall, 2011;Grover, 2010Grover, , 2016. For example, within Indigenous social systems, respect for individual autonomy of decision making can be an important value, further supporting individual diversity within a community, though it is commonly balanced with a foundational regard for the well-being of the collective (Brant, 1990). ...
This study systematically reviewed evidence regarding health program and service evaluations in Indigenous contexts. Following the PRISMA guidelines and combining terms for ‘Indigenous populations’ and ‘health programs and services’. Eight principles emerged: Principle 1: Adopting Indigenous led or co-led approaches is vital to balance power relationships by prioritizing self-determination, Principle 2: Evaluation team should include local Indigenous community members, Principle 3: Indigenous community knowledge and practice should be foundational, Principle 4: Evaluations must be responsive and flexible to meet the needs of the local community, Principle 5: Evaluations should respect and adhere to local Indigenous protocols, culture, wisdom and language, Principle 6: Evaluations should emphasize reciprocity, shared learnings and capacity building, Principle 7: It is important to build strong relationships and trust between and within researcher teams, evaluators and communities, and Principle 8: The evaluation team must acknowledge community capacity and resources by investing in time and relationships.
... External evaluators have imposed and conducted evaluations on Indigenous com munities (LaFrance & Nichols, 2008;Smith, 1999;Taylor, 2003). This has been exacerbated by substandard practices in designing and implementing evaluations, a lack of resources available for conducting Indigenous evaluations ( Chouinard & Cousins, 2007 ), inadequate population health data (Smylie & Anderson, 2006), and incongruent expectations between community and evaluators related to eval uation in Indigenous contexts ( Grover, 2010 ). There is growing recognition that these evaluations are embedded in unique social and historical contexts and must be conducted in culturally relevant and meaningful ways ( Chouinard & Cousins, 2007 ;National Aboriginal Health Organization, 2003). ...
... Nevertheless, principles identified through this review are consistent with the United Nations Declaration on the Rights of Indigenous Peoples ( 2008 ), the Truth and Reconciliation Commission Calls to Action ( 2015 ), and peer-reviewed literature. Specifically, literature indicates that research and/ or evaluation with Indigenous communities should be grounded in Indigenous ways of knowing and traditional Indigenous knowledge (Castellano, 2004;Ko vach, 2010), rooted in the community's context, observing community protocols (Grover, 2010;LaFrance, Nichols, & Kirkhart, 2012;Wilson, 2008), and relevant to and reflective of community needs (PHAC, 2015). Scholars have also identifi ed that researcher-community relationships must exemplify respect and reciprocity (Wilson, 2008), research should benefit communities and support local capacity (Ball & Janyst, 2008), and communities should have access to evaluation data and results (PHAC, 2015;Riddell, Salamanca, Pepler, Cardinal, & McIvor, 2017). ...
... Scholars have also identifi ed that researcher-community relationships must exemplify respect and reciprocity (Wilson, 2008), research should benefit communities and support local capacity (Ball & Janyst, 2008), and communities should have access to evaluation data and results (PHAC, 2015;Riddell, Salamanca, Pepler, Cardinal, & McIvor, 2017). Further, researchers have highlighted the importance of acknowledging and relin quishing power imbalances (LaFrance, 2004), building strong relationships with communities (Flicker & Worthington, 2012), seeking guidance from knowledge keepers and elders (George et al., 2007), and engaging the community at the outset of evaluation (Grover, 2010). Finally, the literature indicates that Indigenous peo ples have a right to participate in research that affects their well-being (Castellano, 2004;United Nations, 2008) and to exercise community self-determination ( First Nations Centre, 2007 ). ...
This article describes findings from a scoping review of the grey literature to identify principles, approaches, methods, tools, and frameworks for conducting program evaluation in Indigenous contexts, reported from 2000–2015 in Canada, the United States, New Zealand, and Australia. It includes consultation with key informants to validate and enrich interpretation of fi ndings. Th e fi fteen guiding principles, and the approaches, methods, tools, and frameworks identifi ed through this review may be used as a starting point for evaluators and communities to initiate discussion about how to conduct their evaluation in their communities, and which approaches, methods, tools, or frameworks would be contextually appropriate.
... Un nombre inquiétant d'enfants autochtones s'initient à la cigarette, à l'alcool et au cannabis dès l'âge scolaire primaire (Coleman, Charles et Collins, 2001 ;Cotton, 2009 ;Cotton et Laventure, 2013). Or, les études canadiennes soulignent le manque, voire l'absence de ressources adaptées pour les Autochtones dans le domaine des dépendances, particulièrement pour ceux habitant dans une communauté (Dell et Lyons, 2007 ;Grover, 2010). En effet, dans certaines communautés, la sélection des programmes de prévention en dépendance serait davantage dictée par les urgences, le manque d'infrastructures et de personnel formé (Kinnon, 2002) que par leur validité scientifique. ...
... En effet, dans certaines communautés, la sélection des programmes de prévention en dépendance serait davantage dictée par les urgences, le manque d'infrastructures et de personnel formé (Kinnon, 2002) que par leur validité scientifique. Ainsi, les programmes de prévention offerts dans les communautés autochtones s'avèrent souvent mal adaptés aux réalités culturelles (Grover, 2010). ...
... C'est pourquoi d'autres auteurs militent plutôt en faveur de l'adaptation des programmes universels prometteurs ou probants, en impliquant des acteurs-clés en provenance des milieux autochtones (Holleran Steiker et al., 2008 ;Okamoto, Helm, McClain et Dinson, 2012). Cette dernière approche, lorsque bien orchestrée, permet d'adapter les programmes d'une façon rigoureuse (Chino et DeBruyn, 2006 ;Dell et al., 2012 ;Grover, 2010 ;McKennitt, 2007 ;Raghupathy et Forth, 2012 ;Whitbeck, Walls et Welch, 2012 ;Moodie, 2010 ;Kumpfer, Pinyuchon, de Melo et Henry, 2008). ...
Plusieurs etudes canadiennes soulignent le manque de programmes adaptes pour les Autochtones dans le domaine des dependances, particulierement en prevention pour les jeunes habitant dans une communaute. A l’aide du modele d’adaptation culturelle de Kumpfer et ses collegues (2008) et d’un devis mixte incorpore, cette etude vise a decrire l’adaptation en « profondeur » du programme de prevention des dependances Systeme d pour des eleves en provenance d’une ecole innue du Quebec. Pour le volet qualitatif, differents acteurs du milieu autochtone ont ete consultes (un directeur d’ecole, deux enseignants, deux intervenants scolaires, six animateurs, neuf parents) afin d’adapter le programme en « profondeur ». Pour le volet quantitatif, des journaux de bord ont permis, d’une part, de cibler les activites ayant ete realisees et, d’autre part, de determiner leur degre d’utilite percue selon les animateurs et les enseignants des activites ayant ete animees. Les principales adaptations apportees au programme concernent la duree des ateliers, les methodes pedagogiques utilisees pour presenter les concepts et l’integration de references culturelles autochtones. A la lumiere de ces resultats, il est recommande de poursuivre l’adaptation culturelle de ce programme pour ensuite valider sa pertinence culturelle au sein d’autres ecoles autochtones et evaluer son efficacite.
... External evaluators have imposed and conducted evaluations on Indigenous com munities (LaFrance & Nichols, 2008;Smith, 1999;Taylor, 2003). This has been exacerbated by substandard practices in designing and implementing evaluations, a lack of resources available for conducting Indigenous evaluations ( Chouinard & Cousins, 2007 ), inadequate population health data (Smylie & Anderson, 2006), and incongruent expectations between community and evaluators related to eval uation in Indigenous contexts ( Grover, 2010 ). There is growing recognition that these evaluations are embedded in unique social and historical contexts and must be conducted in culturally relevant and meaningful ways ( Chouinard & Cousins, 2007 ;National Aboriginal Health Organization, 2003). ...
... Nevertheless, principles identified through this review are consistent with the United Nations Declaration on the Rights of Indigenous Peoples ( 2008 ), the Truth and Reconciliation Commission Calls to Action ( 2015 ), and peer-reviewed literature. Specifically, literature indicates that research and/ or evaluation with Indigenous communities should be grounded in Indigenous ways of knowing and traditional Indigenous knowledge (Castellano, 2004;Ko vach, 2010), rooted in the community's context, observing community protocols (Grover, 2010;LaFrance, Nichols, & Kirkhart, 2012;Wilson, 2008), and relevant to and reflective of community needs (PHAC, 2015). Scholars have also identifi ed that researcher-community relationships must exemplify respect and reciprocity (Wilson, 2008), research should benefit communities and support local capacity (Ball & Janyst, 2008), and communities should have access to evaluation data and results (PHAC, 2015;Riddell, Salamanca, Pepler, Cardinal, & McIvor, 2017). ...
... Scholars have also identifi ed that researcher-community relationships must exemplify respect and reciprocity (Wilson, 2008), research should benefit communities and support local capacity (Ball & Janyst, 2008), and communities should have access to evaluation data and results (PHAC, 2015;Riddell, Salamanca, Pepler, Cardinal, & McIvor, 2017). Further, researchers have highlighted the importance of acknowledging and relin quishing power imbalances (LaFrance, 2004), building strong relationships with communities (Flicker & Worthington, 2012), seeking guidance from knowledge keepers and elders (George et al., 2007), and engaging the community at the outset of evaluation (Grover, 2010). Finally, the literature indicates that Indigenous peo ples have a right to participate in research that affects their well-being (Castellano, 2004;United Nations, 2008) and to exercise community self-determination ( First Nations Centre, 2007 ). ...
Inquiry learning is typically seen as a pedagogical approach, both a method and a process, delivered in small class settings. The basic principles of inquiry are investigative in nature; exploration and discovery are fundamental to student learning. Inquiry is an innovative, institutionally supported teaching development applied at McMaster University, operating in small, first year stand-alone courses in the Faculties of Social Science, Science and Humanities since 1998. The introduction of inquiry learning in first year classes has translated into inquiry as a pedagogical approach in all departments and at all levels in the Faculty of Social Science (Vajoczki et al., 2011). Given McMaster's commitment to inquiry teaching and learning, an exploration of instructor understanding and application of inquiry, is required to enhance how our understanding of how this emphasis has shaped teaching and learning. The focus of this research is to examine instructor perspectives on the pedagogical and methodological role of inquiry learning, as well as variation in approaches between instructors who both taught formal inquiry courses, and those who did not. As part of a broader research project examining the potential of larger inquiry class sizes, interviews were undertaken (N=16) to elicit instructor perceptions of inquiry teaching. Results suggest instructors approached their inquiry teaching differently - some with an intentional and some an unintentional approach. There was consensus that: 1. Teaching and learning academic skills is a valuable academic undertaking; 2. Inquiry instructors are facilitators in the classroom; 3. Technology has only a modest place in the inquiry classroom; and, 4. Larger classes inhibit inquiry teaching and learning. There were mixed responses about the importance of stand-alone inquiry courses. Teaching formal inquiry courses appears to change the instructor's perspective about inquiry: they were more likely to be intentional in their approach to teaching inquiry.
... Indigenous evaluation centralizes tribal context and Indigenous knowledge while building capacity in the community. Indigenous researchers have long called for research and evaluation that is grounded in Indigenous values (Grover & RMC Research Corporation, 2010;LaFrance & Nichols, 2008). While the implementation of Indigenous evaluation may be different depending on the context, the core values remain constant along with the use of culturally valid measures. ...
... While the implementation of Indigenous evaluation may be different depending on the context, the core values remain constant along with the use of culturally valid measures. Culturally valid measures include oral measures, elder review, and community contributions to evaluation (Grover & RMC Research Corporation, 2010). The NS program attended to the relationship between the program and community, while honoring tribal sovereignty, by disseminating evaluation results according to the CARE (collective benefit, authority for control, T A B L E 2 Native Spirit session outline ...
Culturally grounded after-school programs (ASPs), based on local cultural values and practices, are often developed and implemented by and for the local community. Culturally grounded programs promote health and well-being for American Indian and Alaska Native (AI/AN) adolescents by allowing them to reconnect to cultural teachings that have faced attempted historical and contemporary erasure. This article is a first-person account that describes the development and implementation of a culturally grounded ASP, Native Spirit (NS), for AI adolescents (grades 7-12) living on a Southwest urban-based reservation. NS, a 13-session culturally grounded ASP, was developed by an academic-community partnership that focuses on increasing cultural engagement as a form of positive youth development. Each session was guided by one to two local cultural practitioners and community leaders. The development of the NS program contributed to an Indigenous prevention science that emphasizes the positive impacts of Indigenous culture and community on health and well-being. The use of the ASP format, in partnership with the Boys & Girls Club, increased the feasibility of dissemination and refinement of the NS program by tribal communities and organizations.
... Therefore, funding bodies and evaluators are responsible for identifying what they can do to support the tenets and processes set forth in the following document. Meaningful community involvement in the design of Indigenous evaluation must occur at the beginning, must be fully supported with resources, and must be given the time and space to build capacity (Grover, 2010, Grover, Cram & Bowman, 2007LaFrance, 2004). ...
... Recognition of this misalignment in the field of evaluation dates back to the 1970s (Hurworth & Harvey, 2012). Requirements typically do not take into account the need for Indigenous evaluation frameworks, methods, evaluators, and the increased time required to build these models (Grover, 2010). This leads to evaluations that do not meet the underlying needs of the program or the mechanisms through which change typically occurs based on these models. ...
... Mots clé : décolonisation, principes directeurs, autochtone, expérience autochtone, autodétermination BACKGROUND In Canada, evaluations of programs and services that are tailored for and/or include Indigenous peoples are often under-resourced, poorly designed, and fail to take community evaluation priorities into account (Grover, 2008;Scott, 2008). Mainstream evaluations also tend to exclude Indigenous-specific needs from their performance assessments (Grover, 2008). ...
... Mots clé : décolonisation, principes directeurs, autochtone, expérience autochtone, autodétermination BACKGROUND In Canada, evaluations of programs and services that are tailored for and/or include Indigenous peoples are often under-resourced, poorly designed, and fail to take community evaluation priorities into account (Grover, 2008;Scott, 2008). Mainstream evaluations also tend to exclude Indigenous-specific needs from their performance assessments (Grover, 2008). While there has been a move ment toward strengths-based, holistic, and "culturally responsive" evaluations, Indigenous scholars have called for evaluations that are grounded in Indigenous worldviews and ways of knowing and informed by locally defined values, such as sovereignty, reciprocity, and place (Cram, 2018;Lafrance, Nichols, & Kirkhart, 2012;Waapalaneexkweew & Dodge-Francis, 2018). ...
A group of Indigenous health and social service evaluators called the “Three Ribbon” panel came together in Toronto, Canada with the goal of informing a set of evidence-based guidelines for urban Indigenous health and social service and program evaluation. The collective knowledge and experiences of the Three Ribbon panel was gathered through discussion circles and synthesized around the following areas: barriers with conducting Indigenous health and social service evaluation; decolonizing principles and protocols that support community self-determination and centralize Indigenous culture and worldviews; and guidelines to inform health and social service evaluation moving forward. The wisdom and contributions of the Three Ribbon Panel creates space for Indigenous worldviews, values and beliefs within program evaluation practice and has important implications for evaluation research and application.
... Evaluators may do this through strengthening collaborative relationships between evaluators and stakeholders, including both those in traditional positions of power, and those who are traditionally marginalised. Indigenous axiological assumptions align with the transformative principle that the essential characteristics associated with these relationships are that they are respectful and trusting (Grover, 2010). The indigenous paradigm also extends the idea of relationships through indigenous communities' requirements that indigenous evaluators come face-to-face with them, demonstrate caution, listen and look first before speaking, share their skills and knowledge, and be humble and uphold the status of the community (Cram, 2009;Smith, 2012). ...
... Relationships of trust are key as knowledge is sharing within the context of relationships and with the approval of those whose role it is to protect such knowledge. This is also important because the indigenous programmes and services that are evaluated are often built upon, or tailored to respond to, the same indigenous epistemology that indigenous evaluators are aiming to honour and represent within their evaluation reporting (Grover, 2010;Pipi et al., 2003). The same collaborative model expounded here for evaluation is more and more frequently being used by public-health practitioners to infiltrate public-health messages and practices into indigenous communities alongside, and in compatibility, with a community's own cultural norms and wisdom (Symlie, Kaplan-Myrth, & McShane, 2008). ...
... Respect underpins the building of trust relationships that, in turn, are the foundation for the reciprocal sharing of knowledge, expertise and gifts (Grover, 2010). A relationship ethic speaks to journeying together, learning from one another, being comfortable with knowing as well as with not knowing (e.g. when knowledge is out-of-bounds), and being accountable to one another (Ormond et al., 2006). ...
... Community members, viewed as experts, are typically involved in every stage of the process ( Jull et al., 2017 ). Using participatory approaches fosters community buy-in and ownership and successful intervention implementation ( Grover, 2010 ). Indigenous research methodologies embrace partnership approaches to bring together diverse perspectives, knowledge, and expertise, thereby maximizing resources, ef ciency, and the likelihood of genuinely benef ting the communities and honouring local knowledge ( Bainbridge et al., 2015 ;Kurtz, 2013 ). ...
This narrative review highlights evaluation approaches, principles, and frameworks for chronic disease interventions in North American Indig-enous contexts. It aims to inform the co-development of an evaluation frame-work for two studies focused on improving diabetes and obesity outcomes for urban Indigenous communities. This review uses a Two-Eyed Seeing perspective that brings Indigenous and Western ways of being, knowing, and doing together. There is a paucity of published evaluation frameworks inclusive of both per-spectives. The themes identified here suggest that evaluation approaches should address gender equity issues, be participatory, be grounded in local context, traditions, and knowledge, and be responsive to community-identified needs and solutions.
... Firestone et al. (2019) provides a case example of data collection methods that honored "stories and knowledge of those who participated" in the evaluation. Grover (2010) discusses how the IEF was used to evaluate the implementation and outcomes of a public health project, finding that funder requirements for quantitative methods challenged the need of the community to tell their own story using culturally relevant qualitative methods. An Indigenous evaluation approach has also been used in the evaluation of place-based educational programming. ...
This paper presents a case example of the Indigenous Evaluation Framework as applied to a science, technology, engineering, and mathematics (STEM) education pilot program. Indigenous methodologies include knowledge and data that are inclusive of historically marginalized groups, are highly meaningful, valid, and useful for all. A paradigm shift from Western evaluation methodologies to Indigenous evaluation is necessary when evaluating STEM programs that are committed to increasing recruitment, retention, and graduation of students from historically marginalized groups. This paper describes the use of the Indigenous Evaluation Framework during the first two years of the newly created Environmental Stewardship of Indigenous Lands program at the University of Colorado Denver. We discuss the importance of the Indigenous Evaluation Framework and how it informs the development and continued improvements to the program that also provides agency to program leads and participants.
... The team involved in communicating this work has come to realize it takes time and the building of relationships to uncover this knowledge. This is a collaborative space, which resonates with Indigenous epistemologies regarding interconnectedness and the importance of relationships (Corntassel 2012;Grover 2010;LaFrance, Nichols, and Kirkhart 2012;Morelli and Mataira 2010). It calls attention to the principles regarding how knowledge is shared, communicated, and acknowledged, which is perceived and respected differently in Indigenous and Western cultures (Cram 2016;Scott 2010;Smith 2006). ...
This article critically considers the work being done to bring Indigenous Peoples, Knowledges, and perspectives into the dominant structures of engineering education in Canada. We use Gaudry and Lorenz’s (2018. “Indigenization as Inclusion, Reconciliation, and Decolonization: Navigating the Different Visions for Indigenizing the Canadian Academy.” AlterNative: An International Journal of Indigenous PeoplesAlterNative 14 (3): 218–227. doi:10.1177/1177180118785382) spectrum of Indigenization to evaluate self-reported contributions from 25 engineering programs and four engineering organizations. Findings show much of the work being done in Canada is in Indigenous Inclusion and Reconciliation Indigenization, with some Decolonial Indigenization. Efforts in reconciliation and decolonization are seen predominantly in integrated, grassroots initiatives, with institutional initiatives found largely in inclusion. We submit that a diversified strategy and decolonized policies are needed to achieve Decolonial Indigenization. The intention of this work is to create an ethical space where Indigenous and non-Indigenous engineering educators can listen to and learn from one another. Guided by Etuaptmumk (Two-Eyed Seeing), we can advance Indigenous ways of knowing, being, and doing in engineering education in Canada and around the world.
... PAR aligns with Indigenous values and approaches to evaluation [14][15][16][17], aligned with "participatory and empowerment models that allow meaningful impact from the community" [18, p. 36]. It honours the epistemology of "collective interdependence" held by Māori people [19, p. 300] and reflective of many Indigenous cultures [14][ [16][17][18]. PAR enables the co-creation of meaning with participants "to bring about social justice and change" [20, p. 1394]. It is a suitable approach as it supports the participation of a community -in this case engineering educators -and makes space for the interaction of Indigenous and western worldviews. ...
The purpose of this Participatory Action Research (PAR) project is to share with the CEEA-ACEG membership the Indigenous initiatives being taken in CEAB accredited engineering programs across Canada. We received contributions from 24 institutions and 4 organizations, from which 11 categories of initiatives emerged. The intention is to create an ethical space where Indigenous and non-Indigenous engineering educators can learn from one another, and work together guided by Etuaptmumk (Two-Eyed Seeing), to advance Indigenous ways of knowing and being in engineering in Canada. This project is ongoing. Contact us if you wish to contribute and/or engage in the projects arising from this work.
... In general, "conversational and story-telling approaches," which are more consistent with the relational worldview that undergirds many Indigenous epistemologies, tend to be preferred (Ball and Janyst 2008:43;Christensen 2012;Drawson et al. 2017). By contrast, some social science methods of analysis, such as analyzing survey data, may be more difficult to implement if analysis requires formal training of research participants (e.g., Grover 2008). Indigenous communities may also have local norms or ethical principles that clash with Western social science principles and constrain sharing of Indigenous knowledge or stories (Simonds andChristopher 2013:2188-89). ...
Those engaged in community-based participatory research often comment on tensions between social scientific and community values, yet little systematic evidence exists about the relationship between social science research methodologies and community participation. We analyze nearly 500 peer-reviewed articles published between 2005 and 2015 on Indigenous issues in Canada, where policies encourage participatory research methods with disempowered groups. We find that research that includes Indigenous participation is more likely to include Indigenous epistemologies and participatory evidence sources and analysis methods. We also find that peer-reviewed research involving Indigenous participants often fails to go beyond minimum levels of consultation required by policies.
© 2020 Canadian Sociological Association/La Société canadienne de sociologie.
... During implementation, communities expressed concern with the evaluation approach. The NMTPP Leadership Council (LC) and community partners discussed existing culturally incongruent evaluation methods (program evaluation surveys, yearly community-wide surveys, outsider observations/rubrics) that echoed the wariness experienced by many Native communities when feeling "over-researched" with no sense of voice or relevance to the community when data are collected (Couzin-Frankel, 2010;Gray Grover, 2010;Walters et al., 2009). In response, the "Place of Strength" (PoS) evaluation process was born. ...
The Place of Strength (PoS) project represents an effort to Indigenize program evaluation with Indigenous communities by melding art with prevention science. We propose that Native artists as evaluators: (a) opens avenues of communication for Indigenous perspectives; (b) provides opportunities to capture spiritual, relational, and emotional impacts of prevention programming; and (c) maintains Indigenous processes, language, and values at the center of knowledge production. The New Mexico Tribal Prevention Project (NMTPP) funded seven Southwestern tribes to develop substance abuse prevention programs. In response to their expressed negative experiences with evaluation of prevention strategies, NMTPP piloted PoS. PoS shifted systematic knowledge paradigms to Tribal thought, values, and perspectives embodied in art. Art exists in Native communities as a way of documenting lifeways and historical experiences through various cultural forms. We share the process of collaborating with Native artists to document the impact of substance abuse prevention initiatives through their art within a community context. We offer concepts derived from this project as a community psychology model for re‐conceptualizing evaluation utilizing Indigenous knowledge. The Place of Strength project Indigenizes program evaluation with Native artists as evaluators. Art is an example of Indigenous knowledge production that can be applied to program evaluation. We share about a collaboration with Native artists to document substance abuse prevention efforts. Native artist‐evaluators enhanced community engagement, program fidelity, and program outcomes. Evaluation using Indigenous knowledge kept the process, outcomes and impact within the community.
Please Read Here: https://rdcu.be/bUs4d
... Working in more collaborative ways with communities is critical to achieving the goals of inclusiveness, respect, and self-determination (Grover, 2010). In health, for example, Community-Based Participatory Research is demonstrating collaborative ways of designing, implementing, and evaluating initiatives to reduce indigenous health disparities (Wallerstein & Duran, 2006). ...
... The AI/AN literature does not suggest that research and evaluation have typically been conducted through successful partnerships between the community and the researchers and evaluators (Grover, 2010;LaFrance, 2004). Rather, past research and evaluation efforts in AI/AN communities have been met with mistrust and skepticism due to a history of unethical research practices and the imposition of Western research on AI/AN communities (Pacheco et al., 2013). ...
The goal of this current descriptive study was to examine the roles and relationships of evaluators with the tribal communities in which they work. First, we describe a participatory community research model with a strong capacity‐building component as the standard for assessing successful working partnerships between evaluators, programs, tribes, and tribal organizations. This model serves as a yardstick against which we examine the success and challenges of program–evaluation partnerships. Second, we report on a survey of tribal Maternal, Infant, and Early Childhood Home Visiting program leaders and outline their impressions of successes and challenges related to program–evaluation partnerships. Survey participants discussed the importance of working with evaluators who have deep investment in and understanding of the tribal community; respect for cultural relevance and honor for cultural ways; collaboration that includes transparency, trust, and translation of research for community leaders and members; a focus on strength‐based design without losing the need to consider challenges; and relationships of mutual trust that can weather addressing stressors when issues of conflict, limited resources, and/or mixed expectations arise.
... This question points to the need for balancing government requirements for accountability with First Nations' needs that are grounded in their own cultures and epistemologies. Grover (2008) details this issue specifically with respect to the use of culturally valid measures by outlining the need "to balance the state's desire for pre-and post-survey data with measures that will be more credible to the community" (p.47). Fundamentally, this scenario points to the need for viewing evaluation within First Nations education contexts as an inter-cultural process. ...
In our evaluation practice we face the problem of grasping a phenomenon that seems to be so
emergent and fluctuating that it is difficult to keep track of all the changes taking place. This
paper focuses on the theoretical and practical implications of complex interventions on the
practice of realistic evaluation. The challenge of evaluating welfare service reforms is to
identify the overlapping, paradoxical processes and interactions of several organizations and
actors. We have turned our attention to complexity sciences to find new tools to describe the
emergent phenomena. When we adopt a complexity theoretical evaluation frame it has
implications both for the practice of evaluation and for the use of evaluation results. We claim
that in order to succeed evaluators must address the issues of how traditional program
evaluation methods fit the context of complex reforms and what kind of knowledge or tools
are needed in these situations.
Keywords: evaluation, theory, welfare services, complexity, context
Aligning with the Western Door—Do Good Work, this article outlines Urban Indian Health Institute’s (UIHI) Indigenous Evaluation Framework, created to explicitly include and empower urban Indigenous communities to reclaim their narratives by using evaluation as a tool to tell their stories and to build capacity to take ownership of research and evaluation. The framework includes the following core values: Urban Indigenous People Create Communities Wherever They Are, Resilient and Strength-Based, Decolonize Data, and Community Centered. The authors provide an overview of how they applied the framework in collaboration with 18 urban Indian organizations through the UIHI’s community grants program and include a first-hand example of implementation of the framework from the Native American Youth and Family Center, a community grantee. The authors highlight the importance of including urban Indigenous people in evaluation contexts, as evaluation is not just an exercise in methods or logistics but also a political act and an assertion of Indigenous values and sovereignty, one that defines who is counted, how people are counted, and what decisions are made. The UIHI’s Indigenous Evaluation Framework aims to decolonize data to reclaim urban Indigenous narratives from colonial understandings and tell the stories of our communities.
This article focuses on the description, discussion, and problematization of the ways indigenous spirituality is presented in a Brazil’s hospital, in the context of the treatment of chronic illnesses in children under medical supervision. It describes a short extract from a more wide-ranging study, focusing on the analysis of excerpts of the verbal responses of health professionals, and of two specific cases. Adopting a phenomenological approach, the vicissitudes of the hospitalization of indigenous children are described from the perspectives of the professionals, with an emphasis on the outcomes of the reported cases, which are riddled with impasses and challenges, with important implications for the country’s health policy and for the training of health professionals. Grounded on the cosmological explications for the illness, the indigenous child’s family members tend to resort to pajelança, (indigenous traditional healing rituals), even within the hospital environment, with or without the consent of the medical team. This process does not always end harmoniously or in an integrated fashion, evidencing the rifts in the interaction between indigenous curing practices, underpinned by thousands of years of tradition, and medical practices grounded on formal, rational, and scientific understanding. We conclude by indicating perspectives for professional training and the respective ethical considerations.
Báa nnilah is a chronic illness self-management program designed by and for the Apsáalooke (Crow) community. Arising from a collaboration between an Indigenous nonprofit organization and a university-based research team, Báa nnilah's development, implementation, and evaluation have been influenced by both Indigenous and Western research paradigms (WRPs). Báa nnilah was evaluated using a randomized wait-list control group design. In a WRP, contamination, or intervention information shared by the intervention group with the control group, is actively discouraged as it makes ascertaining causality difficult, if not impossible. This approach is not consonant with Apsáalooke cultural values that include the encouragement of sharing helpful information with others, supporting an Indigenous research paradigm's (IRP) goal of benefiting the community. The purpose of this paper is to address contamination and sharing as an area of tension between WRP and IRP. We describe how the concepts of contamination and sharing within Báa nnilah's implementation and evaluation are interpreted differently when viewed from these contrasting paradigms, and set forth a call for greater exploration of Indigenous research approaches for developing, implementing, and evaluating intervention programs in Indigenous communities. (Improving Chronic Illness Management with the Apsáalooke Nation: The Báa nnilah Project.: NCT03036189), ClinicalTrials. gov: NCT03036189).
Highlights
• Within an Indigenous research paradigm, the purpose of research is to improve people's lives and their community and must be relational.
• When one contextualizes Western research notions of contamination within an Indigenous research paradigm, sharing helpful information with others takes on a different meaning.
• We identify a number of ways Indigenous voices and research methods can be amplified within CBPR.
Despite the importance of evaluation to successful programming, a lack of physical activity program (PAP) evaluation for American Indian/Alaska Native (AI/AN) programs exists, which is significant given the high rates of obesity and diabetes in this population. While evaluation barriers have been identified broadly among AI/AN programs, challenges specific to physical activity (PA) programming are unexplored. To address this gap, a research study was conducted involving 17 in-depth interviews with evaluation staff of externally funded AI/AN PAPs. A thematic analysis revealed factors hindering meaningful program evaluation: cultural characteristics that challenge traditional evaluation, the logistics of PA programming, measurement and data collection challenges, lack of resources and support, and lack of alignment between funding agency requirements and the evaluation desired by AI/AN organizations. Some challenges are general to PAPs and others specific to AI/AN organizations. Findings identify ways to improve culturally responsive evaluation for AI/AN PAPs.
Our team has worked closely with First Nations, Métis, and Inuit (FNMI) community partners and our local school board over the past 9 years to develop a range of strengths-based health promotion programs for FNMI youth. This article begins with a brief description of our school-based programming to provide context. Next, we identify challenges in conducting rigorous program evaluation and highlight the requirements of community-based research partnerships. Finally, we identify a number of factors that have helped us achieve a balance between the frameworks of rigour and community-based research partnerships. Throughout the paper we use examples from our projects to illustrate issues.
The objective was to assist with the prevention of Fetal Alcohol Spectrum Disorder through a participatory research approach involving local women and health care workers. Our interest was in understanding how well communities could develop culturally appropriate methods of helping women to reduce their alcohol consumption during pregnancy.
Four geographically distant Aboriginal communities were presented with a task of adapting a standard Brief Alcohol Intervention, and in particular to develop a culturally appropriate means of using the Intervention with women in the community.
Academic and Aboriginal community researchers worked together in partnership, using a participatory action research approach to address alcohol use during pregnancy.
The outcome of the project was the design of four differing models of culturally appropriate community interventions designed to support vulnerable women in their childbearing years. The intervention models developed by the four communities have five core characteristics: (1) identification of women at risk; (2) assessment of the woman's drinking/ drug use; (3) provision of information to the women; (4) delivery method facilitates the decision to adopt healthier behaviors; and (5) means to monitor changes.
This project was considered successful in many respects: (1) each community developed a culturally-relevant prevention tool; (2) community involvement in the design lead to better understanding of its usefulness; (3) considerable knowledge exchange between academic and community partners took place; and (4) most importantly, it was found that community members can be active members in developing and implementing solutions to important public health issues.
Culturally competent evaluation in Indian Country requires an understanding of the rich diversity of tribal peoples and the importance of self-determination and sovereignty. If an evaluation can be embedded within an indigenous framework, it is more responsive to tribal ethics and values. An indigenous orientation to evaluation suggests methodological approaches, a partnership between the evaluator and the program, and reciprocity.