This year’s Canadian Evaluation Society conference centred around the theme of co-creation. Co-creation is fundamentally about producing mutually beneficial outcomes together. Throughout the three packed days in Calgary, we explored new approaches to evaluation, with a focus on Indigenous, collaborative, culturally-responsive, and sustainability-ready approaches. Further, we were challenged to reflect on our privilege, on what we were creating and whose benefiting, and on whose leading and whose following. Being challenged often leads to uncomfortable spaces, but as someone at the conference said: “Ask tough questions. Through discomfort, we will create change”. Below are some of the questions asked, and discussions captured:
What are key principles for working with Indigenous communities?
- Respect for the knowledge, experience, and cultural background each knowledge holder brings
- Listen to hear and learn, and understand not to expropriate
- Be aware of your own biases and understand history with Indigenous Peoples
- Transparency of agenda
- Collaborative partnerships
- Appropriate remuneration
- Ongoing affirmation and validation, while minimizing assumptions
- Communicate to be understood rather than heard
Why Indigenous-led evaluation?
Indigenous-led evaluation offers cultural knowledge which facilitates respectful and appropriate engagement. Further, it offers cultural capital, specifically Indigenous ways of thinking. Indigenous ways of thinking are unique, subjective, unavailable and holistic, while Western ways of thinking are defensible, repeatable, objective, measurable and singular. Finally, cultural validity, ensuring evaluative conclusions are valid to the communities that you work with. Supporting such evaluations through meaningful partnerships (and not tokenistic inclusion) leads to high level of credibility to communities, buy-in to the evaluation process, and affirms community aspirations for transformation.
What does Indigenous-led evaluation look like?
– A strategy to support the development of Indigenous evaluators and Indigenous evaluations
– Partnering with Indigenous people to guide evaluation
– Growing the cultural competencies of non-Indigenous evaluators
– Positioning Indigenous values and principles as foundational to evaluation practice
– Personal evaluation paradigm shift
What does an evaluation paradigm shift look like?
| Evaluator | Indigenous |
| – Destination – Evaluator as expert – Evaluator in control – Independent measures – Silo judgements – Independent – Evaluation standards – Program / contract duration |
– Journey – Indigenous Peoples as experts – Indigenous Peoples in control – Culturally-based measures – Holistic judgements – Relational – Cultural values and principles – Multi-generational timeframes |
Towards culturally-responsive evaluations
What are some steps that evaluators can take? Be respectful of different ways of research. Understand the importance of being welcomed by a mutual exchange – perhaps by asking yourself, what do you bring? This may include knowledge, experiences, and histories. Finally, recognize that a lot of tools that we use are very quantitative and do not capture cultural factors. We need to push people to ask high-level cultural questions.
Towards sustainability-ready evaluations
Sustainability is about how to take from the current world without impairing future generations. How do we shift towards a sustainability-focused approach? First, when we talk about coupled natural/social systems, we need to understand how they work before we start to address them, as to not re-invent the wheel. Secondly, we need to understand the motivations of people on the ground and the larger economic and political forces that drive change. Thirdly, every project will have an impact whether intended or not – it is imperative to pay attention to the impact on natural systems. Finally, strive for question-driven methods and not methods-driven questions.
In the same way that culturally-appropriate evaluation is not just limited to just evaluations with diverse cultures, and sustainability is not applied to just the environment, it is ultimately about our relationship with Mother Earth.
“The environment is the foundational later. If it collapses – everything collapses.”
How can evaluation contribute to change?
In responding to Request for Proposals, do what you need to get your foot in the door, then push for what more can be accomplished. Perhaps include a values-based discussion in the proposal, such as meeting the needs of communities without compromising the environment. Borrow a social innovation mindset – ask not what do we want to measure, ask what do we want to change?
Other ways:
- Paint a broader evaluation canvas. These means include valuing of human and natural systems, cultural capital, interconnectedness, openness, reciprocity, and safe spaces.
- Have a greater tolerance for risk. We hold ourselves back in the name of good research methods or to serve our clients but it stifles our creativity and hence our innovative capacity.
- More developmental evaluation approaches. Recognize that co-creation Is only one of the 8 developmental evaluation characteristics. Others include developmental purpose & context, systems theory, a complexity perspective, utilization focus, etc.
Final takeaway – go beyond the myriad of methods and measurements, to a focus on positionality, privilege, and an understanding of the need for change and how evaluators can facilitate.
What were your takeaways from the conference? Do you agree that this is the direction that evaluation should move towards?
2 responses to “Towards Indigenous-led and culturally-responsive evaluations”
Great read! Thank you for the post Steven.
I could not agree more with going “beyond the myriad of methods and measurements.” I would also add the importance of having a person in the evaluation/research team with an indigenous background or has a affiliation with the community which will ensure the community has some awareness and locus of control on the narrative of the study.
Hi Ten!
Great point, and perhaps arguably the most important. As for non-Indigenous evaluators, there are steps that they can take be better support Indigenous evaluations (e.g. learning the history, building genuine relationships, building capacity). I find that the recent issue in New Directions for Evaluation sums up these steps eloquently.