Around the same time last year, I was struggling to find my place in the world of evaluation. Even though I had been taught evaluation theory and practice in graduate school, and gained relevant work experience outside of academia, I still did not feel comfortable considering myself a “program evaluator”. Evaluations, particularly of complex developmental programs (where my interests lie), are not straightforward. To effectively design a system that can help prove and improve outcomes, the evaluator should have evaluation expertise and a substantial repertoire of methods to call upon… did I really have these attributes? Hmm, I think not initially, but maybe eventually.
“Evaluation is the systematic collection of data that will allow a judgement to be made about the value of a program or intervention, allow reflection about what is happening and provide an assessment of whether you have achieved what you set out to do” (WHO, 2017).
I was motivated to learn everything and anything about evaluation because its notion resonated with me. Shouldn’t all of us try to figure out if our efforts are working, and if so, to what extent? And if not, how can we improve our efforts? I found that evaluation doesn’t just apply to programs, but also to aspects of our everyday lives – our job(s), our relationships, our happiness. We can benefit from assessing ourselves now and look backward to see how things compared, as well as look forward to seeing where we may end up. In a way, everyone can learn from evaluation principles.
During this phase, I ordered way too many books off Amazon.ca, from research methods to evaluation tools. Perhaps embarrassingly, I read all of these books from cover to cover. I am not sure I fully recommend these books yet, but I want to highlight some thoughts (I will do a more in-depth review of these books later).
I found Patton’s books to be helpful and easy to read. If you are having doubts about your expertise as an evaluator, I would recommend Developmental Evaluation or Utilization-Focused Evaluation. It’s an opportunity to see through the lens of a seasoned evaluator, who, at one a point in time, felt “the more I know, the more unsure I am”. Some of my favourite quotes are “evaluation is a learning-by-doing process”, “there is no one best way to do an evaluation”, and “[evaluation] first and foremost is about doing what makes sense”. Although these quotes may read as vague to you, it really puts evaluation into perspective that it is accommodating and can vary widely. But, Patton’s books are not all intuitive – you should have some understanding of evaluation concepts and theories before attempting to read them.
Evaluations may seem scary, but by realizing that others have experienced the same sentiments, make it less so. In fact, this feeling, sometimes called “imposter syndrome”, happens, to a certain extent, in other fields as well. It tends to happen when we enter a new field or achieve some initial progress in a field.
The point is, if you’re like me, you might have a slow, scary, beginning. But, there will come a time when you’ll just start to get it. When you’ll feel like you are no longer wasting time creating program logic models and performance measurement frameworks. And suddenly you will have an evaluation plan that will bear the mark of someone who knows what he’s doing.
Along the way though, it helps to work with more experienced people that have tackled the same problems you’ll soon face. In the process, you can build up your portfolio while learning from them. Note, however, that many evaluators don’t seem to get it right either, even those that have been in the field for decades. So stop doubting yourself. You don’t need to know everything. Just need to know a bit more than your stakeholders. Have the courage to evaluate and learn as you go.
Photo credit: Foter.com