I’ve been enjoying inProgress’ new manual, “Integrated Monitoring: A Practical Manual for Organisations That Want to Achieve Results,” on the metro over the last couple of days. (Thanks to the recommendation from @txtpablo.) The blog, Development That Works, recently discussed the same issue in its article, The M in M&E: Ugly Duckling or Swan?.
As an M&E and organizational learning specialist, it reinforces a message I’m constantly sharing with partners and colleagues: “If you don’t have any sound monitoring data throughout your project, your evaluation is gonna stink anyway.” (Of course delivered with more finesse, credibility, and tact than that – ha!)
Even if you don’t learn anything strikingly new from the inProgress manual, its use is in emphasizing the crucial, yet often glossed-over role of monitoring in aid projects. With what, why, and how explained together, it’s well worth the read.
***
Aside from the manual’s overall purpose, it offers some sound thinking and other gems of wisdom from the author Sonia Herrero that were music to my ears, which I gladly share below:
If the people we want to help are at the centre of everything we do, our motivation for monitoring the project and learning from it is quite a different one.
The need to satisfy donors can create a dynamic in which NGOs hide the problems they encounter in their projects because they are afraid of losing funding if they talk openly about challenges or failures. As a consequence, some NGOs monitor selectively in order to show only the positive aspects of their work. The actual benefit of activities for the communities receiving support is often a secondary concern at best.
…in real life, things don’t happen in such a simple, linear way. We can indeed bring horses to the water tank because, if required, we can force them there, but we cannot make them drink. It is the horses’ own decision and not subject to our control. Now, think of all the training courses, meetings, workshops, and conferences that are organised by the NGOS, all the reports that are being produced, latrines and schools built, and mosquito nets distributed. All of these things are outputs—in this metaphor, about bringing the horse to water.
If we are be successful with our projects, we need to be more modest and much more concrete with the formulation of results. This would help us to move away from the ‘superhero syndrome’ that a lot of non-profits (and donors, too) seem to suffer from. The syndrome manifests itself in an inexplicable optimism that, despite limited time and resources, organizations will somehow manage to solve an impressive number of complex and long-standing problems.
The question ‘What are the right indicators?’ dominates many discussions within the non-profit sector, to an extent that can seem obsessive. People can come down with cases of ‘indicatoritis,’ a sickness that causes one to see things as more complicated than they actually are in an attempt to measure everything.
Many baseline studies suffer from information overload and fail to put all the information collected to good use.
We recommend not to jump into implementation and data collection without first having a clear idea of what information you need, for whom, and why.
Good questions are key to unlocking good learning. Our ability to ask good questions can produce high levels of thinking and engagement…while badly formulated questions are more likely to mislead our thinking and discussions.
Many projects collect tons of “ore” that contains “golden information” about what works, what does not, and why. Unfortunately, like real gold ore, this data will not do you any good if you don’t refine it to produce concentrated nuggets of actionable information.
Learning requires an organization to have a commitment to figuring out how to do its work better and to benefit from its mistakes…Learning not only involves dealing with the situation at hand, but also fundamentally changing the way in which the organization functions, so as to be able to deal with similar situations in the future.
So how can you help people accept changes [that become necessary from monitoring progress]? Guess you’ll have to read inProgress’ “Integrated Monitoring: A Practical Manual for Organisations That Want to Achieve Results” to find out.
***
Related Posts
Accountability in all the wrong places
Oral Reporting With Grassroots Organizations—Here’s How!
Great article. Mining that “gold” requires better tools, in my opinion – something I’m looking for funding to do for the sector.
Also – I think the “M” and the “E” should be separate. One M — many E’s works best. You want consolidation on the M and diversity on the E.
Thanks Marc and I should mention that I also highly recommend the guide that you have developed called “The Realbook: A Manual for Story-based M&E.” Check it out on Marc’s website: http://chewychunks.wordpress.com/2012/05/23/about-the-realbook-manual-for-story-based-monitoring-evaluation/
i just really like the illustrations in InProgress. i was looking to make a presentation/workshop more appealing and memorable to visual learners, and was able to take many clues from the pictures here.
i agree that organisations often have information overload. i think what would be helpful for many small organisations is some pointers or direction on how to compile and analyse the data that they do have, especially qualitative data. what do you do once you’ve interviewed project participants? after you’ve held a focus group? with your direct observations from throughout the project? i think a lot of organisations get to this point, and then aren’t sure what to do next. how do you get from all that information to writing The Report and pulling lessons.
or, in the language of the metaphor in InProgress, once you’ve got the mining part down, how do you refine the nuggets once you’ve dug them out of the earth? (in a way that works for an organisation with limited resources, limited education, no flipcharts to spare, etc…)
all monitoring manuals seem to say “you should do it,” without really saying “here’s how you could do it,” or at least “here are some ways you could start.”