Held in Johannesburg (12 August) and Cape Town (14 August)
Nearly all companies (95%) conduct some monitoring and evaluation, but most track activities, rather than evaluating impact. It is clear that even basic monitoring activities are not rigorously applied. For example, 80% of companies claim to monitor the number of beneficiaries reached by their programmes, yet only 42% of these could provide an exact number of individuals reached.
Trialogue presented its research results and described two approaches to M&E – the logic model and Social Return on Investment (SROI), outlining the pros and cons of each. The discussion that followed explored how corporates assess the impact of their CSI projects and programmes.
A summary of the discussion follows.
- Evaluation studies are vital in shaping the evolution of a programme. For example, a forum participant shared her experience of reviewing a long-standing programme and using the findings to shape the programme into a relevant tool for the Department of Education. The programme received a complete makeover and has shifted from a pure feel-good programme, to one with tangible developmental aspects.
- The logic model can be used to make a business case to present to the board prior to initiating a programme. By articulating a complex issue simply, it can make identifying the strategic business benefits easier.
- M&E can be used to substantiate intuitions you may have about a programme. This can support changes required and decisions that need to be made. Furthermore, when difficult decisions are required, having the knowledge from the M&E process can leave one with a clear conscience.
How to measure impact?
- The value of a baseline study cannot be overstated. One of the participants in Cape Town whose CSI has a strong focus on community development found that the discoveries that arose from their baseline studies were invaluable in correcting preconceptions they had about a community they were moving into.
- Performing a baseline study enables one to measure what would have happened if the company had not supported the project. This is especially true for small and unknown projects.
- There is no ‘one-size fits all’ and it is about finding the M&E model that works for the project. Some projects are easier to evaluate than others due to variables such as the 2 reliability of key stakeholders to monitor and provide the right information, or the general tangible nature of outcomes of specific projects. Trying a number of different approaches to M&E may be necessary to find the one that works best for the project.
- Stakeholder engagement, especially with implementing partners, is required upfront to agree on shared objectives and indicators to assess impact.
- Companies should establish how to help their project partners report on impact by helping them understand M&E and how to tell a story that has value to the business. Engagement with partners and peers can help companies establish a link between the M&E models on paper and what happens on the ground.
- Internal stakeholders in the company should look at what they can bring to the table to add value to the beneficiary organisations. This can be established when project funders visit projects.
- Companies should invest time in the project and conduct site visits in order to really understand the project and its impact.
- Financial monitoring is important to ensure that funds are not misused or diverted.
- There should be a balance between business benefit and developmental impact. When a company does transformation and CSI purely for the sake of BEE points then it is not worth doing it.
- Projects should be categorised according to anticipated impact. For example, if the project is an awareness project, the impact is based on the hope that beneficiaries will master the skills they have been taught based on the number of sessions they attended. The expectations of executives need to be managed so they know upfront the expected impact and don’t question the impact later on the process.
- Lessons learnt should be shared. The Zenex Foundation, for example, has made M&E a science and does it from the start of a project and shares results with peers which can aid collaboration.
- Consider collaboration with other companies who are working on the same project or in the same community. Collaboration helps eliminate duplication and allows for resources to be pulled which can increase overall impact. Partners can then collaborate on M&E. Some examples of collaboration shared in the forum include:
- A number of corporates are working together to establish who is doing what in Diepsloot, Gauteng and to agree each partner’s responsibility (i.e. one corporate responsible for feeding scheme, the other for sports development).
- Tsogo Sun is partnering with the Barclays Africa Group (Absa) where Tsogo Sun provides access to youth who need financial training and Absa provides the financial training.
- Absa partners with GSK in Zambia where they combine their skills and expertise to help eliminate financial barriers to healthcare access, while supporting small business development and job creation.
- Bridge provides a collaboration link in the education sector
- LINC is a networking platform in the children’s sector
What are some of the key challenges associated with M&E?
- Company CSI strategies can change. For example, a company may support education for two years and then change to enterprise development for the next three years which makes it difficult to track impact over a substantial period of time.
- Integrated reporting requirements can lead business executives to pose some questions around impact that are difficult for development practitioners to answer e.g. executives need to know the impact made in terms of jobs created, bursary recipients employed etc.
- Financial challenges can arise when M&E has not been included as a line item in the budget upfront, especially when the project has been in existence and the costs of implementation increase towards the end of the project.
- Impact can be difficult to ascertain when different cultures come into play as what is meaningful to one culture may not have the same value to another culture.
- There is a challenge around measuring real impact and outcomes. Companies find it easier to report on inputs and outputs. When it comes to large NGOs, they are often able to demonstrate the financial aspects of the outcomes but they may not reach the intended beneficiaries. There is need for companies to develop the skills where they can show the impact made by the funding provided.
- A big challenge lies with executives in the donor companies as they often don’t realise that not funding project overheads affects the intended results of the project as a whole.
- Challenges are experienced around measuring short-term projects. Although one can measure the outputs and even the outcomes of a one-year funded project, it isn’t possible to measure the impact.
Who should pay for M&E?
- The general consensus from the two forums was that the funder should pay for M&E and make it one of the project activities in their project plan. The budget should include M&E upfront.
- M&E is dependent on the duration of the project. For short-term projects, monitoring alone is sufficient. When supporting a project for a longer period however, it is worth motivating for an M&E budget. Through the M&E process one is able identify key issues that arise during the process and adapt the project based on the findings
- M&E should be built this into the partnership as a condition of funding from the outset and the requirement to undertake M&E should form part of the partnership agreement. If the development partner doesn’t want to share certain critical information with the funder, it can seriously impair the ability to understand the performance and impact the project is having.
- To avoid making M&E requirements a daunting task for NGOs when partnering on the same project, funders need to collaborate and come up with standardised M&E frameworks.