“The importance of organisational measurement cannot be overstated in the world of CSI, in which millions of rands are spent annually on social development programmes that yield little systemic change or sustained impact.” This, according to Jennifer Bisgard of Khulisa Management Services, in her article on demystifying monitoring and evaluation (M&E), featured in the 20th edition of Trialogue’s Business in Society Handbook.
A breakout session at The Trialogue Business in Society Conference 2018 explored the role that technology could play in improving the collection and interpretation of data, as well as the policies that influence the use of technology in the development sector.
Collecting quality data and creating feedback loops
Bisgard emphasised the importance of using practical, user-friendly and non-intrusive technology – a sentiment expanded by Alyna Wyatt of Genesis Analytics, who spoke about the importance of ensuring that the technology solutions used don’t supersede the delivery of the social service.
Bisgard also underscored the need for the collection of high quality and reliable data with the potential to influence policy change, as well as the importance of reporting back to the communities that the information is collected from.
Citing the effective convergence of these key elements, Bisgard shared an example of work done with the Coca-Cola Africa Foundation who wanted to know about the quality of water in specific areas. “They started off wanting to know everything about communities, but we ended up with only three data points.” Delivery staff were given smartphones and asked to take photos of the wells, test the pH levels and quality of the water, as well as the infrastructural state of the wells in the villages that they went to. The data was uploaded onto Google Maps, results were fed back to the communities and action could then be taken.
In another case, Khulisa Management Services used USSD technology to evaluate a tuberculosis intervention programme by USAID, in partnerships with several local non-profit organisations (NPO), through a simple questionnaire sent to participants’ cell phones. Bisgard attributes the 40% participation rate to the use of familiar technology, language usage options and, most importantly, the lack of cost to participate. This technology-based activity was far less expensive than sending fieldworkers to various regions across the country.
Regulations and the implementation of privacy and security
Wyatt referenced the Principles of Digital Development – a set of guidelines for using technology and digital tools for development, three of which are specific to data: ensuring a feedback loop, privacy and security, and being collaborative and sharing what is collected. “There are about 100 organisations globally that have endorsed these principles and they are trying to mitigate some of the risks that come with using some of these digital platforms,” said Wyatt.
According to Wyatt, there are a variety of ways to implement privacy and security in the field. For example, if an NPO is conducting research and liaising directly with beneficiaries, the privacy settings must be based on the nature of that NPO’s engagement with the participants. However, when publicly accessible platforms such as social media are used, people have already made some of their information available by engaging online.
“The organisations that we work with have underlying principles around ethics and morals as they pertain to their beneficiaries. For example, I asked a large international organisation that we work with about their digital privacy and security policy and they actually said that their Child Protection Policy superseded their Data Policy because it has much higher security limitations.
Given the explosive data breaches that have plagued digital platforms this year, Wyatt also made special mention of the EU General Data Protection Regulation which will affect governments, organisations and partners working with funders from Europe.
Leveraging and improving existing systems of data collection
Ayanda Mtanyana of New Leaders Foundation spoke about the Data Driven District Programme (DDD) which collects various data points from approximately 9.5 million learners in 20 000 schools across seven provinces, except for the Northern and Western Cape. Mtanyana explained that DDD is based on an existing system developed by the Department of Basic Education. The South African School and Administration Management System (SA-SAMS), which is freely available to schools, captures basic administrative data, such as teacher performance, the financial management of schools and learner performance and attendance. However, SA-SAMS is an offline solution, significantly compromising the accessibility and accuracy of the information.
The broad range of challenges in South African schools can be overwhelming and an excess of data can often make it more difficult to know how to respond. This collaboration between New Leaders Foundation, their funder and the Department of Basic Education is focused on improving the quality of data in order to generate targeted insights for the districts and schools that use the DDD.
Useful resources referenced in the discussion:
The Open Data Kit is a free open source app that helps organisations manage data collection using mobile devices. The app only sends data out when there is network coverage or WIFI in range. The advantage of the system is that it allows for meticulous monitoring of the data, so that researchers can assess whether the questions need to be adjusted.
Hosted at the UN Foundation, Digital Impact Alliance advances an inclusive digital economy for the underserved in emerging markets, advocating for a shared space where technology can be rated and evaluated.
MERL Tech, which is hosting a summit in South Africa in August 2018, brings together M&E practitioners and software developers to discuss how best to develop tools for development.
Written by Khumo Ntoane, edited by Zyaan Davids
Fltr: Alyna Wyatt, Genesis Analytics; Jennifer Bisgard, Khulisa Management Services; Cathy Duff, Trialogue; Ayanda Mtanyana, New Leaders Foundation
Image © Brett Eloff