How We Measure What Matters: A Framework For Measuring Job Quality
When it comes to job quality, how can we best measure what matters?
As a mission-driven fund, Upaya Social Ventures aims to create dignified jobs for the poorest of the poor by supporting early-stage entrepreneurs anchored in marginalized communities. Over the last 12 years, we have captured the experiences of jobholders and their communities, and used the constant learnings from this field data to sharpen our investment strategy and approach to impact metrics. To date, our 31 investee companies have cumulatively created more than 27,000 jobs. Nearly 2,400 surveys of jobholders of investee companies have been completed to date, with results (through this dashboard) providing real-time, real-world insights on the jobs created for the extremely poor and how these jobs have or have not impacted the lives of the jobholders.
This year, we used these many years of knowledge and data to create an impact measurement framework to help us report effectively on the number and quality of jobs created through our investments.
The framework also helps establish a common understanding across teams on the fund’s impact measurement approach. Our ambition with this framework is not just to strengthen our own impact performance but to generate insights and learnings for the ecosystem at large on the factors affecting job quality for the extremely poor. Impact investors must demand the creation of quality jobs for the poor and not just count the number of jobs.
It’s worth noting that this is not an entirely new framework. Rather, we have built on Upaya’s well-acknowledged practices of jobs measurement and leveraged knowledge assets created by industry leaders, like FSD Africa’s jobs measurement methodology. We also adopted impact scoring tools, such as the sector-geography placement of investment by the British International Investment, with The operating principles for impact management along with IMP’s 5 dimensions of impact and GIIN’s IRIS+ thematic taxonomy providing the guiding structure. The framework is flexible and open to review and upgradation annually.
The figure below illustrates Upaya’s impact measurement framework. The guiding principles for Upaya are to be transparent, consistent, conservative, and insightful in our impact reporting. We recognise that it is difficult to establish investor contribution in jobs created—hence, our guiding principle is being conservative in what we can claim as an absolute impact of investments made by us in the enterprises. (Watch this space for more on the additionality data we capture as well.)
Breaking Down the Framework
The four pillars of our impact measurement framework — inquire, quantify, validate, and synthesize — are divided across two parts: pre-investment and post-investment.
Pre-investment, we inquire about the impact potential of the pipeline company. At this stage, we use our pipeline assessment tool to capture who the jobholders are, their current level of vulnerability due to living in poverty, and whether the effects of jobs created by the enterprise are material enough to proceed with investment. A structured inquiry into the impact potential of a company at the pipeline stage helps the investment team in aligning impact and business projections with the founders, which are then captured in the shareholder agreements.
One example of this process is our recent investment in Bharat Krushi Seva, an early stage startup that provides full-stack decision support services to farmers through a data driven app-based platform and provides market linkages for their produce. Our team made a systematic enquiry into how the company is targeting and including small and marginal farmers for app usage. We looked at current transaction values from small farmers to BKS and examined the potential growth in that number in future years. Observations from due diligence exercises and field visits were captured through a score sheet that measures a pipeline company’s impact potential; the team also did a sector-geography placement of the investment. This helped us understand the impact potential of the partner based on their alignment with our mission: to support those living in poverty to create a pathway out.
In the post-investment phase, we want to fully understand the impact and also explore ways to deepen it. We first quantify the impact by recording a tally of jobs created, wages paid, and income increase experienced by the jobholders. The source of this data is mostly from the entrepreneurs, who self-report through templates provided. We are now attempting to collect job number data disaggregated by gender through these self reporting templates. Income increase and wage effect on lives of the poor is further verified through on-field surveys of jobholders.
Through these primary surveys of the jobholders, we attempt to validate the quality of the jobs created. The intention is to understand the wage effects on the lives of the jobholders and to gain insights into how that can be improved. So far, we have used quantitative survey tools to validate the quality of impact. However, we are now testing a mixed-methodology approach by including a qualitative approach to capture important aspects like the jobholder's self-perception and dignity at work.
For instance, we began working with the founders of our partner company, Green Worms Waste Management, post-investment to capture jobholder data. We conducted on-field surveys to learn more about the effects of income from Green Worms among the jobholders. The report was found useful by the founders to get insights on how to keep their jobholders motivated, identify pressing issues, address attrition, and design responsive strategies for different geographies based on data.
The fourth pillar is to synthesise our learnings from individual companies and specific sectors. Deep dives into learnings from key sectors of our investments — like agriculture, waste management, and creative businesses (like artisans) — help us finally understand what impact we are creating at the portfolio level. These insights are key in discussing and designing strategies to deepen our partner companies’ impact. More importantly, this information helps us understand the support our partner companies require to achieve their desired impact.
For example, impact data from all companies in our portfolio that are working in waste management provides some useful trends and patterns that can be leveraged by the sector as a whole — like the fact that on average, jobholders in our waste management companies have experienced an income increase of more than 100% over the last three years. During the same period, however, skill development opportunities were not reported to be too high. As the waste management industry transitions into circularity, this could be an important insight to create skilling opportunities for the jobholders to help them increase their income levels. Data from our agriculture portfolio shows that jobholders reported better job stability in terms of predictability of income and learning new skills at work, than those in textiles and waste management. This mostly reflected the shift from traditional agricultural practices to newer, more sustainable and climate-friendly ones.
Developing this framework was an intense exercise. We collaboratively accepted a framework for impact measurement and management, stress-tested it across teams and portfolio companies, and started developing operating processes around it.
And yet, upon reflection it’s clear that we have only scratched the surface when it comes to proving — and improving — the quality and dignity of jobs created for the extremely poor. Going further and deeper requires collaboration. As we move to operationalise this framework, we invite thoughts and comments from impact investors and hope to continue sharing our learnings. We can all learn from each other.
MEDIA CONTACT:
Madlin D’silva
mdsilva@upayasv.org