Social Impact Assessment Reporting — the ground floor

What is Social Impact Assessment?

The workbook's core definition: Social Impact Assessment (SIA) is a process of research, planning and the management of social change or consequences — positive and negative, intended and unintended — arising from policies, plans, developments and projects.

Put simply, SIA looks at every consequence of a social intervention:

• direct and indirect
• intended and unintended
• positive and negative

It lets an organisation enhance the positive impacts and contain the adverse ones. It helps decision makers see the benefits gained against the investment made, and it helps reduce costs.

A naming note: the workbook uses Social Impact Assessment, Social Impact Analysis and Social Impact Evaluation interchangeably.

Where it began. SIA gained formal recognition with the US National Environmental Policy Act (NEPA), 1969, which included social elements in environmental impact studies. It then spread as many international standards made it a crucial component.

The seven steps of SIA

The workbook lists them as (a) to (g):

1. Setting objectives — the scope of analysis, the level, and the desired social change.
2. Analysing stakeholders — who must be involved.
3. Setting relevant parameters — how the social enterprise plans its intervention and how the activity achieves outcomes and impacts.
4. Identifying measurement tools.
5. Measuring the targeted outcomes and impact.
6. Verifying and valuing impact — checking that the logic is strong enough and that key stakeholders value the impact.
7. Monitoring and reporting — feeding the information into the management system and reporting to stakeholders.

The three approaches (Table 7.1)

These steps are a common framework. But the tools built by academics, impact investors and practitioners rest on three approaches.

The workbook's verdict on all three:

• Qualitative is time-consuming.
• Quantitative and monetisation are widely criticised. Behavioural and psychological change is hard to count or price. And counting only the number of beneficiaries is a very narrow view of impact.
• So: understand why the organisation is doing the SIA, then choose the approach and tools. There is no standard format or single tool that captures everything.

Why SIA matters to stakeholders

The workbook lists seven benefits:

• (a) Management learns whether projects run effectively and need changes.
• (b) Management learns whether the intervention is changing the target group and the wider community.
• (c) The organisation can use the SIA report as evidence to advocate and secure future funding.
• (d) Donors and investors can ensure timely and effective achievement of targets.
• (e) Management and others can check the team is keeping to timelines and budgets.
• (f) It builds data on successes and failures and strong data management systems to track long-term impact.
• (g) Concrete evidence motivates internal and external stakeholders to contribute again.

Four organisational models for structuring evaluation

1. Outsourcing model

Why outsource? SIA is time-consuming and needs expertise — output, outcome, impact and methodologies. Many organisations lack that expertise and the money and staff. Also, internal staff and board members contributing to SIA can create a conflict of interest. For an unbiased report, organisations hire external agencies.

Example: DMEO, NITI Aayog. The Development Monitoring and Evaluation Office monitors and evaluates Government of India schemes. Mostly it works internally, but it sometimes outsources to reputed agencies. It keeps a panel of expert agencies in six sectors:

1. Agriculture & Allied
2. Economic
3. Energy
4. Infrastructure
5. Social
6. Social Welfare

Agencies are chosen from the sub-sectors through competitive bidding.

Drawbacks:

• Smaller organisations that live on donations and grants cannot afford external agencies.
• The implementing organisation knows the community and the before-and-after situation better. An outsider may miss impact indicators.

2. Partnership model

Beyond contracts, research teams and policy teams can form mutually beneficial partnerships.

• Researchers want to answer new questions, add to the global evidence base, and push impact evaluation further.
• Policy makers and implementers want a good-quality evaluation of their project.
• The research team may bring co-financing if funders' goals match the research focus.

Risks:

• Innovative research elements may not match immediate local policy goals, though they add value globally.
• Policy teams may not need full scientific rigour and may have a higher tolerance for risks to the evaluation than researchers.

What makes it work: closely aligned objectives, evaluation strategies designed together with clear roles, and a clear evaluation plan as a mutual commitment. This can work even without a contract — but a written agreement is always preferred.

Example: the World Bank. Groups like DIME (Development Impact Evaluation), SIEF (Strategic Impact Evaluation Fund) and GIL (Gender Innovation Lab) fund and support evaluations. In Côte d'Ivoire, the World Bank, J-PAL (Abdul Latif Jameel Poverty Action Lab) and the government jointly evaluated a Youth Employment and Skills Development Project.

3. Fully integrated model

SIA is built into the programme and done on a regular basis, not at the end. Environmental, economic and social indicators are captured at similar points in time. This reduces inconsistencies in data and captures interdependencies between indicators.

Who uses it: usually social venture capital firms and impact investors, to track investments in social enterprises and ensure their sustainability.

Two examples:

• Acumen Fund's BACO Ratio (Best Available Charitable Option) — judges an investment's viability on financial leverage, technological leverage and social impact.
• Robin Hood Foundation (a New York City non-profit) — funds work in Jobs & Economic Security; Education; Early Childhood & Youth; and Survival. Its Benefit-Cost Ratio estimates the collective benefit to poor people per dollar of cost, turning diverse programmes into a single monetised value.

4. DIY (Do-it-Yourself)

Social enterprises build their own tools, customised to their stakeholders' expectations, after studying tools made by others nationally and internationally. The workbook says its benefits and drawbacks are similar to the integrated model.

Impact reporting

Why is it hard? From a beneficiary's view, social impact cannot readily be put in money terms. But not all impact is equally hard. Emergency relief and basic services (sanitation, water, housing) are easier to measure than impact on public policy, good governance, freedom and rights. Even there, it is still useful to measure what an organisation does and whether its strategies are working.

What it is. Impact reporting communicates the difference the organisation made to its stakeholders. It can be:

• an independent impact report,
• part of an annual report, or
• part of a sustainability report.

What shapes it: the objective of the SIA, the time and resources available, and the stakeholders it is written for. If SIA is done under a regulatory mandate, the report follows government guidelines. If it is for one programme's internal needs, it can stand alone or go into the annual report.

How reporting helps:

• reviewing impact against vision and goals
• building a learning organisation
• encouraging employees, volunteers and board members
• building trust and credibility
• sharing lessons with organisations working on the same cause

Formats: trends over a long period (for regular trackers), a ToC map of input, output, outcome and impact, or success stories and case studies. The generally used format is a detailed qualitative report covering objectives, scope, data collection and analysis.

Direct impact and systemic impact

The target community is the area of intervention, mentioned or established by the NPO/SE itself.

What goes into an SIA report

The final report should combine qualitative, quantitative and financial aspects. It should be as short as possible while meeting transparency and materiality. Its components:

• an executive summary for all stakeholders
• the organisation — vision, mission, key stakeholders, activities
• the scope of the SIA
• stakeholder involvement and data collection methods
• for quantitative or monetary approaches, calculations, estimates and assumptions
• an impact map and theory of change map
• assumptions and limitations
• an audit trail — which stakeholders, outcomes or indicators were included and excluded, and why
• an impact scorecard or evaluation criteria using key metrics, past trends, and case studies or participant quotes

Annual impact reports have a dual role: (i) a trend analysis of performance, and (ii) a baseline and midline for later years.

Report the bad news too. Negative impacts must be reported so harm can be reduced. If a trend dips, the organisation is accountable. Causes may be internal (the NPO/SE itself) or external (regulation, natural or man-made calamity, socio-political change). Reasons for underachievement must be reported.

After listing on an SSE, NPOs and FPEs must produce an Annual Impact Report (detailed in Chapter 9), following the Social Impact Assessment Standards issued by the SRO.

Box 7.2: sample indicators by sector

The workbook splits indicators into direct impact and sector-wide impact.

• Livelihoods (direct): outreach with social composition (SC, ST, OBC); SHGs and their tiers; savings and credit; change vectors; capacity building; institution building; specific programmes (WATSAN, land rights, youth); gross annual household income; finance leveraged; community contribution; donor-specific indicators (e.g. trees and carbon sequestration; land treated and water conserved; MGNREGA wage employment and workdays).
• Health (direct): clinical services — patients treated, in-hospital mortality, gender and social group, out-of-pocket expenses; community services — mother and child health, non-communicable diseases, malaria.
• Health (sector-wide): public health system strengthening with the State Government and NHSRC — trainings, mentoring, technical support, accountability, quality improvement.
• Education (direct): coaches trained, schools with functional libraries, active Bal Sansad, new and returning enrolments, volunteers in more than 3 campaigns, improved learning outcomes.
• Education (sector-wide): managers with 21st-century leadership skills, school clusters as units of change, teacher education on 21st-century skills, future-ready curriculum.

The workbook notes a clinical-services problem: the denominator is hard to fix because patients come from wide areas, so lives saved or reduced medical impoverishment are hard to establish.

The seven challenges in conducting SIA

1. Lack of standard terminology — "outcome" and "impact" are used for outputs. Acumen describes BACO as cost per outcome or impact, but it actually focuses on output.
2. Lack of a standard reporting framework — too many national and international standards; donors and investors want their own formats, so teams write multiple reports.
3. Lack of common measures within a sector — some use cost-effectiveness, some benefit-cost ratios, some qualitative methods, so programmes cannot be compared.
4. Unavailable or unreliable data — smaller organisations lack databases; data from primary stakeholders can be biased by conflict of interest.
5. Assumptions — projections assume, for example, that all children in an education programme will pursue higher studies or jobs.
6. Interdependencies — outcomes depend on other interventions or external factors, e.g. a peer-tutoring result also depends on family support.
7. Timeframes — health or microfinance may show impact in 3–5 years; environment or behavioural and cultural change may take 15–20 years. No standard time frame fits all.