Measuring Impact the Hard Way

Authored by Kavya Shah, Impact Leadership Program Fellow at Wadhwani AI Global

It’s a well known example in AI for health, patients with more risk factors receive more attention and resources, and ultimately experience better outcomes. A model trained on this pattern might incorrectly conclude that higher-risk patients are less likely to experience severe illness, an assumption that makes sense statistically, but not clinically, without an understanding of how the system behaves.

Herein lies a core reality of AI for real-world impact: outcomes are shaped as much by human behavior and institutional processes as by the model itself. The path from prediction to impact is rarely linear, and working within this complexity requires more than surface-level assumptions or clean abstractions. It demands clarity about how systems operate, how people make decisions, and how technology influences and integrates with both. Understanding these dynamics isn’t academic, it’s what separates meaningful progress from solutions that don’t translate on the ground.

Designing for complexity

When confronted with development challenges, our approach isn’t optimizing for lab performance, we’re optimizing for usability, fit, and long-term relevance. Doing so requires a clear-eyed view of how systems actually function, the constraints communities navigate daily, and how institutions adapt (or don’t) when new tools enter the workflow.

As we work with partners across the Global Majority, we operate with the understanding that every deployment has consequences, intended and unintended. Too often, the “Global South” has been left behind in discussions about development for its own future, relegated to the sidelines as “minority populations” that will be passive users of interventions. Not only does this view obscure the fact that these populations are in fact the “global majority” but it also misses the opportunity to partner with them in co-creating their own path forward. At WAIG, we take a different approach, ensuring trusted partnership is at the heart of everything we do.

There’s a lot riding on the work we’re doing, and as one of the first organizations of our kind, we take that responsibility seriously. That’s why we measure impact beyond metrics. Our work is judged by what changes on the ground, and what continues to change long after the model is deployed. This understanding of impact comes from harder questions, embedded at each stage in our development process,  questions we ask of ourselves and the partners we engage.

  • Are we innovating with purpose? 
  • Are we designing for sustainability?
  • Are we prioritizing impact?
  • Is our commitment to excellence grounded in reality? 


These principles are not abstract values, they are the conditions under which AI becomes a reliable tool for development rather than an experimental intervention. Because in designing technology for social impact, the real breakthrough must go beyond the algorithm towards the system it helps transform.