As the nation tries to contain and rebound from the pandemic, there is dire need and more opportunity than ever to invest strategically in intelligent developmental programs. Early interventions have a strong evidence base proving profoundly positive and lifelong benefits. But moving from a successful pilot to a state-level program requires careful design in order to achieve the same positive results at scale.

Some of these interventions were discussed in the policy webinar called “The Voltage Effect in Scaling Policies” organized by Merged Areas Governance Project (MAGP) of United Nations Development Programme (UNDP). The webinar is part of an on-going policy seminar series centered around key current development themes, informed by global practice and knowledge.

Government officials, researchers and development practitioners debated the steps of such behavioral interventions and field experiments. In his opening remarks, Professor John A. List (Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago), enlightened the audience by explaining the concept behind scalability, “Scaling is a fragile concept. We’ve found that scalable ideas are ones that hold five key traits—what we call the BIG5. A deficiency in any one can render an idea unscalable, so just being aware of them—knowing what to look out for and ask about—is a critical first step toward avoiding a voltage drop”, he said.

He further elaborated on the same, saying “First, and this sounds obvious, is that there must be adequate evidence to support scaling. The second element of the BIG5 is representativeness of the population. In other words, you can’t assume that the small subset of people for whom an idea worked originally are representative of the general population that needs to be served.

Professor List continued explaining the remaining elements of the BIG5 by saying “Third is the representativeness of the situation. If original research results are dependent on the specific context in which the study was conducted, or if they are not achieved in a policy-relevant environment, we can expect the benefit-cost profile to change at scale. A fourth key aspect pertains to spillovers, which you can think of as a corollary of the Law of Unintended Consequences, or what happens when the implementation of your idea has unplanned effects that backfire and diminish your results. Finally, the fifth element of the BIG5 represents marginal cost or the supply-side economics of scaling—does your idea have economies or diseconomies of scale?”.

Professor List emphasized the fact, that to be truly effective for policy purposes, research studies should be designed, from the very beginning, with an idea of what a successful intervention would look like fully implemented in the field, applied to the entire subject population, sustained over a long period of time.

Studies that enroll representative populations and are implemented in representative contexts should be prioritized. We should encourage replication studies that seek to confirm original impressive findings as well as long-term studies that examine the effects of a program over a sufficient time frame. Jumping the gun and acting, or choosing not to act, based on early evidence can have substantial real-world consequences.

Continuing with the immensely engaging discussion, the honourable Secretary Finance of the Government of Khyber Pakhtunkhwa, Mr Atif Rehman highlighted possible links to Professor List presentation. Even in Pakistan under leadership of the current government, citizens and lawmakers have become increasingly enthusiastic about the adoption of evidence-based programs and policies. “The government has prioritized the development funding in the province, which is witnessing a record increase of 150 billion rupees in 2021-22 in the Annual Development Plan (ADP) allocation and 60 billion rupees have been planned for development expenditure in the Merged Areas, that includes interventions such as the Sehat Card and early childhood education program.” There’s a record of countless interventions that positively impact peoples’ lives. And yet, when expanded, these efforts have often failed to deliver the anticipated dramatic societal impacts. Therefore, it is uplifting to hear that evidence-based policies are still worth pursuing as long as we acknowledge and understand the how, why, and when promising results can be delivered at scale.

Additionally, Dr Mariam Mohsin, Assistant Professor of Public Policy at the Pakistan Institute of Development Economics (PIDE) elaborated further by providing a local context of the small-scale experiments that have not been able to achieve scalability in the country because of the factors explained.