If you are like me, one of the key barriers in adopting cases is finding the right cases for your course. With this in mind, the HKS Case Program created this guide to simplify the process of identifying and selecting a case in the realm of program evaluation.
Cases help bring some of the course material to life and can be a great complement to other pedagogic approaches. For example, while the students in my course learn about different methods to evaluate the impact of the program, using the Jamaica PATH case has helped them more clearly see the advantages and disadvantages of these methods and, perhaps just as importantly, the tradeoffs that are often made in trying to select an evaluation design in the real world.
I hope this guide is as useful to you as it has been to me.
Case Pack: Program Evaluation
These short cases are designed to improve learning on a host of topics, from theory of change to impact evaluations, while underscoring the everyday tradeoffs organizations face when trying to do scientifically rigorous work in the practical constraints of the real world.
A. Theory of Change
Case: Women as Leaders: Lessons from Political Quotas in India
Length: 7 pages
Learning Objective: Students learn how to assess the theory of change of an intervention aimed at improving political representation of women in India and also learn how to (and how not to) assess the causal effect of an intervention.
Case: Scared Straight: Freeport City Council Takes on Juvenile Delinquency
Length: 4 pages
Learning Objective: Students learn what is meant by "evidence" and what different forms of evidence they can draw on.
B. The Role of Evidence in Social Programs
Case: Providing Pensions for the Poor: Targeting Cash Transfers for the Elderly in Mexico
Length: 5 pages
Learning Objective: Introduces students to the use of evidence in social programs in the context of having to decide between 3 possible targeting mechanisms that the government of Mexico considered when launching a pension program for the elderly. Students are put in a position to confront some challenging tradeoffs between targeting accuracy and political, financial, and operational feasibility.
A. Importance of Evaluation Design
Video Case: New York City’s Teen ACTION Program: An Evaluation Gone Awry
Length: 2 videos (total 10 minutes); 4 page background note.
Learning Objective: Highlights the importance of good evaluation design, and raises key issues around statistical power, measurement error, internal validity, etc.
B. How to Design an Impact Evaluation (Non-Experimental Methods)
Case: Designing Impact Evaluations: Assessing Jamaica’s PATH Program
Length: 4 pages
Learning Objective: Students are asked to assess the strengths and weaknesses of three possible evaluation designs using as criteria the scientific rigor, political feasibility, logistical implications, and financial feasibility of each design. They are put in a position to confront the inevitable tradeoffs when trying to balance the scientific rigor of an evaluation design with the practical constraints of implementing it.
C. How to Design an Impact Evaluation (Experimental Methods)
Case: Devil in the Details: Designing a Social Impact Bond Agreement in Medellin
Length: 15 pages
Learning Objective: Under the challenge of crafting a Social Impact Bond to help reduce the teenage pregnancy rate in Medellin (Colombia), students are asked to grapple with key issues in designing an impact evaluation, including selecting treatment and control groups and identifying outcome measures that are both rigorous and practical to collect under the constraints the protagonists of the case face.
Case: The Geography of Poverty: Exploring the Role of Neighborhoods in the Lives of Urban, Adolescent Poor
Length: 5 pages
Learning Objective: Students learn about the functions of qualitative research and closely examine how qualitative research can enhance the interpretation of quantitative data in the context of a mixed methods evaluation of Moving To Opportunity, a very ambitious anti-poverty program that consisted in offering low-income families in five U.S. cities (Baltimore, Boston, Chicago, Los Angeles and New York) the opportunity to move to neighborhoods with low levels of poverty.
Faculty Profile
Dan Levy, Senior Lecturer in Public Policy and Faculty Chair of the Kennedy School's SLATE Initiative, teaches courses in quantitative methods and program evaluation. He has served as a senior researcher at Mathematica Policy Research, a faculty affiliate at the Poverty Action Lab (MIT), and as consultant to several organizations including the World Bank.
Recent Cases by Dan
Betting Private Capital on Fixing Public Ills: Instiglio Brings Social Impact Bonds to Colombia
Providing Pensions for the Poor: Targeting Cash Transfers for the Elderly in Mexico
Devil in the Details: Designing a Social Impact Bond Agreement in Medellin
