Video-Based Case - New York City’s Teen ACTION Program: An Evaluation Gone Awry

New York City’s Teen ACTION Program: An Evaluation Gone Awry


Educator Access
A review copy of this case is available free of charge to educators and trainers. Please create an account or sign in to gain access to this material.

Permission to Reprint
Each purchase of this product entitles the buyer to one digital file and use. If you intend to distribute, teach, or share this item, you must purchase permission for each individual who will be given access. Learn more about purchasing permission to reprint.

  • Product Description

    This video-based case explores how an impact evaluation for a promising after-school program in New York City went awry. In 2006, New York City's Mayor Michael Bloomberg created the Center for Economic Opportunity (CEO). Designed as an innovation fund, CEO tested anti-poverty programs by applying an evidence-based approach. One of CEO's programs, Teen ACTION, was developed to provide after-school service-learning opportunities to at-risk youth. Teen ACTION's model was based on a rigorously evaluated program which had proven to significantly reduce rates of teen pregnancy and course failure among participants when compared to a control group. But when it came time to evaluate Teen ACTION's impacts, CEO ran into significant hurdles. With rare candor, CEO officials Kristin Morse and Carson Hicks discuss on video how flaws in the program's evaluation strategy, coupled with random unforeseen events like the Swine Flu, rendered the evaluation results unusable.

    The case includes a short backgrounder on Teen ACTION and two videos (totaling approximately 9 minutes) designed to be used during the class to frame specific discussion pastures. In the first video, Carson Hicks, Director of Programs and Evaluation at CEO, and Kristin Morse, CEO's Deputy Executive Director, describe the evaluation design and the implementation challenges the team encountered. The second video reveals what the CEO team decided to do once they realized the results of the initial evaluation were flawed. Information on accessing the videos is available to instructors in the teaching plan.

    Learning Objective:
    The challenges the CEO team experienced with the Teen ACTION evaluation illustrate key issues related to conducting impact evaluations, such as importance of good evaluation design, key problems of data collection (including low response rates and social desirability bias), internal validity, external validity, and statistical power.

    Please note: the video portion of this case is included in the teaching plan and is intended for instructors to use in class.  Here is more information on how to access teaching plans.

  • Other Details

    Publication Date: August 29, 2016
    Teaching Plan: Available with Educator Access
    HKS Case Number: 2027.0
    Video Producer: Patricia Garcia-Rios
    Case Author: Anjani Datla
    Faculty Lead: Dan Levy
    Pages (incl. exhibits): 5
    Setting: United States
    Language: English
    _year: 2016-2017
    _pages: 1-15
    _video: Yes
    _geography: US & Canada
  • Warranty Information

    /review/2027.0.EducatorCopy.pdf, /teachingplan/2027_2.pdf