Program Impact Evaluation

Course held Thursday 7 and Friday 8 June 2012 at the Department of Education, Employment and Workplace Relations in Canberra

Presented by Associate Professor Chris Ryan, Principal Research Fellow, Melbourne Institute, The University of Melbourne

Overview

This two day course considers methods for quantifying the causal impacts of social and economic programs or interventions. The focus will be on measuring the impact of  government interventions. What is the impact of a new early-school reading program? How much do their peers influence the school achievement of other students? What is the impact of intensive case management for those who have been out of work for a long time? Does writing a letter that contains a future threat to those receiving government benefits influence their current behaviour? The course will cover a range of experimental and quasi-experimental methods of impact evaluation that can be used to answer questions such as:

  •  randomised control trials (‘field experiments’)
  • lotteries of over-subscribed programs
  • ‘before and after’ and interrupted time-series designs
  • ‘difference in differences’
  •  matching methods (including propensity score matching)
  •  instrumental variables in regression modelling
  •  regression discontinuity design

Exercises (not computer based) and group discussions will complement presentation of the material. A working example, involving a hypothetical intervention, will be used to highlight the properties of the various estimation approaches.

More information

General Enquiries:

Penny Hope
Melbourne Institute
Ph: +61 3 8344 2151
Email: [email protected]

 

Pre-requisites

Participants should be familiar with the basics of statistical inference (simple use of confidence intervals and hypothesis testing) and linear regression analysis. No prior knowledge of evaluation methods will be assumed.

Reading

A detailed guide to material will be given out in the course. Participants who want to familiarise themselves before the course with examples of impact evaluation might consult:

Jeff Borland, Yi-Ping Tseng and Roger Wilkins (2005), Experimental and Quasi-Experimental Methods of Microeconomic Program and Policy Evaluation, Melbourne Institute Working Paper  Series, Working Paper 08/2005, available at  http://melbourneinstitute.com/downloads/working_paper_series/wp2005n08.pdf

Martin Schlotter, Guido Schwerdt and Ludger Woessmann (2010), Econometric Methods for Causal Evaluation of Education Policies and Practices: A Non-Technical Guide, IZA Discussion Paper No. 4725 January 2010, available at http://ftp.iza.org/dp4725.pdf

Gertler, P, Martinez, S, Premand, P, Rawlings, L, and Vermeersch C (2011) Impact Evaluation in Practice, Washington DC: The World Bank (free download – type title into Google).

Instructor

The course will be taught by Chris Ryan. Chris heads the Economics of Education and Child Development research team at Melbourne Institute. He previously worked at the Australian National University and in the Australian Public Service.