This module is targeted toward any student who seeks to understand how best to design effective interventions, big or small, which allow analysis of their success. It aims to provide students with the generalised skills and knowledge to enable them to make informed decisions concerning both the design of such policy interventions, and the evaluation of their impact in a range of settings.
To achieve its aims, the module balances the necessary technicality of rigorous statistical and econometric analysis against the need for an intuitive operational understanding of the required steps when confronted with a policy problem.
Through extensive hands-on use of a range real-world examples and datasets, students gain conceptual knowledge such as frameworks for effective policy design, core statistical knowledge, and a transferrable toolkit of practical analytical skills required for effective impact evaluation and other statistical analysis. Students will then be able to apply this skill set to any situation where an intervention is required – either very locally within a business or organisation, or in terms of national- or international-level issues.
The module is taught via a combination of lecturer-led classroom discussions, problem-based learning, and practical computer lab sessions.
Principal Learning Outcomes
Upon completion of this module, students will be able to:
- Demonstrate an understanding of frameworks for policy design, be able to apply such frameworks to interpret existing policy, and use established frameworks as a tool for designing new policy interventions
- Demonstrate an understanding of the application of policy design frameworks to both localised interventions, and also broader applications
- Interpret, produce, appropriately present, and explain (verbally and in writing) a range of descriptive statistics
- Demonstrate (verbally and in writing) an understanding of the need for rigorous policy or intervention evaluation and be able to provide examples of good practice in this subject area
- Identify and implement appropriate statistical methods for the evaluation of policy intervention based upon the characteristics of the intervention and the available data
- Demonstrate an understanding (in writing and presentation) of core statistical concepts such as (but not limited to) distribution, correlation, estimation, and causality, and how consideration of such concepts is key to selection of appropriate means of impact evaluation
- Independently use a range of computer software for the production of descriptive statistics, graphs and charts, and for the analysis of policy outcomes
The following outline represents the core knowledge and competency gain associated with the course activities.
In order to facilitate the acquisition of knowledge and competency, the course is structured and taught via a number of core problems in the development and evaluation of policy which must be understood and overcome. Where possible, problems will be addressed through a case-study approach which focusses on the design and impact evaluation of real-world policy – including the study, replication, and extension of analysis carried out in reports and/or academic papers.
I. How and why do we consider evaluation when designing policy?
- What frameworks can we use to design effective policy?
- Why do we evaluate?
- What are the links between good policy design and effective impact evaluation?
- The power of data: Descriptive Statistics, Indicators, Indices
II. How can we collect data, what can it tell us, and what can it not?
- Data collection, sampling methods, distributions, sample v population
- Correlation, statistical significance, spurious relationships, causality, formation of hypotheses
III. Are we observing correlation or causation?
- Multivariate analysis: Principles, methods, interpretation, the importance of identification
- Difference in difference approaches and the importance of a control group
- Regression discontinuity
IV. How do we overcome the issue of problematic data?
- Imperfect control groups, matching, and constructing artificial samples
- The frequent issue of endogeneity, instrumental variables and the art of finding a good instrument
V. How can we maximise the impacts of our findings?
Illustrative Reading List
The main text for the course is:
Gertler, P. J., Martines, S., Premand, P., Rawlings, L.B., and Vermeersch, C.M.J. (2016) Impact Evaluation in Practice (2e). The World Bank, Washington, US (freely available from the World Bank website)
This will be supplemented by accessible technical sources such as:
Gujarati, D. (2015) Econometrics by Example (2e). Palgrave , UK
Other technical texts will be employed where necessary to ensure full coverage of the statistical techniques featured in the course
The course will also make extensive use of individual reports and journal articles relevant to the case studies used in the course.
1500 word individual assignment (15%)
4000 word group project (15%)
20 minute in-class group presentation (15%)
9 x weekly online quiz (5%)
2 hour Summer examination (50%)