Page 50

CEP template 2012

Science of delivery: Education outcomes and the World Bank Arianne Wessal, Clay Wescott and Carlos Espindola This article addresses the following research question: do World Bank-supported education projects achieve better outcomes when there is deeper attention to designing monitoring and evaluation systems during project design? The work presented here draws on a recently constructed database of World Bank investment projects. Ratings of education projects’ monitoring and evaluation design, and their outcomes, will be compared to see whether there is a correlation between high monitoring and evaluation design ratings, and better outcomes. This pattern has been observed in other service areas supported by the World Bank. Further analysis will look at other factors associated with better outcomes and determine their relative importance. The next step will be to analyse detailed project components to see if there are design features common to high performing projects with effective monitoring and evaluation, and other outcome-enabling features that are missing in weaker performing projects. Further testing will be carried out using selected case studies. Evaluations of World Bank projects have, in recent years, shown a decline in outcome ratings. Improvements in monitoring and evaluation, and other features, may point the way to reversing this trend. When the enabling environment encourages experimentation, tight feedback mechanisms and constant communication make it possible for project managers to make realtime changes to projects throughout the project cycle. Feedback mechanisms could include such measures as evaluation committees, seminars and workshops, automated systems, reporting and follow-up procedures. It is also essential to tailor project components to local factors, such as implementation capacity and political support. Project managers could be given greater incentives to draw on aspects of past successful projects, try new concepts and adapt to changing conditions. Research on achieving results Many developing country governments are trying to understand why the policies put in place to reduce poverty and build prosperity are not leading to the results they want. One way forward could be a new form of knowledge, the ‘science of delivery’. This concept is borrowed from the health care field, where the previous emphasis on understanding the causes and consequences of health issues is Commonwealth Governance 48 Handbook 2014/15 shifting to give more attention to organising, managing and financing health promotion (Catford, 2009). Applied to the field of public management, the science of delivery should provide mechanism-based explanations of how and why the implementation capability of countries varies, as well as a guide to action (Woolcock, 2013). This approach differs from the institutional reform model that currently dominates the public management field. In the institutional reform model, best practice ‘solutions are often chosen without significant consideration being given to their external validity. In this model, the focus is on inputs delivered rather than on outputs obtained. Another pitfall is that reforms try to take on too much and are stymied by the complexity, lack of clarity, uncertainty and unintended consequences of efforts to change socio-political structures’ (Schuck, 2014: p. 372). The result is that projects frequently fail to achieve their goals, while the specific reasons for this failure remain hard to pin down. In order to remedy these issues, the science of delivery tailors project components based on local factors, such as implementation capacity and political support. As problems arise, consideration is given to concerns at the political, organisational and project levels before deciding on a solution. Project managers are encouraged to draw on processes linked to successful projects, try new concepts and adapt to changing conditions. The science of delivery approach requires experimentation, intensive field research documented in accessible case studies, improved data collection at the project level through the use of good monitoring systems, and the diffusion of ideas to enable these changes in implementation and management. The result of using a science of delivery approach is the incremental creation of localised projects that provide impactful results to the target community as well as useful data and information to the public. This data gives project managers the ability to understand how and why a project was effective rather than just whether it was or not. The science of delivery allows project managers in a region to better understand why their projects fail to achieve their desired impact, as well as giving them the ability to draw on lessons learned from successful projects in other regions. At the same time, there have been recent theoretical advances in many scholarly fields including systems engineering, medicine, economics and public management, which are being exploited to help countries organise the emerging evidence on successful delivery to help them improve development results (Kim, This paper has not undergone the review accorded to official World Bank publications. The findings, interpretations and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the International Bank for Reconstruction and Development, the World Bank and its affiliated organisations, or those of the executive directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. This paper draws on Wessal, A., Treuth, M. and Wescott, C., 2014, Science of Delivery and Implications for Monitoring and Evaluation. An expanded version was presented at City University, Hong Kong, in December 2014 at an International Public Management Network conference, ‘The Science of Delivery’. The authors are grateful for a peer review by Alison Wescott.


CEP template 2012
To see the actual publication please follow the link above