Better Evaluation Systems for Better Post-2015 Development

UN ESCAP Executive Secretary Shamshad Akhtar delivers her remarks during the United Nations Evaluation Group meeting in Bangkok on 2/4/2014.
Photo Credit: UN ESCAP

Delivered to the United Nations Evaluation Group (UNEG) in Bangkok

Dear colleagues, welcome to Bangkok – the Asia-Pacific home of UN agencies.

Introduction

I have learnt to recognize the merits and significance of evaluation. Let me take a minute to share my personal thoughts about the benefits of evaluation.

First, we have to recognize that, like empirical economics, evaluation helps evidence-based analysis of the problem at hand, and feeds into effective policy-making or project design and implementation as the case may be.

Second, evaluation helps get the diagnostics right. I am an ardent believer that without good diagnostics of the problem at hand we cannot embark on development business. Diagnostics are very intricately related to evaluation, because they are themselves a product of evaluation.

Another important dimension has been brought out by the famous Nobel laureate, Milton Friedman – and you can imagine why I value Milton Friedman because he did a lot of monetary work which I admire. He stated that: “One of the great mistakes is to judge policies and programs by their intentions, rather than their results.”

I have to tell you this is very true in monetary policy, with the communication of intentions rather than of outcomes. One can go on, in the same spirit, about the virtues of evaluation in our own development work.

In the United Nations system, and among our development partners, we face shared challenges: that our policies and programs are motivated by the best intentions (and sometimes also driven by conflicting interests), but often we do not know nearly enough about the results our efforts have had, let alone, the impact of our work. Without that crucial information, how can we accurately judge and improve our work?

In UN system we all face some common challenges for our policies and programs, and so even with the best intentions, if we don’t pay attention to rigour in our implementation, then clearly our results are not as credible. So we must pay attention to your evaluation work, because you bring a lot of wisdom and independent judgement.

In this context, the United Nations Evaluation Group (UNEG), to which I have only just been exposed, has special significance for all of us in UN. The vision statement of the UNEG strategy for 2014-2019 rightly states that: “Evidence produced by evaluation informs a more relevant, efficient, and effective UN system, with greater impact on the lives of the people it serves.” We are in the business of the Millennium Development Goals (MDGs) and other development issues which touch on lives of people.

As an economist and development expert, affiliated with multilateral development banks too, I cannot emphasize enough the need to operate on best-available evidence, to ensure the greatest possible relevance, impact, effectiveness, efficiency, and sustainability.

At this pivotal point in global development, conducting a thorough evaluation of the achievements and shortcomings of the MDGs has been timely. Lessons learnt on this front are helping shape the design of the post-2015 development agenda, but our efforts will only bear fruit, if better-informed by credible, reliable, and useful evaluations.

In this context, it is a pity that we will only be evaluating the experience of the MDGs now, when it would have been helpful had we received inputs early on, so that they could have been used in the formulation of the sustainable development agenda. Of course we have gained a lot through consultations and hearing a wide range of voices, but I think the MDG architecture had a number of issues, including some shortcomings in design, a lack of effective partnerships, as well as a need for more supportive frameworks for the means of implementation. This is all captured in the Secretary-General’s report.

Partnership with UNEG

Let me turn now to how much I value the partnership with UNEG. I would like to express my thanks to UNEG, for selecting ESCAP as the official host of your Annual General Meeting and the UNEG Evaluation Week this year. It is an honour for us to host this annual event, taking place for the first time, I believe, in Asia and the Pacific.

From what my colleagues have said, ESCAP has greatly benefitted from its work with UNEG since 2006. The partnership has been invaluable in shaping our monitoring and evaluation systems, and improving the quality and utility of evaluations. ESCAP’s Evaluation Guidelines, were launched in 2009, and they were themselves informed by the norms and standards developed by UNEG.

ESCAP established a separate Evaluation Unit, a couple of years back, with dedicated human and budgetary resources, responsible for managing and implementing ESCAP’s evaluation plan. This unit has allowed ESCAP to more effectively respond to the increasing demand from our member States for evaluative exercises, and to better support organizational reforms, based on evaluation recommendations.

In this spirit of ensuring that our work is more evidence-based, I am pleased to say that, according to the latest assessment by the Office of Internal Oversight Services (OIOS) of evaluation capacity and practice in the secretariat, ESCAP received decent scores. So that part is a delight, but I have to say that I don’t beat the drum only on successes – as a candid evaluator myself I have tried to undertake my own diagnostics of ESCAP’s situation in the last eight weeks. I will certainly take on board all the valuable suggestions of the OIOS evaluation, which concentrated on research and analysis. It’s very welcome, because I have been thinking about how we best take ESCAP to new heights – and I believe that developing its analytical capabilities will be critical. But we can’t do that without input from all of you.

One of the things that’s very important to me is to see the UN ‘work as one’. I am very fortunate that ESCAP is a great platform where I can visibly see the strengths and weaknesses of whether we work as one UN or not. So I take this opportunity with you, as evaluators, to stress this point to your agencies – to drive this message forward: that the more we are united, the stronger we will be, and that to do so in a more profuse manner in analytical work is very-much warranted, because our expertise is quite spread. So rather than produce independent products, I think we should try to do more joint analytical products.

We took this OIOS report to our Senior Management Team meeting, and we got very good feedback from them, which has been communicated to Deborah’s colleagues in OIOS, who did a remarkably good job, and I think that going forward I would not only apply decent rigour in accepting and taking on board those recommendations, but also making sure that we set up a monitoring and tracking system, and strengthen our internal evaluation systems.

Conclusion

To conclude, ESCAP is a valuable shared asset for the people of Asia and the Pacific, and I believe also for the UN system. The Commission serves a region with more than half of the world's population, more than a third of global economic output, the highest average regional growth, with the most disaster-prone countries in the world, and the highest proportion and number of poor.

It offers the most comprehensive platform for the consultations and policy dialogues for shaping regional sustainable development, and contributing to the formulation of the post-2015 development agenda. It is in this spirit that we have organized the Asia-Pacific Regional Sustainable Development Forum, from 19-22 May, as well as an outreach event on Sustainable Financing from 10-11 June, which is a very important area.

For ESCAP to fulfil its potential, we need to focus on our comparative advantages, devise sharper strategies, work more collaboratively with regional institutions and financial agencies, and with all of you. We have to focus on engagement, and concentrate on concrete development outcomes and impact.

In this context, we need an empowered evaluation function to generate information that enriches the strategic planning of our programme of work, improves organizational learning, and strengthens accountability to our member States and other development partners. And hopefully also includes accountability of member States, because as the Secretary-General has often said, our ability to implement development is critically and intricately linked with the member States’ political willingness to strive for change.

Since I began with a quote, allow me end with one too. Peter Drucker, the management theorist who coined the term ‘knowledge worker’, once said that: “The most serious mistakes are not being made as a result of wrong answers. The truly dangerous thing is asking the wrong question.” To which I would add also the wrong diagnostics.

The continued support of UNEG will be crucial in helping ESCAP, and our member States, to ask the right questions, get the right diagnostics, and to unlock the evidence and insights to take our development work to new heights of relevance, efficiency, and impact.

Thank you for being here, and I hope you get some opportunity to see Bangkok.

I thank you.

Listen to Executive Secretary Akhtar discuss the importance of evaluation with Deborah Rugg, Chair of the United Nations Evaluation Group.