1. Introduction and background
2. Issues for discussion
1. compliance
2. standardization and harmonization
3. compilation of indicators
4. reporting
3. The role of ESCAP
4. Concluding remarks

Annex: an assessment of the statistical indicators derived from united nations summit meetings

1. Introduction
2. key issues

2.1 the stakeholders, competing needs, statistical capacity and the burden on countries
2.2 quality and technical properties, continuity and change

3. A technical assessment and framework for indicators
4. Future processes
5. Acknowledgement
6. Summary of recommendations

Table 1: hierarchy of statistical indicators by domain and sub-domain
Table 2: number of indicators by domains, sub-domains and priority levels
Table 3: correspondence of recommended indicators to existing sets by priority level

I. INTRODUCTION AND BACKGROUND

1. This paper aims to summarize how issues in the field of Development Indicators have evolved over the last year. The Working Group may recall that the issue of Development Indicators has already been discussed at past sessions of the Committee on Statistics as well as of the Working Group itself. Development Indicators are defined as those indicators which have been selected to monitor the achievements towards development goals set by global United Nations conferences. There were several such conferences during the nineties, and at the Millennium Summit of September 2000, Heads of State and Prime Ministers reaffirmed their commitment to achieving a number of key development goals.

2. A number of problems affected the possibility of ensuring a coherent process in using indicators to monitor the development goals. These included problems of coordination, both between countries and international organizations and among international organizations themselves; additional problems of burden on national statistical offices; and instances of non-relevance or doubts about the statistical validity of certain indicators. These problems caused the Economic and Social Council to turn to the United Nations Statistical Commission to solve the issue of identifying a limited set of core indicators to be used for monitoring progress towards the development goals. In this regard, progress towards the achievement of goals is usually assessed against the benchmark year 1990, with goals generally set to be achieved by 2015.

3. The Statistical Commission decided to form a "Friends of the Chair" expert group on Development Indicators in order to carry out the work. The "Friends of the Chair" expert group was formed with representatives of national statistical offices worldwide, including some from the Asia-Pacific region, and was chaired by Mr Tim Holt. The process reviewed the goals of the conferences, existing indicator frameworks, and the indicators themselves. The "Friends of the Chair" also benefited from comments and feedback that were provided to the Statistical Commission by other international statistical players, including the ESCAP Committee on Statistics; the Committee's views were most recently conveyed at the thirty-first session of the Statistical Commission, held in March 2001.

4. The "Friends of the Chair" last met at the end of October 2001, and finalized the review of a comprehensive framework of indicators that could serve to monitor achievement towards the goals set by several global conferences and the Millennium Summit. The Working Group may wish to note that a meeting, the "United Nations Workshop on Development Indicators for the ASEAN Countries", took place in Manila from 1 to 5 October 2001, prior to the final meeting of the "Friends of the Chair", and the meeting's views were brought to the attention of the "Friends of the Chair" in its final review. That meeting was organized by the United Nations Statistics Division as part of a broader project for strengthening the statistical capacity of ASEAN countries, and was attended by nine ASEAN countries. The ESCAP secretariat also attended and presented its views on the issue.

5. The issue of Development Indicators will be discussed, but perhaps not entirely settled, at the 2002 session of the Statistical Commission. The final report of the "Friends of the Chair", completed as this paper was being written, appears in the Annex. The list of indicators that have been identified by the "Friends of the Chair" is divided into three tiers, with the first tier being the most important, and the second and third tiers having lower priority, even though they are all necessary for providing a complete picture of the state of development of a country. Indicators on human rights or on governance were found to be either of a non-statistical nature or as having inadequate technical validity; therefore they have been excluded from the framework. The "Friends of the Chair" recommended that further work be done on indicators in these two fields. The Working Group's attention is drawn to the recommendations of the "Friends of the Chair" which are summarized after paragraph 110 of the Annex (page 31). The Working Group's views and comments would be valuable input to discussions on the topic at the next session of the Statistical Commission in March 2002.

6. Development Indicators have already been discussed at previous sessions of the Committee on Statistics and of the Working Group of Statistical Experts. The main conclusions that emerged on this topic are as follows:

• National Statistical Offices (NSOs) of the region have scarce resources that must not be diverted from national priorities;
• Analysis and policy making require statistics of good quality;
• International organizations mandated to monitor progress towards the development goals should not increase the data provision burden on member countries, and every attempt should be made to reduce it;
• International organizations must coordinate among themselves and avoid overlap of work;
• International organizations and their regional agencies should give priority to strengthening statistical capacity of those members which need technical assistance in developing their own statistical systems;
• Among the various lists of indicators none emerged as uniformly better than the others, but it was noted that the Common Country Assessment (CCA) had the widest coverage among the frameworks that were examined;
• Composite indicators, such as the Human Development Index (HDI) of the United Nations Development Programme (UNDP), have a role in advocacy, but are of limited utility as far as decision making and analysis are concerned, and therefore need to be complemented by other sets of statistics;
• Countries need to be involved in the process of selecting the indicators to monitor the development goals;
• The selected indicators must take into account the needs of national as well as international users;
• Comparisons of living standards should be based on Purchasing Power Parities (PPPs).

II. ISSUES FOR DISCUSSION

A. Compliance

7. One of the issues which needs particular attention is that of compliance by signatory countries on the compilation of indicators for monitoring the progress towards the agreed development goals. Governments that subscribe to international declarations, such as the development goals, should also provide their national statistical systems with adequate resources for carrying out the necessary monitoring work. The Working Group might wish to recommend to governments of the region that in order to achieve full compliance with the goals committed to, their statistical offices should be supplied with the necessary funding support.

8. This particular issue also calls for more dialogue between statisticians and policy-makers, as part of this political process; statisticians could provide at an early stage the needed technical support in identifying what is easily, or at least feasibly, collected and what is not. Thus, the issue of dialogue at national level between official statisticians and policy makers remains of prime importance.

9. Another issue of importance is the potential conflict that might arise between country priorities and the three-tier classification recommended by the "Friends of the Chair". There may well be, for instance, situations where indicators belonging to a tier with lower priority in the classification have the highest importance in a particular country's context. The converse is also possible. Thus the three-tier classification, while it tries to take into account information needs from a global point of view, could not be expected to be perfectly suited to all countries. The Working Group might be interested to comment on these potentially conflicting needs.

B. Standardization and harmonization

10. Development Indicators serve two main categories of users: national and international. In designing national plans, or in monitoring the effectiveness of policies, indicators must be consistent among themselves, at least within the national context. One of the problems that countries may face is that of internal consistency, which usually has implications at international level as well. Problems arise when different national agencies collect data and compile statistics, in the process often not adhering to international standards, which are not strictly necessary for national use only. When those indicators need to be used at international level, differences in definitions, practices, quality of sources, and so forth, have a strong impact on the comparability of statistics.

11. With increasing globalization, countries are increasingly interested in how they stand in the international community. It is important that indicators be more closely harmonized among countries. Actually, such adherence of concepts and methodologies to international standards already exists, and is widely accepted as a good characteristic, in economic statistics fields like trade statistics or national accounts. Harmonization is needed between other areas of statistics too: social, demographic, and environmental statistics are fields where stronger efforts at standardization need to be made.

12. The Working Group might wish to reaffirm that Development Indicators should be harmonized in such a way that they serve equally well both national and international needs.

C. Compilation of indicators

13. At the Manila meeting mentioned in paragraph 4, it was noted that not all countries compile the indicators that have been selected for the monitoring of development goals. Sometimes, perhaps because of lack of interest among users, the indicators are not compiled, even though the raw or the underlying data are already available to the statistical offices. In these cases a simple ratio, or other basic manipulation of raw data, could supply the missing indicators. In other cases, relatively sophisticated or time-consuming operations would be needed in order to compute the indicators. Some national statistical offices in the region have also highlighted the fact that sometimes data are available to line ministries, or other government agencies, but not to the NSOs; in such cases it is crucial that good collaboration and communication between government agencies is pursued, especially in the context of decentralized statistical systems. In other instances slight changes to existing survey questionnaires might enable the compilation of indicators, but resistance to changing long-running series has to be overcome if statistical series relevant to today's needs are to be produced. In all such cases, there seems to be a need for a national statistical coordinating agency, which should be enabled by means of an adequate legal framework to manage the compilation of statistical indicators for the country; this would help to ensure that indicators are reliable, timely, relevant and used for monitoring national policies.

14. Therefore, the Working Group might comment on the opportunity for countries to broaden the compilation of indicators to those cases where only slight changes to current practices would be needed. The Working Group might also wish to invite countries to promote arrangements for exchange of data among government agencies, and to attach sufficient metadata to their statistics.

D. Reporting

15. Reporting arrangements on Development Indicators have not been finalized yet, but given the fact that the United Nations Statistics Division has to report to the General Assembly on the level of achievement of the development goals, they will certainly be heavily involved in the international collection of data in this field.

16. Several of the Development Indicators already feature in the data collection activities of UNSD and of other international statistical players. In these cases there will be no additional reporting burden on countries.

17. Indicators that are not part of existing data collection activities will also have to be reported. In order to avoid the risk of overburdening countries, international organizations should expand the existing data sharing arrangements that aim to minimize multiple requests for the same statistics.

18. The Working Group may note that, on their part, countries should also assign an agency to report data to the designated international collector. Government agencies will anyway keep their existing relationships with specialized international organizations like the World Trade Organization (WTO), the United Nations Children Fund (UNICEF), and so forth, and arrangements are necessary in order to have coordinated relationships with international organizations.

III. THE ROLE OF ESCAP

19. The Working Group is invited to comment on the best role for the ESCAP secretariat in this field. The international and national roles seem to be well assigned, as the United Nations Statistics Division has been supporting the efforts of the "Friends of the Chair" expert group towards the identification of a unique set of indicators able to serve the developmental goals set out in United Nations global conferences or in the Millennium Declaration. An additional role of UNSD would seem to be that of international compiler. At national level the United Nations Development Programme might facilitate the collection of raw data through the Country Common Assessment set of indicators, and possibly integrate these raw data with other sources available from countries. Data would be then collected, processed and disseminated at international level by UNSD.

20. This leaves the statistical offices of the Regional Commissions, such as the Statistics Division of ESCAP, with a role that in practical terms remains to be fully elaborated. What emerged from the "United Nations Workshop on Development Indicators for ASEAN countries" was that the role of regional commissions could be two-fold. The first important contribution that regional commissions could give to the process could be on controlling and supplying metadata. This would be needed as a support for the enormous amount of data and information that UNSD will have to collect from all over the world. It was felt that support in collecting metadata information by regional commissions, which are closer and know better the characteristics of their regions, could be a useful role. The second role would be continuing the regional programmes of statistical capacity building. In fact it was recognized that only sustainable capacity in statistics could enable countries to produce the needed indicators. Therefore, Regional Commissions should pursue initiatives that are designed to strengthen statistical capacity rather than promoting ad hoc data collection initiatives. Additionally, the ESCAP secretariat believes that it could integrate the first tier of Development Indicators in its regular publications. Further, as described in document STAT/WGSE.12/2, the Poverty Centre/Unit to which the Statistics Division is contributing will be undertaking analytical work to make country-level indicators more comparable for the purpose of regional synthesis.

21. The secretariat suggests that it might pursue improvement of the coordination of its activities, both data and statistical capacity building work, with the regional arms of concerned international organizations. Furthermore, ESCAP will continue to coordinate its statistical capacity building activities with organizations operating in the region such as ASEAN, the Asian Development Bank (ADB), and the Secretariat of the Pacific Community (SPC). Views on the desirability and feasibility of extending this traditional coordination among regional statistical players to monitoring progress towards the Millennium Development Goals are also sought.

22. The Working Group may wish to comment and advise the secretariat on the above aspects of its role, and on any other means by which ESCAP might contribute to the Development Indicators process.

IV. CONCLUDING REMARKS

23. The Working Group's attention is drawn to the recommendations of the "Friends of the Chair" which are summarized after paragraph 110 of the Annex (page 31). The Working Group's views and comments would be valuable input to discussions on the topic at the next session of the Statistical Commission in March 2002. - Para 5.

24. The Working Group might wish to recommend to governments of the region that in order to achieve full compliance with the goals committed to, their statistical offices should be supplied with the necessary funding support - Para 7.

25. The Working Group might be interested to comment on the potential conflict between country priorities and the three-tier classification recommended by the "Friends of the Chair" - Para 9.

26. The Working Group might wish to reaffirm that Development Indicators should be harmonized in such a way that they serve equally well both national and international needs. - Para 12.

27. The Working Group might comment on the opportunity for countries to broaden the compilation of indicators to those cases where only slight changes to current practices would be needed. It might also wish to invite countries to promote arrangements for exchange of data among government agencies, and to attach sufficient metadata to their statistics. - Para 14.

28. The Working Group may invite countries to enhance their arrangements for having coordinated relationships between their various statistical units and international organizations. - Para 18.

29. The Working Group may wish to comment and advise the secretariat on its suggested role and on any other means by which ESCAP might contribute to the Development Indicators process. The views of the Working Group on the desirability and feasibility of extending the traditional coordination of activities among regional statistical players to monitoring progress towards the Millennium Development Goals are also sought. - Paras 21 and 22.

Annex

AN ASSESSMENT OF THE STATISTICAL INDICATORS DERIVED FROM UNITED NATIONS SUMMIT MEETINGS¹

Prepared by
Friends of the Chair of the United Nations Statistical Commission for the 2002 UNSC Meeting, November, 2001

EXECUTIVE SUMMARY

1. This report results from a request from the Economic and Social Council (ECOSOC) to the UN Statistical Commission (UNSC). It arose out of a concern about the large number of seemingly uncoordinated demands for statistical indicators to monitor a wide range of policy issues that had been agreed at various UN summits and major conferences. At the 2000 meeting (E/CN 3/2000/27) ECOSOC turned to the Statistical Commission (UNSC) as its authoritative technical advisory body to:

• Provide leadership in the field of conference indicators;
• Conduct an in-depth technical analysis of conference indicators;
• Make recommendations regarding a limited list of conference indicators, and
• Develop and recommend to the Council a mechanism of statistical review for future proposed indicators.

2. The technical assessment was carried out on over 280 statistical indicators derived from UN summits and major conferences held over the last 10 years. Seven expert groups were established to cover the common division of policy (e.g. economics, health, education etc) that is reflected in Ministerial responsibility in most countries. The expert groups had members drawn from many countries. The indicators were assessed on technical criteria and the relevance to the policy goals. A web site has been created that contains all of the indicators and the technical assessment of each.

3. In response to the request for a limited list of conference indicators, we propose an indicator framework containing three priority tiers. Each tier contains about 50 statistical indicators. In addition a further category contains indicators that would be useful for a more detailed understanding of any policy area. The framework is arranged to reflect the major policy areas referred to above. However important additional policy areas cut across this arrangement and typically cut across Government Department policy responsibilities in many countries. Such policy areas include Poverty, Gender and Child Welfare. Indicators covering these issues are contained within the framework.

4. Also there are areas in which the indicators need improvement or indeed simply do not exist and need to be developed (e.g. indicators for Human Rights and Good Governance). These tasks were too extensive to undertake in the time available. However we make recommendations to the UNSC to establish processes to do this.

5. A correspondence between the proposed framework and the existing high-level indicator sets is provided.

6. The development of statistical indicators and the statistical capacity that allows higher standards to be met are dynamic. Initiatives exist within international agencies that will require the proposed framework to be reviewed if it is to remain relevant to changing needs. Hence the framework must be kept under review and we make recommendations to achieve this and to improve co-ordination between international agencies.

7. Finally we turn to the question of future summits and major conferences and the need to propose mechanisms that will allow further development of the framework in response to emerging needs. The existing arrangements for indicator development are clearly unsatisfactory. We recommend procedures to improve this situation.

8. These are based on the recognition that the stakeholders in the indicator programme span policy officials and statisticians in both international organisations and member states. Mechanisms are needed to ensure that all can play a full part in indicator development and priority setting. The process of turning a policy goal into a statistical indicator that is feasible to measure and technically sound should involve all stakeholders.

9. Another important issue is the level of statistical capacity needed for countries to support the information needs of national and global policies. Developing statistical capacity goes beyond financial and technical support from international donors narrowly focussed on specific statistical production to monitor a specific global policy. It calls for more support for systemic development.

10. A further issue is the reconciliation of information needs for national and global purposes. In the long run, financial support for statistical programmes must depend upon national rather than international provision. This in turn depends upon national governments using and valuing statistical information in support of policy development, policy monitoring and good public administration in general. Hence it is essential that the national statistical system supports national policy goals.

11. The report contains a series of recommendations that are intended to address these issues. In particular mechanisms are proposed to ensure greater participation for member states in the development and adoption of statistical indicators for global and national purposes.

1. INTRODUCTION

12. In the last decade or so United Nations summits and major conferences (averaging almost two per year) have covered a wide range of economic and social issues. These meetings have resulted in declarations related to future goals and targets that have been endorsed by member states and are intended to improve the wellbeing of the world's population. Goals and targets call for a commitment to monitor progress towards them and, consequently indicators (usually statistical indicators) have been identified in relation to each goal. The intention is to monitor and report on these so that progress towards the declared goals and targets can be measured.

13. However, there is concern that this process has gone on with too little co-ordination between officials concerned with the separate UN summits and major conferences in terms of the number and choice of indicators to be monitored. The meetings have varied considerably in terms of the number of resulting indicators (ranging from a handful or less to as many as 70 being identified from a single UN conference). In total over 280 indicators have been identified.

14. The perception is that this uncoordinated process has resulted in a plethora of indicators of different levels of importance in policy terms. Also there is potential for confusion among users because of an apparent inconsistency and lack of coherence among the indicators. The ongoing addition of indicators has resulted too in a large demand for statistical information from each member state: a demand that has to be set alongside the demands for statistical information for national policy purposes. For countries with less well-developed statistical infrastructure this total demand can be disproportionate to the resources available to meet it.

15. Attempts have been made to distil core sets of indicators that might be afforded greater recognition and therefore higher priority. The United Nations Statistical Commission (UNSC) identified the Minimum National Data Set (MNDS: 15 indicators). The OECD Development Assistance Committee - in co-operation with the UN, World Bank and IMF - identified the International Development Goals (IDG: 21 indicators). This set drew heavily on international summits up to 1995. The United Nations Development Group identified indicators to support Common Country Assessment again based on an analysis of the requirements of UN summits (UNDAF-CCA: 57 indicators). Similarly the need to promote and assess sustainable development has resulted in an additional set (CSD: 57 indicators). There is also Basic Social Services for All (BSSA: 12 indicators). The group has also been aware of the work within the European Union on 35 Structural Indicators.

16. And this process goes on. While this report was in preparation the choice of statistical indicators to support the UN Millennium Goals was announced (MDG: 48 indicators), and constitutes another high-level set of indicators which will be monitored.

17. The Economic and Social Council (ECOSOC) considered these issues in 1999 and 2000 and there is a general recognition that better co-ordination is needed and that full participation and ownership by Member States was needed in all stages of indicator development. At the 2000 meeting (E/CN.3/2000/27) ECOSOC turned to the Statistical Commission (UNSC) as its authoritative technical advisory body to:

• Provide leadership in the field of conference indicators;
• Conduct an in-depth technical analysis of conference indicators;
• Make recommendations regarding a limited list of conference indicators, and
• Develop and recommend to the Council a mechanism of statistical review for future proposed indicators.

18. As a consequence the UNSC, at its 2001 meeting, established a group of 'Friends of the Chair' to consider the issues further and to report back to the UNSC at the 2002 meeting so that the UNSC could report to ECOSOC.

19. The members of the group were:

Tim Holt (United Kingdom): Chair
Guest Charumbira (Botswana)
Claudia Cingolani (Italy)
Francisco Guillen (Mexico)
Hasan Abu Libdeh (Palestine)
Jil Matheson (UK)
Yue Renfeng (China)
Hussain Shakhatreh (Jordan)
Bounthavy Sisouphantong (Laos)
Ken Tallis (Australia)

20. In order to carry out the required in-depth technical review the group subdivided the 280 identified indicators into 7 domains:

• Demography,
• Health and Nutrition,
• Environment and Energy,
• Economics and Poverty,
• Employment and Labour,
• Education, and
• Other Social Indicators.

Seven indicators related to Human Rights and Good Governance were excluded from this framework. This was because all of the indicators were qualitative in nature and no statistical indicators had been identified. We return to this point in due course.

21. The Domains represent major divisions of policy responsibility that are commonly reflected by separate Ministries in many countries (e.g. Economics, Health, Education etc.). Additionally there will be important cross-cutting policy areas such as Poverty, Child Welfare or Gender that are distributed across these Domains.

22. We considered the requirement to make recommendations about a limited number of indicators and, following the UNSC discussion in 2001, approached this by establishing a hierarchy of indicators containing three priority tiers and a category of 'additional' indicators. The first tier contains statistical indicators that might be regarded as of the highest priority and are essential for broad monitoring. This tier includes a small number of indicators in each domain. The second and third tiers contain additional indicators that progressively add to the overall picture and include indicators that allude to additional policy priorities. A fuller description is given in section 3.

23. We are very aware of the need for countries to reconcile their statistical needs for national policy purposes with the international requirements. The hierarchical structure offered is not meant to be mandatory nor to impose a straightjacket on member states although we think that all countries should be encouraged to compile all indicators within the first tier unless there are overwhelming national reasons not to do so. In our view the second tier and many of the indicators in the third tier would be valuable in most countries. However, it is likely that countries with particular concerns or policy initiatives would wish to collect extensive statistics for some domains (including those in the 'additional' indicator category) and less for others. Also the statistical requirements for national policy purposes - in effect most of the output of the national statistical system - would go beyond the indicators identified in the framework. Nonetheless the framework is intended to enable countries to assess their statistical priorities and to reconcile the statistics that are needed for national purposes with the global requirements. As such we hope that countries will find the framework useful.

24. For each domain an expert group was established drawn from member states across the world. Some members of each expert group were official statisticians and others were more concerned with policy issues.

25. In addition useful discussions were held with representatives of UNSD, UNFPA, UNESCO, ILO, OECD and World Bank and we attended the ACC Subcommittee on Statistical Activities meeting in September 2001 at which an initial draft of our report was discussed.

26. As a further consultation phase a draft version of the report was circulated to all National Statistical Offices, UN Regional Commissions and International Agencies. It was also placed on the UNSD web site. The final version of the report takes account of the responses received to this consultation. Representatives of a number of international agencies also attended the meeting of the Friends of the Chair at which the draft report was effectively finalised. We acknowledge and thank all contributors but the final responsibility for this report rests with the Friends of the Chair.

2. KEY ISSUES

27. The request from ECOSOC to the UNSC and the terms of reference established for the Friends of the Chair reflect concern over the current process for identifying indicators. This concern includes the lack of co-ordination between stakeholders, insufficient involvement by member states in the process and the lack of structure of the resulting indicator sets. A number of key issues need to be recognised and taken into account.

2.1 The Stakeholders, Competing Needs, Statistical Capacity and The Burden on Countries

The Stakeholders

28. Identifying statistical indicators for monitoring purposes is neither a pure policy nor a pure statistical issue. The basic expression of the policy goal must drive the monitoring requirement but turning that expression into a statistical indicator that will be relevant, reliable and acceptable to the various stakeholders is a statistical function. The tension between the policy view of what is needed and the statistical view of what is feasible and technically sound should be resolved by joint determination.

29. A second stakeholder issue is that although the statistical indicators that are derived from UN conferences and summits are motivated by international needs, they are based on policy issues that have to be reflected in the national policy agenda if the desired progress is to be achieved. However, there can be differences between national and international priorities and the need to reconcile the national and international priorities needs to be addressed.

30. The third stakeholder issue rests on the simple fact that most of the statistical indicators are derived from national statistical programmes. These are predominantly funded from national resources and reflect a range of user needs of which the international need is but one. It falls to national statisticians to try to respond to often disparate user needs within the resources available. Their ability to respond will depend heavily on the general level of statistical capacity in the country and the extent to which additional demands create a response burden on countries or whether existing statistical sources can be used or adapted to meet additional needs. Thus national statisticians are stakeholders. Their expertise is different from statisticians working within international agencies and they have an important contribution to make to the process of developing statistical indicators.

National and International Priorities

31. Relevance is a dominating requirement of statistical information. If the statistics are not relevant to the policy need, then they will not command the attention, nor have the impact that they should. Failure to meet national needs, in particular, will undermine the requirement to develop sustainable statistical capacity since in the long term this must depend on national governmental funding and support. It will also undermine evidence-based policy as a basis for good governance and public administration within countries. From the UN perspective this would, as a consequence, undermine the provision of statistical indicators for international monitoring purposes.

32. To an extent the tension between national and international needs may be reduced if the statistical system is rich enough and flexible enough to support diverse needs. For example a well-designed Household Budget Survey can estimate the proportion of the population below an international poverty standard and against a national poverty standard. In such cases the conflict between national and international requirements is avoidable. In other cases the resolution may call for additional resources - to collect a wider range of data, or to fund larger sample sizes so as to meet competing needs. In our view all efforts should be made to reconcile national and international needs so as to support both. This implies that countries recognise the need to support international needs and international agencies accept the need to support statistical activities focussed on national, as well as international, needs. Investment in modular frameworks or analytical capacity that allows countries to exploit core sets of survey data for a variety of purposes would be valuable.

33. Thus any rationalised set of indicators should be applicable (or readily adaptable) to both national and international priorities. In the time available we have not been able to assess this as comprehensively as we would have wished although we have drawn upon the experience of the members of the expert groups and international agencies. In our view this assessment should be done more systematically before the proposed framework of indicators and their priority levels are 'set in stone'. The recommendations that we make for the UNSC to maintain the indicator framework will permit this.

Statistical Capacity

34. The ability to produce consistent, reliable statistical information on an ongoing basis requires a sustained statistical capacity. This requirement is not a one-off capability but implies the ability to produce statistics on a regular basis and with the timeliness needed.

35. In particular a sound statistical infrastructure is essential. By this we mean:

• Underpinning systems to create and maintain sampling frames for business and household surveys.
• A critical mass of ongoing statistical activities: survey design, data collection and analysis in order to nurture the basic professional skills.
• The technical and professional capacity to maintain and develop systems in accordance with international standards as these are developed over time.
• A developed analytic capacity.
• Adequate statistical frameworks and IT infrastructure.
• Good management to make the most use of the resources that are available.
• All of the above embedded within a wider legal and administrative structure that recognises the importance of good statistical information and the need to sustain the conditions in which it can be produced with high professionalism and integrity, consistent with the UN Fundamental Principles of Official Statistics.

36. Without this core capacity and the ongoing resources to support it, neither the statistical needs of the country nor those of the international community will be reliably served. In many countries adequate ongoing financial support is a key issue. Where this core capacity is fragile the sporadic provision of additional funds to satisfy a particular statistical need will be much less effective and is no substitute for what one might term 'statistical sustainability'.

37. In this regard, statistical indicators need to be viewed as the end product of often complex statistical infrastructures that are essential if the indicators are to be produced with adequate quality. Population estimates for example, that are fundamental to many indicators that are expressed as rates or per capita estimates, depend on periodic Censuses to provide benchmarks and on systems of vital registration or other sources to permit inter-censal population estimates. Many social statistics depend upon social surveys that need sustained expertise if they are to be well conducted. Complex measures such as GDP require an extensive framework of business surveys, administrative sources and underpinning infrastructure if the statistics are to be of adequate quality. Too much emphasis has been placed on the indicators (which are the end product) and too little on the statistical sources and infrastructure that underpin these. The majority of aid agencies and donors are perceived to provide aid to conduct studies needed to fulfil their objectives without considering the national capacity building.

38. Countries and international donors need to recognise that each statistical initiative depends on the core statistical capacity within the country and that internationally sponsored activities must contribute to this sustainable capacity. It is essential that these activities support both national and international statistical needs rather than being perceived as being driven by international goals alone. The effective use of statistical information within national governments needs to be promoted and ECOSOC and international donors have an important role to play. This is essential if the statistical system is to command consistent financial and political support from the national government of the day.

39. It is important to note that donor resources are often tied to specific (international) objectives while being characterised as supporting statistical capacity building. Whilst these may provide financial support there is a frequently expressed concern that such programmes may consume the statistical expertise available within the country and so distort the overall priorities. If this is so it represent not statistical capacity building but statistical capacity diversion. It is important the donor-supported programmes genuinely add to the sustainable resource within the country.

40. We believe that an indicator of statistical capacity should be developed and monitored. This measure could be based on the level of regular statistical activity within a country, an ongoing critical mass of survey taking and statistical analysis and the existence of basic elements of statistical infrastructure. A task team within the PARIS21 initiative has this work in hand, building on the IMF's Data Quality Assessment Framework. One concern is that this initiative, and the resulting indicator should not be dominated by economic statistics but should span the wide range of statistical areas covered by national statistical systems and the indicators considered in this report. Also the membership of the task force has no country representation. We RECOMMEND that this be remedied and that the eventual proposals be made to the UNSC.

41. Building and monitoring statistical capacity is a systemic issue. In our recommendations we have taken account of this in several ways. First we have focussed on indicators (especially in the first two tiers of the framework) that should be feasible for most countries to compile (perhaps initially with statistical assistance but as part of the ongoing statistical programme in due course). Second we propose a systematic assessment of the availability and frequency of indicators in the priority categories. Third we have in some cases defined a sequence of successive approximations to ideal indicators that countries might compile as their statistical capacities develop. We commend this approach for the maintenance and development of the framework.

The Response Burden on Countries

42. A frequently heard concern is that the uncoordinated demand for a wide range of statistical indicators places a burden on National Statistical Offices (NSO's) that cannot be responded to. Linked to this view is that such a burden is incompatible with the national statistical needs and diverts scarce resources (skills as well as finance) from other priorities. NSO's generally wish to respond to all expressed needs so long as these are technically well founded but the concern is that these cannot be met within the resources (both financial and skills) available.

43. International agencies have taken steps in recent years to align their statistical requirements and to improve the co-ordination when requesting statistics from countries, particularly by establishing joint data collection mechanisms. This process should go on with a view to streamlining the demand on countries further.

44. There are two solutions to the general problem of burgeoning demand: to reduce the demand or increase the resources and hence the statistical capacity. The latter would serve user needs better and is preferred but in the short term both are needed:

Managing demand

A number of steps will help:

• Reconciling the international and national statistical requirements will reduce the burden.
• Establishing a hierarchical structure of statistical indicators so that nations may determine their priorities more systematically.
• Producing more guidance on best practice and measurement processes.
• Further co-ordination between international agencies on data needs and joint data collection from member states.
• Increasing Resources and Enhancing Capacity
• But in addition increasing the funding available for the less well-developed statistical offices is essential and this will be needed on an ongoing basis. In the long term this must come from within the country but in the short term often comes as a partnership between national governments and international donors.
• A climate of support for the statistical system within the country will be developed only if national governments see statistical information as essential in support of national policies and good governance. In seeking efficient and effective public administration governments need to view statistics as part of the solution rather than simply as an additional claim on public expenditure.
• Developing a core statistical infrastructure and a critical mass of professional and technical skills is essential.
• In the case of the donors, they must ensure that all statistical activities strengthen the sustainable statistical capacity and, by taking account of national needs, strengthen the value that national governments place on statistics.

45. The resource implications for new statistical outputs may be very different in different countries and depend on the existing level of statistical capacity. From the lowest additional cost to the highest one may set out a hierarchy of resource implications.

• In some cases it is simply a question of analysing existing data in a different way in order to provide the required output. An analysis by gender is an example of this so long as the basic information on the subject's gender is available for each data record. In such cases the resource requirement (assuming professional skills are available) is small.
• An approach more demanding of professional skills is the use of modelling, synthetic estimation and other analytical techniques applied to exploit existing data sources for new purposes. The financial cost may be low but the technical knowledge to produce high quality outputs is significant.
• In other cases the new requirement may call for a small number of additional items to be collected and analysed using an existing survey. The resource implications are a little higher but so long as the core statistical capacity is in place it is generally feasible to support the requirement.
• More seriously the new requirement may call for a substantial increase in the sample sizes employed. Regional and other sub-national estimates that are often required for national purposes are a good example of this; estimates of population subgroups is another. This can add significantly to the existing costs and the need for analytic skills.
• Even more seriously, the new requirement may call for an entirely new data collection system: for example a new household survey or a new business survey. This is generally an order of magnitude more demanding in terms of time for development, in terms of costs including interviewer and data processing costs, and also in terms of diverting often scarce professional and technical skills from existing programmes to the new survey. In order to avoid this issue, there are examples where existing surveys become overburdened with competing and potentially conflicting data requirements to the extent that one must question whether they are manageable. Also the burden on the respondents who participate in the survey is very severe.
• Where the primary data source is an administrative system new needs may call for the system (or the underpinning software) to be redeveloped. This can be a major undertaking unless the administrative system is being redeveloped for other purposes but for some statistical uses this may be the best long-term strategy for a statistical office.
• Finally some new requirements may call for an infrastructure which simply does not exist in a particular country. For example some administrative systems (e.g. vital registration) may be non-existent or in such poor state that their use for statistical purposes is impractical. Or measurement processes (for example as are often used for some environmental indicators) may not exist. In such cases the basic infrastructure must be established and this can be a long and expensive process.

46. In general the better the core statistical infrastructure the better a country can respond to new statistical requirements. If the national and international goals are to be met then a strengthening of the core is required in many countries.

47. In particular the statistical infrastructure to support estimates of GDP and vital statistics is particularly demanding and complex. Ideally it requires both survey capability and access to effective administrative systems as data sources. Both are cornerstones of the whole indicator programme since many indicators make use of these.

48. Hence we make a set of inter-related RECOMMENDATIONS:

• WGSE, 12th session

2.2 Quality and Technical Properties, Continuity and Change

Quality and Technical Properties

49. It is important that the chosen statistical indicators are relevant to the purpose and satisfy technical criteria. Measurement for statistical purposes is an exacting discipline, calling for specialist development. Definitions and concepts need to be as precise as possible consistent with the intended use. The resulting statistics need to satisfy statistical quality criteria and conform to international standards where established. The development of good quality statistical indicators takes time and may well require field tests and evaluation before a suitable indicator is developed.

50. Over the years, largely independent of the need to monitor Conference goals countries have developed suites of core statistics such as population estimates, GDP or life expectancy. These have been developed through extensive processes over time; international guidelines exist to support best practice and the statistical properties are relatively well understood. So long as such indicators are relevant to the conference goal they are readily available for monitoring purposes. Nevertheless, even for indicators such as these, actual quality varies between one country and another depending on the strength of the statistical infrastructure in each country and the basic statistical capacity.

51. But for new policy areas such as Human Rights and Good Governance no established statistical indicators exist. Their development will take time and the process needs to involve statisticians and policy officials.

52. An additional difficulty for some newly developed indicators is that targets related to future improvements from a baseline date may be agreed (for example reducing by a third the incidence of a particular event within a period of 10 years). If the statistical indicator that is used to monitor this target is not widely available at the baseline time then there is no base value from which to measure progress. There is no easy solution to this problem but when such targets are adopted there is a need for the conference to recognise the need to support the development of baseline measures. If not, it risks bringing discredit to the whole process of target setting. We RECOMMEND that the need for baseline measures be taken into account when targets are adopted that require change to be measured from a specific point in time.

Continuity and Change

53. For all statistics there needs to be a regular process of review and development. As the economic and social environment change so the statistics that are used to monitor development need to change if they are to capture the new situation and so remain relevant. This is as true for statistical indicators monitoring goals as it is for all other statistics. If this process of review and renewal does not occur the statistical indicators will become increasingly less relevant. For global statistics there is another reason for continuous development. The need to establish an indicator quickly may reasonably mean that technical standards are chosen to reflect the reality of what can be achieved in the short term. However, as statistical capacity develops the technical standards that one may apply to any indicator may be increased: definitions may be refined and the quality of the indicator at a global level improved. This process creates a tension between continuity over time and necessary change to improve quality and relevance. This balance needs to be recognised and often will call for continuity but there are established methods, such as statistical revisions, to address the need for consistency of time series.

54. We RECOMMEND

• that all statistical indicators should be subject to periodic review and improvement and
• that when such a review results in change, an approach be provided to support countries in moving to the improved indicator while maintaining continuity with the recent past.

3. A TECHNICAL ASSESSMENT AND FRAMEWORK FOR INDICATORS

55. From the UN Summits of the 1990's about 280 separate indicators were identified of which the overwhelming majority were statistical in nature. This list was based on 15 global conferences listed in the 1999 Report of the Secretary-General. In consultation with the ECOSOC secretariat this list was augmented to take account of the World Conference on Education for All (Dakar 2000) and special sessions of the General Assembly that followed other conferences (e.g. Fourth World Conference on Women, Beijing 1995) up until March 2001. The list included indicators that were identified by cross-conference initiatives: MNSDS, UNDAF-CCA, BSSA and IDG. After this no more conferences were added but the 'road map towards implementation of the UN Millennium Declaration' was included.

56. These indicators covered a wide range of topics. But this set, albeit large, does not include all of the statistical indicators that have been identified as desirable by the UN and other international organisations. It includes only those indicators identified from the UN summits and major conferences. Future meetings will surely identify new areas that require policy monitoring. We address this through the section on future processes. This section is essentially concerned with the 280 indicators identified.

The Expert Groups and their Task

57. As described in paragraph X the indicators were subdivided into 7 Domains and expert groups established for each domain (Demography, Health and Nutrition, Environment and Energy, Economics and Poverty, Employment and Labour, Education, Other Social).

58. The expert groups, with the support of UNSD staff, carried out a technical assessment of each indicator and this material is available on the UNSD Website (address) and will be maintained in future. This is the source that contains detailed definitions and specifications for each indicator and we RECOMMEND that the UNSD web site be the definitive source of technical information about the indicators.

59. This report contains a summary of the findings of the expert groups. A Room Document: Technical Assessment of Statistical Indicators has also been prepared to provide a more comprehensive picture of the work of the expert groups.

60. The expert groups identified Sub-Domains within each Domain as being relatively self-contained separate policy areas. Indicator were allocated to one of three priority tiers:

• A small number of Indicators were allocated to the first tier containing indicators essential for broad monitoring and which all countries are encouraged to compile.
• The second tier contains indicators that add to the information contained in the first tier and which help to convey a fuller picture. These indicators are likely to be vital for both national policy monitoring and internationally comparative purposes.
• The third tier of indicators is needed to gain a more comprehensive picture of the situation in any Domain (depending on national circumstances).

Most of the remaining indicators were allocated to a fourth tier of 'additional' indicators. Many of these would be valuable for portraying additional aspects of the domain and illuminating policy areas further. Some indicators were excluded if there were overriding technical deficiencies or if an alternative was preferred.

The Criteria for Selecting Indicators

61. The process of selecting indicators must be grounded in the policy need but involves balancing a number of criteria surrounding the relevance to policy, the technical properties and current availability (or the feasibility, resource and statistical capacity implications of achieving an acceptable measure in a high proportion of countries). While one may aspire to the situation in which an indicator fully satisfies all of the criteria, in practice this will not be the case. One needs to consider the extent to which the indicator meets the criteria and make a judgement about whether failure against any criterion is of such overriding concern as to disqualify a particular indicator from use. A large number of criteria may be identified but we suggest that the most important are:

Policy Relevance

• Indicators must be relevant to the policy requirement.
• Indicators should measure the real policy objective (or provide a proxy measure that is adequate for policy monitoring).
• Indicators should normally have a global policy relevance.
• Indicators should be straightforward to interpret: changes over time in any direction should not be ambiguous in relation to the policy interpretation and significant differences between countries should be meaningful in terms of the policy goal.
• Technical Properties
• Technical properties of the indicator should be adequate for the purpose, recognising that change over time is often more important than the level of the indicator.
• Indicators that fail to cover the target population fully should have sufficient coverage to ensure that the indicator values are unlikely to mislead policy users (i.e. the potential bias as a measure of the true policy objective should be small).
• If possible, where indicators are difficult to measure for countries with less well-developed statistical capacity, simplified alternatives should be provided for use until the statistical capacity can support the more demanding measure.
• Indicators should be robust to institutional and cultural differences between countries and over time.
• Indicators should exhibit change over time at a rate that would support policy monitoring.
• Indicators should be produced with sufficient frequency and timeliness to support policy monitoring.
• Indicators should conform to international standards if these exist.

Parsimony, Availability and Cost

When considering additional indicators and policy objectives:

• Indicators already established within the priority framework (and in particular contained in the existing major sets) should be used wherever possible.
• Indicators added to the framework should not be closely correlated with other indicators already included and should reflect a new aspect of the policy issue.
• Where indicators are broken down into component indicators (e.g. mortality by cause or school enrolment by age), to merit separate inclusion in the framework the component indicators should: 
• Support separate policy objectives and/or
• Demonstrate important differences in time trends and/or
• Demonstrate important differences between comparable countries.
• The choice of indicator must have regard to the availability of data to support the indicator and the statistical capacities of a wide range of countries.
• Indicators that call for new data sources should not create burdens (cost, opportunity cost, skill requirements etc) disproportionate to the benefit for most countries.

The Criteria for Setting the Priority Levels

62. In our view the priority level for a particular indicator must be driven by the policy need. It must also take account of the technical soundness and data availability for the indicator and the relationship to other indicators within the framework (where relevant). It is natural for anyone associated with a particular topic to regard it as having overriding priority. The priority assessment needs to be strong enough and independent enough to weigh fairly any particular indicator and its policy objective and to resist 'priority inflation' over time. This process should involve policy officials and statisticians and take account of national and international priorities. We have made an attempt to achieve this in the proposed framework as a set of initial proposals but also recommend a process whereby this can be kept under review. We suggest the following criteria:

• First tier priority indicators should be the primary support for monitoring policies of the highest global and national importance. They represent the indicators that, no matter how limited the statistical capacity available, countries and international agencies would find essential for top-level monitoring of policy effectiveness. Whilst the number of indicators in this category should be driven by the policy importance we suggest that, as a guideline, any major domain (e.g. Economics or Health) should aim for no more than 4-8 tier 1 priority indicators. Most domains should have fewer.
• Tier 2 priority indicators should cover different policy objectives (different subdomains) from those covered by the highest priority indicators. These policy objectives should be of sufficient importance to merit a tier 2 priority indicator. Not all subdomains would necessarily do so. As a guideline we suggest that a major policy domain (e.g. Economics or Health) should aim for no more than 8-10 tier 2 priority indicators. Most domains should have fewer.
• Tier 3 priority indicators should support policy needs that are, albeit important, either subsidiary or judged to be less important than others. As a guideline we suggest that each major policy domain should aim for no more than 10-12 tier 3 indicators. Most domains should have fewer.
• The representation of indicators that relate to important cross-cutting issues such as Poverty, Gender or Child Welfare also needs to be taken into account.
• Indicators that support several policy goals should generally command a priority level that reflects this.

In our application of these criteria to create the proposed framework, we found a number of examples where the policy objective was such as to suggest allocation to a particular tier. But the inherent statistical weaknesses of the proposed indicator and/or measurement problems caused us to allocate the indicator to a lower tier. Explanations for these will be found in the technical assessment for each indicator on the web site.

The suggested numbers for each tier reflect the fact that the indicators are not intended to substitute for the mass of detailed statistical outputs from national statistical systems that support users' needs. They are intended as high level indicators for monitoring purposes. The suggested numbers reflect the existing levels of statistical capacity within a wide range of countries that are less well-developed. As the general level of statistical capacity rises, the potential to define a wider set of indicators can be reviewed.

63. In addition to the criteria set out above the expert groups were required to take specific account of the indicators contained in the main existing sets of high level indicators unless there was an overriding technical reason for recommending an alternative.

64. The question of availability was particularly problematic for the expert groups since it was impossible to carry out a detailed assessment of the availability of 280 indicators in all countries of the world in the time available. We RECOMMEND that the UNSD prepare a report to UNSC on the availability of indicators in the highest two tiers (and the third tier where information is available). The report should include an assessment of what might be needed to overcome the shortfall.

The Indicators Considered

65. In general we have considered the 280 indicators identified from the UN summits and major conferences. However in addition we have considered a small number of contextual indicators (such as population numbers in specified age groups by sex) that are essential to provide denominators for the other indicators. Also a very small number of other indicators that are integral to the production of those required.

66. There are more indicators that have been identified by international agencies or are in use inside many countries for policy monitoring purposes. These do not appear in the proposed framework.

67. Within the main framework indicators (or very close equivalents) that are contained in the major indicator sets (MNDS, UNDAF-CCA, IDG, BSSA, MDG and CSD) are referenced to the sets to which they belong. There is an element of judgement in this cross-referencing. In some cases there are technical differences between the indicator as described in the framework and corresponding indicators in the high level set.

68. In addition to the 7 Domains we have set aside a small number of indicators of Human Rights and Good Governance. Similarly two environmental indicators were not included since these, too, were not statistical and were therefore outside our remit.

69. The development of statistical indicators for Human Rights and Good Governance will not be easy and will take time. We RECOMMEND that the UNSC establish a mechanism (perhaps a City Group involving statisticians and others including policy officials) to develop statistical indicators of Human Rights and Good Governance. Whatever is established needs to take account of existing initiatives in this field and in particular follow-up activities from the IAOS conference held in Montreux. Whilst we recognise the importance of this area, we take the view that it would be better to 'get it right' rather than 'get it quick' if wide ownership of the indicators is to be established across the world.

General and Domain-Related Issues

70. Comparative Measures: International comparisons require that statistics be put on a basis that is immediately comparable and for this reason almost all of the indicators are presented as rates or proportions or in per capita terms. These require a denominator (often a population figure of some kind). Economic and some other measures use GDP as a denominator in the same way. This raises a number of important issues:

• The pervasive use of GDP and of population estimates in this way underlines the importance on the quality of these estimates, and the statistical infrastructure to support them, if a wide range of indicators are to be sufficiently reliable.
• Both GDP and Population estimates require a strong statistical capacity and infrastructure if they are to be regularly produced.
• Whilst the immediate population indicators call for population counts by gender and broad age group (0-4, 5-14, 15-64, 65+) the reality is that finer estimates are required to support a range of other indicators. For example 5-year age categories strengthen age specific mortality or fertility rates (and hence measures such as the Total Fertility Rate). Also other age groups are needed to support rates for indicators such as educational participation or HIV/AIDS infection rates.
• An added difficulty is that the numerator of such indicators and the population denominator are often provided from different sources within a country and may be inconsistent. Hence the rates, when calculated, may not be recognised within the lead policy Ministry. In extreme cases different population denominators may be used for different policy areas. This is clearly unsatisfactory and when it occurs may imply a systemic problem of consistency and quality assurance. International agencies have an important quality assurance role in identifying such situations and may act as a catalyst in helping countries to resolve them.

71. We RECOMMEND:

• That when considering statistical capacity, international donors and countries themselves take particular account of the importance of a core set of demographic statistics and GDP estimates as an integral component of many statistical indicators.
• That the need for coherent statistics used in the numerator and denominator of indicators is recognised and that international agencies work to identify inconsistencies and act as a catalyst in helping countries to resolve them.

72. There are a large number of indicators which are usually derived from administrative systems in countries where these are well established (e.g. mortality rates by cause, fertility rates, net enrolment rates in education and many health indicators concerned with health services and provision). In countries where these systems are unavailable survey based measures are available and widely used in which both the numerator and denominator of the indicator may be derived from survey estimates. In this case a special survey devoted to one particular area of interest (e.g. health and fertility history) could provide a wide range of indicator values. These could easily extend beyond those contained in the three priority levels of the framework and this is a viable possibility particularly when countries want a more comprehensive picture of a situation.

73. However, ad hoc surveys cannot provide the ongoing information needed to track important indicators. To ensure that critical information will be available on an ongoing basis it is necessary to invest resources into the statistical infrastructure. Such infrastructure to include administrative data bases as well as survey capabilities.

74. In addition the priority indicators should be few enough that all countries have the potential to produce them.

75. Meta-Data: This is essential if users are to understand any particular issues affecting the statistical indicator values for any country. Good meta-data (such as is required by the IMF's SDDS and GDDS) is a general requirement but there are specific situations when countries should ensure that specific meta-data is provided.

• When national priorities result in an indicator which is not fully comparable with those produced by other countries. Failure to provide informative meta-data will fail those users who seek to use the indicator for comparative purposes.
• Where national standards or targets are adopted (for example in setting a national poverty standard) the basis of this measure needs to be available to users.
• Population forecasts will depend crucially on the assumptions made about age-specific fertility rates for example. A clear specification of the underpinning assumptions is essential to users.
• Assumptions underpinning inter-censal population estimates in countries where vital registration systems are unreliable or unusable.

We RECOMMEND that member states supply adequate meta-data to support users needs and in particular where national norms differ from international measures or underpinning assumptions may affect the indicator value materially.

76. Gender statistics: A number of indicators call for separate analyses by sex. As a general rule we RECOMMEND that if the data source supports an analysis by gender then this should be provided for all indicators. To add emphasis we have identified in the framework the indicators for which separate estimates by sex are particularly needed.

77. Distributional Measures: There is a general issue about providing indicators that measure inequality and distribution within each country. There are a rather small number of indicators that focus on distributional issues (e.g. share of consumption by lowest quintile of population) but the large majority of indicators are based on national averages. Whilst it is beyond our remit, we feel bound to observe that much deprivation and inequality in the world will be masked by such indicators. Analysis by subgroups (e.g. by gender, region, age group, income groups, ethnic or social classifications) where feasible would illuminate this issue much more. Similarly, additional measures of inequality, such as the ratio of consumption by the highest 20% of households to the lowest 20% have much to commend them.

78. Frequency: As a general issue we comment on the frequency of provision of indicator values. In many countries with well developed statistical systems annual estimates will be available and we regard this as the desirable goal. However not all countries can sustain this. The frequency with which indicators should be measured will vary according to the importance of the topic and the rate of change that the indicator is likely to display. We RECOMMEND that, in general, indicators should be measured every 3-5 years, but some should be measured annually, while others (particularly those that are census based) will be updated every 10 years. Failure to produce indicators at the desired frequency may be one sign of inadequate statistical capacity.

79. Demography Domain: The choice of indicators in the first two tiers was relatively simple for the expert group because many are common to the needs of the UN Conferences. They are well established nationally and internationally, are relatively widely available and are relatively few. They depend upon a good infrastructure for population statistics and vital registration. The indicators provide important contextual information for the indicators in other domains.

80. Health and Nutrition Domain: This Domain has certain characteristics that makes prioritising very challenging. It includes an exceptionally large number of indicators. However, the indicators do not span all major health sub-domains resulting in significant gaps in the final indicator set. This final set cannot be viewed as a core indicator set for the health domain. The fact that the indicators are at very different levels of specificity also makes it difficult to identify a coherent high priority set. In addition, many of the indicators in the Demography Domain may also be considered major health indicators and these need to be evaluated along with the Health and Nutrition indicators. The sheer number of indicators made it difficult to carry out a full technical assessment and this activity will need to continue in the future.

81. The technical evaluation of some of the indicators raised problems of data availability. While this evaluation is done on an indicator by indicator basis, strategies for supplying the needed data can be developed in a more coordinated way. Investing in components of the statistical infrastructure, for example the vital statistics system or national health surveys, would provide data for several of the high priority indicators. This would also allow countries to collect more extensive statistical information on particular aspects of health covered by the full set of conference indicators if needed.

82. Given the large number of indicator, we felt it necessary to provide clear guidance on a relatively small number of indicators that countries could measure. We have made an initial attempt to carry out the technical assessment and to provide some structure and, as a result, a large number of indicators have been allocated to the 'Additional' category. While many of these indicators present significant measurement challenges, countries that have the statistical capacity and the need for the more detailed information may wish to produce some of these indicators. We RECOMMEND that the UNSC establish a process involving official statisticians and others, including officials from the WHO, to review the hierarchical framework and priorities in the Health Domain with the intention of addressing the substantive gaps in the indicator set; determining if there are infrastructure investments that can address a range of data gaps; and creating linkages between the short list of priority indicators and the large number of other indicators in the domain.

83. Environment and Energy Domain: Environmental indicators span a very wide range of different issues and as such it is more unlikely that one indicator will serve as a proxy for others. This results in rather more indicators for the Domain than one might initially expect. Also, environmental factors vary enormously with climate and there will be issues such as desertification or forest loss that are not highly relevant to all countries. Nevertheless these indicators are concerned with global issues as well as national policy areas. Comparability of indicators across countries is particularly difficult for some environmental indicators. It is often not the absolute level of the indicator so much as the trend over time that is the key focus of policy.

84. Economics and Poverty Domain. Except for GDP, which is provided as a contextual indicator, it is recommended that monetary indicators be expressed, not as a level, but in general as a percentage of current price GDP. In the main, GDP is recommended rather than GNI for this purpose. We RECOMMEND that the indicators adopted in the major sets be amended to be consistent with the use of GDP/GNI in the framework.

85. A number of indicators depend on a poverty measure that may be an global standard (e.g. $1 per day or $2 per day) or may be a nationally determined poverty threshold. Also measures may be based upon income or expenditure. The group favour an expenditure measure and for international comparison proposes an international standard. In all cases the indicator should employ PPP conversion. Countries may also wish to utilise nationally determined poverty thresholds if appropriate. A well-designed household budget and consumption survey can be used for both universal and national measures. Where these are produced we RECOMMEND that the meta-data must make the basis of Poverty indicators clear and, in the case of national poverty lines, the meta-data should contain an explanation of the methodology employed.

86. When the indicator is measured in a financial unit (e.g. GDP or GDP/Capita) the group favour the use of PPP conversion for international comparisons but recognise that for some countries this may not be available and exchange rate conversion may be the only option. PPP estimation is virtually unique in the sense that its primary purpose is to convert monetary aggregates to a common unit for international comparison purposes. As such, and with related measurement capability especially for developing countries, it may fall relatively low on national priorities. These measures need continued effort if quality is to be improved and this has been recognised by the UNSC and among international agencies, the World Bank. Given the nature of the measure, international assistance is essential. We RECOMMEND that all efforts be made to fulfil the UNSC resolutions of 2001 in respect of Purchasing Power Parity measurement.

87. Employment and Labour Domain. The majority of indicators in this Domain derive from the ILO Key Indicators of the Labour Market (KILM) project. Establishing priorities for the indicators is complicated by two factors. First, Labour Market indicators support both economic and social policies (e.g. earnings are a primary determinant of family wellbeing and the principal cost of production). Hence a wide range of policy issues is at stake. The second factor is the fundamental difference between labour markets in most industrialised and developing countries. For example indicators relating to the informal sector are essential to policy developments in most developing countries but of little relevance to the analysis of labour markets in industrialised countries. The choice of tier 1 indicators reflects an attempt to optimise at the global level.

88. Education Domain: Indicators in the Employment and Labour Domain are based upon a sound theoretical and conceptual base and half a century of continual measurement and refinement. In comparison the proposed indicators in the Education domain lack a similar solid conceptual foundation and measurement history. The first task of the expert group was to classify the indicators using the OECD's INES framework as a rough guide. Once classified it became clear that the majority of the proposed indicators reflect educational inputs rather than educational process or output/outcome measures. Also outcome indicators used grade level attainment as a proxy for real levels of literacy achievement. The main challenges for the expert group were, therefore, to try to strike a more balanced reflection of educational inputs, processes and output/outcomes and to recommend more reliable methods for indicators in the literacy domain. We RECOMMEND that the UNSC establish a process involving educational statistics experts from international agencies and member states to investigate the feasibility of adapting skill assessment methods employed in the developed world for use in developing nations.

89. Other Social Domain. This Domain is necessarily diverse since it comprises the social policy issues not allocated to other Domains. Many of the topics do not have a strong framework of international statistical standards and guidelines as other areas. Hence a number of the proposed indicators need further conceptual and statistical development if they are to be well based. We have drawn attention to these in the web site and have suggested some additional indicators that may be considered further through the process described in section 4. There range of indicators derived from UN summits and major conferences in the 'Other Social' Domain appears to be deficient. In particular the indicators for female participation and gender equality focus on political participation. There are no corresponding indicators for female participation in professional and senior administrative levels of the labour force. Also there are virtually no social indicators focussed on the social and housing conditions in which children are raised.

Indicator Recommendations

90. Table 1 contains the recommendations of the expert groups for a classification of the indicators into the three priority tiers. The framework includes the structure of Domains and Sub-Domains so that one may see how any indicator fits into the wider framework. Table 1 also contains a key to the indicators that appear in the major indicator sets. Detailed information on all the indicators considered is available on the UNSD web site (address). Additionally, the Room Document Technical Assessment of Statistical Indicators contains a description of the work of the expert groups and comments on the indicator framework, issues and perceived deficiencies.

91. Table 1 has been structured to reflect the major policy areas that are common in most national governments (e.g. Economics, Health, Education etc). Inevitably there are important cross-cutting policy areas that are contained within the hierarchy. For example gender statistics, or statistics relating to children are contained within a wide number of Domains and Sub-Domains. Similarly the economic aspects of Poverty are contained within the Economics and Poverty Domain but indicators that reflect other aspects of poverty are contained in other Domains.

92. Table 2 provides a breakdown between the Domains by priority tier. For convenience some Demography Sub-Domains (Mortality and Fertility) include appropriate Health indicators. The three priority tiers contain 38, 42 and 43 indicators respectively.

93. Table 3 provided an analysis of the relationship between the priority recommendations in Table 1 and the lists of indicators comprising the high level sets (MNDS: 15 indicators, MDG: 48 indicators, IDG: 29 indicators, UNDAF-CCA: 58 indicators, BSSA: 12 indicators and CSD: 58 indicators). These counts include cases where the expert group recommended a technical change in an indicator already identified by UN summits or where a direct replacement was judged to be preferred. For example the substitution of GDP for GNI for economics indicators as described in paragraph x.

94. A number of non-statistical indicators (Human Rights and Environment) have been excluded from our considerations and hence from Table 1. Also a number of the indicators associated with the Millennium Goals Declaration will be monitored for specific sets of countries only. These too have been excluded on the basis that they are not global indicators. The table shows that a high proportion of the various major sets are included in the three priority tiers of the framework. The remainder are generally included in the category of additional indicators unless the proposed indicator was sufficiently flawed technically as to be omitted altogether.

95. The MNDS was established by the UNSC as an attempt to provide guidance to countries on a high priority set of indicators that reflected UN summit and major conference priorities at that time. We believe that this function has been superseded by the proposed priority framework and hence we RECOMMEND that the MNDS be withdrawn.

4. FUTURE PROCESSES

96. The third aspect required of the report is to develop and recommend to ECOSOC a mechanism of statistical review for future proposed indicators.

97. Future processes are undoubtedly needed for several important reasons. First future UN summits and conferences will inevitably address new policy areas or, when reviewing progress on existing policy areas, will see the need to modify or elaborate the policy objectives in such a way that new or revised indicators are needed. The second important reason is that international agencies must review and develop the indicator frameworks that relate to their sphere of interest as policy objectives change or new issues rise up the agenda. Indeed we are aware of such reviews taking place in several agencies at the present time and the indicator framework should be updated as these reviews come to fruition. The third reason is that international standards, definitions and best practices must evolve over time as technical standards increase (and as the global statistical capacity develops and can support more demanding standards). For all these reasons it is vital that the framework is kept up-to-date and we so RECOMMEND.

98. We see the need to consider three related issues:

• Establishing new indicators in response to future major conferences and summits.
• Keeping under review the proposed hierarchical framework and priorities
• Reviewing and Refining Existing Indicators over time

New Summits and Major Conferences and New Indicator Initiatives

99. The work to establish new indicators should begin as part of the preparation for any forthcoming major conference or summit and should involve both policy officials and statisticians from both international organisations and from member states. In our view there is not necessarily a need for new mechanisms but there is a need to make existing mechanisms work more effectively. A number of principles need to be applied.

• Although policy officials for a particular conference will see themselves as being in the lead on indicators related to a particular topic, many other officials from other parts of the UN and other international agencies have a legitimate interest in the development of indicators in any field.
• The indicator requirement should be seen within the wider context of the totality of indicator needs. As such, emerging needs must be set alongside existing needs.
• The development of new indicators should be reconciled with national policy needs for statistics and take account of the statistical capacity of countries to produce them.
• Statisticians from international organisations have a role to play in assessing quality and reconciling the definitions of proposed indicators with the range of policy uses that may exist. National statisticians should be involved to contribute to this too. But in addition, since they are closer to the raw data, they have a special perspective on the technical properties of indicators, the availability of any proposed indicator and the data collection and resource implications.
• International consultation takes time as does the development of high quality statistical outputs. This needs to be recognised by the UN and other international agencies. However, if the wish of the Economic and Social Council to build agreement and ownership across the international community and member states is to be realised then this time is time well spent.

100. Tthe ideal arrangements are not easy to articulate. In general most (but not all) parts of the international community have made considerable efforts to improve co-ordination between themselves but the problem of drawing member states into the indicator development process in a full and genuine way is much more difficult. Token consultation by international agencies when key decisions have already been made is not sufficient. At the national level there is often good communication between statisticians and policy officials since national statistical plans need to be grounded in policy needs. In many countries statistical work for some domains is often located in the lead ministry concerned rather than in a central statistical office. This improves communication between the statistician and the policy officials but often disrupts it between these statisticians and others in the national statistical system. Lack of co-ordination and coherent planning across the national statistical system is a sign of inadequate statistical capacity and/or legal and administrative frameworks for national statistics in a country.

101. From the national statistical perspective it is the statisticians directly responsible that should make inputs into the indicator development process. But it is also essential that national statistical offices are informed if the co-ordination between statistical activities is to be achieved. We have tried to address this point in our recommendations.

102. We RECOMMEND that advanced planning for UN summits or major conferences, or the significant review of indicators within any international agency should trigger the consultation process below.

103. We RECOMMEND that the identification of statistical indicators for monitoring purposes should involve both policy officials and statisticians and each of these groups should draw upon international agencies and member states:

• That the identification and development of new indicators should be co-ordinated by the appropriate lead policy area.
• That it should take account of the capacity of countries to produce them.
• That the officials concerned should have a clear responsibility to involve other agencies and parts of the UN organisation who have a legitimate interest at the earliest stage.
• That the liaison should involve both policy officials and statisticians within the international agencies. UNSD should be involved from the outset.
• That a number of representatives (statisticians and policy officials) from member states should be invited to join any development team as full participating members and that, additionally, wider (electronic) consultation should be undertaken.
• That within member states the statisticians consulted should be those responsible for the relevant area but that the UNSD should ensure that national statistical agencies are involved in co-ordination issues.
• That the UNSD should use the regional UN statistical commissions and direct electronic communication with national statistics offices to ensure that national statisticians were consulted during the development process.
• That national statisticians, in turn, should use their regular contacts with their user communities (and particularly national policy officials) to provide feedback on the reconciliation of national and international requirements and that UNSD should provide feedback to the development process through these mechanisms.
• That in due course the lead policy area should, in consultation with UNSD, make proposals to UNSC who would report to ECOSOC.

104. We RECOMMEND that the responsibility for maintaining the indicator framework and for extending this to take account of new requirements should rest with UNSC who would recommend to ECOSOC the adoption of new indicators and their position within the hierarchical framework.

105. In this connection, we RECOMMEND that UNSC establish a standing committee to take responsibility for indicator issues and to act on behalf of the UNSC between meetings so that no undue delay occurs.

106. We RECOMMEND that UNSD shall, in close consultation with the lead policy officials and as a result of the consultations recommended, prepare recommendations for UNSC (or the standing committee as appropriate).

107. We RECOMMEND that in developing indicators and placing them within the framework the criteria listed in paragraph xx be applied.

Technical Improvements and New International Standards

108. The process of making technical improvements to statistics and of updating international statistical standards is long established. Hence we RECOMMEND that periodic reviews of individual statistical indicators within the framework should be included within the appropriate work programmes of statistical review and revision that are regularly reported to the UNSC.

5. ACKNOWLEDGEMENT

109. This report depends heavily on the contributions generously made by a large number of individuals from across the world. People from X countries contributed. Also members of various international agencies made valuable inputs to the process. We wish to thank all who contributed. But in particular the chairpersons of the expert groups made an invaluable contribution and deserve special mention:

Demography: David Pearce (UK)
Health: Jennifer Madans (USA)
Environment and Energy: Hilary Hilier (UK) / Bob Harrison (Aus)
Economy and Poverty: Rob Edwards (Aus)
Employment and Labour: Ian Macredie (Can)
Education: Scott Murray (Can)
Other Social: Linda Sabbadini (Italy)

Finally, a significant burden fell onto the staff of the UN Statistics Division and we sincerely thank them for their support.

110. In our view the future development of the indicator framework should be based on this report and we RECOMMEND that the Friends of the Chair group be discharged.

6. SUMMARY OF RECOMMENDATIONS

Development of Indicators and Maintenance of the Indicator Framework

1. That the indicator framework be maintained in response to future UN summits and major international conferences, developments of the indicator framework within international agencies and advances in technical standards.

2. That advanced planning for UN summits or major conferences, or the significant review of indicators within any international agency should trigger the consultation processes recommended.

3. That the need for baseline measures be taken into account when targets are adopted that require change to be measured from a specific point in time.

4. That the identification of statistical indicators for monitoring purposes should involve both policy officials and statisticians and each of these groups should draw upon international agencies and member states:

• that the identification and development of new indicators should be co-ordinated by the appropriate lead policy area.
• that it should take account of the capacity of countries to produce them.
• that the officials concerned should have a clear responsibility to involve other agencies and parts of the UN organisation who have a legitimate interest at the earliest stage.
• that the liaison should involve both policy officials and statisticians within the international agencies and in particular UNSD should be involved from the outset.
• that a number of representatives (statisticians and policy officials) from member states should be invited to join any development team as full participating members and that, additionally, wider electronic consultation should be undertaken.
• that within member states the statisticians consulted should be those responsible for the relevant area but that UNSD should ensure that national statistical agencies were involved in co-ordination areas.
• that UNSD should use the regional UN statistical commissions and direct electronic communication with national statistics offices to ensure that national statisticians were consulted during the development process.
• that national statisticians, in turn, should use their regular contacts with their user communities (and particularly national policy officials) to provide feedback on the reconciliation of national and international requirements and that UNSD should provide feedback to the development process through these mechanisms.
• that in due course the lead policy area should, in consultation with UNSD, make proposals to UNSC who would report to ECOSOC.

5. That the responsibility for maintaining the indicator framework and for extending this to take account of new requirements should rest with UNSC who would recommend to ECOSOC the adoption of new indicators and their position within the hierarchical framework.

6. That UNSC establish a standing committee to take responsibility for indicator issues and to act on behalf of UNSC between meetings to ensure that no undue delay occurs.

7. That UNSD, in close consultation with the lead policy officials and as a result of the consultation process recommended, prepare recommendations for the UNSC (or its standing committee as appropriate).

8. That in developing indicators and placing them within the framework the criteria listed in paragraph xx be applied.

9. That the UNSC establish a process involving official statisticians and others, including officials from the WHO, to review the hierarchical framework and priorities in the Health Domain with the intention of addressing the substantive gaps in the indicator set; determining if there are infrastructure investments that can address a range of data gaps; and creating linkages between the short list of priority indicators and the large number of other indicators in the domain.

10. That the UNSC establish a process involving educational statistics experts from international agencies and member states to investigate the feasibility of adapting skill assessment methods employed in the developed world for use in developing nations.

11. That the UNSC establish a mechanism (perhaps a City Group involving statisticians and others including policy officials) to develop statistical indicators of Human Rights and Good Governance.

12. That the indicators adopted in the major sets be amended to be consistent with the use of GDP/GNI in the framework.

13. That the MNDS be withdrawn

Production of Indicators

14. That the UNSD prepare a report to UNSC on the availability of indicators in the highest two tiers (and the third tier where information is available). The report should include an assessment of what might be needed to overcome the shortfall.

15. That, in general, indicators should be measured every 3-5 years, but some should be measured annually, while others (particularly those that are census based) will be updated every 10 years.

16. That where the data source supports an analysis by gender then this should be provided for all indicators as a general practice.

17. That the need for coherent statistics used in the numerator and denominator of indicators is recognised and that international agencies work to identify such inconsistencies and act as a catalyst in helping countries to resolve them.

18. That member states supply adequate meta-data to support users needs and in particular where national norms differ from international measures or underpinning assumptions may affect the indicator value materially.

19. That meta-data must make the basis of Poverty indicators clear and, for national poverty lines, the meta-data should contain an explanation of the methodology employed.

20. That the international agencies strive to improve co-ordination for data collection from countries.

Technical and Quality Issues

21. That the UNSD web site be the definitive source of technical information about the indicators.

22. That UNSD promotes the development of standards and guidance on best practice for indicators where needed.

23. That all statistical indicators should be subject to periodic review and improvement and that when such a review results in change, an approach be provided to support countries in moving to the improved indicator while maintaining continuity with the recent past.

24. That periodic reviews of individual statistical indicators within the framework should be included within the appropriate work programmes of statistical review and revision that are regularly reported to the UNSC and that appropriate policy officials have been involved in the process.

Statistical Capacity

25. That ECOSOC and international donors recognise the need to support and develop the core statistical capacity within member states, including the statistical infrastructure, and that all donor activity for statistics should recognise the need to address both national and international statistical requirements.

26. That, as part of this recognition, ECOSOC and international organisations and donors promote the use of statistics to support effective national policy development and good public administration.

27. That donor supported programmes should genuinely add to the statistical capacity within the country rather than divert it.

28. That when considering statistical capacity, international donors and countries themselves take particular account of the importance of a core set of demographic statistics and GDP estimates as an integral component of many statistical indicators.

29. That the initiative to develop an indicator of national statistical capacity through the PARIS21 initiative be modified to involve member states and that final proposals be made to the UNSC.

30. That all efforts be made to fulfil the UNSC resolutions of 2001 in respect of Purchasing Power Parity measurement.

Miscellaneous

31. That the Friends of the Chair be discharged.

Table 1: Hierarchy of Statistical Indicators by Domain and Sub-Domain

Table 2: Number of indicators by domains, sub-domains and priority levels

Table 3: Correspondence of Recommended Indicators to Existing Sets by Priority Level

¹Total cou755nt for UNDAF-CCA excludes 7 proposed non-statistical indicators on Human Rights and Good Governance and one on the Environment.
²Total count for CSD excludes 2 non-statistical indicators on the Environment
³ Total count for MDG excludes 12 indicators that will be monitored for specific groups of countries only.