1. List of abbreviations
2. Background.
3. Measurement challenges
4. Key players in statistical standards development
5. Definition and classification issues
   1. ICT sector and ICT product
   2. Content product and content sector
   3. Electronic commerce
6. Sector surveys on the use of ICT
   1. Use of ICT by individuals and households
   2. Use of ICT by enterprises
   3. Use of ICT in the public sector
7. Challenges in measuring expenditures and prices
   1. Quantification of ICT expenditures
   2. Quality improvements and prices of ICT products and services
8. Country comparisons and technology assessments
   1. Technology achievement index of UNDP
   2. Other assessments
   3. Sample indicator sets
9. Measurement issues for developing countries - points for discussion

Annex 1. Europe benchmarking indicators

Annex 2. IT indicators in Japan 2001

I. Background

1. A year ago the twelfth session of the Committee on Statistics recommended that statistics on the information society should be included on the agenda of the next sessions of the Committee and its subsidiary bodies.  For the basis of discussion at the 2001 session of the Working Group of Statistical Experts, the secretariat was requested to prepare a paper exploring alternatives for developing statistical frameworks to measure the field of information technology.  The Committee advised the secretariat to take into account the relevant work done elsewhere, in particular by UNDP, OECD and countries in the region.  The present document has been prepared in response to that request, except that it does not propose any frameworks; that content has been contributed by the Australian Bureau of Statistics in document E/ESCAP/WGSE.12/6.

2. The Working Group may also recall that the Committee recommended adding information and communication technologies (ICT) and the knowledge-based economy as a priority item in the programme of work in statistics, 2002-2003 and to make a corresponding modification in the statistics medium-term plan 2002-2005.  In the substantive discussion at the Committee, a suggestion was made that the United Nations Statistics Division or ESCAP should prepare a manual on measuring IT related activities.

3. While endorsing the above recommendations at its fifty-seventh session in April 2001, the Commission requested the secretariat to facilitate the sharing of regional experiences on the measurement of the digital economy through training and other means, and to obtain extrabudgetary funding for that purpose, if necessary.  The secretariat was also requested to produce a document on best practices for the measurement of electronic commerce activities in the region.

4. Having recently started planning its work on the topic of measuring the information society and the knowledge-based economy, the secretariat hopes to receive guidance from the Working Group on specific points listed in paragraph 57 of this document.

II. Measurement challenges

5. By and large, national statistical systems are geared towards capturing social and economic data using tried and tested methods, techniques and classifications.  Focus areas in economic statistics, for instance, include the production of goods and services, international trade, value added, labour force and capital, in other words the fundamentals that are needed for measuring an economy and its productivity.

6. Starting with mass production of microcomputers in the mid-1980s and accelerating after the popularization of the Internet in the latter half of the 1990s, the latest technological revolution poses one of the toughest measurement challenges in recent history.  Information and communication technologies (ICT) have done away with many borders that in earlier days conveniently circumscribed the measurement process.  Networking and mobile communications have made any place a workplace and blurred the boundaries between working time, leisure and learning time.  The divergence and vertical integration of products and services is accelerating and the latter increasingly fail to fit into standard industry and product classifications.  Significantly, ICT also offer opportunities for increased participation of often marginalized groups, disabled people, rural residents, etc.

7. As all these developments are very rapid, probably faster than any statistical system could ever adapt to (considering available resources), NSOs are challenged to provide statistics before conceptual frameworks are in place.  Another difficulty is on the data side.  Obtaining reliable data from enterprises, households and the public is not easy, especially about ICT expenditures and prices.

8. As the Committee on Statistics recognized at its twelfth session, it was natural that the first data on the information society would be restricted to indicators that were easy to obtain, for instance by extraction from traditional data sources or by collection through regular household or price surveys. Also, before moving to emerging industries and trying to capture conceptually difficult changes, it has proven easier to survey among established industries.

9. Statisticians and researchers prefer precise definitions for the phenomena, sectors, factors and indicators they are trying to measure and analyse.  In the case of the information society or the knowledge-based economy, they will have to wait for ongoing methodological and empirical work to provide precise definitions for such common terms as the ICT sector, the information society, the knowledge-based economy, etc.  The international community and institutes working on terminology and classifications have taken a pragmatic approach by starting from where the measurement is easiest and indicators comparable, and by testing and progressing gradually.

10. As the present document primarily attempts to capture those developments, less emphasis is given to describing the measurement of the impact that ICT has on the society and people in general. To assess the impact of ICT in organizations, for instance, related expenditures and uses of equipment and software should be analysed together with the skills of employees.  If organizations are not changing their ways of working, the potential productivity gains from ICT could be marginal.

11. For the term 'information society', a large number of definitions have been offered. www.information-society.org.uk, for instance, defines it as a society in which the creation, distribution, and manipulation of information has become the most significant economic and cultural activity. The often quoted IBM Community Development Foundation in a 1997 report, "The Net Result - Report of the National Working Party for Social Inclusion" defines it as a society characterized by a high level of information intensity in the everyday life of most citizens, in most organizations and workplaces; by the use of common or compatible technology for a wide range of personal, social, educational and business activities, and by the ability to transmit, receive and exchange digital data rapidly between places irrespective of distance.  Some others contrast it with societies where the economic underpinning is primarily industrial or agrarian.

12. For statistical development purposes, the presentation of the elements of the information society by a Nordic expert group might be useful.  It is reproduced below from the paper "ICT Statistics at the New Millennium-Developing Official Statistics-Measuring the Diffusion of ICT and its Impacts", presented by Jeskanen-Sundström at the IAOS Satellite Meeting on Statistics for the Information Society, Tokyo, 30-31 August 2001 (abstract available at http://www.stat.go.jp/english/iaos/paper/jeskanen.pdf).

Figure 1. The elements of the information society

13. Whichever definition is used, statisticians clearly have many dimensions to capture, which will eventually lead to using a relatively large number of indicators.  Due to differences in economies and statistical systems, not all of them are likely to be internationally comparable.

III. Key players in statistical standards development

14. As the rest of this paper is structured by subject matter, it is first perhaps useful to note the most active players in the development of statistical definitions, frameworks and classifications for the measurement of the ICT sector, the information society and the knowledge-based economy.

Table 1.  Key participants and their roles in the development of statistics on the information society

IV. Definition and classification issues

A. ICT sector and ICT product

15. The currently used classifications of economic activities (ISIC Rev 3) and products (CPC v. 1.0) do not have a clearly recognizable ICT sector and ICT product category, which shortcoming needs to be corrected in their next revisions.  In the meantime, statistics on the information society are being compiled using existing classifications.

16. In 1998, the OECD Working Party on Indicators for the Information Society (WPIIS) adopted an activity-based definition of the ICT sector.

For manufacturing industries, the products of a candidate industry:

• must be intended to fulfil the function of information processing and communication including transmission and display
• must use electronic processing to detect, measure and/or record physical phenomena or to control a physical process

For services industries, the products of a candidate industry:

• must be intended to enable the function of information processing and communication by electronic means

17. Using those principles, the ICT sector was defined from Revision 3 of the International Standard Industrial Classification (ISIC) as shown in the table below (left-hand column).  Subsequent detailed reviews on the degree of ICT specialization of the selected classes by the Nordic countries and Australia, and a comparison with the more detailed North American Industrial Classification System (NAICS), point to a need to revise the 1998 definition in respect of classes indicated in the right-hand column.

18. The OECD WPIIS has acknowledged that the inclusion of class 5150, in particular, would have meant that the ICT specialization rate would not be as high as desirable, thus leading to a need to establish subclasses where activities related to a mixture of ICT and non-ICT goods and services (OECD proposal for subdividing the class in 2002, see http://www.un.org/Depts/unsd/class/tsg-11.htm).  In this connection, it may be noted that class 5233 (other retail traded of new goods in speciality stores) has not been included in the ICT services sector as very few retailers exclusively sell ICT products.

19. When compiling the first edition of "Measuring the ICT Sector" a year ago, the OECD secretariat noted that in particular classes 5150, 6420 and 7123 had been treated in different ways, leading to incomparability of the sector statistics across its member countries.  Reviews by the Nordic countries and Australia are pointing to exclusions of class 3130 from the sector as the ICT content, most notably cables, was estimated to be only about a quarter of total class activity. Table 2. Activity-based (ISIC Rev 3) definition of the ICT sector

20. A paper prepared by OECD for the Voorburg Group meeting in September 2001 'Reviewing the ICT sector definition: issues for discussion', contains detailed information about the classes under review.  That and other documents presented at the Örebro meeting in Sweden are available at http://www.voorburg.scb.se/.

21. An alternative approach to defining the ICT sector was last discussed at the Voorburg Group meeting (Aufrant 2001, 'Some grass roots concepts to describe and measure information economy').  It was based on the definition of 'ICT product' and then simply labelling the group of industries primarily engaged in producing ICT products as the 'ICT sector'.  The WPIIS had initially evaluated such an approach but the idea was abandoned since it proved very difficult to quickly extract a list of ICT products from the ageing Central Product Classification (CPC).

22. While there is no commonly adopted definition for 'ICT product' among the OECD, it is understood to be an instrument that supports the electronic display, processing, storage and transmission of information or a tool to handle information by electronic means.  In other words, ICT products do not include the objects (content, information) handled with the tool.

B. Content product and content sector

23. There is a general agreement that the ISIC needs need urgent improvement in respect of information content industries.  It does not have a class for "publishing video games" or more generally "publishing leisure software"; the classification of audio-visual industries is ageing, and the "data base activities" class is poorly defined.

24. A paper presented at the September Voorburg Group meeting contains a good discussion and a proposal to update "the group of industries primarily engaged in the publishing and/or the electronic distribution of content product" (Nivlet 2001, 'Industry classification revision and the content sector').  In the proposal, new content products have been filed according to the type of content delivered and not according to the type of supporting medium.  Analogous to the proposal of deriving the ICT sector from the definition of ICT product, the content sector could be defined as a group of industries primarily engaged in the publishing and/or the electronic distribution of content products.

25. The above approach represents a departure from the initial thinking of WPIIS to define electronic content, electronic content products and the electronic content sector.  The 'electronic' track proved conceptually unsuccessful as what really needs to be observed is the content available on an electronic communication medium, or content combined with an electronic communication medium. A newspaper and its Internet version are an example where (one) content is produced but which is published using printed and (one type of) electronic media.  Although still commonly used, expressions such as electronic product, digital product, electronic content or digital content cannot be easily and explicitly defined (for statistical purposes) and their use can be misleading.

C. Electronic commerce

26. The Committee on Statistics observed last year that e-commerce transactions still appeared to represent a marginal share of all transactions in both countries studied (Australia and the Republic of Korea) and that the take-off of e-commerce had been slower than had been anticipated a couple of years earlier.  However, the volume of business-to-business transactions was growing and business-to-consumers e-commerce was increasing rapidly, although the latter continued to be restricted to selected goods, such as books, compact discs and IT ware.  With regard to cross-border transactions, the Committee observed that, while much of the physical flow of goods was recorded in customs statistics, it was difficult to distinguish which transactions fell under e-commerce.

27. Another problem that the Committee observed was the lack of consensus on whether conventional electronic data interchange (EDI) should be treated as e-commerce.  Soon afterwards, a solution was suggested by the WPIIS and its dedicated expert group on defining and measuring e-commerce. Based on experiences gained from e-commerce surveys carried out by the member states, the group concluded that one definition could not serve all and adopted a narrow and broad definition of e-commerce transactions (see table below, and Nielsen 2001, http://www.stat.go.jp/english/iaos/paper/nielsen.pdf).  Basically, the narrow definition covers all sales and purchases of goods and services through the Internet, whereas the broad definition includes also non-Internet EDI. Table 3.  OECD definition of electronic commerce

Scope	Definition (2000)	Proposed operational definition to be used in surveys
Narrow definition	An Internet transaction is the sale or purchase of goods or services, whether between businesses, households, individuals, governments, and other public or private organisations, conducted over the Internet. The goods and services are ordered over the Internet, but the payment and the ultimate delivery of the good or service may be conducted on or off-line.	Internet Transactions. Orders received/placed on a Web page, over extranets and other applications that run over the Internet, such as EDI over the Internet, Minitel over the Internet, or over any other Web enabled application regardless of how the Web is accessed (e.g. through a mobile or a TV set, etc.). The payment and the ultimate delivery of the goods or services may be conducted on or off-line.
Broad definition	An electronic transaction is the sale or purchase of goods or services, whether between businesses, households, individuals, governments, and other public or private organisations, conducted over computer-mediated networks. The goods and services are ordered over those networks, but the payment and the ultimate delivery of the good or service may be conducted on or off-line.	Electronic transactions. Includes all Internet transactions as defined above plus orders received/placed over EDI or any other online applications used in automated transactions (e.g. Minitel, interactive telephone systems). Orders received/placed using facsimile, telephone or non-interactive e-mail should not be included. The payment and the ultimate delivery of the goods or services may be conducted on or off-line.

V. Sector surveys on the use of ICT

28. A number of statistical offices have been focusing on developing definitions of the ICT sector, ICT products and services and defining e-commerce.  That conceptual work has been assisted by empirical statistical data collection by using established and new vehicles.  A rather concerted effort can be observed among a number of OECD countries in designing and testing model questionnaires on ICT usage in enterprises, households and the public sector.

A. Use of ICT by individuals and households

29. The OECD's Working Party on Indicators for the Information Society (WPIIS) has been reviewing model questionnaires for surveys of household ICT use.  The model questionnaire has been simplified during the course of work.  The latest revision, presented to the Voorburg Group in September 2001 by Australia (see http://www.voorburg.scb.se/Australia_HouseholdITUse.pdf for details), included the following modules:

1. Household access to computers and the Internet
2. Household barriers to adoption of the Internet
3. Adult use of computers and the Internet: location and frequency of use
4. Purpose and nature of adult activities on the Internet
5. Internet-commerce details: adult activities and barriers.

30. It is suggested that additional components of the questionnaire be added over time as technologies, usage practices and policy interests change.  As differences in scope and coverage between countries are unavoidable because of the use of existing population surveys as vehicles, it is suggested that countries note any exclusions from scope, or areas of poor coverage.

31. To eliminate survey error, the Australian proposal suggests using well designed samples which are of sufficient size to produce reliable data; careful design and testing of questions and question sequences (whether interview-based or self-administered); intensive training and checking of interviewers; reducing non-response as far as possible; and minimizing data entry, editing and other processing errors.

32. The suggested classification variables are 

33. WPIIS, which is expected to continue working on the questionnaire and approve a final version in the 2002 meeting, has already generally agreed that it should include questions on frequency, barriers and Internet activities, whereas questions on intentions and information in respect of children should not be included.

34. A number of views have been expressed regarding the statistical unit, with some countries preferring individuals as the appropriate unit, while other countries ask household questions on access to computers and the Internet.  Opinions differ also on the length of a recall period for value of Internet purchases such that they can produce unbiased annual aggregate data.  There are a number of ways to rate barriers to Internet access and the purposes of its use, the current model favouring relatively simple classification models, with additional notes for assisting inter-country comparisons.

35. Discussion also continues on whether the purpose and activities questions should be restricted to home use only or whether the questions should be asked separately for home and other locations.  The latest revision of the survey asks questions separately in respect of home and other sites of Internet use.  Another question being reviewed relates to dealing with the location when respondents have used a mobile access device.

B. Use of ICT by enterprises

36. Having specialized in survey methodology concerning the services sector, the Voorburg Group took the responsibility of developing a model survey on ICT usage by enterprises. Based on the experiences from the Nordic countries, the Voorburg Group started the elaboration of the model survey in 1999, followed by discussions in WPIIS and the Group in 2000.  The model questionnaire was finally adopted at the April 2001 meeting of the WPIIS.  It has the following modules and a few background questions about the enterprise:

1. General information about ICT systems (three questions)
2. Use of Internet (six questions)
3. E-commerce via Internet (twelve questions)
4. E-commerce via other computer-mediated networks (four questions)
5. Barriers on use of e-commerce, Internet and ICT in general (three questions)

37. It has been designed

• to be a flexible tool composed of modules allowing country specific features to be included
• to allow easy revision to reflect the rapid changes in ICT and its use
• to be a general survey tool for all economic activities
• based on qualitative core questions, an approach which is considered to provide the most harmonized basis for country comparisons.

C. Use of ICT in the public sector

38. There is great interest in statistics on the use of ICT in the public sector as those technologies offer new means of communication and facilitate provision of public services.  Another aspect of interest is the efficiency of resource use.  Compilation of required data and analysis is however not as easy as might be thought.

39. The Danish experience reported at the Voorburg Group indicated that the main problems were related to the tracking down of the ICT expenditure (see Lundø 2001, "Use of ICT in the public sector - a questionnaire based survey of Danish municipalities", http://www.voorburg.scb.se/UseICTDanishPublic.pdf).  It was not always easy to define from the existing accounting systems what constituted ICT expenditure.  In particular, the school sector was assumed to account for a large, insufficiently known part of ICT expenditure.

Table 4. Items included in the survey on the use of ICT by municipalities in Denmark (August 2001)

Module	Topic
1. Website/electronic services	- Homepage - Information on homepage - Communication via homepage
2. Intranet	- Having Intranet - Organizational coverage - Contents
3. Other ICT systems	- Electronic filing system . - EDI
4. ICT expenditure	- Previous year - Current year - Forecast
5. Strategy and cooperation	- Strategy - Coverage - Cooperation with other municipalities
6. Barriers to usage	- In general - Electronic services

VI. Challenges in measuring expenditures and prices

A. Quantification of ICT expenditures

40. Experiences gained from dedicated surveys indicate that it is difficult to obtain reliable and comparable data about ICT expenditures.  The Swedish experience in 1993 and 1997 was that the response rate for questions on ICT expenditures remained below the rate that would allow publication of the results (see http://www.voorburg.scb.se/Paper Voorburg 2020010821.pdf ).  The enterprise respondents found it generally difficult

• to separate ICT expenditure from other expenditure due to the lack of consensus on the definition of what belongs to ICT.  For instance, some enterprises considered telephony costs as ICT expenditure while others did not.
• to determine who or which part of the enterprise (for instance, IT managers or the accounting department) should respond to the survey and be in a position to provide the best information.
• to separate IT-related education expenditure from accounting systems.
• to separate out the cost of embedded software in machinery

41. Moreover, the Swedish legislation had depreciation options for PCs.  While some enterprises considered them as direct costs, others treated them as investments, depreciating them over three years.  Because of the multitude of problems reported above, the Nordic countries' harmonized survey concentrates on the use of ICT, excluding questions on IT expenditures.

42. The difficulties in quantifying ICT expenditures are reflected in national accounts.  The lack of expenditure data and data on own software development makes the estimation of gross fixed capital formation (GFCF) in computer software challenging. The above-mentioned paper discusses the Swedish methods in estimating GFCF in software purchased and in software developed on own account.

B. Quality improvements and prices of ICT products and services

43. The life cycle of PCs and many other ICT products is very short. New computer models offer faster performance for the same, or reduced prices, and the maintenance of old models is not supported.  The change in quality poses a difficult problem in creating price indices from sample prices.  Any price difference between the new item and the old one reflects the value of the quality change in addition to the pure price change.  Implicit and explicit methods exist for the quality adjustment, or separating out the value of the quality change from the prices.  The former include simple judgment and production cost estimates, and the latter various linking methods.

44. A technically more challenging and more costly method, hedonic adjustment is seen as a solution by many economists.  This method requires a large amount of data on prices and characteristics of computers and a substantial amount of econometric modelling in the estimation of hedonic functions.  Recent studies are nevertheless pointing out that the observed large international variations in price indices of computers among OECD countries, which use various methods of quality adjustment, are implausible and that the extrapolation of the hedonic price indexes of the United States would be better than accepting the measures that are now published for many OECD countries.

45. The impact of ICT prices on economic indicators and growth estimates is significant in increasingly ICT-dependent economies.  To explore cost-effective solutions to the substantial problems that arise in computing quality-adjusted price indexes for IT products, an international collaboration project has been launched between the OECD, the European Hedonic Center at Eurostat, Statistics Canada and the Australian Bureau of Statistics.

46. The above information is from Triplett 2001 ("IT, hedonic price indexes, and productivity", http://www.stat.go.jp/english/iaos/paper/triplett.pdf), which can be referred to for more details about hedonic adjustment.

VII. Country comparisons and technology assessments

A. Technology achievement index of UNDP

47. The Human Development Report 2001 published by the United Nations Development Programme introduced the technology achievement index (TAI), which aims to capture how well a country is creating and diffusing technology and building a human skill base.  Countries are classified as Leaders, Potential Leaders, Dynamic Adopters and Marginalized according to the level of the TAI.  The TAI comprises four dimensions of technological capacity, each measured by two different indicators.

1. To capture the level of innovation in a society, the index uses (i) the number of  patents granted per capita (to reflect the current level of invention activities) and (ii) the receipts of royalty and license fees from abroad per capita (to reflect the stock of successful innovations).
2. Diffusion of recent innovations is measured by (iii) the number of Internet hosts and (iv) exports of high- and medium-technology products as a share of all exports.
3. The third dimension, diffusion of old innovations, is gauged by (v) the number of telephone connections and (vi) the consumption of electricity per capita.
4. For human skills needed to create and absorb innovations, the indicators are (vii) the mean years of schooling and (viii) the gross enrolment ratio of tertiary students enrolled in science, mathematics and engineering.

48. It is relatively easy to simulate index calculations following the Report's example on composing the index.  Each indicator and each dimension have proportionate weights.  The energy consumption and telephone indicators are expressed as logarithms and capped at the average OECD level, which effectively ensures that as the level increases they contribute less to the index.

49. The Report acknowledges that many aspects of technology creation, diffusion and human skills are hard to quantify.  And even if they could be quantified, a lack of reliable data makes it impossible to fully reflect them.  In other words, the selection of indicators is much affected by the availability of measurable data.

50. One main issue that arises from the report is the lack of comparable data for the Asia-Pacific region.  Even after allowing substitution of missing data values for the first dimension indicators (patents and received royalties) with zeroes (indicating that little formal innovation is occurring), it is possible to calculate the TAI for only 16 of the 57 regional member and associate member economies of ESCAP (see table).  For the rest, some critical data values were missing or were not available at all. 

Table 5.  TAI ranking and the number of its missing component values for ESCAP members and associate members

Country/Area (Number of indicator values missing out of 8)	TAI rank (among 72 countries/areas)
Leaders
Japan	4
Republic of Korea	5
Australia	9
Singapore	10
New Zealand	15
Potential leaders
Hong Kong, China (1)	24
Malaysia (1)	30
Dynamic adopters
Thailand	40
Philippines	44
China	45
Islamic Republic of Iran	50
Indonesia (2)	60
Sri Lanka (2)	62
India	63
Marginalized
Pakistan (1)	65
Nepal (1)	69
Source: UNDP Human Development Report 2001
Inadequate data to calculate TAI (no ranking) Bangladesh (1), Russian Federation (1), Turkey (1), Armenia (2), Kazakhstan (2), Kyrgyzstan (2), Mongolia (2), Myanmar (2), Azerbaijan (3), Georgia (3), Tajikistan (3), Brunei Darussalam (4), Turkmenistan (4), Uzbekistan (4), Cambodia (5), Fiji (5), Maldives (5), Papua New Guinea (5), Viet Nam (5), Bhutan (6), Lao People's Democratic Republic (6), Samoa (7)
No data presented in HDR2001 Afghanistan; American Samoa; Cook Islands; Democratic People's Republic of Korea; French Polynesia; Guam; Kiribati; Macao, China; Marshall Islands; Micronesia; Nauru; New Caledonia; Niue; Northern Mariana Islands; Palau; Solomon Islands; Tonga; Tuvalu; and Vanuatu

51. The Report makes several qualitative statements about the index, which could be discussed by the Working Group:

• It is intended to help policy-makers define technology strategies.
• Its design reflects two particular concerns.  It focuses on indicators that reflect policy concerns for all countries, regardless of the level of technological development.  To be useful for developing countries, the index must be able to discriminate between countries at the lower end of the range.
• The composite index helps a country situate itself relative to others, especially those farther ahead.
• The report further argues that an overall assessment of a country's technological achievement is more easily made based on a single composite measure than on dozens of different measures.
• Like other composite indices in the HDRs, the TAI offers a starting point to make an overall assessment, to be followed by an examination of different indicators in greater detail.
• It is further argued that TAI rankings do not shadow income rankings and show considerable dynamism in several countries with rising technological achievement - for example, the Republic of Korea ranks above the United Kingdom, Canada and other established industrial economies whereas large developing countries -Brazil, China, India -do less well than one might expect because this is not a ranking of "technological might " of a country.
• The TAI is not a measure of which country is leading in global technology development, but focuses on how well the country as a whole is participating in creating and using technology.

B. Other assessments

52. As official statistical systems are unable to respond fully to the demand for statistics on rapidly changing society, the private sector has undertaken numerous surveys and 'e?readiness' assessments at national and international levels.  Organizations engaged in comparative assessments between countries include UNDP (as mentioned above), ITU, the World Bank, the World Economic Forum, and several national donor agencies.  The e-readiness assessments have been conducted using a variety of tools.  The Bridges.org organization has assessed some of the work done and compared the methods used in selected assessments (see details in http://www.bridges.org/ereadiness/Readiness v8.13f.rtf).

53. The Bridges.org also surveyed which countries had been assessed and by whom (see "E-readiness Assessment:  Who is Doing What and Where", http://www.bridges.org/ereadiness/where.html).  Although many of the assessment results were not publicly available or easily accessible, it found out that many countries had already been assessed more than once.  While there appeared to be significant duplication of effort regarding some countries, others lacked useful data.  Of the total of 84 countries that had been assessed by at least one tool, sixteen had been assessed at least five times by different organizations.  On the other hand, many of the poorest countries that had most to gain from the information technology revolution had not been assessed at all.  It is highly likely that the lack of data is a key reason for their being ignored.

54. As new data sources emerge and estimation methods improve, the variance in estimates and assessment results is likely to decrease.  For instance, the official estimates of the value of e-commerce in some countries, which have appeared recently, have been warmly welcomed by data users who were earlier puzzled by the huge variance of estimates made by various private sector organizations.

C. Sample indicator sets

55. It is not feasible to review global or regional indicator development in this document.  Two recent examples are provided instead, eEurope benchmarking by the European Union in Annex 1 and IT indicators in Japan 2001 in Annex 2. The eEurope indicators were selected to measure comparative progress in implementing the eEurope action plan, an ambitious initiative launched in 1999 to make all parts of the society benefit from information technology.  The Japanese indicators describe the recent diffusion of information technology in society and give a better overview of the current status of IT in Japan than similar publications in the past.

VIII. Measurement issues for developing countries - points for discussion

56. The twelfth session of the Committee on Statistics noted that low computer ownership and poor access to the Internet in developing countries explained the relatively low attention paid to IT indicators.  The Committee recommended, however, that all statistical offices should prepare for the eventuality that policy makers and other influential data users could at any time request data on the stage of digitization in the country.

57. The Working Group is requested to discuss how statistical offices, particularly in developing countries, could step up their responses to the high interest in statistics on the information society.  The following questions might provide starting points:

1. Is the described direction of conceptual, methodological and classifications development desirable?  Is there a need for developing countries to participate in those processes?  If there is, how can they participate effectively?
2. The development of statistics on the information society is no doubt a gradual process, but would the adoption of the models of OECD countries, for instance, accelerate their development?  Could the utilization of existing survey vehicles be higher and the frequency of dedicated ICT surveys correspondingly lower?
3. Are there particular early indicators that would be useful in monitoring the progress of economies that are only starting mainstream adoption of ICT?
4. Should the need for measurement of cross-border flows affect the development of related national legislation (e-commerce, taxation, etc)?
5. How critical is the national measurement of changes in the quality and functionality of ICT products in the region?
6. Does the adoption of ICTs create difficulties for the measurement of working hours in the region?
7. Are there deficiencies in registration of new firms in the ICT sector in the region?  How should any deficiency be assessed?
8. Certain countries in the region are major global contributors of software engineers.  Are there any statistical issues to be resolved?
9. What are the quality adjustment options for the price indices of ICT products in developing countries?
10. Are international technology assessments and similar inter-country comparisons desirable and sufficiently methodologically sound?
11. As basic data collection tools and systems are becoming established, the attention of statisticians in OECD countries appears to be turning to the statistical capture of the impact of ICT.  How important is early measurement of this impact in developing countries?

Annex 1. Europe benchmarking indicators

Document http://europa.eu.int/information_society/eeurope/benchmarking/ indicator_list.pdf contains metadata about the indicators; it also includes a number of supplementary indicators that could be used in the benchmarking.

1. Percentage of population who regularly use the Internet
2. Percentage of households with internet access at home
3. Internet access costs
4. Speed of interconnections and services available between and within national
5. Number of secure servers per million inhabitants
6. Percentage of Internet-using public that have experienced security problems
7. Number of computers per 100 pupils in primary/secondary/ tertiary levels
8. Number of computers connected to the Internet per 100 pupils in primary/secondary/tertiary levels
9. Number of computers with high speed connections to the Internet per 100 pupils in primary/secondary/ tertiary levels
10. Percentage of teachers using the Internet for non-computing teaching on a  regular basis
11. Percentage of workforce with (at least) basic IT training
12. Number of places and graduates in ICT related third level education
13. Percentage of workforce using telework>
14. Number of Public Internet Points (PIAP) per 1000 inhabitants
15. Percentage of central government websites that conform to the WAI accessibility guidelines at A level
16. Percentage of companies that buy and sell over the Internet
17. Percentage of basic public services available on-line
18. Public use of government on-line services - for information/ for submission of forms
19. Percentage of public procurement which can be carried out on-line
20. Percentage of health professionals with Internet access
21. Use of different categories of web content by health professionals
22. Percentage of EU web sites in the national top 50 visited
23. Percentage of the motorway network (vs. total length of network) equipped with congestion information and management systems

Annex 2. IT indicators in Japan 2001

Household Sector