Workshop on Application of New Information Technology to Population Data
12-20 October 1999
Bangkok

• Sourcing the Maps
• The Quick Count System
• Linking the Quick Count data to the Maps
• Access Control
• Hardware and Software Requirements
• System Design
• Training of the System Developers

• Software Selection
• Preparation of the Maps
• Rendering the thematic layer on the map and generating the GIF image
• User Interface

A census undertaking is a wide-scale data gathering operation within a fixed enumeration period. Of utmost concern during the field operations is the ability to effectively monitor and track the status or progress of enumeration work. Management action relies mainly on this key status information to reallocate and mobilize resources when and where needed. In the past, the NSO management had to rely on traditional methods of data presentation - reports, tabulations by geographic area, historical information and charts even, to learn about the extent of enumeration that has taken place since the start of the field operations and to spot potential problem areas. However, none of these methods present the spatial dimension of the situation. For instance, the decision maker has to make an effort to spot the problem areas if he is not provided a visual information on where to source enumerators when there is a need to add more enumerators to a specific area. Mobilization and reallocation becomes difficult.

GIS technology has been around for several years now. However, its requirements on the hardware has prevented many agencies with a limited budget from adopting this technology. And then again, there is the problem on the sourcing of maps. Depending on the type of GIS application to be developed, the accuracy of maps to be used as base maps may or may not matter.

The explosion in the number of Internet users worldwide has propagated the use of the World Wide Web as an effective tool for disseminating information. The web ability to manage multi-media inputs and link to other applications and databases more than cements its role as the ideal tool for dissemination at the start of the next century. Anything that can be presented in any of the forms that are supported in the web is assured of getting a wide base.

The pilot application merges the GIS and World Wide Web technologies to support the forthcoming 2000 Census of the Population and Housing operations which will cover the entire Philippines and will last for about 30 days. During this period, the NSO management has to be kept constantly updated of the progress of the operations.

The pilot application is part of the overall system being prepared for the Philippine Census 2000. More specifically, it is part of the Quick Count System responsible for generating the tabulations and other reports which will be the source of the preliminary counts for the Census 2000. The Quick Count System is based on the Household Listing Sheet to be used in the census. It has data entry, evaluation, summarization, transmission and progress tracking modules of its own different from that of the main census questionnaire.

The pilot GIS application will track the progress of the census enumeration process. It will be responsible for presenting the status of the operations by geographic area by showing this information with the use of maps. There are two inputs to the pilot GIS system: the digitized maps of barangays throughout the country and the database containing the characteristics or attributes to be displayed on the map. The characteristics to be displayed on the map include the data from the geographic area enumerated (usually at the barangay or village level), the actual and expected counts of the population and households, and other secondary information such as the number of enumerators. 

Users will be able to interface with this GIS using a web browser. More specifically, the application system will:

• Track the following critical indicators and statistics for the Census 2000:
Enumeration Areas Completed
Total Population
Number of Males
Number of Females
Number of Households Enumerated
Number of Institutions Enumerated
• Allow user to query the status of the operation for a particular region, province, municipality or barangay by specifying the area and the characteristic to be tracked.
• Dynamically generate thematic maps using a shareware program to generate on-the-fly GIF images of thematic maps by reading a vector file of the map coordinates.

The web-based tracking system will only be made accessible to NSO staff in the central and field offices through a password protection system. Authorized NSO offices, both from the central and field offices, will then be able to view the latest status of the census operations. Since the Quick Count system is expected to regularly provide updated data inputs to the GIS application on a daily basis, progress tracking and monitoring can therefore be done almost on real-time.

The user is also given several means of selecting the area he wants to track. One way is by clicking on the specific area on the map to display the lower level of disaggregation. The other is by clicking on the display of area names. (The names are organized in a tree-like manner similar to the Windows style of displaying its directory structure.) The third method is by selecting from a pull-down list.

The maps used in this pilot application are the scaled provincial and municipal boundary maps of the NSO. The municipal maps also include delineation of barangay boundaries. The digitization of these maps had already been completed by the NSO using the Atlas software much earlier.

Nearly all commercial mapping software allow the export of a map's coordinates into a vector file, which could be in plain ASCII format, which is easily accessible to another application program. The vector file consisting of the latitude-longitude coordinates of the base barangay maps provincial files were earlier used as bases in generating the maps distributed with the 1995 POPCEN PUF¹. These files had already been edited for spikes and other maps errors. The geographic codes that link a particular polygon in the map to a specific region, province, municipality or barangay had likewise been added to the file.

The Quick Count system is designed to provide the preliminary municipal-level counts from the 2000 Census and at the same time provide information for monitoring the progress of the operation. The counts as well as the demographic indicators that will be derived from the Quick Count system shall be compared against secondary information and provide an initial evaluation of the quality of the data gathered in the 2000 Census.

The Quick Count will get its data from the form containing the summary of information in the household listing sheet. The accomplished summary form is submitted to the team supervisor every four days. When these forms reach the Census Area Supervisor (CAS), these will be folioed for transmission to the provincial office and, if the CAS has a handheld PC, the data in the summary forms will also keyed into the handheld PC. The handheld computers are then brought to the provincial offices for uploading of the data. The data entry of summary forms from CAS without handheld computers will be done at the provincial office. After the data submitted have been verified and reviewed at the provincial office, a summary file is generated. Updated summary files are e-mailed to the central office at least twice a week. Provinces without e-mail will bring the diskette copy of the summary files to the regional offices which will e-mail these to the central office. At the central office, the program which updates the Quick Count database used as input for the Tracking System will be scheduleds to run twice daily.

The characteristics being tracked are stored in a database that has barangay level data as well as municipal, provincial, and regional aggregates of the characteristics being tracked. The database of the data inputs is tied to the maps database through the use of standard geographic codes.

The webpages of the Census 2000 Tracking System will be protected through a password authentication system so that these are accessible only to the NSO. There will also be several levels of access to the system. A user logging in with the "administrator" password can generate updated images through a web interface while one logged in as a "guest" can only view the files already available to the system.

A Pentium PC will be configured during the actual 2000 Operation to host the webserver for the pilot application. Windows NT and MS Internet Information Server (IIS) will run on this server. Although IIS will be used for the pilot application, other webservers may also be used so long as they can run CGI applications and Java Scripts. The CGI scripts required by the program are written in Perl. The program which converts the map coordinates to x and y coordinates is written in CLIPPER.

The GIF images of the maps are created using the FLY program, a public domain software developed by Martin Gleeson. FLY is a C program that creates GIF images on the fly from CGI and other programs. It uses Thomas Boutel's gd graphics library for fast GIF creation. It provides a command-file interface for creating and modifying GIF images. More information about this program can be obtained from its homesite http://www.unimelb.edu.au/fly/.

Since the development of the application will require the pool of the web development team and the Census 2000 system development team, members of the Census 2000 team were given a brief training on web authoring and coding of CGI scripts using Perl.

1. Selection of software to be used for the pilot GIS application.
2. Transformation of the digitized maps 
3. Rendering the thematic layer of the maps and generating the GIF images
4. Choice of the user interface

Initially, the office considered using commercially available GIS software for the pilot application. Among the factors considered in the selection of the software were: 

1. Software Cost;
2. GIS/Database features;
3. Hardware resource requirements;
4. Development time; and
5. Usability of existing digital map holdings.

At least 4 commercial GIS software were considered for the application system. At least two of these nearly had all the desirable features of a GIS software. However, cost is a major consideration. And, in general, commercial softwares which offer the features desired for the pilot application were all priced above the normal budgets. To be able to access the GIS application using a browser, an Internet server module also has to be acquired in addition to the base module in order to make the GIS accessible through the web. Some still require add-on modules on the client side to make the GIS information accessible to web clients. In the end, the NSO had to custom build a system centered around a software that is freely available in the Internet for the pilot application.

For the pilot application, one major task is the convertion of the coordinate system of the maps from latitude-longitude coordinate system into a cartesian coordinate system suitable for displaying into a windows environment. In particular, the coordinates were transformed or "rescaled" to fit a 256x256 coordinate system in order to produce a GIF image that is 256 pixels wide and 256 pixels long. A simple application program was developed to convert this latitude-longitude coordinate pairs into x and y coordinate pairs. For purposes of demonstrating the capabilities of the pilot application, only the maps for areas covered in one pilot province were converted.

To test the coordinate conversion program, the x and y coordinates were used to create a command file for the FLY program and then the FLY command was invoked to create the GIF file. The GIF image created was then displayed on the screen and compared against the original map. This was done to determine the appropriate factor to be used in computing the x and y coordinates so that the maps will appear in the correct proportions when displayed in the PCs monitor - that is, a map does not appear elongated or stretched or displayed on the screen using a wrong aspect ratio.

Of specific interest to this pilot application is the FLY program's ability to draw polygons based on x and y coordinates read from a file and fill a given area with a specified color (colors are specified using the R,G,B system).

An application program developed reads the data input and determines the color of the area based on the level of the characteristics being tracked. This application program generates the command-file code to be read by the FLY program and invokes the FLY program to generate the GIF image. To make the GIF image of the map clickable, the GIF image has to be displayed by an HTML or HTM file as an image map. Another function of the program is the generation of this HTML code.

Early on, it was already decided that the user interface to the GIS will be made through browser softwares. One can never discount the obvious advantages of using the World Wide Web as the interface to the GIS. Besides the web's wide acceptance, using the web as a medium for tracking the operations will not require additional programs on the client side. Since most of the field offices of the NSO now has access to the Internet, many of the field offices can also keep track of the progress of operations in other areas as well. Browsers also provide a very user-friendly navigational feature through the web's "point-and-click" feature.

The development of the pilot application system afforded yet another learning opportunity for the IT staff of the NSO. Some of the difficulties encountered were:

The FLY program which was used to produce the GIF image files also has its limitations. The FLY command which draws the polygons can only process a limited number of x and y pairs. To get around this limitation, some of the points of certain polygons had to be deleted. This workaround is expected though to introduce small distortions on the shape of the displayed map.

It would be ideal if all GIF images can be generated on the fly and summaries generated through a web interface. However, under the pilot system, both are server-side processes and thus, will require considerable server resources especially if several users are allowed to run these processes at the same time. If all those who have access to the system will be allowed to dynamically generate the files, the server's crunching engines have to be beefed up. Thus, to balance the dynamic generation of files and the server load and capability, the option to generate dynamic pages has been limited to a few users by using a multi-level access control mechanism.

A major input to the application system which are the digitized maps are already existing at NSO. Geographic compositions are expected to change before and even during the actual census operations. Such changes are expected for municipalities and barangays. There may be splitting of barangays, transfers of barangays to another municipalities and other similar changes. Although the number of such changes is only a small percentage of the total, any such occurrence will mean repetition of the map transformation process for the affected areas.

So far, only the maps for the three provinces included in the Pilot Census have been converted. All maps must be converted before February 2000. The development of all modules of the pilot applications is expected for completion by the end of this year.

Activities expected to take place in the next few months include:

• Map Conversion
• Completion of the Systems Development, in particular, the completion of the following modules:
• Summarization of Quick Count data to produce barangay-level data suited for the barangay-level maps
• Automation of the Updating of the Quick Count database
• Coding the menu system for the display of the area names

• Configuration of the Server to be used for the GIS Tracking System
• Training of Users
• Preparation of the System Reference Manuals for the maintenance of the system

The pilot system is intended to track key indicators of the Census 2000 operations. Additional functionalities can be added to the pilot application later. This includes the capability of highlighting the areas or polygons in the map which fit user specified criterion. For example, the system can be expanded to accept request of this form:

Although the GIS application is being developed as tracking system for the 2000 Census, the database can be expanded to include other demographic characteristics or replaced with other sets of socio-economic statistics.

The images and files generated by the system is not only useful while these are on-line. The generated GIFs and web pages can be bundled together and sufficient documentation added to produce a GIS product that can be distributed in other electronic media like the CD-ROMs and diskettes. The browser is the only software requirement for the user.

The only external program being used by the pilot system is the publicly available GIF generating program which was sourced from the Internet. The rest of the application programs were developed at the NSO. Hence, the office can freely use this tracking system in its other field operations.