The need for a generalised survey processing
system has been apparent in Statistics New Zealand
for some time. Key requirements were the ability
to rotate processing staff between surveys,
improving publication times, improving quality
without adding to costs, reducing the risks
of errors, as well as reducing development costs.
However, the development of a generic system,
wasn't considered achievable, given the inherent
differences between surveys and the specialised
nature of software. Our solution was the development
of a "standard" survey processing system template,
that could easily be reused. Lotus Notes lends
itself well to this approach, in that common
functionality can be added simply using "cut
and paste".
The standard template is to be used for several
surveys over the next 18 months, including,
QMS (Quarterly Manufacturing Survey), QES (Quarterly
Employment Survey), ERBS (Employment Related
Business Statistics), RT. (Monthly Retail Trade)
and AES (Annual Enterprise Survey). Other simpler
production surveys will also use the template.
This will eventually become the standard way
to process all business surveys.
The template was designed for business surveys
but could also be used for surveys of individuals.
The feasibility of using the template for interview
surveys has not been tested, but the capture
from paper questionnaires or electronic data
and all subsequent survey phases is not fundamentally
different to business surveys.
Progress
to date
The quarterly Accommodation Occupancy survey
has been using an early version for two years.
The QMS (Quarterly Manufacturing Economic Survey)
using the production version of Sprocet has
completed its second quarter in production,
while the QES (Quarterly Employment Survey)
version is being used for the first time for
the November 1998 survey.
Our expectations for the system which seemed
ambitious two years ago have been exceeded and
further additional benefits and development
opportunities have now become evident.
Key
Strategic advances
The main advantage of Sprocet is integration:
being written in Lotus Notes has facilitated
fully integrated desk to processing. Lotus Notes
is the SNZ gateway to discussion databases and
email as well as survey processing using Sprocet.
Sprocet has utilised the power of Lotus Notes
views to integrate all survey processing phases
from data capture through to output editing.
At all stages of the process management information
can be produced which gives the staff involved
and any other approved user in the organisation
the ability to balance work flows and monitor
progress against contractual agreements. The
dynamic nature of views along with the use of
categories, sorting and drill down, have integrated
and transformed processes that were previously
large batch jobs, into interactive processing.
Sprocet is also very integrated with the Business
Frame and the Business Surveys database of respondent
information. Both of these written in 'SQL Windows'
and located on Sybase databases. The minimum
of information is replicated onto Sprocet, the
rest is retrieved dynamically during processing
and changes identified during the survey process
are updated back to the Sybase databases.
The functions of system described below uses
the quarterly manufacturing survey (QMS) as
the example. The QMS is a sample panel survey
of 1,600 manufacturers. The survey collects
19 data items on sales, stocks, expenditures,
profits, hours worked. Small manufacturers complete
a short form with four data items. Results are
published for 10 industry groups and are available
10 weeks after the end of the quarter. The survey
is a key component in producing GDP estimates.
The survey design allows for births each quarter
form the business frame and imputation for non
and partial response.
Main
Functions
Sprocet itself is organised to match the
workflows of a standard survey. The system is
organised into 5 key areas, These are:
Management activities,
Unit respondent functions,
Data capture functions,
Macro Checking functions
and finally the Survey Results area.
Movement around the system is controlled via a
Navigation menu on the left hand side of the display.
Individual details are then displayed on the right
hand side of the display.
The
Navigator Display
This screen provides access to each of the
five over arching functions and also access
to the Business Frame, and also access to standard
Notes applications.
The
Management Area
This groups together the management functions
of the survey. These tend to be the large processing
agents, such as the creation of postouts, reminder
runs, non-response unit record imputation and
sample maintenance. In addition the system also
records the run status of these processes, allowing
a history of processing information to be built-up
over time.
The following processes (were created in
Lotus Notes) are included amongst those carried
out by the agent scheduler.
1. Imputation
The QMS survey uses two methods for imputing
for cases where non-response exists. These are:
Historical imputation - the
values from the previous quarter are used, after
adjusting by an estimate of the "trend" calculated
from respondents.
Weighted mean imputation - a mean
is calculated from respondents, and used as
an estimate for non-respondents.
Each KAU is assigned to an imputation cell.
Imputation is carried out within each imputation
cell. If there are not enough respondents in
an imputation cell, the system combines cells.
2. Estimation
Rate-up estimation is used to produce population
estimates in the non-full coverage strata of
the various industry groups. Rate-up estimation
is where each KAU's value is multiplied by its
weight and these are summed up to produce a
population estimate. Each KAU in the sample
is assigned a weight. The selection cell weight
is formed by considering the number of KAUs
in the survey population for the particular
selection cell, dividing by the actual number
of KAUs selected within that selection cell.
The selection cell weights are updated quarterly
by the addition of births in the survey. Any
non-full coverage KAUs which may distort the
estimates if we use their selection cell weight
are specially treated. Special treatment is
the process of reducing a KAU's selection cell
weight to 1.
3.Grossing-Up Procedure
for Small Firms
This is a special imputation procedure which
is applied to all QMS respondents receiving
short form questionnaires. The variables not
collected from respondents for these KAUs are
imputed by multiplying their values for quarterlysalaries
and wages for employees by ratios created using
past Census data.
The
Units (Respondents) Area
This area allows quick access to the units
contained within the system. Access can be via
specialised views, grouping together respondents
with similar characteristics such key firm status,
recent births, all live units, all units etc.
By selecting the respondent in the view, their
basic survey information is displayed, e.g.
name and address information, sample weights,
survey processing flags, postout and reminder
dates, classification information, size details,
as well as any comments recorded by the processing
sections.
Notes
A "KAU" is the Business
Frame reference number given to units that
can supply full profit and loss accounting
information. A "KAU" is usually equivalent
to an Enterprise, but some large enterprises
have more than one "KAU".
The names of individual
respondents have been deleted to meet confidentiality
requirements.
The
Answers Area
This area groups together the data capture
and edit functions. The initial view records
the current rate of progress in processing the
incoming questionnaires. Individual data capture
of respondents is started from this view. In
addition on-line editing rules are created and
maintained by the client sections directly into
the system.
The
Macro Checks
This area groups together the macro checking
of the data captured. It includes those checks
done across all the units of the survey. The
checks can be carried out at any stage of the
survey processing cycle as results are progressively
updated as the data is captured. Such checks
allow early and consistent checking of the survey
trends and likely results. Unusual or unexpected
results can be discovered earlier on in the
processing cycle.
The
Results Area
The last area provides quick access to the
results of the survey. A variety of results
are valuable including weighted results by key
classifications, as well as the ability to drill
down on the results set e.g. from national level
to industry level , to sample strata, to individual
respondent.
As much of the system is dynamic a change in
one area is reflected in other areas. e.g. an
amendment to the data from one respondent is
then reflected dynamically in the results views.
Likewise a re-run of the non-response imputation
system would also be reflected in both the macro
checks and results views.
The
Benefits of Sprocet
The integrated nature of Sprocet has the
benefits of reducing survey costs, while improving
timeliness and survey quality. The concept of
reusing a template, and then customising it,
gives a standardised client interface across
surveys, as well as, reduced development costs.
There are also significant business benefits
with using Sprocet. Feedback from the users
has been positive with our major clients also
re-organising their operations to take advantage
of the Sprocet features. The significant feature
impacting on clients are:
The system contains all
the process for the survey within the one
system. All the input process, output processes
and the associated run time logs are held
within Sprocet. There is no movement of data
or change of platforms within the processing
cycle. The results of all processing runs
are available, back over time, for each processing
cycle.
Easy access to all the
processes, regardless of work site or section.
Both the input processing staff and the output
quality analysis staff are using the same
system and viewing data in a similar fashion.
For the QMS there are four groups of staff
using the system. The respondent liaison section
is in our Auckland office, data input, editing
is carried out by another group of staff in
Auckland. The survey methods staff who have
an audit role are in Wellington and the Output
analysis and publication is carried out in
Christchurch. Each group works on the same
database and can view processes carried out
by all groups as they occur.
The use of Lotus Notes
is already universal within Statistics New
Zealand, therefore having the survey processing
system also contained within Lotus Notes significantly
improves the basic familiarity of staff with
the system and reduces training time required
to a minimum.
With the system containing
both micro and macro data checks, there has
been a significant move away from the checking
of micro level data and a significant increase
in the amount of macro level data checks.
The macro editing checks
follow a 'top - down' approach, allowing data
to be checked at the survey level, then the
industry level and then a drill down feature
to view the individual respondents of interest.
Finally the system is
dynamic in nature. This means that clients
have the ability to look at survey level results,
drill down to an individual units, alter the
data and then have the results now reflect
the new data values.
Future
developments
Future technical directions are likely to focus
on electronic interfaces, with:
Electronic means for respondents
to supply data, e.g. the supply of input data
by fax, the web, email, respondent direct
data dumps or via the use of touch tone data
entry etc.
New options for sending
reminders, using email, efax as well as the
traditional reminder letter.
On-line questionnaire
image storage & retrieval;
Tax data access: the
use of administrative data to impute non response
and edit survey data from questionnaires;
Direct updating of results
into Time Series Manager (TSM).
Contacts for more information
David Archer
Manager Business Development
Internet: david_archer@stats.govt.nz
