Archive
_________________________________________________________________________________
Document Name: Resource Manual for Customer Surveys Part 5
Date: 10/01/93
Owner: OMB
_________________________________________________________________________________
Title: Resource Manual for Customer Surveys Part 5
Author: OMB
Date: Oct 1993
6. DIRECTORY FOR ASSISTANCE FROM
FEDERAL STATISTICAL AGENCIES
Many of the decisions required to develop a customer satisfaction
program draw heavily on knowledge of agency operations involved
in providing services. In many cases, however, technical survey
expertise and statistical services may be more efficiently
provided by sources outside the affected program.
In support of the Executive Order, Federal statistical agencies
will offer both advice and specific services to assist in the
design and execution of customer satisfaction surveys. The
information in this section of the manual provides a reference
for the kinds of expertise and services available and the names
of appropriate contacts.
The weekly Group Consultation Sessions described in Section 7 of
this manual will be attended by many of the experts listed here.
These group sessions will be organized both to limit the demands
on the time of these specialists and to provide users of this
manual with "one-stop" opportunities to discuss particular
problems with experts representing a wide range of statistical
methodological experience.
Users are encouraged to take advantage of these group sessions by
registering for them through the procedure described in Section 7.
The names and phone numbers of selected contacts are provided
here for follow-up and for future reference.
6.1 Bureau of Labor Statistics
The Bureau of Labor Statistics, Office of Survey Methods Research
(OSMR), has staff with expertise and experience in the following
areas of survey design:
- The development of protocols for conducting focus
groups,
- Procedures for developing and testing questionnaires
(both in the laboratory and in the field), and
- Sample design.
Since the primary function of OSMR staff members is related to
research on methods for improving Bureau programs, their time
available for consulting outside BLS is limited. However, they
are willing to provide advice to the extent they can fit requests
around their primary work assignments. For example, staff can
provide advice on how to develop a focus group protocol or define
a sampling design, but can only participate in the development of
a protocol or sample design on a reimbursable basis. Potential
clients interested in such services should consider contracting
with the Office of Survey Methods for research under an
interagency agreement.
Currently, OSMR has a contract with the Internal Revenue Service,
which includes services related to some of their Customer
Satisfaction Surveys. OSMR also has a Behavioral Science
Research Laboratory, which may be used by Federal agencies on a
space-available basis. For information on these services, please
contact --
Clyde Tucker (202) 606-7370.
6.2 Bureau of Justice Statistics
The Bureau of Justice Statistics will be available to consult and
advise other agencies in the areas of questionnaire development
and design; sample design, implementation, and follow-up methods;
the procurement of high-quality statistical services; and the
analysis of opinion questions.
The Bureau of Justice Statistics does not have a unit dedicated
to planning and implementing customer surveys. However, staff
can be made available to assist other agencies during the
planning phase of customer survey development.
For further information or assistance, please contact
Charles R. Kindermann (202) 616-3489.
6.3 Bureau of the Census
The Census Bureau collects, processes, analyzes and disseminates
statistics on most aspects of the Nation's social and economic
life. It has committed resources and is using its survey
expertise to achieve its own customer service objectives:
- Meet or exceed customer expectations, and
- Improve product lines to meet customer needs.
It has undertaken the design and implementation of external
customer surveys to find out what Census customers expect, how
well products and services are delivered, and how to improve.
This process has provided hands-on experience in most areas
covered by this manual, including both management strategy and
specific methods.
The Census Bureau can provide a wide array of services to other
Federal agencies interested in conducting customer or other type
surveys, including statistical consulting, focus groups,
cognitive research, questionnaire development and testing, and
analysis assistance.
These services are performed, via interagency agreement, on a
reimbursable basis. For further information, please contact
Ken Riccini, (301)-763-5734.
6.4 Energy Information Administration
The Energy Information Administration (EIA), an independent
entity within the Department of Energy, is the principal and
authoritative source of comprehensive energy data for the
Congress, the Federal Government, the States, and the public. To
fulfill its mandate EIA fields approximately 75 surveys
collecting energy information, primarily from establishments.
EIA has considerable experience in the areas of questionnaire
design, sampling, development and implementation of survey
procedures, questionnaire follow-up, and the evaluation of
nonsampling errors. Many of EIA's surveys use contractor support
during the implementation phase, and the agency has considerable
experience in contracting out for statistical services. In the
past several years, EIA has also begun to work with focus groups
to determine user needs and design questionnaires.
For assistance or advice in the area of focus groups, please
contact Lynda Carlson (202) 586-1112, Charles Heath (202) 586-
6860, or Diane Lique (202) 586-6090.
For assistance or advice in the other statistical areas, please
contact Nancy Kirkendall (202) 254-5363.
6.5 Environmental Protection Agency
The Environmental Protection Agency has developed several
customer surveys to assess the effectiveness of EPA programs and
policies. The survey plans are developed by individual offices.
They are reviewed in both the Regulatory Management Division for
their programmatic content and the Environmental Statistics and
Information Division for their adherence to accepted statistical
practices. ESID also has statistical contractors available for
the Agency should extra measures of statistical design or
analysis be appropriate, and has had experience in developing,
reviewing, and contracting for such surveys.
On questions of a statistical nature, please contact
Barry D. Nussbaum (202) 260-2680.
On questions of a programmatic nature, please contact
David S. Schwarz (202) 260-2706.
6.6 National Agricultural Statistics Service
The National Agricultural Statistics Service (NASS), USDA, is
organized, trained, and equipped to collect and disseminate
statistical information. Its mission is to provide accurate,
timely statistical information and services for and about
agriculture and rural America.
To meet the above mission, NASS maintains extensive survey
resources, including list and area sampling frames, quick-
turnaround survey capabilities, field staff and distributed data
processing facilities, and a professional staff of statisticians
that are able to consult on a wide range of issues.
For consulting service in any of the following areas, please
contact Dave Aune, (202) 720-4008 for General Survey Methodology;
Ray Halley, (202) 720-2248 for Data Collection Modes and
Associated Costs, or Survey Activities; Dave Aune, (202) 720-4008
or Carol House, (202) 720-3895 for Edit Analysis and
Summarization of Statistical Data; Jack Nealon, (202) 720-5269
for Questionnaire Design Issues; Carol House, (202) 720-3895 or
Phil Kott, (703) 235-5211,(x102) for Sampling Techniques and
related Summarization Issues; Asa Manning, (202) 720-9485 or Mark
Pierzchala (703) 235- 5218,(x114) for Computer Assisted Data
Collection and Interactive Edit.
6.7 National Center for Education Statistics
The National Center for Education Statistics (NCES) collects
statistics on the condition of education in the United States,
analyzes and reports the meaning and significance of these
statistics, and assists States and local education agencies in
improving their statistical systems. NCES collects data from
both individuals and institutions and provides technical
assistance to the Education Department, to States and to local
education agencies.
NCES technical assistance is available in survey design and
implementation, and in statistical analyses and development.
Expert assistance can be provided on focus groups; cognitive
research; questionnaire development and testing; sample design,
implementation and follow-up methods; procurement of high-quality
statistical services; and analysis of opinion data.
Please address your requests for assistance to Lucille Reifman
(voice: (202) 219-2056, FAX: (202) 219-2061, Internet:
lreifman@INET.ED.GOV.) If you are unable to reach Ms. Reifman,
please call Samuel Peng (202) 219-1643.
6.8 National Center for Health Statistics
The National Center for Health Statistics (NCHS), of the Centers
for Disease Control and Prevention, has a long history of
providing advice and services on all aspects of survey design,
implementation, and analysis. For questions and requests
requiring minimal resource expenditures, immediate advice is
often available. For more involved inquiries and requests for
services, NCHS currently has a reimbursable program that enables
requesting agencies to provide the resources needed.
The Center is particularly strong in developing and pretesting
data collection instruments through its Questionnaire Design and
Research Laboratory and has staff experts in cognition, sampling,
estimation, other survey design and implementation procedures,
health data analysis, and presentation of data. Requests for
advice and services are considered individually and responses are
subject to the availability of appropriate staff and other
resources.
For inquiries related to conducting customer surveys, please
contact Andrew White (301) 436-7904.
6.9 National Science Foundation
The Division of Science Resources Studies of the National Science
Foundation collects and analyzes data related to science
resources. In this capacity they have extensive experience in
designing and overseeing the conduct of surveys of individuals
and institutions. The Division is currently evaluating alternate
means of collecting information about its own customers. Staff
are available to provide expert assistance in the design of
customer surveys and the selection of contractors to perform
them.
For further information or assistance, please contact
Carolyn Shettle (703) 306-1780.
6.10 Statistics of Income Division (IRS)
The Statistics of Income Division of IRS has as its primary
mission the responsibility to publish statistical information on
the operation of the U.S. Federal tax system. The Division also
provides general statistical consultation to many areas of the
IRS, notably on sample design and implementation, designing
measurement systems, developing questionnaires, analyzing data
from existing administrative data bases and special collections,
and providing training in basic statistical methodology and in
the use of the measurement systems it helps develop.
Currently the Division also employs the skills and facilities of
the Bureau of Labor Statistics cognitive lab to conduct small
group and one-on-one research for the design of questionnaires
and forms.
For further information or assistance, please contact Mary
Batcher (202) 874-0684, Jeri Mulrow (202) 874-0681, or Glenn
White (202) 874-1114.
7. TRAINING OPPORTUNITIES TO BE OFFERED BY
THE JOINT PROGRAM IN SURVEY METHODOLOGY
The Joint University of Maryland - University of Michigan Program
in Survey Methodology (JPSM) was founded in 1992 on the College
Park campus of the University of Maryland, with funding from the
National Science Foundation. Its mission is to promote
continuous improvement in government statistical activities by
ongoing enhancement of staff skills. Its major activities are to
--
- Provide continuing professional training and graduate degree
programs in survey statistics and survey methodology, and
- Stimulate (and conduct) advanced research in survey
methodology.
The JPSM was founded to enhance the likelihood that government
sponsored surveys would use state-of-the-art methods that are
both cost efficient and of high quality. At the present time the
curriculum offers an MS Program in Survey Methodology and
noncredit short courses useful to survey practitioners.
The Joint Program plans three activities aimed at increasing
expertise in customer surveys within Federal government agencies:
____________________________________________________________
- Short courses presenting latest techniques on
components of survey methodology
- A lecture series with talks by prominent researchers
and practitioners in customer surveys
- Group consultations for agency staff charged with the
responsibility for customer surveys, conducted by
experts in the Federal statistical agencies
____________________________________________________________
7.1 Short Courses in Survey Techniques
These educational activities are not focused on customer surveys
alone, but communicate the theory and applied skills necessary to
mount surveys on all subject matters. Relevant short courses,
scheduled prior to the September 11, 1993, Executive Order, are --
Introduction to Questionnaire Design (Dr. Nora Cate Schaeffer,
University of Wisconsin), November 17-18, 1993
This course gives practical guidance to those who have
written questionnaires but who are not familiar with
research on question design, who are beginning to
design questionnaires, or who use survey data but do
not themselves design questionnaires. The course
focuses on paper interview schedules, but some issues
affecting self-administered and computer-assisted
instruments are discussed at several points.
Cognitive and Communicative Aspects of Survey Measurement
(Dr. Norbert Schwarz, University of Michigan),
December 6-7, 1993
This course provides an introduction to selected
theoretical principles of cognitive and social
psychology and their implications for questionnaire
construction. This course is designed to alert
participants to common problems in questionnaires and
to apply cognitive principles in developing possible
solutions. No prior background in cognitive psychology
is assumed.
Self-Administered/Mail Surveys (Dr. Don A. Dillman, Washington
State University), February 23-24, 1994
This course presents practical aspects of designing and
implementing high quality self-administered/mail
surveys. The course provides the state-of-the-art
perspective on obtaining high response rates with low
measurement error from self-administered/mail surveys.
It presents the latest findings on differences between
telephone and self-administered survey responses. It
provides practical advice on designing questionnaires
and implementation procedures.
All three of these short courses will be held in a downtown
Washington, D.C. hotel. There are registration fees of $225 to
$350 for each 2-day course. For further information call 1-800-
937-9320 and ask for registration forms for the short courses of
particular interest to you.
There is also a credit bearing course in Applied Sampling from
January 6 - April 19, to be held on the College Park campus. For
more information, call (301) 314-7911 or fax (301) 314-7912.
At the request of OMB for assistance in the area of customer
surveys, additional short courses are being planned that will
provide training in relevant survey methodology. Among these are
--
An Introduction to Surveys (now being planned for December 1993)
This course assumes no knowledge on the part of
participants about surveys. It reviews the key steps
of survey design and implementation in a format not
unlike the step-by-step approach contained in Section 3
of this Resource Manual. Development of sampling
frames, sample design, questionnaire design, data
collection methods, and analytic issues are reviewed in
brief. For each of the steps key questions that must
be addressed are reviewed and key pitfalls that new
survey teams face are described. The course is
designed both for those who are considering mounting
their own surveys and those who will oversee the work
of outside contractors.
Focus Groups as a Tool in Developing Survey Questionnaires
(now being planned for January 1994)
This course will introduce participants to the design
and conduct of focus groups, the technique proposed for
investigating customer perspectives on agency services
and products. The course will describe recruitment
strategies for focus group participants, scripting the
group discussion, the role of the moderator, methods of
recording the group discussion, analysis of the
discussion, summarization of the results, and
presentation of focus group results. It will
concentrate on how focus groups can inform the process
of developing a survey questionnaire to assure that the
questions use language relevant to the population
studied.
An Introduction to Survey Sampling
(now being planned for January 1994)
This course assumes no statistical training among the
participants, but is designed to describe key issues in
the design and implementation of probability samples.
The course describes concepts and terminology key to
understanding why carefully designed samples are
necessary to create surveys that provide accurate
information. The course will address a variety of list
formats for populations that can be used for selecting
samples, with lessons on how to avoid various pitfalls.
Some additional courses that may be offered in the early months
of 1994 are --
An Introduction to Measurement of Attitudes
This course will review alternative concepts of
attitude measurement with focus on selection of
multiple indicators, impact of alternative response
scales on answers, order effects on responses, impacts
of response labels, filter questions for respondents
who claim to have no opinion on an issue, and other
features of attitude questions.
Practical Graphical Techniques to Summarize Data
This course describes methods to summarize data
visually. Graphical techniques that are simple to
understand are reviewed, with descriptions of what
software can be used to create the images. The
instructor will also alert participants to visually
deceptive presentations in order to avoid
miscommunication graphically.
Other courses will be planned as needed.
To get on a mailing list for these short courses, telephone