Document Name: Resource Manual for Customer Surveys Part 5
Date: 10/01/93
Owner: OMB
Title: Resource Manual for Customer Surveys Part 5

Author: OMB

Date: Oct 1993



Many of the decisions required to develop a customer satisfaction

program draw heavily on knowledge of agency operations involved

in providing services. In many cases, however, technical survey

expertise and statistical services may be more efficiently

provided by sources outside the affected program.

In support of the Executive Order, Federal statistical agencies

will offer both advice and specific services to assist in the

design and execution of customer satisfaction surveys. The

information in this section of the manual provides a reference

for the kinds of expertise and services available and the names

of appropriate contacts.

The weekly Group Consultation Sessions described in Section 7 of

this manual will be attended by many of the experts listed here.

These group sessions will be organized both to limit the demands

on the time of these specialists and to provide users of this

manual with "one-stop" opportunities to discuss particular

problems with experts representing a wide range of statistical

methodological experience.

Users are encouraged to take advantage of these group sessions by

registering for them through the procedure described in Section 7.

The names and phone numbers of selected contacts are provided

here for follow-up and for future reference.

6.1 Bureau of Labor Statistics

The Bureau of Labor Statistics, Office of Survey Methods Research

(OSMR), has staff with expertise and experience in the following

areas of survey design:

- The development of protocols for conducting focus


- Procedures for developing and testing questionnaires

(both in the laboratory and in the field), and

- Sample design.

Since the primary function of OSMR staff members is related to

research on methods for improving Bureau programs, their time

available for consulting outside BLS is limited. However, they

are willing to provide advice to the extent they can fit requests

around their primary work assignments. For example, staff can

provide advice on how to develop a focus group protocol or define

a sampling design, but can only participate in the development of

a protocol or sample design on a reimbursable basis. Potential

clients interested in such services should consider contracting

with the Office of Survey Methods for research under an

interagency agreement.

Currently, OSMR has a contract with the Internal Revenue Service,

which includes services related to some of their Customer

Satisfaction Surveys. OSMR also has a Behavioral Science

Research Laboratory, which may be used by Federal agencies on a

space-available basis. For information on these services, please

contact --

Clyde Tucker (202) 606-7370.

6.2 Bureau of Justice Statistics

The Bureau of Justice Statistics will be available to consult and

advise other agencies in the areas of questionnaire development

and design; sample design, implementation, and follow-up methods;

the procurement of high-quality statistical services; and the

analysis of opinion questions.

The Bureau of Justice Statistics does not have a unit dedicated

to planning and implementing customer surveys. However, staff

can be made available to assist other agencies during the

planning phase of customer survey development.

For further information or assistance, please contact

Charles R. Kindermann (202) 616-3489.

6.3 Bureau of the Census

The Census Bureau collects, processes, analyzes and disseminates

statistics on most aspects of the Nation's social and economic

life. It has committed resources and is using its survey

expertise to achieve its own customer service objectives:

- Meet or exceed customer expectations, and

- Improve product lines to meet customer needs.

It has undertaken the design and implementation of external

customer surveys to find out what Census customers expect, how

well products and services are delivered, and how to improve.

This process has provided hands-on experience in most areas

covered by this manual, including both management strategy and

specific methods.

The Census Bureau can provide a wide array of services to other

Federal agencies interested in conducting customer or other type

surveys, including statistical consulting, focus groups,

cognitive research, questionnaire development and testing, and

analysis assistance.

These services are performed, via interagency agreement, on a

reimbursable basis. For further information, please contact

Ken Riccini, (301)-763-5734.

6.4 Energy Information Administration

The Energy Information Administration (EIA), an independent

entity within the Department of Energy, is the principal and

authoritative source of comprehensive energy data for the

Congress, the Federal Government, the States, and the public. To

fulfill its mandate EIA fields approximately 75 surveys

collecting energy information, primarily from establishments.

EIA has considerable experience in the areas of questionnaire

design, sampling, development and implementation of survey

procedures, questionnaire follow-up, and the evaluation of

nonsampling errors. Many of EIA's surveys use contractor support

during the implementation phase, and the agency has considerable

experience in contracting out for statistical services. In the

past several years, EIA has also begun to work with focus groups

to determine user needs and design questionnaires.

For assistance or advice in the area of focus groups, please

contact Lynda Carlson (202) 586-1112, Charles Heath (202) 586-

6860, or Diane Lique (202) 586-6090.

For assistance or advice in the other statistical areas, please

contact Nancy Kirkendall (202) 254-5363.

6.5 Environmental Protection Agency

The Environmental Protection Agency has developed several

customer surveys to assess the effectiveness of EPA programs and

policies. The survey plans are developed by individual offices.

They are reviewed in both the Regulatory Management Division for

their programmatic content and the Environmental Statistics and

Information Division for their adherence to accepted statistical

practices. ESID also has statistical contractors available for

the Agency should extra measures of statistical design or

analysis be appropriate, and has had experience in developing,

reviewing, and contracting for such surveys.

On questions of a statistical nature, please contact

Barry D. Nussbaum (202) 260-2680.

On questions of a programmatic nature, please contact

David S. Schwarz (202) 260-2706.

6.6 National Agricultural Statistics Service

The National Agricultural Statistics Service (NASS), USDA, is

organized, trained, and equipped to collect and disseminate

statistical information. Its mission is to provide accurate,

timely statistical information and services for and about

agriculture and rural America.

To meet the above mission, NASS maintains extensive survey

resources, including list and area sampling frames, quick-

turnaround survey capabilities, field staff and distributed data

processing facilities, and a professional staff of statisticians

that are able to consult on a wide range of issues.

For consulting service in any of the following areas, please

contact Dave Aune, (202) 720-4008 for General Survey Methodology;

Ray Halley, (202) 720-2248 for Data Collection Modes and

Associated Costs, or Survey Activities; Dave Aune, (202) 720-4008

or Carol House, (202) 720-3895 for Edit Analysis and

Summarization of Statistical Data; Jack Nealon, (202) 720-5269

for Questionnaire Design Issues; Carol House, (202) 720-3895 or

Phil Kott, (703) 235-5211,(x102) for Sampling Techniques and

related Summarization Issues; Asa Manning, (202) 720-9485 or Mark

Pierzchala (703) 235- 5218,(x114) for Computer Assisted Data

Collection and Interactive Edit.

6.7 National Center for Education Statistics

The National Center for Education Statistics (NCES) collects

statistics on the condition of education in the United States,

analyzes and reports the meaning and significance of these

statistics, and assists States and local education agencies in

improving their statistical systems. NCES collects data from

both individuals and institutions and provides technical

assistance to the Education Department, to States and to local

education agencies.

NCES technical assistance is available in survey design and

implementation, and in statistical analyses and development.

Expert assistance can be provided on focus groups; cognitive

research; questionnaire development and testing; sample design,

implementation and follow-up methods; procurement of high-quality

statistical services; and analysis of opinion data.

Please address your requests for assistance to Lucille Reifman

(voice: (202) 219-2056, FAX: (202) 219-2061, Internet:

lreifman@INET.ED.GOV.) If you are unable to reach Ms. Reifman,

please call Samuel Peng (202) 219-1643.

6.8 National Center for Health Statistics

The National Center for Health Statistics (NCHS), of the Centers

for Disease Control and Prevention, has a long history of

providing advice and services on all aspects of survey design,

implementation, and analysis. For questions and requests

requiring minimal resource expenditures, immediate advice is

often available. For more involved inquiries and requests for

services, NCHS currently has a reimbursable program that enables

requesting agencies to provide the resources needed.

The Center is particularly strong in developing and pretesting

data collection instruments through its Questionnaire Design and

Research Laboratory and has staff experts in cognition, sampling,

estimation, other survey design and implementation procedures,

health data analysis, and presentation of data. Requests for

advice and services are considered individually and responses are

subject to the availability of appropriate staff and other


For inquiries related to conducting customer surveys, please

contact Andrew White (301) 436-7904.

6.9 National Science Foundation

The Division of Science Resources Studies of the National Science

Foundation collects and analyzes data related to science

resources. In this capacity they have extensive experience in

designing and overseeing the conduct of surveys of individuals

and institutions. The Division is currently evaluating alternate

means of collecting information about its own customers. Staff

are available to provide expert assistance in the design of

customer surveys and the selection of contractors to perform


For further information or assistance, please contact

Carolyn Shettle (703) 306-1780.

6.10 Statistics of Income Division (IRS)

The Statistics of Income Division of IRS has as its primary

mission the responsibility to publish statistical information on

the operation of the U.S. Federal tax system. The Division also

provides general statistical consultation to many areas of the

IRS, notably on sample design and implementation, designing

measurement systems, developing questionnaires, analyzing data

from existing administrative data bases and special collections,

and providing training in basic statistical methodology and in

the use of the measurement systems it helps develop.

Currently the Division also employs the skills and facilities of

the Bureau of Labor Statistics cognitive lab to conduct small

group and one-on-one research for the design of questionnaires

and forms.

For further information or assistance, please contact Mary

Batcher (202) 874-0684, Jeri Mulrow (202) 874-0681, or Glenn

White (202) 874-1114.



The Joint University of Maryland - University of Michigan Program

in Survey Methodology (JPSM) was founded in 1992 on the College

Park campus of the University of Maryland, with funding from the

National Science Foundation. Its mission is to promote

continuous improvement in government statistical activities by

ongoing enhancement of staff skills. Its major activities are to


- Provide continuing professional training and graduate degree

programs in survey statistics and survey methodology, and

- Stimulate (and conduct) advanced research in survey


The JPSM was founded to enhance the likelihood that government

sponsored surveys would use state-of-the-art methods that are

both cost efficient and of high quality. At the present time the

curriculum offers an MS Program in Survey Methodology and

noncredit short courses useful to survey practitioners.

The Joint Program plans three activities aimed at increasing

expertise in customer surveys within Federal government agencies:


- Short courses presenting latest techniques on

components of survey methodology

- A lecture series with talks by prominent researchers

and practitioners in customer surveys

- Group consultations for agency staff charged with the

responsibility for customer surveys, conducted by

experts in the Federal statistical agencies


7.1 Short Courses in Survey Techniques

These educational activities are not focused on customer surveys

alone, but communicate the theory and applied skills necessary to

mount surveys on all subject matters. Relevant short courses,

scheduled prior to the September 11, 1993, Executive Order, are --

Introduction to Questionnaire Design (Dr. Nora Cate Schaeffer,

University of Wisconsin), November 17-18, 1993

This course gives practical guidance to those who have

written questionnaires but who are not familiar with

research on question design, who are beginning to

design questionnaires, or who use survey data but do

not themselves design questionnaires. The course

focuses on paper interview schedules, but some issues

affecting self-administered and computer-assisted

instruments are discussed at several points.

Cognitive and Communicative Aspects of Survey Measurement

(Dr. Norbert Schwarz, University of Michigan),

December 6-7, 1993

This course provides an introduction to selected

theoretical principles of cognitive and social

psychology and their implications for questionnaire

construction. This course is designed to alert

participants to common problems in questionnaires and

to apply cognitive principles in developing possible

solutions. No prior background in cognitive psychology

is assumed.

Self-Administered/Mail Surveys (Dr. Don A. Dillman, Washington

State University), February 23-24, 1994

This course presents practical aspects of designing and

implementing high quality self-administered/mail

surveys. The course provides the state-of-the-art

perspective on obtaining high response rates with low

measurement error from self-administered/mail surveys.

It presents the latest findings on differences between

telephone and self-administered survey responses. It

provides practical advice on designing questionnaires

and implementation procedures.

All three of these short courses will be held in a downtown

Washington, D.C. hotel. There are registration fees of $225 to

$350 for each 2-day course. For further information call 1-800-

937-9320 and ask for registration forms for the short courses of

particular interest to you.

There is also a credit bearing course in Applied Sampling from

January 6 - April 19, to be held on the College Park campus. For

more information, call (301) 314-7911 or fax (301) 314-7912.

At the request of OMB for assistance in the area of customer

surveys, additional short courses are being planned that will

provide training in relevant survey methodology. Among these are


An Introduction to Surveys (now being planned for December 1993)

This course assumes no knowledge on the part of

participants about surveys. It reviews the key steps

of survey design and implementation in a format not

unlike the step-by-step approach contained in Section 3

of this Resource Manual. Development of sampling

frames, sample design, questionnaire design, data

collection methods, and analytic issues are reviewed in

brief. For each of the steps key questions that must

be addressed are reviewed and key pitfalls that new

survey teams face are described. The course is

designed both for those who are considering mounting

their own surveys and those who will oversee the work

of outside contractors.

Focus Groups as a Tool in Developing Survey Questionnaires

(now being planned for January 1994)

This course will introduce participants to the design

and conduct of focus groups, the technique proposed for

investigating customer perspectives on agency services

and products. The course will describe recruitment

strategies for focus group participants, scripting the

group discussion, the role of the moderator, methods of

recording the group discussion, analysis of the

discussion, summarization of the results, and

presentation of focus group results. It will

concentrate on how focus groups can inform the process

of developing a survey questionnaire to assure that the

questions use language relevant to the population


An Introduction to Survey Sampling

(now being planned for January 1994)

This course assumes no statistical training among the

participants, but is designed to describe key issues in

the design and implementation of probability samples.

The course describes concepts and terminology key to

understanding why carefully designed samples are

necessary to create surveys that provide accurate

information. The course will address a variety of list

formats for populations that can be used for selecting

samples, with lessons on how to avoid various pitfalls.

Some additional courses that may be offered in the early months

of 1994 are --

An Introduction to Measurement of Attitudes

This course will review alternative concepts of

attitude measurement with focus on selection of

multiple indicators, impact of alternative response

scales on answers, order effects on responses, impacts

of response labels, filter questions for respondents

who claim to have no opinion on an issue, and other

features of attitude questions.

Practical Graphical Techniques to Summarize Data

This course describes methods to summarize data

visually. Graphical techniques that are simple to

understand are reviewed, with descriptions of what

software can be used to create the images. The

instructor will also alert participants to visually

deceptive presentations in order to avoid

miscommunication graphically.

Other courses will be planned as needed.

To get on a mailing list for these short courses, telephone
Navigation Bar For NPR site Back To The NPR Main Page Search the NPR Site NPR Initiatives Links to Other Reinvention Web Sites Reinvention Tools Frequently Asked Questions NPR Speeches NPR News Releases