Archive
_________________________________________________________________________________
Document Name: Resource Manual for Customer Surveys Part 1
Date: 10/01/93
Owner: OMB
_________________________________________________________________________________
Title: Resource Manual for Customer Surveys Part 1
Author: OMB
Date: Oct 1993
Content-Type: text
Content-Length:15877
Statistical Policy Office
Office of Management and Budget
Executive Office of the President
October 1993
OFFICE OF INFORMATION AND REGULATORY AFFAIRS
Sally Katzen, Administrator
Katherine K. Wallman, Chief Statistician
Jerry Coffey, Editor,
Resource Manual for Customer Surveys
Second Printing - November, 1993
PREFACE
On September 29, 1993, Director Leon E. Panetta issued a
memorandum to the heads of departments and agencies outlining
initiatives the Office of Management and Budget (OMB) is
undertaking to facilitate the devleopment, review, and operation
of customer surveys. These activities include:
- preparing and disseminating a Resource Manual for Customer
Surveys;
- employing "generic" clearances to expedite approval of
certain voluntary customer surveys; and
- offering through the Joint Program in Survey Methodology
courses and consultations to enhance agency expetrise in
conducting customer surveys.
Together, these efforts have been designed to support surveys of
Federal agency customers "to determine the kind and quality of
services they want and their level of satisfaction with existing
services," as called for in President Clinton's September 11,
1993, Executive Order No. 12862.
This edition of the Resource Manual for Customer Surveys
includes information on techniques for designing and implementing
customer satisfaction surveys, strategies for expediting their
approval under the Paperwork Reduction Act, and opportunities for
learning how to improve their quality. Our decision to issue the
manual in loose-leaf format reflects both philosophical and practical
considerations. From a philosophical perspective, we recognize
that customer surveys represent a new venture for many agencies;
from a practical standpoint, we acknowledge that this first
edition has been assembled in a time frmae that has limited our
sources of information. We plan to supplement this manual as we
learn together. In this connection, we particularly look
forward to benefitting from your insights and experiences. This
process begins with the feedback sheet for updates and suggestions
included at the end of the manual.
In the truest sense, the production of this manual reflects a
partnership of government, academic, and business organizations.
The many individuals who shared their time and talents as the
Resource Manual for Customer Surveys was developed under the
leadership of Jerry Coffey are listed on the following page. We
are particularly indebted to Robert Groves oft he Joint Program
in Survey Methodology and to Fritz Scheuren and Wendy Alvey of
the Internal Revenue Service for their exceptional dedication to
this effort.
ACKNOWLEDGEMENTS
This manual represents the contributions of a great many
individuals and organizations who provided ideas, time and
talent to the project. The idea for the manual grew out of a
Symposium on Client Satisfaction Surveys sponsored by the Social
Security Administration. It drew support from the heads of Federal
statistical agencies who designated key staff to participate in
an Interest Group on Customer Surveys. Robert Groves of the
Joint Program in Survey Methodology organized a consultant panel
representing the private sector, academia, and Fdeeral agencies
which provided invaluable advice and expertise.
Interest Group on Customer Surveys
Jerry Coffey, OMB Statistical Policy Office, Chair
Douglas Fox, Bureau of Economic Analysis
Stanley Freedman, Energy Information Administration
Jack Galvin, Bureau of Labor Statistics
Ray Halley, National Agricultural Statistics Service
and Economic Research Service
Charles Kindermann, Bureau of Justice Statistics
Barry Nussbaum, Environmental Protection Agency
Kenneth A. Riccini, Bureau of the Census
Lucille Riefman, National Center for Education Statistics
Carolyn Shettle, National Science Foundation
Andrew White, National Center for Health Statistics
Glenn White, Internal Revenue Service
Ernest Wilcox, Bureau of Economic Analysis
Consultant Panel for the Resource Manual
Louise Bindseil, Social Security Administration
Marbue Brown, Bell Communications Research (Bellcore)
Jerry Coffey, OMB Statistical Poliyc Office
Claes Fornell, University of Michigan,
School of Business Administration
Robert Groves, Joint Program in Survey Methodology
Lynne Heltman, Veterans' Affairs Department
Graham Kalton, Joint Program in Survey Methodology
Jacob Ludwig, The Gallup Organization
Elizabeth Martin, Bureau of the Censsu
A. Parasuraman, Texas A. & M. University,
College of Business Administration
Stanley Presser, Joint Program in Survey Methodology
Fritz Scheuren, Internal Revenue Service
Clyde Tucker, Bureau of Labor Statistics
Katherine Wallman, OMB Statistical Policy Office
TABLE OF CONTENTS
1. INTRODUCTION
Executive Order 12862
Goals and Organization of the Manual
Future Manual Updates
2. GENERAL OBSERVATIONS ON
CUSTOMER SATISFACTION SURVEYS
What are Customer Satisfaction Surveys?
How are Customer Satisfactino Surveys Used?
How are Customer Satisfaction Surveys Connected
to Agency Operations?
3. ACTIVITIES IN CONDUCTING A CUSTOMER
SATISFACTION SURVEY
Introduction
Top Management Role
Presentation of Steps (the twelve steps)
4. SOME OTHER CONSIDERATIONS
5. EFFICIENTLY MANAGING THE
REVIEW OF SURVEY PLANS
Public and Private Sector Surveys
Delegation
Less Difficult Alternatives
Generic Clearance for Qualitative Studies
Generic Clearance for Quantitative Surveys
Simplified Generic Clearance for Voluntary Customer
Surveys
6. DIRECTORY FOR ASSISTANCE FROM FEDERAL
STATISTICAL AGENCIES (listings by agency)
7. TRAINING OPPORTUNITIES TO BE OFFERED BY THE
JOINT PROGRAM IN SURVEY METHODOLOGY
Short Courses in Survey Techniques
A Lecture Series
Group Consultation on Customer Survey Design Problems
BIBLIOGRAPHY
APPENDIX A -- POLICY DOCUMENTS
Executive Order No. 12862
"Setting Custome rService Standards"
OMB Memorandum M-93-14
"Facilitating Customer Surveys"
APPENDIX B -- SELECTED TECHNICAL NOTES
Introduction
Focus Groups and the PRA
Opinion Scales
Representativeness of the Customer Survey
Some Other Issues
APPENDIX C -- CASE STUDIES (to be supplied as available)
APPENDIX D -- CONTRACTING FOR SURVEYS
STATISTICAL POLICY WORKING PAPER NO. 9
"CONTRACTING FOR SURVEYS"
Feedback sheet for Updates and Suggestions
1. INTRODUCTION
Executive Order 12862
"Setting Customer Service Standards"
On September 11, 1993, the President signed Executive Order
12862 aimed at --
"Ensuring that the Federal Government provides the
highest quality service possible to the American people."
The Executive Order establishes an explicit goal for the quality
of service --
"Customer service equal to the best in business."
That is --
"The highest quality of service delivered to customers by
private organizations providing a comparable or analogous
service."
The Executive Order requires three survey tasks as steps in
establishing and implementing customer service standards --
- Surveying "customers to determine hte kind
and quality of services that they want."
- Surveying customers to determine "their level
of satisfaction with existing services."
- Surveying "front-line employees on barriers to,
and ideas for, matching the best in business."
Our primary focus in this manual is on the first two of these
survey tasks.
1.1 Goals and Organization of the Manual
The Executive Order sets a goal of matching the best customer
service achieved in the private sector. The goal of this manual
is to present methods and practices equal to the best customer
satisfaction research and measurement performances achieved in
the private sector.
To meet this goal the manual is organized around the following
objectives --
- To describe a general approach to customer
surveys (Section 2).
- To lay out the specific steps and issues involved in a data
collection program (Section 3).
- To explore some further considerations in developing a plan
(Section 4).
- To examine ways to streamline the statutory review process
for those data collections covered by the Paperwork
Reduction Act (Section 5).
- To document sources of assistance in Federal statistical
agencies for planning and executing customer surveys
(Section 6).
- To outline training opportunities available from the Joint
Program in Survey Methodology (Section 7).
The manual also provides bibliographical information and
technical appendices on pertinent topics, copies of policy
documents, and a reference report on contracting for statistical
services.
1.2 Future Manual Updates
Just as customer service improvement is an iterative learning
process, this manual is intended to be updated and expanded as
Federal agencies gain useful experience.
The manual has been produced in loose-leaf format so that
additional material can be added from time to time. Each
subsection begins on a new page to facilitate insertion of new
or revised material.
Planned supplements will include brief case studies prepared by
participating agencies and information on new resources as it
becomes available.
2. GENERAL OBSERVATIONS ON
CUSTOMER SATISFACTION SURVEYS
2.1 What are Customer Satisfaction Surveys?
Customer satisfaction surveys are a tool for learning about
agency services from a customer's perspective. They are a form
of evaluation pioneered in the private sector, not in goevrnment.
In fact, customer satisfaction surveys differ substantially from
methods traditionally used in evaluation of Federal programs.
Traditional program evaluation addresses the objective outcomes
of a program, while customer satisfaction surveys focus on
customer perceptions, preferences, and related opinions about an
agency's performance.
Not only do customer surveys provide insight into customer
perceptions, they also can help identify agency operations that
need quality improvement, provide early detection of problems,
and focus attention on areas where remedial training or changes
in existing operations might improve delivery of products or
services.
The one thing that all customer satisfaction surveys have in
common is that they solicit opinions. Both quantitative and
qualitative studies of opinions play important roles:
- Quantitative Studies -- In the ucrrent context, the
object of a quantitative study is to produce
statistical descriptions (careful, repeatable
measurements) of customer satisfaction related
to a fixed set of agency perceptions or
activities. For example,
a set of questions concerning satisfaction
with timeliness, courtesy, accuracy and
other particular aspects of your agency's
operations, administered to a random
sample drawn from a complete, current list
of customers for a particular program --
with the object of making comparative
measurements over time.
- QualitativeS tudies -- Qualitative studies can play
many roles, from the basic task of
understanding customer perceptions and
expectations, to the task of developing survey
instruments (e.g., questionnaires) for a
quantitative study. For example,
a focus group of customers, sasembled to
discuss a specific set of questions -- to
respond in their own words about their
expectations, to relate their own
experiences with agency services, or to
discuss improvements they would like to
see;
or
a cognitive laboratory experiment that
asks volunteer subjects to tell what they
are thinking as they read a survey
question -- what they understand the words
to mean and what they think the question
is asking.
The most important distinction is that qualitative methods are
intended to produce understanding and insight, while
quantitative methods are intended to produce statistical
measurements describing large populations.
2.2 How are Customer Satisfaction Surveys Used?
One example of the way the private sector uses customer
satisfaction surveys is to look at the overall gap between
customer expectations and their perceptions of actual service
performance. These overall gaps are then related to a series of
gaps within an organization that may contribute to the shortfall
in perceived performance:
- The difference between actual customer
expectations and management perceptions of
those expectations.
- Errors in translating management perceptions
into quality standards.
- The shortfall of service or product delivery
relative to standards.
- External communications to clients (e.g.,
advertising) that inflate or otherwise alter
expectations.
While these ideas have generated much discussion and technical
debate (identifying the components of satisfaction, the
consequences of a separate measure of expectations, etc.), they
have also motivated a valuable set of management precepts that
have been applied broadly to programs to improve customer
satisfaction. Among these are to --
- Involve and ensure the support of top
management.
- Define client satisfaction in the client's
terms.
- Establish focused and measurable objectives.
- Define measurements that are "actionable."
- Build customer awareness at all levels of an
organization.
- Tailor measurements to actual operations.
- Carry out satisfaction measurement
continuously.
- Incorporate the results into the regular
management information system.
One final point to consider in applying customer satisfaction
surveys to agency operations is that private sector practice may
need some modification. Given the new ground agencies may have
to break in mounting customer surveys, it is expected that
whatever measurement process is put in place initially, your
agency may need to make a major investment to sharpen the focus
and otherwise improve on beginning efforts.
2.3 How are Customer Satisfaction Surveys Connected
to Agency Operations?
The focus of this manual is on surveys of customers, but these
surveys are not an end in themselves. The ultimate objective is
to use survey data to direct actions to improve customer
service.
To reach this objective, many other kinds of studies, both
qualitative and quantitative, often are required. Among the
sources of information valuable for a program to improve
customer satisfaction are --
- Surveys of employees.
- Inputs from all levels of management.
- Reviews of agency operations.
- Internal performance measurement systems.
- Complaint and suggestion systems.
To be useful (actionable), customer satisfaction surveys need to
be designed so the results obtained can be linked with these
other sources of information. This may affect, for instance,
the frequency of customer satisfaction measurement and the degree to
which such measures are specific to identifiable agency missions
or activities.
It is possible that some existing agency information systems may
need to be redesigned as well. More will be said about
integration issues in the next section, which provides a
step-by-step approach to designing a customer satisfaction
survey.