Archive

_________________________________________________________________________________
Document Name: Resource Manual for Customer Surveys Part 1
Date: 10/01/93
Owner: OMB
_________________________________________________________________________________
Title: Resource Manual for Customer Surveys Part 1

Author: OMB

Date: Oct 1993

Content-Type: text

Content-Length:15877

Statistical Policy Office

Office of Management and Budget

Executive Office of the President

October 1993

OFFICE OF INFORMATION AND REGULATORY AFFAIRS

Sally Katzen, Administrator

Katherine K. Wallman, Chief Statistician

Jerry Coffey, Editor,

Resource Manual for Customer Surveys

Second Printing - November, 1993

PREFACE

On September 29, 1993, Director Leon E. Panetta issued a

memorandum to the heads of departments and agencies outlining

initiatives the Office of Management and Budget (OMB) is

undertaking to facilitate the devleopment, review, and operation

of customer surveys. These activities include:

- preparing and disseminating a Resource Manual for Customer

Surveys;

- employing "generic" clearances to expedite approval of

certain voluntary customer surveys; and

- offering through the Joint Program in Survey Methodology

courses and consultations to enhance agency expetrise in

conducting customer surveys.

Together, these efforts have been designed to support surveys of

Federal agency customers "to determine the kind and quality of

services they want and their level of satisfaction with existing

services," as called for in President Clinton's September 11,

1993, Executive Order No. 12862.

This edition of the Resource Manual for Customer Surveys

includes information on techniques for designing and implementing

customer satisfaction surveys, strategies for expediting their

approval under the Paperwork Reduction Act, and opportunities for

learning how to improve their quality. Our decision to issue the

manual in loose-leaf format reflects both philosophical and practical

considerations. From a philosophical perspective, we recognize

that customer surveys represent a new venture for many agencies;

from a practical standpoint, we acknowledge that this first

edition has been assembled in a time frmae that has limited our

sources of information. We plan to supplement this manual as we

learn together. In this connection, we particularly look

forward to benefitting from your insights and experiences. This

process begins with the feedback sheet for updates and suggestions

included at the end of the manual.

In the truest sense, the production of this manual reflects a

partnership of government, academic, and business organizations.

The many individuals who shared their time and talents as the

Resource Manual for Customer Surveys was developed under the

leadership of Jerry Coffey are listed on the following page. We

are particularly indebted to Robert Groves oft he Joint Program

in Survey Methodology and to Fritz Scheuren and Wendy Alvey of

the Internal Revenue Service for their exceptional dedication to

this effort.

ACKNOWLEDGEMENTS

This manual represents the contributions of a great many

individuals and organizations who provided ideas, time and

talent to the project. The idea for the manual grew out of a

Symposium on Client Satisfaction Surveys sponsored by the Social

Security Administration. It drew support from the heads of Federal

statistical agencies who designated key staff to participate in

an Interest Group on Customer Surveys. Robert Groves of the

Joint Program in Survey Methodology organized a consultant panel

representing the private sector, academia, and Fdeeral agencies

which provided invaluable advice and expertise.

Interest Group on Customer Surveys

Jerry Coffey, OMB Statistical Policy Office, Chair

Douglas Fox, Bureau of Economic Analysis

Stanley Freedman, Energy Information Administration

Jack Galvin, Bureau of Labor Statistics

Ray Halley, National Agricultural Statistics Service

and Economic Research Service

Charles Kindermann, Bureau of Justice Statistics

Barry Nussbaum, Environmental Protection Agency

Kenneth A. Riccini, Bureau of the Census

Lucille Riefman, National Center for Education Statistics

Carolyn Shettle, National Science Foundation

Andrew White, National Center for Health Statistics

Glenn White, Internal Revenue Service

Ernest Wilcox, Bureau of Economic Analysis

Consultant Panel for the Resource Manual

Louise Bindseil, Social Security Administration

Marbue Brown, Bell Communications Research (Bellcore)

Jerry Coffey, OMB Statistical Poliyc Office

Claes Fornell, University of Michigan,

School of Business Administration

Robert Groves, Joint Program in Survey Methodology

Lynne Heltman, Veterans' Affairs Department

Graham Kalton, Joint Program in Survey Methodology

Jacob Ludwig, The Gallup Organization

Elizabeth Martin, Bureau of the Censsu

A. Parasuraman, Texas A. & M. University,

College of Business Administration

Stanley Presser, Joint Program in Survey Methodology

Fritz Scheuren, Internal Revenue Service

Clyde Tucker, Bureau of Labor Statistics

Katherine Wallman, OMB Statistical Policy Office

TABLE OF CONTENTS

1. INTRODUCTION

Executive Order 12862

Goals and Organization of the Manual

Future Manual Updates

2. GENERAL OBSERVATIONS ON

CUSTOMER SATISFACTION SURVEYS

What are Customer Satisfaction Surveys?

How are Customer Satisfactino Surveys Used?

How are Customer Satisfaction Surveys Connected

to Agency Operations?

3. ACTIVITIES IN CONDUCTING A CUSTOMER

SATISFACTION SURVEY

Introduction

Top Management Role

Presentation of Steps (the twelve steps)

4. SOME OTHER CONSIDERATIONS

5. EFFICIENTLY MANAGING THE

REVIEW OF SURVEY PLANS

Public and Private Sector Surveys

Delegation

Less Difficult Alternatives

Generic Clearance for Qualitative Studies

Generic Clearance for Quantitative Surveys

Simplified Generic Clearance for Voluntary Customer

Surveys

6. DIRECTORY FOR ASSISTANCE FROM FEDERAL

STATISTICAL AGENCIES (listings by agency)

7. TRAINING OPPORTUNITIES TO BE OFFERED BY THE

JOINT PROGRAM IN SURVEY METHODOLOGY

Short Courses in Survey Techniques

A Lecture Series

Group Consultation on Customer Survey Design Problems

BIBLIOGRAPHY

APPENDIX A -- POLICY DOCUMENTS

Executive Order No. 12862

"Setting Custome rService Standards"

OMB Memorandum M-93-14

"Facilitating Customer Surveys"

APPENDIX B -- SELECTED TECHNICAL NOTES

Introduction

Focus Groups and the PRA

Opinion Scales

Representativeness of the Customer Survey

Some Other Issues

APPENDIX C -- CASE STUDIES (to be supplied as available)

APPENDIX D -- CONTRACTING FOR SURVEYS

STATISTICAL POLICY WORKING PAPER NO. 9

"CONTRACTING FOR SURVEYS"

Feedback sheet for Updates and Suggestions

1. INTRODUCTION

Executive Order 12862

"Setting Customer Service Standards"

On September 11, 1993, the President signed Executive Order

12862 aimed at --

"Ensuring that the Federal Government provides the

highest quality service possible to the American people."

The Executive Order establishes an explicit goal for the quality

of service --

"Customer service equal to the best in business."

That is --

"The highest quality of service delivered to customers by

private organizations providing a comparable or analogous

service."

The Executive Order requires three survey tasks as steps in

establishing and implementing customer service standards --

- Surveying "customers to determine hte kind

and quality of services that they want."

- Surveying customers to determine "their level

of satisfaction with existing services."

- Surveying "front-line employees on barriers to,

and ideas for, matching the best in business."

Our primary focus in this manual is on the first two of these

survey tasks.

1.1 Goals and Organization of the Manual

The Executive Order sets a goal of matching the best customer

service achieved in the private sector. The goal of this manual

is to present methods and practices equal to the best customer

satisfaction research and measurement performances achieved in

the private sector.

To meet this goal the manual is organized around the following

objectives --

- To describe a general approach to customer

surveys (Section 2).

- To lay out the specific steps and issues involved in a data

collection program (Section 3).

- To explore some further considerations in developing a plan

(Section 4).

- To examine ways to streamline the statutory review process

for those data collections covered by the Paperwork

Reduction Act (Section 5).

- To document sources of assistance in Federal statistical

agencies for planning and executing customer surveys

(Section 6).

- To outline training opportunities available from the Joint

Program in Survey Methodology (Section 7).

The manual also provides bibliographical information and

technical appendices on pertinent topics, copies of policy

documents, and a reference report on contracting for statistical

services.

1.2 Future Manual Updates

Just as customer service improvement is an iterative learning

process, this manual is intended to be updated and expanded as

Federal agencies gain useful experience.

The manual has been produced in loose-leaf format so that

additional material can be added from time to time. Each

subsection begins on a new page to facilitate insertion of new

or revised material.

Planned supplements will include brief case studies prepared by

participating agencies and information on new resources as it

becomes available.

2. GENERAL OBSERVATIONS ON

CUSTOMER SATISFACTION SURVEYS

2.1 What are Customer Satisfaction Surveys?

Customer satisfaction surveys are a tool for learning about

agency services from a customer's perspective. They are a form

of evaluation pioneered in the private sector, not in goevrnment.

In fact, customer satisfaction surveys differ substantially from

methods traditionally used in evaluation of Federal programs.

Traditional program evaluation addresses the objective outcomes

of a program, while customer satisfaction surveys focus on

customer perceptions, preferences, and related opinions about an

agency's performance.

Not only do customer surveys provide insight into customer

perceptions, they also can help identify agency operations that

need quality improvement, provide early detection of problems,

and focus attention on areas where remedial training or changes

in existing operations might improve delivery of products or

services.

The one thing that all customer satisfaction surveys have in

common is that they solicit opinions. Both quantitative and

qualitative studies of opinions play important roles:

- Quantitative Studies -- In the ucrrent context, the

object of a quantitative study is to produce

statistical descriptions (careful, repeatable

measurements) of customer satisfaction related

to a fixed set of agency perceptions or

activities. For example,

a set of questions concerning satisfaction

with timeliness, courtesy, accuracy and

other particular aspects of your agency's

operations, administered to a random

sample drawn from a complete, current list

of customers for a particular program --

with the object of making comparative

measurements over time.

- QualitativeS tudies -- Qualitative studies can play

many roles, from the basic task of

understanding customer perceptions and

expectations, to the task of developing survey

instruments (e.g., questionnaires) for a

quantitative study. For example,

a focus group of customers, sasembled to

discuss a specific set of questions -- to

respond in their own words about their

expectations, to relate their own

experiences with agency services, or to

discuss improvements they would like to

see;

or

a cognitive laboratory experiment that

asks volunteer subjects to tell what they

are thinking as they read a survey

question -- what they understand the words

to mean and what they think the question

is asking.

The most important distinction is that qualitative methods are

intended to produce understanding and insight, while

quantitative methods are intended to produce statistical

measurements describing large populations.

2.2 How are Customer Satisfaction Surveys Used?

One example of the way the private sector uses customer

satisfaction surveys is to look at the overall gap between

customer expectations and their perceptions of actual service

performance. These overall gaps are then related to a series of

gaps within an organization that may contribute to the shortfall

in perceived performance:

- The difference between actual customer

expectations and management perceptions of

those expectations.

- Errors in translating management perceptions

into quality standards.

- The shortfall of service or product delivery

relative to standards.

- External communications to clients (e.g.,

advertising) that inflate or otherwise alter

expectations.

While these ideas have generated much discussion and technical

debate (identifying the components of satisfaction, the

consequences of a separate measure of expectations, etc.), they

have also motivated a valuable set of management precepts that

have been applied broadly to programs to improve customer

satisfaction. Among these are to --

- Involve and ensure the support of top

management.

- Define client satisfaction in the client's

terms.

- Establish focused and measurable objectives.

- Define measurements that are "actionable."

- Build customer awareness at all levels of an

organization.

- Tailor measurements to actual operations.

- Carry out satisfaction measurement

continuously.

- Incorporate the results into the regular

management information system.

One final point to consider in applying customer satisfaction

surveys to agency operations is that private sector practice may

need some modification. Given the new ground agencies may have

to break in mounting customer surveys, it is expected that

whatever measurement process is put in place initially, your

agency may need to make a major investment to sharpen the focus

and otherwise improve on beginning efforts.

2.3 How are Customer Satisfaction Surveys Connected

to Agency Operations?

The focus of this manual is on surveys of customers, but these

surveys are not an end in themselves. The ultimate objective is

to use survey data to direct actions to improve customer

service.

To reach this objective, many other kinds of studies, both

qualitative and quantitative, often are required. Among the

sources of information valuable for a program to improve

customer satisfaction are --

- Surveys of employees.

- Inputs from all levels of management.

- Reviews of agency operations.

- Internal performance measurement systems.

- Complaint and suggestion systems.

To be useful (actionable), customer satisfaction surveys need to

be designed so the results obtained can be linked with these

other sources of information. This may affect, for instance,

the frequency of customer satisfaction measurement and the degree to

which such measures are specific to identifiable agency missions

or activities.

It is possible that some existing agency information systems may

need to be redesigned as well. More will be said about

integration issues in the next section, which provides a

step-by-step approach to designing a customer satisfaction

survey.

Navigation Bar For NPR site Back To The NPR Main Page Search the NPR Site NPR Initiatives Links to Other Reinvention Web Sites Reinvention Tools Frequently Asked Questions NPR Speeches NPR News Releases