____________________________________________________________________________
Establishing national aims for improvement alone will not lead to better quality health care. Standardized measures of quality are needed to track the health care industry's progress in reaching the aims established for the nation and to guide public health planning and policy making. Comparative information on quality also is needed for individual consumers, group purchasers, and others to use in selecting health care providers and health plans. Furthermore, valid and stable quality measures are integral to health plans' and providers' efforts to improve performance, and, when standardized, encourage health care organizations to learn from each other through a process of benchmarking.
Despite a growing number of efforts to measure and report on health care quality, useful information is neither uniformly nor widely available. Improving our ability to measure quality has been the object of significant public and private-sector activity over the last decade, reflecting the expectation that measurement can serve both as a catalyst and a tool for improvement. While considerable advancements have been made in the quality measurement field in recent years, current efforts fall short of fully meeting users' needs, and often are duplicative and unduly burdensome on health care providers, plans, and others.
The Commission has identified several steps critical to advancement. First, core sets of standardized quality measures should be identified to address the common information needs of individual consumers, group purchasers, health plans and providers, oversight organizations, and public health and policy officials. These measures must focus on areas that can have a meaningful impact on the quality of health care and be consistent with national aims for improvement. In addition, new and better quality measures should be developed to fill important gaps in existing measurement sets, and steps should be taken to assure that comparative information on health care quality is valid, reliable, meaningful, comprehensible, and widely available through multiple means of dissemination.
Core sets of quality measures should be identified for standardized reporting by each sector of the health care industry. These sets should reflect measurement priorities developed by taking into account both national aims for improvement and the information needs of consumers (especially vulnerable populations), purchasers, providers, health organizations, and public health and policy officials. Pertinent measures developed by the research community, quality oversight organizations, health care organizations, and others should be evaluated for potential inclusion in these reporting sets.
A framework and capacity for quality measurement and reporting should be developed in tandem with the standardization of quality measures for reporting. Specific functions to be undertaken include:
All sectors of the health care industry should support the focused development of quality measures that enhance and improve the ability to evaluate and improve health care. Comprehensive sets of quality measures are needed to reflect the full continuum of health care, but measures currently are lacking in a number of important areas. Types of measures that are needed include:
Quality measurement requirements and the detailed specifications for measurement and reporting should be determined through a stable and predictable mechanism. The Commission's specific recommendations on the creation of such a stable and predictable mechanism -- a Forum for Quality Measurement and Reporting -- are described in Chapter Five.
This entity, which would be responsible for designating core sets of measures for reporting for each sector of the health care industry, should:
Steps should be taken to assure that comparative information on health care quality is valid, reliable, comprehensible, and widely available in the public domain. Specifically:
Measurement is a critical tool for improving quality and for supplying the information consumers and purchasers need in a market-driven health system. A large and growing number of health care quality measurement and reporting initiatives have recently developed in an effort to address the demand for information.
Quality measures can influence health care quality in a number of distinct ways. They can serve to provide individual consumers and group purchasers with better information for making health care choices. They can also help oversight organizations in the public and private sectors safeguard consumers from poor quality. They can inform and support public health initiatives and public policy making, and they can serve in health care providers' efforts to improve the quality of care they furnish. In conjunction with each other, these approaches for using quality measures can provide a framework for assuring, fostering, and improving health care quality.
Quality Measurement to Support Marketplace Decisionmaking. Many health economists and other experts believe that disseminating information about the quality of care furnished by competing providers and organizations can help improve quality overall. If, individually or collectively, purchasers reward those that demonstrate better performance and higher quality, competition on those dimensions should increase. Health care consumers would choose the providers that furnish the level of quality they want or that perform well in areas important to them. Providers and health care organizations would then respond to choices made in the health care marketplace by striving to deliver the kind of quality that consumers want.
For quality measures to play such a role in health care decisionmaking, comparative information generated through measurement must be available to those who face choices among alternative health care providers or organizations. One group of potential users of this type of information is individual consumers, who usually have at least some choice of primary care practitioner and other providers within their health plan, and who sometimes also have a choice of plans. Public and private-sector group purchasers also face choices among potential contractors, normally health plans but sometimes also providers. Finally, health plans themselves can use quality measures in their decisionmaking by using them to select providers for participation in their networks.
Quality Measurement to Support Oversight Efforts. Public and private oversight organizations utilize quality measurement in their efforts to safeguard the public from poor quality health care, distinguish those who furnish better care, and stimulate ongoing quality improvement efforts. Various entities have been established to license, accredit, or certify health care providers and organizations (see Chapter 9). Quality measurement provides these entities with information to feed into their determinations as to whether a provider or organization can serve a defined market, whether financial or other sanctions should be imposed, or whether an entity should be rewarded in some manner for providing exemplary care.
Quality Measurement for Public Health Initiatives and Policymaking. Quality measures can be used to support public health initiatives and public policymaking, providing a means of evaluating the quality of care provided through alternative financing and delivery systems, in particular geographic areas, and to vulnerable groups within the population. Public health planners can use information from quality measures to examine the need for educational campaigns or other initiatives in specific areas, and to gauge the subsequent effects of those initiatives. Public policymakers can use measures to examine aspects of health care quality that might be affected by changes in policies, like those that dictate methods of paying providers that serve beneficiaries of public programs.
Role of Quality Measures in Internal Quality Improvement Initiatives. Quality measures also can provide the tools that allow health care providers and organizations to undertake efforts to improve the quality of care they furnish. They thereby provide a means of influencing or responding to the decisions of purchasers and oversight entities, as well as any actions in the public health and public policy arenas. The measures can serve in focused improvement programs, whereby quality is measured both before and after an intervention designed specifically to improve the way in which care is provided or its outcome.
In response to the multiple needs and audiences for information on health care quality, considerable effort has been devoted to improving measurement and reporting ability. Both the number and the variety of health care quality measures have grown at a remarkable rate. For example:
The work of these groups has been supplemented by additional quality measures and sets of measures developed by numerous professional associations, such as the American Nurses Association (ANA 1995); organized health systems and providers, including both health plans and hospitals; and research and consulting organizations. Development of new measures and measurement initiatives is continuously being undertaken to fill perceived gaps in the existing measurement sets, whether in the areas of health care addressed (e.g., treatment of chronic conditions, preventive care), the units of analysis (e.g., health plan, hospital, medical group), the user orientation (e.g., providers, consumers, purchasers), or the populations of interest (e.g., commercial health plan enrollees, Medicare beneficiaries, Medicaid beneficiaries).
This continuing growth has been so substantial that efforts are under way to create electronic databases to catalog and describe the variety of measures. One of these, an AHCPR-sponsored project known as CONQUEST, has identified more than 53 separate measurement sets containing more than 1,100 clinical quality measures. Separate projects have also been undertaken to create an inventory of population and health systems performance measures.
Measures of health care quality are being used by a growing number of private and public-sector organizations at the national, state, and local levels that have undertaken initiatives to collect and disseminate information on quality. These groups include the Maryland Health Care Access and Cost Commission, which has used data from enrollee and practitioner surveys and that from health plans to generate comparative information on performance; the Minnesota Health Data Institute, a not-for-profit entity that distributed comparative performance reports on health plans; the Alabama Health Care Council, an employer purchasing coalition that collected and analyzed outcome and cost information on hospitals; and the Cleveland Health Quality Choice project, a coalition of employers and providers that examined the technical quality of care and patient satisfaction with local hospitals. The JCAHO and NCQA have developed national databases of information on the performance of institutional providers and health plans across sets of quality measures. The American Medical Association also is taking steps to establish a national database of information on the performance of individual practitioners.
Existing mechanisms for quality measurement and reporting do not fully meet the needs of the potential users of information on health care quality. Despite recent development of improved measures, initiation of new measurement efforts, and advancements in methodology, information that can be used by consumers, group purchasers, and others to make health care choices is neither widely nor uniformly available (see Chapters 6 and 7).
Nor do current measurement initiatives serve public health and quality improvement objectives as well as they might. Incentives to improve quality have been diluted by measurement efforts that vary widely in their aims and scope, and that have been, at best, only informally coordinated. A lack ofclearly articulated national priorities for improving health care quality has meant that the superior ability to measure quality in specific areas or along certain dimensions may have diverted attention from issues of greater importance with respect to quality.
Furthermore, the lack of widely agreed-upon priorities and standards for quality measurement has been a source of frustration and inefficiency. Most health plans and providers now produce data in response to multiple, customized requests from purchasers, oversight bodies, consumer groups, and others. For example, a health plan might be required to calculate and report on how it cares for individuals with diabetes in several different ways to respond to uncoordinated demands. Similarly, a health care practitioner who participates in several health plans may need to fulfill various requests for data of different types and formulations. The resulting information is not always comparable, meaningful, valid, or reliable, and is disseminated in different ways and through various means.
To increase the effectiveness and efficiency of measurement, core sets of quality measures for each sector of the health care industry should be identified for reporting in a standardized way. Measurement sets should be designed to meet the information needs of potential audiences and should reflect defined national aims for improvement in health care (see Chapter 3). In this way, their development would assure the availability of comparable information on quality and increase the potential impact of measurement while alleviating the burden of complying with reporting requests. Developing standards for measuring and reporting on a core set of indicators also would allow health care providers and organizations to reallocate resources to focused quality improvement and to the generation of additional information, as needed, to supplement the core measures. Finally, core measurement sets would assist in tracking progress in addressing high-priority areas established as national aims for quality improvement.
Prior to and concurrent with the identification of core measurement sets, a framework for measurement and reporting must be developed. A key part of that effort will be to articulate a strategy for quality measurement and reporting to make the results of comparative quality measurement available in the public domain. Other steps are also needed to increase the effectiveness and efficiency of quality measurement. For instance, measurement priorities must be identified, credible and open processes for specifying core sets of measures must be implemented, policy issues associated with measurement and reporting must be addressed, and public understanding of the value and use of quality measures must be advanced. There is also a need to encourage widespread compliance with the measurement and reporting specifications identified, and to build capacity for quality measurement and reporting across the industry.
The Commission has identified the following principles for guiding the development of a strategy to be utilized by the many parties involved in quality measurement and reporting:
Evaluate Quality at Multiple Levels. Consumer interests, quality improvement needs, and health system changes all point to the value of focusing measurement at the individual provider level, for at least some aspects of quality. There are technical challenges to doing so, however. For instance, individual practitioners and small group practices may not have enough patients to permit reliable measurements of nonroutine health care or rare outcomes. Furthermore, current approaches for data collection and analysis focus primarily on the plan or organization level and do not facilitate disaggregation of quality measurement information to the provider level. Increased standardization and automation of key data sets is needed to facilitate concurrent quality measurement at multiple levels. Such changes could also minimize marginal costs associated with processing additional data for measuring at finer levels of analysis.
Create Synergy of Improvement Efforts Throughout the Health System. Core sets of health care quality measures should help to create synergy of quality improvement efforts undertaken throughout the health system. While individual measures and measurement methods specified for each sector (e.g., health plans, hospitals, long-term care facilities) will vary to account for differences in the provision of care and in the health data and information systems available, common measures and areas of focus should be used where possible. Common measures will need to be used across delivery systems, including both fee-for-service and managed-care models. Use of consistent measures over time will be needed to foster the ability to track trends, establish targets for quality improvement, and assess the levels of increases in quality.
Capture Multiple Dimensions of Quality. To provide a comprehensive and balanced assessment of health care quality, measurement efforts will need to capture multiple dimensions of quality. These dimensions include the structures of a health care organization or system of care that can influence quality, the processes associated with the delivery or care, and the outcomes obtained (Donabedian, 1966). Another framework for classifying dimensions of quality delineates three dimensions of quality:
These dimensions might be modified or enhanced to reflect the expected uses of the information generated through quality measurement and reporting efforts. For example, value might be included as an additional aspect of quality relevant to individual consumers and purchasers who would use the information in health care decisionmaking. Furthermore, the dimension of acceptability might be broadened to include the perspectives of health care practitioners so as to obtain a richer view of health care quality.
Serve the Needs of Consumers. Quality measurement should focus on serving the needs of consumers, who can use information on quality to make choices among doctors and other practitioners, institutional providers, health plans, and systems of care. Orientation toward consumers does not preclude addressing the needs of other users of information on quality. A sound measurement strategy must take into account the interests of health care providers, purchasers, plans and other health organizations, oversight groups, policymakers, public health representatives, and others whose decisions directly affect the well-being of patients.
Address Quality of Care for Vulnerable Groups. Quality measurement should account for concerns about vulnerable groups within the population (see Chapter 8). There is a clear need to increase the level of attention paid to these groups, including both those who, because of chronic illness or disability, have many interactions with the health system, and those who have difficult accessing the system and may be most likely to fall through the cracks during this period of rapid health system change. In addition, there is a particular need to pay attention to the measurement of quality for children's health care.
An increased focus on vulnerable groups could be accomplished through two measurement strategies. First, measures focused on specific quality issues that disproportionately affect vulnerable groups (e.g., measures of the quality of chronic care) could be included in reporting sets. Second, separate analyses of the quality of care provided to vulnerable groups could be conducted and reported using the full reporting sets, to the extent feasible in light of methodological issues and cost constraints.
Encourage Innovation. Identification of core sets of quality measures is not intended to lock in approaches to quality measurement and reporting. Innovation can be fostered by calling for the development of new measures to augment, complement, and enhance the required measures; updating core measurement sets periodically to reflect advances in measurement techniques; and placing limits on the breadth of standardization efforts (e.g., number of measures) to assure that compliance with the core reporting sets does not consume resources better devoted to innovation or internal quality improvement.
Provide Efficient Mechanisms for Data Reporting and Collection. A centralized data repository may need to be created to facilitate the reporting of core information on quality and the dissemination of such information to the public. Use of computer technology -- particularly the Internet -- could allow for such a repository to be affordable and usable by both the reporters and the users of the information.
The need to set priorities for quality measurement stems from the inability to assess the results, processes, and subjective experiences associated with every interaction with the health system, or even the treatment of every disease. Identification of the areas to be addressed through measurement should be closely guided by established national aims for improvement and the objectives established for addressing those aims. Consideration of the information needs of patients and other expected users should also inform the priority-setting process, with particular attention paid to the information needs of vulnerable populations. Other questions to be addressed include: (1) the extent to which measurement should focus on health care outcomes relative to processes used in providing care or the way in which the delivery of care is organized or structured; and (2) the relative focus on dimensions such as technical quality of care, acceptability of care from the perspective of patients and providers, and accessibility of care.
Credible and open processes for specifying core sets of measures will need to be established. Several steps will be involved in establishing core sets of quality measures, including: obtaining measures for consideration, evaluating individual measures and sets of measures according to established criteria, and promulgating sets of measures and specifications for measurement and reporting.
Obtain Measures for Consideration. Selection of measures for inclusion in core measurement sets should draw upon the wealth of work already under way and previously accomplished in both the public and private sectors. Creation of sets of standard measures for reporting can build on this work by issuing a call for measures in specific areas identified through a priority-setting process. Where adequate measures in particular areas are not available, development of measures could be sponsored by the entity charged with developing the core measurement sets.
Evaluate Measures and Measurement Sets. Measures need to be evaluated, both individually and as sets of measures. An important part of the process will be to agree upon the criteria used to evaluate individual measures and measurement sets. Examples of criteria that might be identified for evaluating individual measures include:
Somewhat different criteria would be established for the evaluation of measurement sets. For instance, various criteria might be identified to assess whether a potential set was sufficiently comprehensive. Does the set address the full spectrum of health care? Does it incorporate measures of multiple dimensions of quality (e.g., technical quality, accessibility, acceptability)? Does it include various types of measures (e.g., structure, process, outcome)? The representativeness of measurement sets might also be of interest, given that an undue concentration of measures in a specific clinical area that could provide a distorted view of performance. Another key evaluation criteria might be the measurement burden associated with a set. Is the set as concise as possible? Is it redundant? Can measurement be conducted efficiently, with a minimal burden on providers and health care organizations?
Promulgate Measures and Specifications for Measurement and Reporting. Measurement sets will need to be formulated, evaluated, and promulgated, following a field trial or test period as needed for validation. Measurement specifications need to be developed that address source data sets and data elements to be used in making measurements; the measurement methods, including risk adjustment formulas, to be used; instructions for an independent, external audit prior to public reporting; and reporting formats and communications modes to be used. Finally, measures will need to be reassessed subsequent to their promulgation to determine their utility in practice.
Address Public Reporting Issues
Several important issues need to be addressed with respect to the public reporting of information on quality. Changes in payment systems will be needed to address concerns that health care organizations or providers may have about demonstrating superior performance in areas likely to attract sicker, more costly patients (see discussion of risk adjustment of payments in Chapter 8). Furthermore, confidentiality issues will need to be explicitly addressed (see Chapter 14).
Another issue to be addressed is the question of how best to promote reporting of clinical errors or adverse outcomes (see Chapter 10). Measurement of clinical errors or adverse outcomes can offer significant opportunities to improve health care, but public reporting of health care quality problems is likely to be problematic. Public reporting of such information could lead to underreporting of problems by health care providers who are concerned about potential legal liability or affected by social, cultural, and financial barriers to acknowledging mistakes. Underreporting would diminish opportunities to identify areas in which quality improvement is needed, and would yield inferior information to use in health care decisionmaking.
A number of alternatives for addressing this issue could be considered. Legal protection from liability claims associated with reporting of adverse events also should be considered to foster complete and accurate reporting. Further analyses are needed to identify how changes in medical liability and quality oversight systems might affect both the likelihood and the accurate reporting of adverse events. An alternative strategy would be to distinguish between measures to be used for internal quality improvement and those to be used for external reporting. Finally, steps could be taken to ensure that publicly released information was aggregated in such a way as to render it not individually identifiable.
Continued advancement in the field of quality measurement should be fostered by assuring that there is an ongoing agenda for research and development, as well as adequate, stable funding for such work. A forum is needed for assuring continuous development, testing, and improvement of quality measurement methods, and for furthering the interpretation and application of the data generated through these activities. The identification of areas in which better quality measures or tools for disseminating or using quality information are needed should be a top priority. Steps to promote needed research could include sponsoring projects to develop and test particular types of quality measures, such as for treatment of traumatic injuries, mental health, reproductive care, or dental care. Another needed effort would be to develop an agenda for needed research in areas such as:
Increase Public Awareness and Understanding of Comparative Quality Information
Efforts will be needed to promote and support appropriate use of information on health care quality. The availability of comparative information on quality should be advertised to reach a wide audience. Potential audiences will also need to be educated about the value of using such information in making health care decisions and about the parameters of appropriate use.
For quality measurement efforts to achieve the aims that have been set for them, they must better meet the needs of those who would use the information they generate. While considerable advancements have been made in the quality measurement field in recent years, current efforts fall short of fully meeting users' needs. Several elements are critical to advancement. New and better quality measures are needed to fill important gaps in existing measurement sets, and improved approaches are needed to fully utilize existing measures. Steps should be taken to assure that comparative information on health care quality is valid, reliable, comprehensible, and widely available. Perhaps most important, a significant investment in data collection, database management, and information systems must be undertaken to support improved health care quality measurement initiatives (see Chapter 14).
The science of quality measurement still is in an early stage of development and the continued promulgation of measures on many fronts is critical to advance the field quickly. Innovation and experimentation should be encouraged, accompanied by rigorous testing and evaluation of potential new measures. To derive the greatest benefit from these development efforts, enhanced processes will be needed to more effectively disseminate new measures shown to be valid and reliable throughout the health care industry.
Several aspects of health care are ripe for the development and testing of new and better quality measures. For instance, many more measures of preventive and acute care have been established, as compared with measures of chronic conditions. The need for measures of health care outcomes, including functional status, also has been identified as an important area to be addressed. In terms of specific clinical areas, mental health and substance abuse care, pediatric care, care for specific disabilities, geriatric care, and care for injuries or other traumas have all been identified as gaps. Better measures of the interpersonal aspects of health care that influence patients' perceptions of quality are also needed. Finally, as discussed further in Chapter Eight, there is a need for measures designed to assess the health care procured by vulnerable populations. Quality measures are lacking in these areas for a variety of reasons, including problems with the available data and data systems (see Chapter 14), lack of accepted clinical practice guidelines, and an insufficient investment of resources. Fully addressing all aspects of health care in quality measurement sets will be not be possible, but a wider variety of measures are needed to allow for more comprehensive assessments.
Existing measures also must be adapted to a broader scope of use. For example, many quality measures were developed for use in managed care plans, which serve a defined population that is relatively large in comparison with the number of patients served by an individual physician or medical group. To meet measurement needs imposed by changes in the financing and delivery of health care, measures must now be adapted or newly developed to measure the quality of care provided by individual practitioners and obtained in fee-for-service or differently defined systems of care. In the same vein, quality measures that rely on patients' self-reports must be translated to other languages to be meaningful for the full population.
This need for additional quality measures raises concerns about the potential for information overload as well as risking an imbalance of resources allocated to measurement at the expense of improvement. Group purchasers report that the variety and amount of performance information to be processed in making purchasing decisions is a barrier to doing so effectively (Hibbard et al., 1997). Summary measures that combine information on quality across multiple dimensions could help in addressing this concern, as could the use of cross-cutting measures that provide information about the quality of care obtained by patients across multiple conditions (e.g., measures of access to specialty care). A defined set of national aims for improvement and an articulated framework for measurement that takes those aims and users' information needs into account would provide the sense of priorities needed to process information adequately.
For information on quality to be most useful to health care consumers, purchasers, providers, organizations, and others, it must be valid, accurate, reliable, comprehensible, and widely available. Several important steps should be undertaken to increase the integrity and utility of information on quality.
Standardize Measurement Specifications. Standardization of requests for quality information would do much to improve the utility of measurement and reporting efforts. This could be accomplished by developing and using a single, standard approach for measuring each specific aspect of quality subject to measurement. A registry of health data elements that facilitates sharing of common concepts, definitions, and measurement standards could help in this undertaking.
At present, there is very little consistency in how measures with the same conceptual definition are being operationally defined and technically specified (Coltin, 1997; McCormick, et al., 1997). For instance, alternative quality measures might be available to address cesarean section rates that differ in terms of their operational definitions (e.g., the age range of women to be included in measurement) or technical specifications (e.g., the time frame or type of data to be used). Health care organizations and providers today are called upon to report on pediatric immunizations in one way for a consultant acting on behalf of a large employer, another way for the state Medicaid program, a third way for the state regulators, a fourth way for accreditation purposes, and a fifth way for the local business coalition (Udvarhelyi, 1997). These variations do little to add to the overall store of information on health care quality and detract from the ability to compare results.
Assurance that health care providers and organizations were using the same types of data, including the same segments of the population, and adjusting for risk in the same ways in their calculations would allow for greater confidence in the comparability of reported information. It would also provide greater ability to trend improvement in quality over time, a capacity that is necessary for setting targets for improvement and gradually increasing the overall level of quality. Agreement on appropriate measurement specifications for conceptual measures would be valuable in reducing unnecessary duplication of effort and the wasted resources that this entails.
Other industries, as well as the health care industries of other countries, have begun to address inconsistencies in the specifications and operational definitions of data concepts by establishing data registries. These registries serve to promote consistent use of data that need to be interchanged between organizations. Examples of such efforts include the National Health Information Knowledgebase of the Australian Institute of Health and Welfare and the Basic Semantic Repository Project of the International Organization for Standardization of Data Elements.
Make Appropriate Adjustments for Risk. Risk adjustment is sometimes needed to ensure that fair and valid comparisons are made on the basis of legitimate differences in health care quality, since characteristics of populations that are outside of the control of a health plan or provider may influence measurement results. Methods used to adjust quality information for risk should be subject to external scrutiny to assure that they meet explicit standards of clinical credibility and statistical rigor. Evidence that the need for risk adjustment has been considered in the development of a specific measure should be examined prior to promulgating a measure for reporting. Specifications for risk adjustment should be included, or not included, in measurement standards as appropriate to each measure.
The objective of risk adjustment is to account for differences in the mix of patient factors across plans or providers so that comparisons of how their patients fare can be fairly made. The underlying assumption is that outcomes result from a complex mix of factors that relate, in part, to intrinsic patient characteristics. The mix of persons covered by different health plans or treated by different providers varies, reflecting the multiple factors that affect the way persons select their sources of care (e.g., the nature of their health needs, financial considerations, preferences, and expectations). These differences in mix are important for quality measurement: persons with complex illnesses, multiple coexisting diseases, or other significant risk factors are more likely to do poorly, even with the best care, than healthy individuals (Iezzoni, 1997).
The appropriate use of risk adjustment techniques is essential because using measures that fail to account for important differences in the populations served runs the risk of penalizing in the marketplace those plans and providers that serve the sickest and neediest populations. Failure to risk-adjust before comparing performance across health plans or providers can have several consequences. Without adequate risk adjustment, resulting information could be inaccurate or misleading. Consumers, policymakers, and other health care stakeholders will not have valid information for decisionmaking. Another goal of quality measurement is to motivate improvement of care. Without risk adjustment, providers often perceive that they are treated unfairly. Providers could avoid high-risk patients who are more likely to do poorly. Furthermore, failure to adjust for risk hampers attempts to engage providers in a meaningful dialogue about improving performance.
Despite the need for risk adjustment methods, current techniques inherently are imperfect and likely to remain so for several reasons: lack of requisite data, especially pertaining to important nonclinical attributes of patients (e.g., socioeconomic status, psychosocial issues, functional abilities, preferences and expectations about health care); problems with existing data (e.g., completeness and accuracy); unavailability of appropriate data sets on which to develop and validate risk-adjustment methods; and, perhaps most important, the absence of well thought-out, evidence-based conceptual models of risk factors for performance indicators of proven validity (Iezzoni, 1997).
Similarly, there are limits in what can reasonably be expected of risk adjustment. It will never be possible (or even desirable) to adjust for every characteristic of patient populations that could affect quality measurement. Gathering the necessary data is expensive and logistically complicated. Collecting some types of information may even be impossible, by any practicable means. Given that science cannot guarantee that risk adjustment will be perfect, decisions about risk adjustment will inevitably engender controversy -- with legitimate arguments for and against using methods with inevitable shortcomings.
Provide for an Independent, External Audit. Conducting an independent audit of processes used in data collection and analysis is necessary to ensure that publicly reported information is comparable and accurate. There are numerous reasons why externally reported data may be inaccurate: measurement specifications that fail to provide detailed instructions; errors on the part of health care organizations in interpreting specifications; use of different source data sets to undertake measurement; incomplete or inaccurate source data sets; and differences in clinical coding systems (National Committee for Quality Assurance, 1995). Analytic mistakes can also lead to reporting errors. By providing assurance that information is accurate, auditing can increase the integrity and usefulness of publicly reported information on health care quality.
If undertaken by an entity that is independent and external to the organization whose performance is being examined, auditing may serve to allay concerns about the accuracy and validity of reported information on health care quality. Recent studies have suggested that one reason many large health care purchasers have refrained from making decisions based on performance data is concerns about whether they are accurate (GAO, 1997; Hibbard et al., 1997). Certification of accuracy by an entity that is known to be independent and external to the organization whose performance is being measured would provide an assurance that reported information provides a sound basis for decisions.
Independent, external audits can be resource-intensive and are not required for all uses of quality data. For instance, health care organizations might determine individually, on a case-by-case basis, whether information to be used internally for quality improvement efforts required an external check on its validity. Other types of uses of quality information, such as research uses, could rely on peer-review processes or alternative validation methods.
Establish Effective Approaches for Dissemination and Transmission of Information. Comparable quality information should be placed in the public domain and made available to the full array of interested audiences. Problems attaining information known to be comparable and reliable has been among the greatest obstacles to the work of consumer advocacy organizations and others interested in assisting consumers with their health care choices (Krughoff, 1997). Routine release of this type of information -- and assurance that such information is easily accessible and affordable -- would greatly facilitate the work of employee benefits departments, professional and trade associations, national and local media, and others who would disseminate the information to those who could use it in their decisionmaking (see Chapter 7). It might also reduce the burden on health care organizations and providers of responding to a number of uncoordinated requests for information on quality.
Placing measurement results in the public domain is necessary, but not sufficient, to promote their use in health care decisionmaking. More research is needed to determine how best to make information on quality accessible and usable. To foster widespread awareness and use of the information, quality measurement findings should be reported through multiple modes and dissemination channels. Standard reporting formats and accessible language will need to be used. Further, steps may need to be taken to facilitate interpretation of the information provided, particularly for certain vulnerable groups within the population. Various types of consumer assistance programs, such as those provided through employers' benefits departments, labor unions, or independent ombuds organizations, may play a role in this effort.
An information campaign -- targeted to consumers and other information users -- will also be needed to promote and support appropriate use of information on health care quality. The availability of comparative information on quality should be advertised to reach a wide audience. Potential audiences will also need to be educated about the value of using such information in making health care decisions, the limitations of measurement, and the parameters of appropriate use. Strategies for disseminating quality information to consumers are discussed in Chapter Seven.
American Nurses Association, Nursing Report Card for Acute Care (Washington, DC: American Nurses Publishing, 1995).
Coltin, Kathryn, testimony before the Subcommittee on Quality Measurement of the Advisory Commission on Consumer Protection and Quality in the Health Care Industry, July 21, 1997.
Donabedian, Avedis, "Evaluating the Quality of Medical Care," Milbank Memorial Fund Quarterly 44:166-203, 1966.
Greenfield, Sheldon, testimony before the Subcommittee on Quality Measurement of the Advisory Commission on Consumer Protection and Quality in the Health Care Industry, July 21, 1997.
Hibbard, Judith H., Jacquelyn J. Jewett, Mark W., Legnini, and Martin Tusler, "Choosing a Health Plan: Do Large Employers Use the Data?" Health Affairs 16(6):172-180, November/December 1997.
Iezzoni, Lisa, ed. Risk Adjustment for Measuring Healthcare Outcomes (Chicago: Health Administration Press, 1997).
Krughoff, Robert, testimony before the Subcommittee on Roles and Responsibilities of Group Purchasers and Quality Oversight Organizations of the Advisory Commission on Consumer Protection and Quality in the Health Care Industry, November 18, 1997.
McCormick, K., A.L. Renner, R. Mayes, J. Regan, and M. Greenberg, "The Federal and Private Sector Roles in the Development of Minimum Data Sets and Core Health Data Elements," Computers in Nursing 15(2) Supplement:S23-S32, 1997.
McGlynn, Elizabeth A., "Six Challenges in Measuring the Quality of Health Care," Health Affairs 16(3):7-21, May/June 1997.
Palmer, R. Heather, "Considerations in Defining Quality of Health Care," Part I. In Palmer, R. H., Donabedian, A., and Povar, G.J. (eds.), Striving for Quality in Health Care: An Inquiry into Policy and Practice (Ann Arbor, MI: Health Administration Press, 1991).
Udvarhelyi, Steven, testimony before the joint Subcommittees of the Advisory Commission on Consumer Protection and Quality in the Health Care Industry, June 25, 1997.
U.S. General Accounting Office, Management Strategies Used by Large Employers to Control Costs, GAO/HEHS-97-71, May 1997.