Part 1—Use of Strategic Planning
Part 2—Development and Use of
June 9, 1997
in the National Weather Service
of the National Oceanic and Atmospheric
Outcome Information in NOAA's
National Weather Service
Part 2—Development and Use of
June 9, 1997
This case study was developed by Curtis L. Marshall, of the NOAA Office of Policy and Strategic Planning, and Robert H. Stockman, Strategic Planner of the National Weather Service. It was prepared in conjunction with several other case studies which examine federal agencies' activities in the areas of strategic planning and performance measurement. The preparation of these case studies was coordinated by the National Academy of Public Administration, the Office of Management and Budget, and the American Society of Public Administration. This is a draft case study, and its purpose is to foster interagency consultation on matters related to the Government Performance and Results Act. The authors would like to acknowledge several contributors to this paper, including Lou Boezi, Larry Klingensmith, Jill Meldon, Annie Millar, Jeffrey Payne, and Dennis Staley.
USE OF STRATEGIC PLANNING
IN THE NATIONAL WEATHER SERVICE
OF THE NATIONAL OCEANIC AND ATMOSPHERIC
The National Weather Service (NWS) is one of the five Line Offices within NOAA, which comprises over half of the budget of the U.S. Department of Commerce. The NWS has approximately 5,200 employees nationwide, and its FY 1997 budget is approximately $638 million. Even prior to the Government Performance and Results Act (GPRA), the NWS had done extensive strategic planning as part of its weather service modernization program, and explored innovative approaches to guide and measure organizational performance in a continuing way. There are several reasons why the NWS has done this.
First, NWS has always been a customer-oriented government agency focused on service to the public. NWS delivers a large number of products (weather forecasts, warnings, and advisories) every day that are used by virtually every American at least in some way.
Second, as a professional science-based agency operating 24 hours a day, 365 days a year, strategic planning and verification of organizational performance is an integral part of the way that NWS does business -- and has been doing business for a number of years. Accordingly, strategic planning and performance measurement has been institutionalized in NWS operations to maintain quality control to a degree that is probably unusual among federal agencies.
Third, the ten-year, $4.5 billion modernization plan for the NWS during the 1990's was driven in large measure by an objective determination that the performance of NWS using the science and technology of the 1980's was clearly deficient in terms of fulfilling public needs for accurate and timely weather warnings. For example, until the 1990's, the limitations of NWS's operational capabilities were such that some one third of the severe storms in the country were not accompanied by an NWS warning for the protection of the public. Moreover, approximately two thirds of the NWS's severe storm warnings that were issued turned out to be "false alarms", thereby eroding public willingness to take protective action in the face of a warning. In other words, strategic planning and performance management drove the NWS modernization, beginning as early as 1979.
II. Strategic Planning Process
While strategic planning for the modernization of NWS commenced as early as 1979, the actual budgeting for the modernization itself, as well as a full-scale NWS-wide strategic planning effort took form in the mid-1980's. An official NWS Strategic Plan, which included all of the elements of the NWS modernization was completed in 1987, and this strategic plan has provided the basis for the implementation of the modernization as well as the associated annual budgets and operating plans.
This existing strategic plan also provided the basis for a continuation of NWS and NOAA strategic planning for the 1990's. In 1990-1992, many of the elements of the earlier NWS Strategic Plan were incorporated into a broader "Draft" NOAA Strategic Plan, which was reviewed and updated throughout the NWS and NOAA leadership.
After the transition to the new Administration, a new NOAA-wide planning effort commenced in late 1992/early 1993, based upon a matrix-style management approach; and again, the NWS modernization elements, with their performance management approach, were incorporated in the new NOAA Strategic Plan -- A Vision for 2005. Also, because of the matrix-style management approach, the NWS elements were integrated with other Line Office functions of NOAA, as part of the NOAA Environmental Assessment and Prediction Mission. (The NOAA Strategic Plan contains two principal missions --environmental assessment and prediction, and environmental stewardship.)
In the new NOAA Strategic Plan, the environmental assessment and prediction mission includes the following goals:
While the NWS focused principally on the first three of the above goals, there was an effort, through the establishment of strategic planning teams for each of the seven NOAA goals, to carefully integrate each of the goals into a NOAA-wide planning effort. This process is still underway. Each of the strategic planning teams includes a mix of NOAA Line Office and Staff Office representatives who articulate the various NOAA interests for each of the objectives within that particular goal. Each team is also required to develop and update each year a 5-Year Implementation Plan for their goal. This Implementation Plan includes budget scenarios for each fiscal year, and is used as a starting point for the next fiscal year budget formulation process. In this way, the NOAA Strategic Plan is intended to drive the budget process. The Implementation Plan also contains performance measures and milestones associated with each goal.
In addition, NOAA uses an Annual Operating Plan which also contains the performance measures and milestones. As part of a Quarterly Review process, Line Offices report on progress in meeting milestones and moving toward accomplishment of outcome measures. The Strategic Plan Team Leads are asked to assess the performance of the Line Offices in accomplishing proposed work. (See Discussion in Part 2.)
III. Strategic Plan Summary
NOAA's vision for 2005 for the "Advance Short-Term Warning and Forecast Services" Goal, which is the principal focus of this case study, is to provide significantly improved short-term warning and forecast products and services for a broad spectrum of environmental events, including tornadoes, hurricanes, floods, tsunamis, and geomagnetic storms, to reduce the catastrophic impacts of natural disasters, and improve the public safety and the economic productivity of the Nation. NOAA's plan states that it will enhance its ability to observe, understand, and model the environment, and effectively disseminate products and services to users.
NOAA's plan also states that NOAA will achieve this goal by meeting the following objectives:
The NOAA Strategic Plan stresses the benefits to be obtained by the implementation of this particular goal. Increasing our understanding of the environment through research and investing in new technologies will provide more accurate and timely warnings and forecasts required by the Nation. Improved forecasts will support management of water resources, and help avoid flood damage. Extended forecasts of solar and geomagnetic disturbances will increase efficiencies for space operations, and power generation and satellite communications networks. Advanced modeling techniques and more complete observations will reduce uncertainties in hurricane track prediction, saving millions of dollars through evacuation costs avoided. Accurate outlooks of future conditions will provide better information for planning weather sensitive activities over land and the oceans.
Improvements associated with the NWS modernization will more than pay for themselves. A National Institute of Standards and Technology cost-benefit analysis for the modernized NWS estimates that economic benefits to the Nation will be about 8 times greater than the costs involved. Once modernization is completed, the Nation should realize annual benefits beginning at $7 billion.
IV. Use and Impact of Strategic Planning
The NOAA Strategic Plan for the NWS has been the vehicle to design, seek approval, and implement a $4.5 billion modernization effort which includes a fundamental consolidation of NWS facilities nationwide. When the modernization process is completed, this consolidation will result in more than 50% reduction in the number of facilities (from 300 to approximately 119 Weather Forecast Offices and 13 River Forecast Centers) and a totally revamped workforce profile (professionalization of the workforce). In the current NOAA Strategic Plan, the NWS modernization is principally contained in the "Advance Short-term Warning and Forecast Services" Goal, and comprised about 59% of the total NOAA FY 1997 appropriation.
Numerous NWS and NOAA personnel now conduct strategic planning activities as a vital component of their normal work responsibilities. There has been much reliance on the existing planning and budget staffs, as well as programmatic staff and policy officials and the matrix style management process within NOAA to develop and implement this strategic plan. This modest investment is in stark contrast to the actual costs of the $4.5 billion modernization effort, which is leading to a dramatic improvement of forecast and warning services to the public.
VI. Lessons Learned
NOAA is well underway toward completing the largest modernization and restructuring effort in its history to improve the severe weather, flood warning and forecast services in the U.S. Major technological advances in meteorology and hydrology, coupled with the need to replace obsolete and increasingly unreliable equipment provide an unprecedented opportunity to improve weather and hydrologic services. These replacement systems, along with observations from advanced satellites, and newly developed storm-scale forecasting techniques, will continue to improve the timeliness and accuracy of severe weather and flood warnings to the U.S. public.
The NOAA Strategic Plan for the NWS became the vehicle for the systematic fleshing out of goals for modernization of the NWS, including competing goals between and within the Executive Branch and the Congress. While the implementation of new technology was the driving force at the outset, during the course of the strategic planning effort, levels of service, quality and accuracy also became focal points. The development of the goals of the plan forced an honest policy debate about ways to implement new technology, including major restructuring to take advantage of the efficiencies available, and also ways to improve the level and quality of services to the public.
VII. Next Steps
The modernization and restructuring of the NWS is about two-thirds completed. In addition to continued monitoring, updating, and evaluation of the modernization occuring within the Executive Branch, the Congress is also involved in overseeing the final implementation of the plan over the next 5 years. Under P.L. 100-685, and an amended version, P.L. 102-567, Congress recognized, and in effect, adopted the NOAA Strategic Plan for the NWS, but also established a certification process for the closure of weather service facilities. For each closure of a facility, Congress required a certification by the Secretary of Commerce that no degradation of services to the public would result for that particular area. Although NOAA has characterized this process as being too burdensome, and has proposed legislation to streamline the process, the mechanism has assisted in "legitimizing" the closure of outdated weather service facilities by allowing for greater public and Congressional participation.
Maintaining NWS Modernization is only one of the objectives of the "Advance Short-Term Warning and Forecast Services" Goal of the NOAA Strategic Plan -- A Vision for 2005. The other objectives are as follows:
In addition, NOAA will be actively moving ahead with two other separate, but related goals of its strategic plan which include:
DEVELOPMENT AND USE OF OUTCOME INFORMATION
IN NOAA'S NATIONAL WEATHER SERVICE I. Context
With the completion of the NWS Strategic Plan in 1987, which would totally modernize and recast the way in which the NWS did business, there was also an effort to develop performance measures which could address progress toward modernization and measure the actual impact of new prediction and forecast capabilities. While these performance measures have evolved during the modernization process, there has been a continuing effort to make them outcome-based and focused on the public's needs for prediction and forecast services.
II. Development of Indicators
Performance measurement in the NWS takes several forms at the present time. Measures range from relatively routine quantitative parameters of operational readiness to ad hoc assessments of NWS's actual conduct in specific severe weather events, such as "lead time" and % accuracy for tornado, severe thunderstorm, and flash flood warnings. NWS is also attempting to move beyond these traditional performance measures in order to achieve a higher degree of performance in terms of important national goals where weather is a factor. Each type of performance measurement will be reviewed briefly.
Unlike the first two types of performance measures which target objective, quantitative measures of internal NWS performance, the NWS disaster survey seeks to determine the external impact of NWS forecasts and warnings to protect life and property. In a typical year, no more than several NWS disaster surveys are implemented and the primary purpose is to identify any mistakes that were made, any breakdowns in communication that occurred or generally what lessons could be learned for future action.
III. Indicators of Outcome/Results
For the most part, the performance measures collected by the NWS are quantitative, and the NWS has considerable experience in collecting and verifying the data to support them. (See Attachment 1). These are largely "lead-time" and accuracy indicators which provide a very quantitative, measurable, and reliable benchmark -- and in most cases, a benchmark which is outcome-based.
Building upon the conceptual approach of the NWS disaster survey which focuses on the external impact of NWS forecasts and warnings, there are opportunities to extend NWS performance measures to more than a few ad hoc cases and to the socioeconomic utility of weather information that goes beyond cases of severe weather. The overriding goal will be to seek higher levels of NWS performance in satisfying national needs where weather is a factor.
IV. Use and Impact of Information on Outcomes/Results
The performance data currently collected by the NWS has been collected and verified for a number of years. Since the GPRA, there have been further efforts to refine and validate the existing performance measures being used by the NWS, but because of the earlier strategic planning process, most of the measures remain valid and provide a fairly reliable measure of accountability. The performance measures (see Attachment A) are the basis for the collection of data by region and by Weather Service Forecast Office, and can therefore provide a basis for performance and accountability at those levels. Of course, inherent geographical variations exist in the "difficulty of forecasting" weather and hydrologic events, and these are taken into account by NWS and NOAA leadership in their reliance on these tools.
These performance measures, which are for the most part quantitative, have been fairly reliable instruments to evaluate the effects of the NWS modernization. For example, modernization efforts to date already have demonstrated changes in warning lead times for severe weather events from less than zero to 10 to 15 minutes. Final improvements in the accuracy and timeliness of these weather and flood warning and forecasts will result in saved lives and reductions in losses from weather disasters. These improvements, which are already being borne out by the performance measures, will also result in very large economic gains for highly weather dependent sectors of the U.S. economy such as transportation and agriculture.
Although few NOAA and NWS staff are dedicated solely to these efforts, there is strong reliance on the existing planning and budget staffs, as well as programmatic staff to develop, validate, and verify existing performance measures. It is estimated, in the aggregate, that the NWS commits approximately $1.5 million and 20 to 30 FTE's per year to the task of compiling and verifying performance measurement data. This modest investment is in stark contrast to the actual costs of the $4.5 billion modernization effort, which is leading to a dramatic improvement in forecast and warning services to the public.
VI. Lessons Learned
The lessons are still being learned as the NWS proceeds through its process of modernization and continues to validate and verify its performance measures, while continually improving its short-term forecasting and prediction capabilities. Also, the NWS is evaluating additional areas of performance related to its many customers and potential new partnerships with the users of NWS data.
The NWS has benefitted greatly from its strategic plan developed at the outset of the NWS modernization. The plan simplified the performance measurement process -- because everyone was aware of the goals they were trying to achieve. The performance measures are a logical outgrowth of the strategic plan, and the results of the modernization are already measurable. For example, the use of new Doppler radars has resulted in positive "lead-times" for the first time ever in the case of some severe weather events.
VII. Next Steps
Several principles and concepts offer promise for a more sophisticated approach for NWS to manage and measure its performance as an agency to serve the public. These principles and concepts include the following:
For example, if more Americans die in highway accidents when weather is a factor than for any other weather related risk, the NOAA could work with a State Governor, the U.S. Department of Transportation and the American Automobile Association to seek a reduction in these highway fatalities through public education and other steps beyond issuing accurate NWS forecasts.
It should be stressed that outyear performance measures are dependent on a stable funding profile and take into account improved use of the Weather Surveillance Radar (WSR-88D), new satellites, improved forecast models, new and continued research of the U.S. Weather Research Program, investments in critical observing systems, Coastal Forecast System activities, and the implementation of the Automated Weather Interactive Processing System (AWIPS).
As the National Weather Service continues to change, upgrade its capabilities, and reorganize, so too are the various sectors of the economy. To keep pace with changing demands and to assist in its strategic planning process, NOAA has contracted The Kanawha Institute for the Study of the Future to conduct a study of the present and future needs for weather data and information by business and industry. The societal and economic trends identified in this study will enable NOAA to re-evaluate its current strategic planning goals and objectives, and continually assess its performance measurement process.
KEY FY 1998 PERFORMANCE MEASURES
National Oceanic and Atmospheric Administration Goal: Advance Short-Term Warning and Forecast Services
|Flash Flood Warnings|
|Lead Time (Min.)||17||21||27||30|
This measures the lead time (in minutes) and accuracy (in percent) for successfully forecasting a Flash Flood Warning. A Flash Flood Warning is issued when flooding has been reported or is imminent. Flash flooding is the #1 weather-related killer in the United States, with nearly 140 fatalities each year. Advance warning of flash floods will enable people to take steps toward protecting life and property.
|Severe Thunderstorm Warning|
|Lead Time (Min.)||16||18||18||18|
This measures the lead time (in minutes) and accuracy (in percent)for successfully forecasting a Severe Thunderstorm Warning. A SevereThunderstorm Warning is issued when thunderstorms produce hail at least 3/4 inch in diameter, winds of 58 mph or higher, or tornadoes. Of the estimated 100,000 thunderstorms that occur each year in the United States, about 10 percent are classified as severe (with its associated lightning, flash floods, straight-lined winds, large hail, and possible tornadoes). Advance warning of these severe storms will enable people to take steps toward the protection of life and property.
|Lead Time (Min.)||9||10||11||12|
This measures the lead time (in minutes) and accuracy (in percent) for successfully forecasting a Tornado Warning. A Tornado Warning is issued when a tornado has been sighted or indicated by radar. Tornadoes are the most destructive storms in the United States, with over 800 tornadoes, 80 deaths, and over a thousand injuries reported each year. Advance warning of these violent storms will enable people to take steps toward the protection of life and property.
|Lead Time for 1" Precipitation Forecast|
|with the Same Accuracy as a 1-Day Forecast|
|in 1971 (Days in Advance)||2.3||2.3||2.3||2.3|
This measures the lead time (days in advance) for successfully forecasting one inch of precipitation as compared to a 1971 precipitation forecast. The NWS is now successfully forecasting one inch of precipitation 2.3 days in advance as compared to 1.0 day in advance in 1971. In light of the devastating socio-economic impacts of flooding and significant snowfalls, quantitative precipitation forecasts (QPFs) have emerged as a critical facet of the end-to-end forecast process. Improved and early lead time of one inch of precipitation or more will promote timely and accurate issuance of flood and winter storm forecasts which are essential for the preservation of life and property.
|Accuracy of Landfall (km) with|
|24-hour Lead Time||134||130||145||140|
This measures the accuracy of a landfalling hurricane (in kilometers) with a 24-hour lead time. Hurricane Warnings are issued when hurricane conditions are expected within 24 hours. Hurricanes are powerful, tightly coiled weather systems with high winds, heavy rains, tornadoes, and a battering storm surge along the coastline near where a hurricane makes landfall. Earlier and improved predictions of storm track and intensity will enable people to take steps toward the protection of life and property, as well as reduce the size of the warning area.