ACCIDENT INVESTIGATION BOARD
WEDNESDAY, APRIL 23, 2003
3000 NASA Road 1
BOARD MEMBERS PRESENT:
Admiral Hal Gehman
Rear Admiral Stephen Turcotte
Major General John Barry
Major General Ken Hess
Dr. John Logsdon
Dr. Sheila Widnall
Mr. G. Scott Hubbard
Mr. Steven Wallace
Dr. Jean Gebman
Mr. Robert P. Ernst
Dr. Diane Vaughan
ADM. GEHMAN: Good afternoon. The afternoon session
of the Columbia Accident Investigation Board public hearing
is in session. This afternoon we're going to hear from two
experts on the subject of aircraft aging, which is another
risk element in the shuttle program which wasn't originally
foreseen -- at least I don't think it was. The shuttles
were originally designed to last ten years and now we're
passing 20 and headed toward 30 and the shuttle vehicle
then is facing issues which need to be looked at to determine
whether or not the shuttle can operate safely. We're very
pleased to have you two gentlemen join us.
Dr. Jean Gebman is a senior engineer at the Rand Corporation;
and Mr. Robert Ernst is the head of the Aging Aircraft Program
at the Naval Air Systems Command, Patuxent River. We're
glad to have you both with us.
I would invite you to introduce yourselves and say a little
bit about your present job and your background; and then
if you have an opening statement or a presentation, please
go ahead and proceed. Why don't you both introduce yourselves
first, and then we'll go ahead with the presentation.
JEAN GEBMAN and ROBERT ERNST testified as follows:
DR. GEBMAN: I'm Jean Gebman, senior engineer at Rand,
working on the aging Aircraft Project. My educational background
is in aerospace. My doctoral work majored in structural
dynamics with minors in fluids and control engineering.
MR. ERNST: I'm Bob Ernst, the head of the Nav Air
Aging Aircraft Program and also representing the Joint Council
on Aging Aircraft which is a DOD, FAA, NASA, and industry
consortium trying to work on age issues. I don't have the
storied credentials and degrees that my counterpart has,
but I've got a lot of years of experience working on old
platforms and rust and corrosion and obsolescence and those
sorts of things.
ADM. GEHMAN: Thank you very much. Go ahead and proceed.
DR. GEBMAN: Thank you, Mr. Chair. Bob and I are going
to present two briefings that are very complementary. I'm
going to talk about some technical details to give you a
somewhat hurried landscape technically, and then Bob's presentation
is going to deal with some of the cultural and programmatic
Next chart, please. This is simply a bit of background.
In the interest of time, we'll just press on ahead. Next
The examples that I've selected do have a methodology behind
them, and this chart is an attempt to try to capture the
essence of that. We're going to focus on the top set of
items, although aging aircraft do involve all of the functional
areas that are listed on the left-hand side of the chart.
Next chart, please. So this is going to be the focus.
Next chart. Whether or not this focus proves helpful to
you is, of course, a matter to be determined as your investigation
moves forward. So my purpose here today is more to share
with you some areas where the aging aircraft experience
might prove helpful as you move down the road.
Next chart, please. You all have seen the various diagrams
of the shuttle. I'm going to focus on the left side.
Next chart. And simply make a couple of points. We have
four main spars that go through; and when we talk about
structures and structural dynamics, one of the things we
often quickly look at is the wing route where the spars
go through. That's just simply one area that one is always
Next chart. Another area that's of interest and will be
touched on by one of my examples subsequently has to do
with the aluminum honeycomb. This is simply a cross-section
showing at the top there the interior face sheet, which
is aluminum; the corrugation, which is aluminum; and the
piece of bond between the corrugation and the exterior face
sheet; and then, of course, the thermal protection system
underneath. A very sophisticated system. And one of the
things we will be talking about later is the matter of adhesion
as a method of joining structural materials together.
Next chart, please. This is a list of the samplers. Let's
get right to it.
Next chart. B-52 is a very interesting story. This often
is pointed to as here is why it is possible to maintain
a fleet for a very long period of time. We need, though,
to be cautious and acknowledge how it was we got to that
situation, because you may note that the G model and the
D model have long since gone to the boneyard. Corrosion
was the principal culprit. The basing at Guam was about
the worst base you could be at for an Air Force aircraft
from a corrosion standpoint.
Next chart. Even the H model, to get it to where it is today,
has been significantly rebuilt in many areas, as these various
shaded areas demonstrate. Moreover, it has been based at
a location that is relatively benign from a corrosion hazard
standpoint and the maintenance people learned a good lesson
from the experience of the G model and there has literally
been a zero tolerance for corrosion. If they see corrosion,
it must be removed.
When we visited the depo about six years ago, we looked
B-52 and the KC-135s. I was challenging the technicians
on the B-52, "Show me the corrosion."
They said, "Dr. Gebman, there is none."
I said, "Folks, it's an old airplane. We know there must
Finally, they were able to show me a detail at the back
of the airplane and they acknowledged, well, we ground out
a little bit back here but this is not even significant.
This airplane is very different from the 135. Next chart,
ADM. GEHMAN: Could I ask you to go back a second.
In that first bullet, what is a full-scale fatigue test,
what's a damage tolerance analysis, and what's a tear-down
THE WITNESS: The full-scale fatigue test is where
you take an article that could be flown in flight and, instead
of doing that, you set it up to be loaded cyclically by
attaching various jacks and an enormous hydraulic contraption
and typically you will try to simulate two -- in the old
days, four -- equivalent lifetimes that identify where the
fatigue vulnerabilities are so that they can be addressed
during production and/or during maintenance.
ADM. GEHMAN: And I assume also recognize -- I mean,
in other words if you have a fatigue indicator like a crack
or something like that, the idea is that you would then
be able to recognize that if that were to happen in a service
DR. GEBMAN: One of the most important things you
learn from the test is where the cracks are taking place
and so that you can set up a maintenance program or do a
modification so you don't have to set up a maintenance program.
The damage tolerance analysis is a method of studying the
growth of fatigue cracks and their significance, giving
you further information that you use for fleet management
and modification purposes.
The tear-down inspection took place in the 1990s, largely
to identify places with corrosion was going on in areas
that could not otherwise be seen. When we do heavy maintenance,
we don't take the airplane totally apart. The notion of
a tear-down inspection is to take a high-time airplane which
you're prepared to sacrifice and literally take every part,
open it up, and see where you have challenges.
MR. WALLACE: Is the concept of damage tolerance that
you will be able to detect cracks and things and also make
predictions as to their growth rates in such a way that
you can easily detect them before they become critical?
DR. GEBMAN: Yes. And I would encourage, if I might,
that we try to speed through the examples because you will
have an opportunity to see illustrations of some of these
With the board's permission. Next chart, please. Moving
on to the 135, corrosion is the principal challenge with
Next chart. This is an example of a tear-down inspection.
What you're looking at is a drawing of the top view of the
full fuselage. Each square is an area that they took the
structure apart, opened it up, looked at it sometimes under
a microscope. If you see color in the square, it means they
found at least light corrosion present. Just about every
square that they did a detailed examination of, they found
some indications of corrosion with that fleet. That is a
result of the materials that were selected, the environment
in which it is operated, and the maintenance program which
it had through its lifetime.
Next chart, please. Similar view. This time it's the wing
Next chart, please. As a consequence, over time when these
airplanes go in for heavy maintenance now on a five-year
cycle, it can take a year to do the complete job.
Next chart, please. This chart shows declining labor hours
required. We are now at a point where the labor hours to
do that heavy work is eight times what it was the first
time it was done when the airplane was about eight years
Next chart, please. Until very recently it was the Air Force's
intent to keep all KC-135s to the year 2040 or thereabouts,
at which point the fleet would be 80 years of age. Recently
the senior leadership has decided that the older airplanes,
the E models of which there are somewhat more than 100,
need to be retired sooner than that; and they are now looking
at leasing perhaps a 767 to fill this particular function.
So one's perspective about life can change significantly
as you learn more and more about the growing burdens before
Next chart, please. Moving on now to a new decade. Next
chart. I share this example with you that illustrates some
of the complexities and depth and breadth of endeavor one
can get into when dealing with life issues. Now, the irony
is that this is dealing with the new C-5A in the early Seventies.
It had a very unfortunate experience in its full-scale fatigue
tests. Fatigue cracks throughout the airplane, especially
in the area of the wing.
The Air Force Scientific Advisory Panel convened a study
in 1970 for the Air Force, made some recommendations. The
following year, a major engineering effort was launch. Independent
review team. One hundred people worked for one year, going
through the results of the full scale fatigue tests, looking
at the different options that the Air Force might consider,
analyzing Options A through H, and presenting them to the
leadership. Ultimately Option H, wing redesign and replacement,
was selected. Once you open up the area of structures, the
number of things that you can end up having to examine can
be considerable. That's the lesson from this particular
Next chart, please. This example is a little bit different.
We're focusing on a specific technical issue. It's honeycomb
composite material, and it proved, in those few areas where
it's used on the F-15, to be quite challenging.
Next chart. These are some of the methods in which the water
and the corrosion and cracking and durability issues arose
with that particular fleet. To the extent that this proves
of interest, the area of honeycomb composites, this particular
fleet -- and there are some other examples -- might be worth
GEN. BARRY: One comment on that. This is also the
leading edge of a lot of the wing forms in the F-15s, particularly
in the tail as a point. So might be of interest in the board.
DR. GEBMAN: Yes, sir. Thank you.
Next chart. Moving on to the Seventies, here we have two
examples dealing with the loads that actually occurred,
exceeding what the designers thought they would be.
Next chart. This is a classic. The F-16 was designed for
both air-to-air and air-to-ground work; and it turned out
that in the air-to-ground mission area, the loads that the
structure encountered very quickly exceeded the capacity
of the structure as it was designed. This illustrates the
importance of really monitoring your loads through your
life cycle so that you take that load information and update
your expectations as regards fatigue cracking.
Next chart, please. This is the process. This is the durability
and damage tolerance analysis process and I'm certainly
not going to lecture on this today, but this is a summary
that you might find useful as your work moves forward. When
I look at this, I look at it from not only a structures
viewpoint but also from a systems viewpoint. You can literally
go through that chart and change its orientation from fatigue,
which it was designed for, to corrosion or other kinds of
things that affect an aircraft as it ages. Indeed, today
people are working on the development of what's called a
functional integrity program approach, which mirrors this
aircraft structural integrity kind of program.
Next chart, please. The B-1 example is a little bit different.
Here we were dealing with acoustic fatigue, which is a dynamic
phenomenon and it's a bit like the tuning fork. If you hit
the tuning fork, it will vibrate at a natural frequency.
Well, aircraft structures, if excited at their natural frequency,
will engage in vibration; and this can great accelerate
the propagation of fatigue cracks. That's the essence of
that particular story. It's an interesting one from you
all's perspective to an extent because it involved both
thermal, aerodynamic, and structural dynamics. It turns
out that the designers deliberately had hot exhaust from
the engines going over the control systems at low-speed
flight to increase the control authority of the control
Next chart, please. Now for our final example. Next chart.
This is an airplane that served quite long in terms of landings.
It was designed for 75,000, and in flight hours it was not
all that high. It was designed actually for 50,000. This
example illustrates the three things listed on the chart.
Next chart, please. Imagine yourself flying over the Pacific
in this particular airplane. You're in Row No. 5. You have
the seat next to the window, and over your left-hand shoulder
there's a fatigue crack. From the NTSB's excellent work,
it appears that the sequence we're going to talk about started
at the fastener hole indicated here. What's important to
focus on here is the length of the fatigue crack. The blue
is supposed to depict the sky. From the outside of the airplane
that crack was only a tenth of an inch long, and yet it
contributed to a sequence of events that we're going to
look through in the subsequent charts.
Next chart, please. Part of the problem is that it wasn't
just one crack at that fastener. There was one on the opposite
side, as well. It was only .11 inches. So from a detection
standpoint, this would have been a bit of a challenge to
be detected visually just from a casual walk-around kind
of inspection. From a fracture mechanics standpoint, though,
the crack is really a half inch long because when you look
at the stress intensity at the tip of the crack, what it
depends upon is that total length, that .53 inches. And
fatigue cracks, we now know, grow at a rate that is a function
of how long they are. So the longer the crack, the more
rabidly it will grow as that part of the structure goes
through its next cycle of loading up and down.
Next chart, please. Not only was Fastener Hole 5 cracked
on both sides but there were also adjoining fastener holes
numbered 3 through 9 that also had these kinds of cracks.
Next chart, please. Consequently, Fastener Holes 3 through
9 simply zipped across one afternoon when the loads hit
a particular level; and this particular sheet of middle
separated from its counterpart.
Next chart, please. The problem is -- and I must apologize,
this chart didn't quite make the translation from Macintosh
to PC the way I had hoped -- this chart is intended to illustrate
two pieces of skin with an adhesive material between the
skins. You see, the fasteners were never designed to carry
the load. The load was supposed to be carried by the adhesive.
The adhesive broke down. There was corrosion that took place.
So we have a combination of adhesion failure and corrosion
going on, destroying the primary joining mechanism. The
fasteners picked up the load, but cracks developed very
quickly because they really weren't intended to carry the
load for very long.
Next chart, please. The failure next was supposed to be
stopped by what's called a fail-safe strap. These are spaced
every couple of feet; but it also was glued, if you will,
to this skin. The glue had eroded over time. Corrosion was
taking place. So when the came load came zipping down to
the fail-safe strap, it too broke.
Next chart, please. Indeed, all of the fail-safe straps
broke between the two major bulkheads that define the boundaries
of this particular failure. Fortunately, there was only
one fatality, although there were a number of other injuries.
The silver lining to this particular cloud is it caught
the attention of the aerospace community, and since then
there have been a whole series of efforts that really were
stimulated by this and some subsequent events.
Next chart, please. One of the matters you all will be talking
about later, I think, might be somewhat related to this
chart. This was not a matter that was brand new in 1988.
The first signs of it were back in 1970, and the bullets
in this chart sort of trace some of that history.
Next chart, please. So in closing, two more charts. Next
chart. In looking back at the life cycle management of fleets
over time, there are some things that seem to serve us well,
and they're highlighted here. We talked about the durability
and damage tolerance analysis, the full-scale fatigue tests,
tear-down inspections, updating the damage tolerance analysis
with new loads data because loading environments change
over time with flight vehicles, and maintaining high levels
of system integrity.
Next chart, please. In closing, many fleets have flown way
beyond the traditional points of retirement. In studying
these flights, each seems to have its own unique story in
terms of the challenges it had and how those challenges
were dealt with. We hope, we at Rand on the Aging Aircraft
Team, that this quick survey of the landscape may prove
of some aid to the board as you continue your important
ADM. GEHMAN: Thank you very much.
MR. ERNST: I'm hoping to see a slide here in a minute
that comes up.
I want to thank you for the opportunity to talk to you a
bit more about the cultural issues. Dr. Gebman and I compared
slides for the first time about two hours ago, and you'll
see some tie-ins to his slides that is more by coincidence
in our mutual experience by preplanned coordination.
One of the things I want to focus on is cultural, and it
goes back to part of the problems that I saw in Dr. McDonald's
Shuttle Independent Assessment Team back in 1999 and some
changes that I think need to be made in the aerospace industry.
Next slide, please. I also want to offer the apologies of
Colonel Mike Carpenter, my counterpart in the Air Force
Aging Aircraft Program, who was still stuck at Wright Patterson.
You'll see these slides we kind of do interchangeably on
here. This one's a little dated, but it shows the growth
of the age, the average age of our fleets over the last
10 or 12 years, most of it from the DOD side from a procurement
holiday. When you're talking about an aircraft reaching
20 years of age, that's an average age. You've got some
like the B-52 and the KC-135, H-46, they're getting up in
the late 30s.
We are in unprecedented areas in dealing with aging aircraft.
It's not like we can go back and find the predecessor of
the B-52 and see how it did in its forty-fifth year. There
isn't that data. As you can see from Dr. Gebman's presentation,
there are a lot of complex issues. I use the phrase, "This
isn't rocket science," but it really is a complex issue,
an age type of rocket science in there. Even though we have
a lot of very, very talented individuals working on these
issues, we're kind of a 1-of-1 type of scenario. We're out
in new areas in there.
I also want to show that the systems, even that are old,
it doesn't mean they can't be effective. I think all we
have to do is look to the recent aircraft performance in
Operation Iraqi Freedom to see that our legacy platforms,
when they're put in the hands of qualified operators and
maintainers that are dedicated to their jobs, can do a tremendous
job and do a great performance. But sometimes those aircraft,
when they get up in age, we have new issues that we have
to handle in there.
The challenge we need to do is balance when can we recapitalize.
There's no idiot light that just sits here and goes, ding,
"Replace this aircraft and buy new aircraft." We have to
look at a variety of factor, things such as fatigue tests,
tear-down inspections, load surveys, complex issues. And
frankly, they aren't very sexy. When you talk about I want
you to go study corrosion and rust propagation in aircraft,
that's not the thing that the young kid out of school necessarily
wants to focus on. So there's some challenges there.
Next slide, please. One of my other hats that I put on to
cover my bald head is part of the Joint Council on Aging
Aircraft. I wanted to explain a little bit about this. This
was a grassroots group that got together a little less than
two years ago because we all realized in the Air Force and
the Navy and the Army and Coast Guard and DLA and NASA that
we did not have enough resources. You can read resources
as people, money, and time to be able to handle all the
issues adequately but we said, you know, we're taxpayers
and every April 15th I look at my tax statement and say,
gee, I'd like to see if I can reduce that tax burden somehow.
So we decided to cooperate and graduate and see if we could
share things together and work together on certain issues
in here. This group met in about August of 2001, the Joint
Aeronautical Commanders group said, "Hey, what are you doing
on aging? Let's get together and formally charter this group."
Next slide, please. So if you know anything about the Joint
Aeronautical Commanders Group, the service three stars,
at the systems command they report to the Joint Logistics
Commanders group in there. They have a series of boards,
and we were adopted by them and became one of their boards.
Click it again for me and bring up my next pretty picture.
There's the people we have from the leadership of the different
aging aircraft communities. And we are a board and what
we're trying to do now is bringing the attention of aging
aircraft issues up to the other members of the board and
to try to get things changed.
For example, training. We went around and we found out that
sometimes our maintenance training wasn't up to snuff in
some areas. So we went back and said, "Hey, how does that
training curriculum that was done when the S-3 that Admiral
Turcotte flew was delivered in 1974, how should that change?"
And we went through and looked at seeing some of those things
because aging is going to change some of your core functions
and logistics and engineering and supply support and those
issues and our job is to bring focus to those.
Click it again for me, please. Next slide, please. What
is the mission of the JCA? Twofold really. One is to identify
and investigate issues. But we're not just a think tank.
We're not going to put a pretty little report that says
you really need to go, you know, build this or you need
to do this. We're also serving as program managers that
are fielding products, especially in the transition area,
taking a lot of the new technologies that are out there
and look really good, putting them on aircraft and making
sure what application they work. That's our focus. And that's
one of the biggest pitfalls we have on an aging side is
taking all that really neat stuff out there, all those science
fair projects, and putting them on platforms.
Next slide, please. Ironically, I sat in the airplane late
last night and said what are some of the characteristics
of a robust, good successful program; and you'll see a lot
of similarities to what Dr. Gebman presented. The first
thing we have to do is understand how all of the components,
whether it be an O-ring, a structure, an ejection seat in
a fighter aircraft, whatever you need, how does that age.
If you look at the way we classically develop air vehicles,
we spend a lot of time focusing on the development side,
getting it up to initial operational capability, and then
we've qualified all those issues, they're good, we just
kind of do some monitoring of our data but we really don't
know all the interdependencies of all those different materials
and how they age as a function of time, how they age as
a function of changes in environmental regulations, how
the load changes, the pilots are going to fly the airplanes
differently. We have mission changes on there and we now
want to be able to do this or do this or drop this bomb.
You can look at all the views of the airplane over time
and see the mission changes. So we have to understand how
each of those subsystems are effective in the system of
The next thing is monitoring our fleet usage data. You give
a pilot an aircraft, and he's going to find unique ways
to be able to fly that airplane in an environment, especially
with new mission growths that we've got to counter. The
way you do a fatigue test is you go and you estimate how
many 1G, 1 1/2, 2G maneuvers, how many landings, how many
takeoffs, how many pressurization cycles, and you put it
all in there and you literally, you know, bend this thing
like it's a piece of silly putty to see where it cracks
going in. But you're guessing how that airplane is going
to be used 20 and 25 years in advance. And one of the changes
that we've seen is we need to go and monitor that fleet
usage, collect that data, and then update that fatigue testing
because, you know, I guarantee things are going to be different
ten years from now, just as they were ten years ago.
You need to utilize that fleet data to go back and not just
collect it in some big data morgue but go back and say:
How's your original calculations? Are you using up your
service life earlier? You know, the Navy went and bought
some F-16s for their adversary squadrons, and we used them
up in about four years because they were all doing the shooting
down their watch type of stuff very quickly in there. The
mission changes, the requirements change, and we have to
be able to make sure our original predictions -- they're
not wrong, but they've got to be validated. It's kind of
like me taking my two thumbs and going like this and saying,
yeah, I can figure out and calculate how I'm going to go
to the moon. You've got a lot of mid-course corrections
you have to do.
The last issue which was brought up before, I found it amusing
to hear the previous panel talk about the daily report systems
in PRACA. We need to collect good data, but we need to have
that data resident at the subject matter expert's fingertips,
not in some type of huge data base in the sky that nobody
can get to. And all those elements need to be in there.
It's more than just neat technology. You have to have all
these elements and, folks, this ain't sexy but this is the
core that allows you to manage a fleet effectively.
Next slide, please. The Joint Council on Aging Aircraft,
working together, try to run their own programs and share
this data together, is trying to make process recommendations
and not just field issues. Microcircuit obsolescence was
brought up today. What data do we need to buy in our acquisition
programs to make sure that we can support the rapid changeover
in technology, because we're not going to drive it in Department
of Defense or NASA anymore. When we have to get with the
industry and figure out what data we need, what's the best
approach, that's going to require some acquisition changes,
some process changes -- again, not just technology -- but
yet we will take those technologies, evaluate them, and
say these are the ones we need to select.
I once told a group that I was walking along the beach and
picked up a pretty seashell and out ran three guys selling
corrosion solutions. I mean, there literally are hundreds
of technologies; and I think I broke my corrosion lead's
pencil when he got up to about 84 different areas. I said
let's get six out there and be successful. We like good
ideas. That's what fuels the reduction of our problems with
aging aircraft, but we need to also make sure that we are
pushing not all of them but we are pushing the top couple
We are facilitating the transition, making sure that we
are prototyping them on the aircraft. We do not fly what
we have not tested; and I can show you story after story
after story when it approached that test, something else
happened, either we had a sealant or we had a compound,
or wash cleaning fluid that interacted and we need to be
able to evaluate those issues.
Of course, we're promoting knowledge management. What is
the cost of aging? Where is that big idiot light that says:
"Buy more F-18EFs and retire S-3s for tankers"? Where is
that point that we can make the right economical decision?
And there's a paucity of data on those issues and it's kind
of like everybody has their own way of calculating it and
we're working with Rand, trying to get all those groups
So we're working together on a variety of issues from process
to technology to acquisition to knowledge management type
Next slide, please. That's what I do on my part-time job.
We've been tasked by the Aeronautical Commanders Group to
try to foster a national strategy, working DOD, NASA, FAA,
and industry. What do we need to do? A lot of our effort,
about 80 percent of our time, is on what I call tactical
initiatives, what is the best way of inspecting wire, what
is the best corrosion compound, yada, yada. About 15 percent
of our time or more is strategic areas. What do we need
to do to handle diminishing manufacturing sources and obsolescence?
About 5 percent of our time is on things like what is the
right amount of sustaining engineering that we need to have
on our team. How much emphasis do we need to have on our
data systems? What data do we need to collect?
We just recently partnered with NDIA and AIA, two industry
consortiums, so that we can get feedback from industry,
because I'm not going to say that I'm clairvoyant and have
all the answers. I've made enough mistakes, I have nine
lives based on my mistakes, but I want to get from industry
that partnership of where do they think we need to change.
Do we need to change our process for buying, for supporting?
What amount of balance is there in the government and industry
Next slide. You purposely can't read this. I don't want
anybody to read this because it's an early version. But
we've actually gone to doing road maps where we've surveyed
-- and this is from wiring -- from both a technology point
of view, an acquisition, a logistics, a training, all those
areas, all the different programs that are out there. When
you see those pretty little red things, well, green is good,
yellow is ehhh, and red is real bad. You see where we need
to build a strategy, and we're trying to make sure that
all of our funding and resources, they're not joint but
they're at least lined up and all pointing in the same direction
and we're pulling in the same way.
Next slide. What are some of the successful models of teams
that we've stood up. Too often we have a hearing like this
and we go in there and Congress passes a new law and we
anoint a new person to be the czar of something and he comes
out, or she, and puts out lots of mandates. And maybe I'm
a cynic -- well, I know I'm a cynic -- maybe I'll admit
it -- but that doesn't always work.
One example I want to point out is what we did with the
JCAA corrosion steering group. The reason it was successful
is we took the materials experts in each of the sites and
married them up with the program teams, put in logistics
people for publications and training, a cross-functional
IDT, and said, "You guys tell us what to do." My role now
becomes less of a messenger and more of a barrier-removal
expert. At least that's what I call myself. They call me
something, other things, but we can't say those in public.
So we need to build those from the bottoms up and not just
create something from the top down that puts more unfunded
mandates on us.
Next slide, please. Summary. I think our aging aircraft
problem's a serious threat. I think it's something that
requires an infusion of resources, an infusion of capital,
and a national strategy to be done. At the Joint Council
on Aging Aircraft, we're trying to coordinate those different
areas. You can come back and judge whether we're successful
or not. I think the industry cooperation is critical. We're
not going to say that this is a government-only issue, but
we're listening from the best practices. I will steal from
anybody and any group and, as Winston Churchill said, he
would even say a kind word for the devil in the House of
Commons if he would help him against the Nazis. I'll even
partner with the devil if he'll help us with our aging aircraft
strategies, and I think we need a strategic process that
requires that collaboration. And the last time I checked,
we need NASA's involvement in there. Their involvement's
increasing, but we need to remind NASA that one of those
A's stands for aeronautics and we need them and their expertise.
ADM. GEHMAN: Thank you very much.
MR. WALLACE: I think the focus has been mostly on
structures, although Mr. Ernst did talk about avionics and
wiring. I know that in the civil sector where I came from,
after Aloha we launched, of course, a very extensive aging
airplane program. I feel like the structural part, at least
perhaps in the less challenging field of civil aircraft
operations, is reasonably well handled or at least that
we currently feel that the aging systems challenge is greater
-- and wiring in particular.
I wondered if you have any sort of conceptual thoughts on
aging systems, wiring, and whether or not there's a different
approach. You talked about the need for accurate reporting
and that sort of thing. But in many respects those seem
to be some of the more difficult challenges.
MR. ERNST: You could pick any subsystem that you
want and the process that was set in place -- from analysis,
technology, investments, prototyping, data collection --
that Dr. Gebman showed, needs to be followed through. And
I believe that the FAA's wiring non-structural program follows
some of those classic issues. In having been part of it
and actually teaming with the FAA on some of those areas
in wiring, you can see that it follows the same type of
elements in there.
Wiring is a major issue. We made some mistakes when we selected
the wire types in some of our vehicles in the Eighties.
We did some qualification testing on it, and it had some
very adverse characteristics. I'm trying to be nice. We
now need to make sure that we're developing things that
are not just saying, yeah, throw that one away, build all
new aircraft, but can inspect it to make sure the bad characteristics,
i.e., the tracking that was associated with aromatic polyimide
insulation is not prevalent. But all those elements require
smart people work together and the success story is -- I'm
not sure you aware of this, but the FAA has spent a fair
amount of money really investigating the different types
of inspection technologies, whether it be frequency domain
reflectometry, time domain reflectometry, scanning wave
ratio, and a whole bunch of things that make my brain hurt.
And the Navy is actually doing some of the transition and
manufacturing of those systems and buying and fielding them
initially in our depots and our organization of the troops.
The Air Force is doing the same thing. We're working together
on these issues and eventually we're going to get products
that the commercial industry can take back in on. So you
see the FAA do the early R&D, the Navy and the Air Force
do some of the tech transition of prototyping and measuring
and quantifying what percentage of wire chafing is now degraded
that you have to replace -- what are those red, yellow,
green thresholds -- and then the commercial aircraft industry
can pick up and procure those things without having to develop
all those issues. The process is pretty much the same, but
we need to make sure we have a robust in all those issues.
Wiring is in pretty good shape. Corrosion in structures
is in pretty good shape. If you want to talk about helicopters
and all those rotating machinery, it's a pocket of poverty.
MR. WALLACE: Well, following up on one of your points
about the type of detailed inspections required, I mean,
can you speak to the issue which I know was very much discussed
sort of in the post-Aloha inspection implementations of
just sort of numbingly monotonous maintenance tasks and
the human factors associated with that?
MR. ERNST: I like the choice of words. One thing
that when I got a chance to sit inside or look at the internal
bay, cargo bay of the Columbia in '99 was at Palmdale and
there were wiring issues, the primary method of inspection
of wiring was the Mark 1 motto, eyeball in a mirror. And
I sat there with the Air Force wire technologist on a team,
George Zelinski, a very detailed, knowledgeable individual,
and I tried to see if I could find those myself because
I'm an engineer. I've been around wiring enough times. I
couldn't see those issues that they were required to pick
up. And we had a system then that was mind-numbing, that
required a lot of expertise and experience and there's technology
out there that can do that better and, more importantly,
can do that as a precursor to failure. You don't have to
wait until you see insulation to say, yes, it's through.
What we need to get to is a prognostic system where we can
check non-intrusively, not pulling bundles apart, but we
can check those wiring bundles and say I'm starting to get
some breakdown whether it be due to hydrolysis, whether
it be to chafe, vibration, wear, gremlins, whatever, and
say now I've got 80 percent through. At 20 percent I now
ought to go on a scheduled maintenance procedure and put
that together. And that's where we need to go and that's
part of a holistic wiring strategy that I believe we have
right now. We just have to get it funded and implemented.
MR. HUBBARD: I have a question for Mr. Ernst. You
made a passing reference to NASA's PRACA problem-reporting
system. Could you characterize for us what you think are
the best characteristics of the kind of accurate problem-reporting
system you referred to in your slides?
MR. ERNST: A system has to be realtime. It cannot
be a system that takes 18 months to collect data. It's got
to be something that is easy for the operator or maintainer
to input. The Navy system, years ago, was a paper system
where the poor guy, after working a lot of hours fixing
the aircraft, would fill out the paperwork and, because
of that, there were inaccuracies once in a while -- not
in Admiral Turcotte's squadron, of course -- but there were
inaccuracies that every once in a while we went back and
looked at those things.
ADM. TURCOTTE: We go back.
MR. WALLACE: Are you trying to sell him something?
MR. ERNST: I could tell stories, but I won't.
It has to be a system that is easy, simple, robust, and
it has to be something that tells you something about the
failure, not bug-in-the-cockpit type of issue and then say,
"I removed the bug." You need to go in there and say, "Hey,
I had some failure issue," and it needs to tie back in from
the operator what his perception of the failure, because
he's going to describe it, "Hey, this didn't work." He's
not going to say that you had a corrosion on Pin 5 of your
connector which stopped your data flow. That's going to
be the engineer, and it has to tie those systems together
with some software that can easily do some trend analysis.
And another point we have to do is we have to keep the data
long enough to do trend analysis. And there has been a push
to throw systems and data away after 18 months and we need
to go back five or six or seven or eight or ten years to
get a statistical sample size. So those are some of the
characteristics, and we're working to get some of those
systems implemented now.
MR. HUBBARD: On the report that my predecessor Harry
McDonald did, one of the shortcomings that he found was
that the PRACA system did not appear to have all of these
characteristics you just mentioned.
MR. ERNST: Harry called it the data morgue.
MR. HUBBARD: Data morgue. Yes. One of the things
that you commented on just a few minutes ago was getting
the material to the subject matter experts at their fingertips.
Can you expand on that a little bit?
MR. ERNST: Sure. Let's switch to an avionics box
failure. We need to not only have it so that a data expert
who knows the system can write trend reports but the information
if we get a failure back, let's say, on an INS system that
failed, that individual who's cognizant of that system needs
to go in there and say, "Have I had other failures on this
system? Can I find some trending? Is it just recently or
periodic? Can I go in and find out if memory chips or whatever
type of chip is failing in other systems?" He needs to be
able to do that research, that forensic science at his computer
terminal and a lot of times our data systems will give us
great reports on how many maintenance manhours we spent,
three months late. And when we get a mishap in, when we
get a box that's been failed, we need to understand and
have that information right there at our fingertips.
MR. HUBBARD: It would be as if you only got a report
on your checking account every three or four months.
MR. ERNST: Yes, sir.
MR. HUBBARD: Thank you.
ADM. GEHMAN: Mr. Gebman, in one of your viewgraphs
that you presented on the heavy maintenance work days per
depot for KC-135s and also in the heavy maintenance workload
ratio which showed how much depot-level maintenance is required,
how it's grown over the years, in your experience -- and
I'll ask both of you this -- is that an accurate indicator
that there's something else working below the system that
you need to go look at? Just keeping track of how much depot-level
maintenance is required and how it's growing, how does that
relate to characterization of aging?
DR. GEBMAN: Excellent question.
ADM. GEHMAN: Or is it just interesting?
DR. GEBMAN: Excellent question. We have studied now
all of the Air Force's fleets and have compiled the statistics
for, in particular, the labor hour growth over time; and
it seems that once you get beyond 15 years, you're almost
certainly facing a future of climbing work to be done --
some fleets that will start a bit sooner, the fighters tend
to start sooner, their lives being somewhat shorter than
the larger aircraft. It just seems to be a feature of aging.
It might well be somewhat analogous to people. In the older
years, we find ourselves going to the doctors somewhat more
often than in our teenage years.
So if you want to have a sense of the age of a fleet, one
measure that you might look at is, well, how is the maintenance
workload changing over time. And when you see that steep
part of the curve, like the presidential transport, the
old 707 known as the BC-137 in Air Force nomenclature, that
one literally exploded over a couple-year period and those
airplanes are no longer with us.
So it's certainly something to watch. We've tried regression
analysis, various statistical methods to try to correlate
the rate of rise, the characteristics of fleets. We're making
some progress in that area, but this is an area where there's
a lot that's not known.
MR. ERNST: You want to mention the cost-of-aging
ADM. GEHMAN: Go ahead.
MR. ERNST: One of the issues is I had seen the Rand
data almost when I started in the aging aircraft program
about four years ago and we've shared back and forth and
just recently the Joint Aeronautical Commanders Group Aviation
Logistics Board has kicked off an effort that we're part
of to look at what are these factors, can we translate the
KC-135 experience to other Navy aircraft and other Air Force
and Army helos and try to understand what are those factors
so we can get a better understanding of what's causing it
and what the trend lines are. Just having information that
says my cost is going up is not sufficient to be able to
correct the problem. You need to then drill down and say,
okay, but why. You know, I think on the KC-135 they have
a pretty good idea of that. But that's what you need to
do is not just look and say, yes, it's going up by 7 percent
but you need to understand why is it going up 7 percent
and what can you do to try to mitigate that curve.
ADM. GEHMAN: So my understanding is that, unlike
the Dow Jones Industrial Average, the fact that older aircraft
require more maintenance is not remarkable in and of itself
and is not an indicator that anything's breaking or anything's
going wrong. You've got to have much, much better indices
at the system, subsystem component level in order to determine
MR. ERNST: And it's not just age. I'll give you an
example. We were talking this cost of aging. I don't remember
the numbers off the top of my head but one of the folks
at Tinker said it's costing them X number of hours to paint
a KC-135 now and it cost them a lot less ten years ago.
And they said we're not adding one more ounce of paint.
The problem is that you've had different changes in environmental
regulations over those years, and you've got to make sure
you're accounting for things properly. I mean, those environmental
regulations aren't bad, but we've decided that this hurts
Bambi and Flipper and those sorts of things and we want
to take them off and it requires different steps and you've
got to factor that in there. A lot of the cost growth you're
seeing is due to things that are not age, either environmental
or fleet usage. Yes, they're going to go up, but they may
go up in a certain time to a manageable point and then where
that curve breaks, that's what we have to figure out.
DR. GEBMAN: I'd just like to basically add that Bob
is absolutely right. You need to look at the underlying
mechanism. If the workload is climbing because you now have
to tend more to corrosion and you're satisfied that you're
able to see the corrosion and tend to it, that's manageable.
In the area of fatigue cracking, you have to be a little
bit more careful. Rising workload may indicate that you're
getting more and more cracks closer and closer together,
and one of the very important assumptions that we make in
managing fatigue cracks is that the neighborhood is healthy.
So as the population density of cracks starts to get too
high, you run into a situation where you might have thought
you were fail-safe but, in point of fact, the neighboring
structure can't carry the load.
DR. WIDNALL: I'm sort of sensitive to this issue
of aging aircraft because I worked on the B-52G when I was
a freshman and I worked on the KC-135 when I was a sophomore.
So my friends are still out there.
What I want to talk about is composite materials. I was
a little sorry that you sort of excluded that from your
chart, but I'd like to get a sense from you about some of
the challenges associated with these composite materials.
How well do we really understand their fatigue properties?
Do we really understand their properties as well as we understand
metals? What about their exposure to UV radiation and high
temperatures and corrosive chemicals and all those sorts
of issues? And I know we're using these more and more in
our aircraft fleets in general and in particular on the
shuttle. They're obviously a key part of it. And it's just
not composite materials but other kinds of brittle materials,
sort of what I would call nonstandard terms.
DR. GEBMAN: Thank you for asking this question because
when I was thinking about what to talk about today, I really
struggled with do I talk about the areas where we have depth
of knowledge that might be useful to your investigation
or do I talk equally across the areas even though the depth
of knowledge is shallow. Clearly, with metals there's a
lot that we know, especially on fatigue, and we're learning
rapidly in corrosion.
In the area of composites, I think that Charlie Harris from
NASA Langley at the conference earlier this month of the
AIAA, American Institute of Aeronautics and Astronautics,
this big gathering, 780 people, 525 papers, Charlie gave
a talk about the progress in composites and he was very
positive and upbeat about all the good technical work going
on. And that was all appropriate. But then he shared with
the group a round robin exercise where they sent problems
around to people, the same problem to work on, and people
came back with different answers. And then they did another
exercise where they even told people what the problem was
and they still came back with different answers in terms
of the methods and the assessments.
So the whole area of composite materials is one that might
be analogous to where we were with metals back in the 1950s.
Back in the 1950s, we had the alloy-of-the-month club; and
that's where the B-52 and the KC-135 came from. The young
engineers were finding out better ways to do the chemistry
to get strength, but they didn't have time to understand
the durability, the fatigue properties, and the corrosion
properties. I'm somewhat sensing that, with composites,
we're still inventing cleverer ways to get strength but
we don't yet understand the long-term durable characteristics.
The science is far more complicated because with metals
it's homogeneous, it's the same material, with composites
you've got fibers and glues or resins and it's a very complex
interaction to try to model and we're not good at it yet.
So anything that is made of composite requires even more
circumspection and attention than probably the metals.
DR. WIDNALL: I was afraid of that.
GEN. BARRY: Excellent presentations by both of you
and raises a lot of questions. As you know, the board has
taken a very serious approach to aging spacecraft in this
what we call R&D development test, however you want to call
A couple of comments. Your references to the Air Force,
as obviously I'm familiar with, where we are older than
we've ever been before. We've never been in this era in
the United States Air Force -- as is the Navy. We're approaching
ages where the average age of our platforms of 6,000 is
22 years old. So even within the data experience base that
we have, flying airplanes, we're approaching new environments.
Now, let's translate that over to spacecraft. We are entering
a new era in spacecraft, with reusable vehicles in an environment
of aging. We've never been there before. So we've got two
parallel efforts going on that certainly can kind of cooperate
and graduate, as we've seen evidenced by the Navy and the
Air Force here.
I've got a couple of quick questions and then a rather larger
one. First question is: Is NASA involved in any of this
Joint Council on Aircraft Aging, as far as you know?
MR. ERNST: Yes, sir. NASA has been involved in the
aging aircraft effort since Aloha, prior to me being in
it. The efforts at Langley in structures and corrosion NDI
have been solid. Just recently, Christmas time frame, before
Columbia, they said, hey, we recognize we need to help you
in that national strategy; and they're getting more involved.
We need even more. I need to fill in gaps.
GEN. BARRY: On your side as well as the space side?
I mean, are they translating lessons learned to both aero
MR. ERNST: Yes. I'm not going to tell you it's even
and homogeneous throughout, but I know that in wiring, the
shuttle folks at Kennedy are in lock step with my guys and
the FAA and I know the aerospace side and structures are
working real well together. We're trying to see where the
gaps are and plug them in there. We need more involvement,
but they have been involved.
GEN. BARRY: All right. Let me ask this. Two things.
Let's just talk about corrosion and let's talk about fatigue
cracking that, Jean, you mentioned earlier. Right now we
have capabilities within our aircraft to do stress-testing
that you mentioned as an example. We have programs that
are not only based in the United States -- Australia has
an excellent one on how do this. I think we all recognize
that who are in the industry. What can we do insofar as
spacecraft are concerned because obviously they are larger
and we translate that to our larger aircraft insofar as
dynamic testing is concerned, because I don't think it's
unfair to say that managing aging spacecraft in NASA, for
the large part, is done by inspection. So how do we translate
that, what we've learned in aircraft, over into NASA as
a possible recommendation?
MR. ERNST: I think you need to break it down into
the subsystem component areas. For example, we had this
discussion on the McDonald's team three years ago now, on
the SIAT team on wiring, where we had totally different
environments but we could take the Air Force and Navy's
experience with aromatic polyimide insulation and say here's
what we saw under these load conditions. Now, under a probably
higher vibration, higher thermal but shorter duration environment,
how is that going to translate? We know how that fatigue,
so to speak, environment can translate and run a new model
to see what it should do with the shuttle program.
That's the kind of transformation that could be done, but
only if you knew how each of those subsystems and the materials
of those subsystems is going to behave as a function of
time and age and environment over a number of years. The
problem is a lot of times we don't know that information.
So we know how it works here, we know the loads are different,
but we don't know how the age is going to translate as those
factors are translated, if that makes sense to you. I don't
think it's hard to do that, but you have to invest in some
age-related studies and that's not necessarily the top of
GEN. BARRY: One of the concerns we have is to be
able to analyze how the orbiters have been shaken, rattled,
and rolled over these many years, especially when we take
into consideration that this was a spacecraft that was designed
to be flown 100 times in ten years ago and now we're multiple
years, decades past that and we are still only at the 20s
and 30s. A question then is, you know, how do we maybe translate
some of the lessons learned on how the spacecraft are flown
within spec but, you know, after a while, get some kind
of stress loads on them that can be accumulated over time
and measured. Now, translate, if you could, the lessons
learned that we've developed on aircraft that might be able
to be translated over to NASA.
DR. GEBMAN: Could I have Chart No. 24, please.
MR. ERNST: You guys are going to learn this chart,
because he wanted to show this to you.
DR. WIDNALL: He's ready for you.
DR. GEBMAN: This is a really tough question. Obviously,
with the shuttle we don't have the luxury of a full-scale
fatigue test. Obviously doing a tear-down, if this was an
aircraft fleet and when we had hundreds or even tens, we'd
consider taking the oldest one and tear it apart and see
what ails it and then use that to guide future work. When
you're down to three, that's not an option.
So then you ask yourself, well, what might we do? And when
you look at this diagram, on the top row, the matter of
force tracking data and loads analysis, there may be some
things you could do in terms of assuring that NASA has developed
all of the effort that it can, evaluating the strain gauge
recordings and pressure recordings from prior flights, and
that you really have as excellent a record, historically,
of the loads that have been imposed on the structure as
you can possibly get each.
The next thing you then could consider doing is, given the
best loads data, to go back and, using more current finite
element analysis methods which have improved greatly over
the decades, to go in and do some spot checks on your stress
computations to make sure that you've got the best that
we can do in terms of estimating stresses from the given
loads and then take it the next step and go in for the fatigue
part to check on the crack growth calculations, the fracture
toughness issues, and to make sure that the engineering
community has really been resourced and tasked to do everything
that we can to understand the health analytically of the
Then the final thing you might consider doing. From the
debris that you do have, in effect, you have already a partial
tear-down circumstance and to go in there at some point
and literally take apart that which is still connected together
and really check for like adhesion on honeycomb, how is
that, that waffle still adhering to the face plates, and
just get as much mileage as you can out of your debris in
terms of understanding what the health of the remaining
fleet may be.
MR. ERNST: Slice up your poles, your joints, rivet
holes, things like that. That's what we do routinely.
To follow on with the chart that Dr. Gebman put up, you'll
notice a couple of things. One, do a mid-life assessment
of the loads. You know, the Columbia originally was kind
of a flight-test bird and I believe had some several hundred
pounds of instrumentation and sensors in there to measure
fleet loads. To give you an example, the P-3 and S-3 program
just recently completed mid-life fatigue testing at Lockheed,
and we found drastic changes to both loads from what they
were anticipating. The maneuvers were a little different.
The theoretical issues, the early introduction issues slowly
change over time. You know, it's like boiling a pot of water.
It doesn't boil all at once. And I think you need to go
back and really do those load surveys.
You also need to do some type of tear-down. You can't cut
up, you know, the Atlantis and make it a series of razor
blades and fractographic analysis and stuff; but the Columbia,
when they had wiring problems in '99, NASA did go and remove
certain wire segments. You can go in without cutting the
whole thing up and remove certain panels, remove tiles to
see adhesion, remove subsystems. When a part's going through
an overhaul, take this part on overhaul and do those types
of things. So there are things that you can do; but again,
you've got to have a proper program to get that environment
and see how we're doing.
The S-3 example in fatigue tests, we had 12 points that
we considered life-limiting on the aircraft. Four of those
they knew in the original fatigue tests and the odds were
out of there. We found an additional eight points that were
due to the loads, and due to the tear-downs we saw microscopic
cracks. We were able to go in and cold-work fastener holes
in that aircraft and give it fatigue life back. Real simple
operation, real cheap, and not have the 305-inch wing cracks
we had in the P-3. So you're able to do some of those things
if you invest in the time and the resource and have a robust
DR. GEBMAN: If I might, I'd just like to follow up.
Could I have Chart No. 7, please. There's an important aspect
that I neglected in my answer, and that is that we're dealing
with a spacecraft. And I apologize. Obviously with something
like the shuttle, you have thermodynamics acting as well
as structural dynamics; and in addition to getting a solid
characterization of the historical loads, you also want
to get a solid characterization of the historical thermodynamic
exposure because -- take a spar cap, any one of those four
spar caps that are identified with the arrows. If, in the
course of the history of a particular spar cap, it has been
exposed to temperatures different than the other spar caps,
then the loads in that part of the structure are going to
be different by virtue of the thermal expansion of the material.
So this is a very complex thermal as well as structural
ADM. GEHMAN: Let me follow up on that before I call
on another board member. Do I understand that you are suggesting
that it's useful in the study of aging aircraft to establish
some measurements of what I would call stress cycles or
something like that? We understand age. We understand landings
and takeoffs. But there are other events which cyclically
stress the aircraft, particularly in the case of the shuttle.
And it's useful to keep track of those, in addition to the
obvious ones like landings and takeoffs and how many months,
hours and all those kind of obvious things.
DR. GEBMAN: These things with aircraft are tracked
routinely. Exceedance curves are developed which are a statistical
way of representing even the small variations. My most recent
comment suggests that we should also construct a thermal
exceedance spectrum, as best we can from the historical
data, so that to the extent that we've got differential
thermal expansion of structure going on, we can factor that
into the loads that the members receive.
You see, there's two load levels. One is the aerodynamic
load and the inertial loads applied to the gross structure.
The other issue of load is, for a particular structure member,
what load does it see over its lifetime; and that can be
driven by thermal expansion issues, just as it can be driven
by the aerodynamics. And given the historical records of
the temperatures, the engineers should be able to construct
and may already have done thermal exceedance curves to go
along with load exceedance curves.
MR. ERNST: I think you need to look at every environmental
factor and see if is there a similar type of a correlation
in there. We've done a good job of fatigue tracking. We're
tracking a lot more parts than we used to. The models are
a hundred times more detailed than they used to be. We can
calculate things a lot finer, but I think you need to be
able to look at all the different loads in environments
that any vehicle goes on and say, okay, what's changing,
what's the effect of that over time.
ADM. TURCOTTE: For both. Kind of the 3Cs in aging
-- you know, Kapton, Koropon, and corrosion -- which go
back a long time in finding problems with Kapton wiring,
with Koropon bonding, de-bonding, heat translation, all
of those things. That's Part 1 of the question. Could you,
kind of both of you, talk a little bit about major lessons
learned from both fleet usage, commercial usage, and your
knowledge to the extent of findings on the shuttle, both,
you know, galvanic or intergranular types of corrosion.
Part 2 question. If you were king for a day with your knowledge
of the PRACA data base, what would you do to improve it?
MR. ERNST: You're going to get me in trouble. I was
very nonpolitically correct about the PRACA data base in
1999. And I have not seen it since then but I think if you
go back and you read the Shuttle Independent Assessment
Team report, you will find that the comments of the group
were less than favorable on PRACA. I'm not saying that the
Navy and the Air Force and the Army's data systems are perfect,
but we're taking steps in the right direction. So I really
can't comment on what they're doing today. I know they made
some improvements, but it was pretty abysmal back in 1999
and, I think, masked some of the issues that feed into your
risk equation that we saw back then. I think that was a
As far as handling some of the materials and some of the
issues with Kapton, aromatic polyimide insulation manufactured
under the Dupont trade name Kapton -- get that correct --
we didn't do a good job on establishing realistic life cycle
testing for that material when it was introduced. Kapton
has a lot of good properties. I don't believe I said that,
but it has a lot of good properties. It's very, very tough.
It has some very adverse characteristics that we never tested
for. But I think you can go through several other tests
and I know there's been arguments with the FAA on the flammability
tests, whether that's applicable, and there's lots of different
tests and we didn't do a good job of running a qualification
test and an aging test that's run on a short period of time
that's trying to cover 20 or 40 years. So we made some mistakes
The other issue is once we had problems with the wiring
insulation, I don't think we developed realistic scenarios.
If you look at the cost of replacing and rewiring a whole
aircraft, it's several million dollars. Well, do I really
need to do it? Do I need to do it in all areas? Which platforms
do I need to do first? And what we have done now is develop
a bouquet of options. Whatever color of flowers you want
and whatever kind of room, it goes together. Because what
my wiring options on the F-14 Tomcats, which are going to
be retired in the next four or five years, is totally different
than what I would do on earlier production F-18s or P-3s
that are going to be around a little longer. So you have
to develop options based on risk so that you can do things
quickly, cheaply, easily, and get it done and not just give
one option is all.
So I think two issues. One, we didn't do a good qualification
testing and we need to continue, just like the life cycle
testing, just like the fatigue tracking where you update
it and you get better; and the second issue is we didn't
develop any options.
DR. GEBMAN: On the matter of wiring, the Air Force
in the case of the KC-135 embarked on a major rewiring program
about five years ago; and that is going to probably continue
for the next four to five years, at which point they will
have substantially replaced the wiring on the 135. The basis
for this was an accumulation of maintenance action that
was becoming increasingly costly to exercise and a concern
for flight safety, and those two factors together seemed
to have driven the train on that fleet.
Unfortunately, our ability to predict life, we don't have
the engineering tools that we have with fatigue cracks,
either with composites yet or, for sure, with wiring, which
makes those areas very difficult to feel comfortable about
with an aging fleet.
ADM. GEHMAN: How comfortable would you feel with
the study of the aging characteristics of a main engine
that's fueled by liquid hydrogen and burns a thousand gallons
a second and produces a million pounds of thrust? How's
our data base on how that baby ages?
DR. GEBMAN: Well, on my chart I did include a line
that said propulsion; and it didn't get extremely high grades
for data or methods or people that really understand life
issues in that area. So you've hit another excellent nail
squarely on the head. For those areas, going back to General
Casey's comments about understanding margins and managing
to margins, you really have to worry that as time goes by,
you're eating into those design margins and at some point
the ice become thinner than what you're comfortable with.
And that's a technical judgment probably more than an engineering
MR. ERNST: Follow up. One of the successful programs
that the Air Force and the Navy has is on aircraft engines.
And they've realized that you've got a lot of moving parts,
a lot of high temperatures, a lot of complex interactions
in there. And they have what they call CIP, Component Improvement
Program, where they go back in and they test and they see
where their problem areas are and they incrementally try
to infuse newer technologies and fixes in the early parts
of the service.
Again, that's one of those that's always fighting to try
to get resources adequately in there, but if we follow what
the commercial industry does, we can really improve the
reliability and we can have a pretty good idea and almost
get to a scheduled maintenance type of inspection so we're
not flying and say, yeah, lost an engine or had a shutdown
but, okay, now at 7, 8 hundred hours I have an 8,000-hour
interval period and know exactly what to replace. So that's
another example where we've taken the methodology that Dr.
Gebman talked on structures and we've transferred over to
the engines, and I think both the commercial and the military
have very good experience in that being successful.
DR. GEBMAN: I certainly wouldn't quarrel with my
distinguished colleague, but I would hasten to add that
the commercial engine and even the military engine circumstance
with aircraft is far different than the circumstance we're
talking about here.
DR. LOGSDON: This is all very far away from the experience
of a Washington policy wonk. So excuse me if these are really
naive questions. What does the fleet size of three do to
the ability to do the sorts of things that you think ought
to be done?
And the second question, I think it's really for Mr. Ernst,
coming out of his independent assessment experience. Is
NASA routinely collecting the kind of data that would feed
into the kinds of trend analyses? You know, outside of faults,
PRACA and that, is there a data base that you could apply
some of these methodologies to?
MR. ERNST: Well, I think all the agencies and commercial
are collecting a fair amount of data.
DR. LOGSDON: On shuttle.
MR. ERNST: On shuttle? I mean, you look at the Navy
programs and Air Force programs. We're collecting 80 percent
of what we need. I still think we need to do more the cause
For example, if I went into the Navy's data base on wiring
chafing, there is no failure code for chafe right now. What's
the primary failure mode for wiring? We're fixing that,
by the way. So I can say that, but that's one of the issues.
I mean, we're not recording the right type of information
in all cases. We're about 80 percent there.
My beef with PRACA at the time was you couldn't go in there
easily and extract anything to make decisions. I at least
can go into some of the services' data bases and pull some
information and get a pretty good idea and then at some
point I have to play archeologist or forensic scientist
and go back through and do some more. But it works out to
that 80 percent. There need to be some other changes; and,
unfortunately, data is the one thing that everybody wants
to cut in the budget crunch. We don't want to pay for that
DR. LOGSDON: If I understand PRACA correctly, you
have to have a problem or perceive a problem to even get
in the system. I'm saying is the shuttle even instrumented
to capture the kind of data that you would like to have
to measure various elements of its aging.
MR. ERNST: Not in all cases, but I think you can
probably do some work-arounds with that and be able to check
things. I mean, you don't have to do everything in flight.
You can do engine warmup cycle times and check temperature-wise
in there, check component issues, and test things. Things
like that. You can capture that information if you need
You need the maintenance-reporting information, which PRACA
primarily did. You need to trend analysis like if I get
to this certain load level, this is going to impact my fatigue
life. And then you need to be able to do periodic instrumentation
at times. And it doesn't always mean a full-scale in-flight
test. It means capturing some of the data. And that data
was available. Could you get that. Was it easily, readily
available? No, it wasn't readily available.
DR. GEBMAN: Putting my engineering hat on relative
to your data question, given that the instrumentation and
wiring in the shuttle and the systems were designed in an
earlier era in terms of electronics, it might well be worthwhile
rethinking the matter of what are we interested in observing
during future flights in order that be might create a more
complete record of environment and loads so that we can
better manage the remaining lives of the fleet.
MR. ERNST: Health management, health monitoring for
DR. GEBMAN: And regarding your observation of the
number three, what does it mean to have three in a fleet?
From an operational perspective, one of the early lessons
I learned at Rand was that whenever you visit a unit, you
always expect -- and Admiral Turcotte will appreciate this
-- you always expect at least the Nth airplane to be a source
of supply for the others, if you're lucky. Sometimes it's
more than just the Nth airplane. So if you have a fleet
of three, from an operational perspective, one of the three
is needed to support the operation of the remaining two.
And to have an operating fleet with just two means that
you only have one backup and that's very thin.
MR. ERNST: And I think it makes your correlation.
A lot of times when you have how many hundred F-15s and
F-16s, you can start looking at the gross number of failures
and say I need to look at something. When you have three,
you can't rely on that. You have to take a little bit different
systems approach to be able to capture your data.
The Navy flies some type model series, you know, that are
12. Twelve EP-3s. And each one of them is a slightly different
configuration. But you can capture that information. It
just requires a little different approach, and sometimes
it's not as robust, predictive, leading edge because you
don't have that significant sample size.
MR. WALLACE: Were you suggesting, Dr. Gebman, that
sort of the fleet leader concept; or were you suggesting
cannibalizing parts? I wasn't entirely clear.
DR. GEBMAN: No matter how good your supply system
is in terms of providing parts, you always end up in a circumstance
where you have a first-time demand for a part and the last
airplane of the unit then becomes the offer of that replacement
part. I think that if you talk to the NASA folks regarding
the matter that's referred to commonly as cannibalization,
it's borrowing a part from one aircraft or spacecraft in
order to be able to launch one that's scheduled to go.
MR. WALLACE: Another question. This is jumping subjects
a bit. Should the goal of an aging aircraft program grow
beyond maintaining the aircraft to be as good as new? What
I mean by that is: Should it meld in with sort of obsolescence
issues, issues where the technology has simply gotten to
be so far behind the state of the art that it either makes
sense for economic or safety reasons to upgrade or even
reasons of simply maintainability?
DR. GEBMAN: You're raising the issue of replacement,
fleet replacement; and we have struggled at Rand with the
Air Force long and hard on that matter because, for example,
the tanker fleet. It's a very important fleet. Without the
tankers, the Air Force doesn't go places. They don't have
aircraft carriers to carry their airplanes. So they're very
dependent upon their tankers; and to have almost all of
your tanker fleet wrapped up in one type of aircraft that's
40 years old now and to be planning to do so for another
40 really raises questions.
The first thing we looked at, well, is there a case on economic
grounds for replacing the fleet. There was an economic service
life study done and it shows rising costs, but it doesn't
show the rising cost by themselves being a sufficient basis
for justifying a new fleet, whereupon then you start asking
questions along the line of obsolescence issues, foregone
capability improvements that you can't have without substantial
investment in an aging fleet. So this whole question about
when is it wise to replace a fleet is one for which we still
don't have a good methodology for dealing with.
MR. WALLACE: I really didn't intend to ask that question
about replacement. Well, it was a good answer. But about
replacing the fleet as opposed to simply upgrading, particularly,
I mean, fleet replacement, you know, lots of smart bean-counters
with spreadsheets do that for the civil aircraft industry
but I think there's a whole set of different issues with
next-generation spacecraft. My question really is more about
MR. ERNST: To address that -- and you picked on obsolescence.
When you get to the microcircuit obsolescence issue, which
has become a science fair, pet rock project of mine over
the last 10 or 12 years, there are lots of different options
and right now we are system-incentivized to find this chip
to put in this box in a lot of cases. We found about a third
of the time that doesn't make sense because not only is
that part obsolete but the three around it are terminal
and the whole board's wearing out because we keep replacing
it so many times because of poor reliability. So it's probably
better at that time to take the whole thing, take the cards
out, and make it a lobster trap somewhere and then put a
new system in there. That really happens about a third of
the time. But we need to again, I think, balance some of
the different pots and stovepipes of money that are available,
especially in DOD, to be able to optimize those issues and
have the best understanding of the age effects, where they're
going to be two years from now, because I may make a replacement
today and I've got three more downstream. I need to look
where I'm going to be three years from now and say this
is time to replace this 1988 Tercel that I had with 189,000
miles and go buy something new because this is just the
tip of the iceberg. And I don't think we're doing a real
good job of that but's one of the challenges of not just
maintaining status quo but looking and saying what capabilities,
what mission growth areas, where am I going in some reliability
issues and balance all of those into like a triangle of
a decision matrix.
DR. GEBMAN: There's a fleet that we're looking at
now that has the potential for receiving an upgrade to its
aviation electronics to give it capabilities to continue
its military relevance. And there are also a series of mods
being considered to upgrade the engine so that its flight
safety features remain appropriate. And similarly with the
air frame. And as we're going through the arithmetic on
this particular fleet, one of the things that we're seeing
is that by the time you're done making whichever of the
three mods or all three of them to the fleet, the years
remaining becomes very significant to your choice. And when
you go to the operator and you ask the operator, well, how
long do you want to retain this fleet, well, they're really
not sure. So this question is almost as difficult as the
fleet replacement question.
MR. ERNST: And you look at the mission changes in
the Department of Defense in the last couple of years where
we've gone from a Cold War scenario to more of a small conflict
and now global war on terrorism and it changes. We have
planes that, to pick on Admiral Turcotte's S-3, that were
designed to hunt subs that were doing surveillance and tanking
and dropping weapons and doing, you know, partridge in a
pear tree and everything else. And you need to look at those
mission changes as a function of age too and say, you know,
I may be able to keep this aircraft doing what it did five
years ago but you know I need to replace it. I need to go
over here. And we don't always balance all those issues.
I know the Air Force is really trying to look at that decision
and set up a fleet viability board to weigh the aging factors
in these mission scenarios. I'm monitoring that for the
Navy to see what they do; and then after they get all the
kinks worked out, we'll steal it. But that's kind of the
approach. I think that answers that it's not a simple answer
but that's what needs to be looked at. I think the shuttle
has the same issue: Where does it need to be ten years from
MR. HUBBARD: I heard one of you mention or whisper
the term "vehicle health monitoring," I think. The notion
of a fleet of three. I'd just like you to think out loud
for a minute or two about how vehicle health monitoring
would apply in this case along three lines. One, what would
a systems approach be to that, given that we have a fleet
of three? Second, realtime versus recorded measurements?
Third, what other measurements could you imagine? I mean,
we've got a thermal protection system, for example, that
is pretty unique to the orbiter versus the military aircraft
you mentioned. We've got pressure, strain, and temperature.
Can you imagine, in this kind of systematic approach to
vehicle health monitoring, what one might do?
MR. ERNST: Let me answer in reverse order. I don't
want to bad-mouth technology. And I've talked about some
cultural issues but there's some real technology advancements.
I know some of the DOE labs have now started looking at
electronic signature analysis for failures in motors, predicting
when motors are going to fail. There are all kinds of things.
I mean, you can literally go around to the different areas
and find better ways that people can get precursors to failures
if they measure data and give you good information. That
would help us understand. From an overhaul interview, it
would let us know if you had a degraded flight mode issue
so that we're not having, yes, that system failed, we have
to do something else. It would really help you manage your
redundancy a lot better, too. So there are a lot of new
technologies beyond the strain engages that I learned about
in college that need to do.
I think the realtime versus recorded is something you need
to use a system engineering approach in analyzing. There
are oil analysis systems that I remember we had a vapor
cycle system and by the time you got oil in the filter,
you had basically eaten the whole system; it was too late.
So putting an oil analysis system that you measure it every
ten hours wasn't doing any good. It needed to be realtime.
Not everything needs to be realtime and any information
at all, whether it be on one unit or on three units, is
a lot more than no information and I think that having some
health monitoring systems on any fleet -- shuttle, the F-18,
the S-3, or whatever, F-15 -- gives you information if you
use a good systems engineering approach, not just collect
data for data's sake but see what are you trying to do with
the data and then drive what you need to collect to get
data or what technology best does that, I think, is helpful.
DR. GEBMAN: I would like to speak both as a proponent
and also share a word of caution. The engineering in me
would prompt me to want to put strain gauges and instrumentation
in many places. Probably too many. There's a trade-off between
the disease and the cure, and it's possible to overdo a
good thing. We need to remember that, with this instrumentation,
comes wires; and we've already been talking about the vulnerability
that wiring can introduce into the system. So what I would
think might be helpful is to try to understand what are
the critical issues that we're concerned about or we should
be concerned about and then ask, for those critical issues,
what initially at least modest amount of additional instrumentation
might be appropriate and try to really focus on the core
vulnerabilities and not to go too quickly too far overboard.
MR. ERNST: We can't be kids in the candy shop. I
ADM. GEHMAN: Thank you, sir. I'm going to ask the
last question myself; and, hopefully, it's a brief one.
I think probably, Mr. Gebman, your Chart No. 3 answers this
question; but I want to allow us to listen to it for a second.
Would you list the aircraft aging areas of examination as
to which of them appear to be mature technologies and which
of them appear to be not so mature? Obviously the detection
of corrosion, of course, is obviously a big one and I suspect
we'd probably know a lot about that.
DR. GEBMAN: Probably the quickest answer to the question
would be to focus on the first column and the last three
columns. In the last three columns, we have my subjective
assessment of where we stand in terms of data, methods,
and people. The metals area for structure, we're in very
good comparative shape to the others.
In corrosion, our data and our methods are still what embryonic
but now, thanks to the various laboratories really engaging
the last several years in a more aggressive way, we're building
a core of people that are knowledgeable in the area.
The business of adhesion, we haven't paid much attention
to it. And my sense is that our data and methods are below
low and even the number of people really knowledgeable in
that area is not great.
Moving down to the composites, there's a lot of people out
there. There's a fair number of people doing excellent,
promising research; but the fruits of that research in terms
of data and methods is still forthcoming.
In the area of propulsion, the general area strikes me,
especially when we're thinking about shuttle types of applications,
as not particularly high. The whole area of high-cycle fatigue
is still a challenge for the engine community, even for
Then the "Other" category. This is the one that worries
me most because it's oftentimes the one that's not getting
the attention that's the one that bites you the hardest.
Functional systems, pumps, motors. The TWA 800 killed more
people than metal structures in recent times, and that may
well have been down in this "Other" category, either the
wiring or the functional systems.
So as the board moves forward with its good work, attention
to all of the technical areas. And all that I've tried to
accomplish here today is to bring forward that there are
some areas where the aging aircraft community really has
depth. If that proves to be relevant or of interest, the
community is certainly prepared to help. In the others,
it's going to be more challenging.
ADM. GEHMAN: Well, thank you very much. On behalf
of the board, I would like to express your appreciation
for your attendance here today and your complete and helpful
replies to our questions and the information that you've
given. You're obviously great experts and we've learned
a lot and we hope that we can apply it to this problem.
We appreciate your attendance.
We're going to take about a ten-minute break while we seat
the next panel, and we'll be right back.
ADM. GEHMAN: All right. We're ready to begin our
last session for the day.
It's a privilege for the board to recognize Dr. Diane Vaughan
from Boston College. Dr. Vaughan has written an influential
and well-read book on the Challenger accident. We are continuing
our look into the business of risk assessment and risk management.
This is one of the classic studies on the Challenger accident.
Most of the board members have at least read parts of your
book, Professor Vaughan; and we're delighted to have you
DR. VAUGHAN: Thank you.
GEN. BARRY: And we're ready for a test.
ADM. GEHMAN: I would like you to please, if you would,
before we get started, introduce yourself by telling us
a little bit about your background; and then if you would
like to say something to get us started, we would be delighted
to hear you.
DIANE VAUGHAN testified as follows:
DR. VAUGHAN: Thank you. I'm a sociologist. I received
all of my education at Ohio State University, getting my
Ph.D. in 1979. After that, I had a post-doctoral fellowship
at Yale; and I began teaching at Boston College in 1984,
where I am currently a full professor.
My research interest is organizations. I'm, in particular,
interested in how organizational systems affect the actions
and understandings of the people who work in them. So it's
what we call, in my trade, making the macro-micro connection,
how do you understand the importance and effect of being
in an organization as it guides the actions of individuals.
My research methods are typically what we could call qualitative,
which are interviews, archival documents, and ethnographic
observations. So using these methods, I have written three
books, the last of which was The Challenger Launch Decision,
which was published in 1996.
ADM. GEHMAN: Thank you very much. You may proceed.
DR. VAUGHAN: All right. I want to start from the
point of view of Sally Ride's now famous statement. She
hears echoes of Challenger in Columbia. The question is:
What do these echoes mean? When you have problems that persist
over time, in spite of the change in personnel, it means
that something systematic is going on in the organizations
where these people work.
This is an O-ring -- not The O-ring, but it is an O-ring.
I want to make the point that, in fact, Challenger was not
just an O-ring failure but it was the failure of the organizational
system. What the echoes mean is that the problems that existed
at the time of Challenger have not been fixed, despite all
the resources and all the insights the presidential commission
found, that these problems have still remained.
So one of the things that we need to think about is when
an organizational system creates problems, the strategies
to make the changes have to, in fact, address the causes
in the system. If you don't do that, then the problems repeat;
and I believe that's what happened with Columbia.
What I would like to do is begin by looking at what were
the causes of Challenger and, based on my research, to point
out how the organizational system affected the decisions
that were made, and then make some comparisons with Columbia
and then think about what it might mean, taking that information,
to make changes in an organization to reduce the probability
that this happens.
One of the things that we have learned in organizational
ADM. GEHMAN: Excuse me for interrupting. If I may
ask a question while we're still on this subject. On your
first viewgraph, the first bullet, you said when you find
patterns that repeat over time despite changes in personnel,
something systemic is going on in the organization. There
are no negative connotations in that sentence. You didn't
say something wrong is going on in the organization. I assume
the obverse is also true. If patterns repeat over time and
you keep changing people and you keep getting good results
DR. VAUGHAN: The system is working. Right. It's the
fact that there is a bad outcome that we're looking at here.
ADM. GEHMAN: Thank you. Sorry for the interruption.
DR. VAUGHAN: I wanted to begin and go back over just
really briefly what happened in Challenger. First, the presidential
commission reported that there was a controversial eve-of-the-launch
teleconference during which worried engineers at Morton
Thiokol, the solid rocket booster contractor in Utah, had
then objected to the launch, given that there was going
to be an unprecedented cold temperature at launch time the
Marshall management, however, went ahead and launched, overriding
the protests of these engineers. Not only did the commission
discover that but also the fact that they discovered that
NASA had been flying with known flaws on the solid rocket
boosters O-rings since early in the shuttle program, that
these flaws were known, and known to everybody within the
May I have the next slide, please. What happened was what
I called an incremental descent into poor judgment. This
was a design in which there were predicted to be no problems
with the O-rings, no damage. An anomaly occurred early in
flights of the shuttle and they accepted that anomaly and
then they continued to have anomalies and accepted more
and more. This was not just blind acceptance, but they analyzed
them thoroughly and on the basis of their engineering analysis
and their tests, they concluded that it was not a threat
to flight safety. It's important to understand, then, that
this history was a background in which they made decisions
on the eve of the teleconference; and that was one more
step in which they again gradually had expanded the bounds
of acceptable risk.
Next slide, please. One of the things that's critical with
Challenger, and also now, is the fact that we tend to look
at bad outcomes and work backwards and we're able to then
put in line all of the bad decisions and apparently foolish
moves that led up to it. It becomes very important to look
at the problems as they were unfolding and how people saw
them at the time and try to reconstruct their definition
of the situation based on the information they had when
they made their choices.
Next slide, please. The Challenger launch decision was,
in fact, a failure of the organizational system; and I hope,
by going through the explanation, it will show why it was
not groupthink, it was not incompetent engineers, unethical
or incompetent managers.
Next slide, please. So what happened? Richard Feynman called
it Russian roulette, which implies that there is a knowing
risk-taking going on. The result of my research, I called
it something else, the normalization of deviance; and I
want to use the organizational system perspective to explain
how this happened.
The idea of an organizational system is that there are different
levels at which you have to do your investigation. So the
first is the people doing the work, their interactions,
and what they see; the second level is the organization
itself; and the third level has to do with the environment
outside the organization and the other players that affect
what's going on internally.
So let's start with the bottom layer, the people doing the
interaction. First, it's important to know that they were
making decisions against a backdrop where problems were
expected. Because the shuttle was designed to be reusable,
they knew it was going to come back from outer space with
damage; and so there was damage on every mission. So simply
an environment like that, to have a problem is itself normal.
So what to us in hindsight seemed to be clear signals of
danger that should have been heeded -- that is, the number
of flaws and O-ring erosion that had happened prior to Challenger
-- looked different to them. The next slide will show how
they looked as the problem unfolded.
What we saw as signals of danger, they saw as mixed signals.
They would have a problem flight. It would be followed with
a flight for which there was no problem. They would have
weak signals. Something that in retrospect seemed to us
to be a flight-stopper, to them was interpreted differently
at the time. For example, cold, which was a problem with
the Challenger flight, was not a clear problem and not a
clear caught on an earlier launch. Finally, what we saw
as signals of danger came to be routine. In the year before
Challenger, they were having O-ring erosion on 7 out of
9 flights. At this time it became a routine signal, not
a warning sign.
The next slide, please. That's what's going on on the ground
floor. So the question is then how does the organizational
system in which they're working affect what they're doing
and how they're interpreting this information and how their
decisions move forward. This is what I call the trickle-down
effect. Congress and the White House were major players
in making decisions, and their policy decisions affected
how people were making decisions in the project.
The budget, the problem of Challenger starting out with
insufficient resources, meant that the only way the program
got going was by Challenger, by the shuttle program being
responsible in part for its own livelihood. That is, it
would carry payloads. The number of payloads it would get
paid for annually were expected to contribute to its budget.
So early on, the space shuttle project was converted from
what during the Apollo era had been an R&D organization
into a business. Contracting out and regulation both had
altered the shuttle program so that it was much more bureaucratic.
There were a lot of people who had been in pure engineering
positions were reversed in the sense that they became more
administrative. They were put in oversight positions, and
they had a lot of desk work to do.
Finally, when the program was announced, it was announced
that it would be routine to fly shuttles into space. It
would operate like a bus. So the expectation that it would
be routine also had an effect in the workplace. The effect
was to transform really a culture that had been pure R&D,
with emphasis only on the technological discovery, into
one that had to operate more like a business in that cost
was a problem, production pressures were a problem.
The notion of bureaucratic accountability made the agency
what some people told me was bureau-pathological. That is,
there were so many rules, there were so many forms to be
filled out that these kinds of tasks deflected attention
from the main job of cutting-edge engineering. It wasn't
that the original technical culture died but that, in fact,
it was harder to follow it through with these other influences
on the shuttle program.
How did these actually play out on the ground? Next slide.
The original technical culture called for rigorous scientific
and quantitative engineering, real solid data in the form
of numbers to back up all engineering arguments; and that
was still true. However, also with the original technical
culture, there was a lot of deference to engineering and
engineering expertise based on the opinions, valued opinions,
of the people who were doing the hands-on work.
The latter was harder to achieve within a bureaucratic organization
where hierarchy dominated. The schedule became a problem
interfering with the decisions by compelling turn-arounds
in time to meet the schedule, so that expected research
on hardware problems sometimes continued past the next launch.
So they were still getting more information while a new
launch was in process.
It also affected them in that the engineers and managers
truly followed all the rules. In the midst of a system that
many people at the time said was about to come down under
its own weight before Challenger, what was happening was
the fact that they followed all the rules in terms of having
the numbers, in terms of procedures, gave them a kind of
belief that it was safe to fly. Engineering concerns had
to be backed up with hard data or there couldn't be money
set aside to do a correction to the program. Hunch and intuition
and concern were not enough.
Next slide, please. The third part is to say, well, there
was a long incubation period here. Why didn't someone notice
the trend that was going on with the solid rocket booster
project in terms of O-ring flaws and intervene? This is
where the organization's structure was at that time a problem.
The safety system had been weakened. One safety unit had
been completely dissolved, and staffing had been cut back.
Top administrators, because of extra work in an expanding
programming, were no longer able to maintain what in the
Apollo program was known as the dirty-hands approach --
that is, keeping in touch with the technology, the problems,
and the riskiness of it.
And the anomaly tracking system, which was another way that
you could get warning signs, made it very difficult for
administrators to isolate serious problems. At one time
under their Criticality 1 category, which is the most serious
label that you give to a technical problem, they had 978
items on it. So how, of those, do you sort out which are
the most serious?
Next slide, please. With this as an outline, I'd like to
move to some comparisons, the echoes that Sally Ride talked
about. First, here I'm drawing analogies. I spent nine years
on the Challenger book and I haven't done this on this case
and your investigation is still underway. So where I'm able
easily to identify the similarities, it's harder to define
the differences; and what we see now as similarities are
yet to be proved. So my goal here is just to maybe point
you in some ways to look, and not come to any conclusions.
First, in both circumstances, Columbia and Challenger, a
crisis -- well, let's say it was a crisis of uncertainty.
Circumstances happened for which they had no background
experience. They came to this condition of high uncertainty
with a belief in acceptable risk -- that is, based on all
the flight readiness reviewed decisions that had proceeded,
they believed they were flying with a vehicle that did not
have a problem that was related to, in Challenger, O-rings
and, in Columbia, the foam problems. They believed in their
own analysis. That was this background, and they had engineering
reasons for believing that.
Second, in each of those cases, Challenger and Columbia,
there had been an event in the recent past that had some
import for their decision-making that night. For Challenger,
the year before the launch, STS 51B was launched in January.
The condition that the engineers on the eve of the Challenger
launch were concerned about was the cold temperature, which
for the next day was predicted to be at an all-time launch-time
low. The STS 51B, which was launched in January of 1985,
was launched where also cold temperature mattered but not
on the launch pad. The cold temperature had been the three
previous nights when the vehicle was sitting on the launch
pad and the temperature was down 19 to 22 degrees at that
The foam strike in Atlantis. There had been several foam
strikes preceding the Columbia launch. The Atlantis foam
strike, which happened in October of 2002, was the most
recent. The history in the foam strikes was that they had
problems with imagery, that they couldn't see so much the
location of the strikes and so on. So that was part of the
history which led to the fact that that night they didn't
have or that -- when they discovered the foam strike, that
they didn't have good data.
For the cold temperature on 51B, there was a similar effect.
At the time when they did the analysis, the engineer who
went to the Cape and looked at the vehicle when it was disassembled
and looked at the solid rocket boosters was alarmed because
he saw that in the base of the putty in the groove in which
the O-rings lay, the grease was charred black like charcoal;
and he believed that this was significant. But when they
came forth after that with their analysis of 51B for the
next Flight Readiness Review, their analysis showed them
that it was still safe to fly. They had had damage of the
O-ring, they had serious O-ring erosion, and they had had
for the first time hot gases that had gone beyond the primary
O-ring to its backup, the secondary O-ring, and their analysis
told them that in a worst-case scenario, it would still
work. It would still work.
Where does cold come into this? The engineer who saw the
charcoaled grease had this feeling that, intuitively, this
was bad. So when he argued that cold should be a serious
concern, they had at that point had many things happening
with O-rings. The smallest thing could cause damage. So,
for example, a piece of lint in the bed of putty in which
an O-ring lay could cause erosion. Each time something different
had happened. They believed that there was no generic problem
because they were not having damage on every ring on every
mission. Sometimes they would not have any, so that he could
not prove that cold was a correlation with the O-ring damage.
They decided at that point that they should get some cold
temperature data; but they didn't scramble to get it, as
this engineer said. The reason they didn't was they believed
it was a unique incident, that the chance of overnight temperatures
of that low for three nights running in Florida was, in
his words, the equivalent of having a 100-year storm two
years in a row. So there was no scramble to get temperature
data. They did some resiliency tests, but they did not have
systematic temperature data. So in both circumstances, when
the condition of high uncertainty came up for both Columbia
and Challenger, they did not have a lot of supporting data,
they didn't have the best data available to them and this,
it turned out, mattered.
The third point is that the organization's structure interfered
with good communication, and it interfered in several ways
in which there seem to be parallels across cases. There
were, in this case, missing signals. People who had information,
if that information had been relayed up the hierarchy, might
have made a difference. People in the Challenger evening
teleconference were in three different locations, and they
were in telephone communication but not video. People were
in different locations who did not speak up, so their message
didn't get across on the main teleconference line.
Why didn't they speak up? Some people felt that that was
their specialization, they hadn't worked on it recently,
and therefore though they had some input and they had some
information, they didn't know what the most recent data
was. Some people didn't speak up because it simply wasn't
their specialization. Other people didn't speak up because
they trusted in the hierarchy, they trusted in the key people
who were running the teleconference to guide it in the right
direction, they trusted the engineers at Thiokol to do the
analysis. Those were some of the reasons.
One of the parallels with Columbia comes up in the accounts
of the E-mails that were circulated from approximately the
21st on, worries of concerned engineers. From newspaper
accounts that I've been able to conclude and the E-mails
themselves, that in a sense they were marginal to the process,
they had not been brought in early on, this was a conversation
they were having among themselves. They were also specialized
and felt that perhaps they didn't have the same information
that other people had. There was a trust in the hierarchy;
and, as one of them said after a press conference early
in your investigation, "We didn't have the data." That is,
they were concerned they didn't have any hard numbers.
One of the characteristics of the conversion from the Apollo-era
culture to the Challenger-era culture was that intuition
and hunch didn't carry any weight. They carried weight in
everyday, daily decision-making and batting around ideas,
but when it came to formal decisions like the Flight Readiness
Review, it was hard data, it was numbers that were required.
And in this case it was significant to me that he said we
didn't have the data and therefore, not having the data,
they didn't feel empowered to speak up in these E-mails
and carry them upward farther.
There is evidence of production pressure in the Challenger
case that I haven't seen yet in Columbia. In Challenger,
there was a deadline for the engineers to make their preparation
for their eve-of-the-launch teleconference engineering recommendation
about the relationship between the cold temperature and
O-ring erosion and what they expected, what they were recommending
in terms of launch. They scrambled to put their analysis
together, dividing up the work, and began faxing their charts
over the telecon line without having the time to look through
them, and if they had taken that time, they might have noticed
ahead of time -- if they had collectively looked through
them, they might have noticed ahead of time that they didn't
have a strong correlational argument. So as a consequence,
it was a weak argument in terms of the engineering culture
at NASA. The hard numbers didn't hold together. They couldn't
prove that there was a cold temperature correlation with
At one point the key engineer said, "You know, I can't prove
it. I just know it's away from goodness in our data base."
But in that culture, that was considered an emotional argument,
a subjective argument, it was not considered a strong quantitative
data argument in keeping with the technical tradition at
So far there isn't any evidence of engineering concerns
during the history of the foam problem like there was with
Challenger either. Afterwards, there had surfaced some memos
in Challenger, the previous year in particular, as engineers
at Thiokol was trying to get through the bureaucratic rigmarole
in order to get the help they needed to try to analyze the
problem; and they were working on a fix at the time.
The other point I wanted to make was about bureaucratic
accountability. What was obvious with Challenger was that
on the eve of the launch that the concerns of the engineers
were not prioritized. It also seems to be the case in the
requests for the imagery from Columbia that concerned engineers
discovering the foam strike at this point described it as
it was large. There was nothing in their experience like
this. It was the size of a Coke cooler. This was unique.
They met, a team of approximately 37 engineers, and made
a request for better visuals than the ones that they had
from ground camera; but somebody up the hierarchy canceled
the request. In a condition of high uncertainty. One of
the comments that I read in the newspaper -- and I don't
claim to have all information on this -- was that the request
had not gone through proper channels, which points to me
the significance of rules and hierarchy over deference to
technical expertise in this particular case.
There are many conclusions we can think about from this,
but one of them is that in both of these situations, following
the normal rules and procedures seemed to take precedence;
and we know that, in fact, in conditions of uncertainty,
people do follow habits and routines. However, under these
circumstances where you have something without precedent,
it would seem that this would be a time not for hierarchical
decision-making but for a more collective, collaborative,
what does everybody think, let's open the floodgates and
not pull on the usual people but especially what are the
concerns of our engineers and also to let up on the idea
that you have to have hard data. Engineering hunches and
intuitions are not what you want to launch a mission with;
but when you have a problem that occurs that's a crisis
and you don't have adequate information, this is a reverse
of the pro-launch situation, in which engineering hunches
and intuitions ought to be enough to cause concerns, without
asking for hard data.
So what's to be done if it turns out in this investigation
that you do, in fact, find a failure of the organizational
system? Could I have the next slide, please.
Typically in the results of an accident investigation two
things happen. One is that the technical culprit is found
and a technical fix is recommended and achieved; second,
that key decision-makers are identified who had important
roles and where they might have prevented a bad outcome
but didn't. More typically, the organizational system goes
untouched. It is, in fact, more difficult to identify the
flaws in the organizational system. It's harder to pin it
down and it's more challenging to try to correct it, but,
in fact, there are many people who are experts in how to
build high-reliability systems and what are the problems
with systems from an organizational system that might help
in advice in circumstances like this.
Next slide, please. Just looking at the model that I put
up earlier where we looked at the trickle-down effect, it
leaves three levels at which you might target changes. First,
the beauty of operator error is that it deflects attention
from key policy decisions made in the past that have affected
a program and affected the daily operations. Policy leaders
need to be concerned and aware of their responsibility with
risky systems and be aware of how their choices affect the
hands-on work. They also are responsible and implicated.
Cultures, for example, are hard to change; but leaders might
try to change them, even if they weren't the ones who created
them. It's important that they remain in touch with the
hazards of the workplace. Whereas in the modern NASA it
may be more difficult for administrators to stay in touch
with the hazards of the workplace and the dirty-hands approach
cannot be carried out like it was in the time of Apollo,
still it's important to stay in touch with those.
For example, prior to Challenger, the shuttle was declared
as an operational system. As a result of that and the belief
and the expectation it would be routine, citizens were allowed
to be taken on for ride. The people at the top of the organization
apparently believed that it was not a risky technology and
therefore it was safe to take along ordinary citizens. The
engineers who were doing the daily work did not believe
that it was -- I mean, they were aware of all the problems
in the system on a day-to-day basis. They were the ones
who had the dirty hands. They were not the ones who made
the decision to put a teacher on the space shuttle.
Another aspect of concern for top leaders is changes are
often made in an organization's structure for budgetary
reasons, for better coordination, without thinking about
how that might affect the people who are having to make
decisions at the bottom. What does it mean, for example,
when you have an International Space Station and NASA is
now dividing up the work so that there are two combined
structures and projects in which decisions have to be made?
How are these priorities getting sorted out? Does that affect
what's going on in the program?
Contracting-out had a serious effect on the work of people
making technical risk analyses. We know hospitals, when
they have mergers, often let people go and it loses the
institutional memory and there are startup costs in people
getting going again. These kinds of changes should not be
made without looking at their implications.
Second. Please, next slide. Target culture. You can't really
make assumptions about your culture. We think we understand
our cultures, but they act invisibly on us, and so we cannot
really identify what their effects are. In one of the comments
post Columbia concerning the E-mails, "We have a safety
culture and we strongly encourage everyone to speak up at
every opportunity." And I'm sure that they believe that.
But when you look at the chronology of events, even in skeletal
form in which I'm aware of them, the fact that these what-ifs
didn't percolate up the hierarchy, the fact that the engineering
requests did not get fulfilled indicates that there are
some things that suppress or that are acting to suppress
It's also significant, I think, in terms of culture to understand
the power of rules. The things that we put in organizations
that do good also can have a dark side. It is really important
at NASA, because of the complexity of the agency and its
projects, to have rules. You couldn't run it without rules.
It's impossible. But then there are times when maybe the
normal rules don't apply. So how do you train people to
recognize circumstances when you have to expedite matters
without going through the hierarchy, and how do you empower
engineers to get their requests filled?
Finally, targeting signals. Missing signals are obvious
in both cases. What does it mean to try to reduce missing
signals? One is to truly create a system in which engineers
have more visibility, their concerns have more visibility
on a formal and informal basis. Second, the safety system.
The parallel with Challenger and the reduction of safety
personnel is also a parallel with Columbia. When you reduce
a safety system, you reduce the possibility that other people
are going to be able to identify something that insiders
have seen and normalized the technical deviation. And the
slippery slope. When you're working in a situation where
problems are expected, you have problems every day, and
people are busy with daily engineering decisions, it becomes
very difficult to identify and stay in touch with the big
How do you identify the trend so that people are aware when
they are gradually increasing the bounds of acceptable risk?
It is certainly true, based on what we know about organizations
and accidents in the social sciences that this is a risky
system and what we know is the greater the complexity of
the organization, the greater the possibility of a failure.
The same is true of organizations. Organizations are also
complex systems. The greater the complexity of the organizational
system, also the greater the possibility of a failure. When
you have a complex organization working a complex technology,
you're never going to be able to completely prevent accidents,
but the idea is to be more fully aware of the connection
between the two so that you can reduce the probability that
a failure would occur.
That's it. Your turn.
ADM. GEHMAN: All right. Well, that's a bucket full.
Since you studied the Challenger decision so carefully,
and even though we're talking about Columbia here, let me
ask a Challenger question, even though it's loaded because
it has Columbia implications. Several things you said struck
me, and they're related to each other. One is that you can't
change the behavior unless you change the organization.
You can change the people, but you're going to get the same
outcome if the organization doesn't change. Yet in another
place up there, you said beware of changing organizations,
because of the law of unintended consequences. You've got
to be real careful when you change organizations.
What do you make of the post-Challenger organizational changes
that took place, particularly in the area of more centralization
and program management oversight? What do you make of all
DR. VAUGHAN: The changes that I am most familiar
with are the ones related to launch decisions. That is that
immediately following, they put an astronaut, former astronaut
in charge of the final "go" outcome of the Flight Readiness
Review procedure and they tried to integrate engineers,
working engineers, into the flight readiness process more.
I'd say that there is always a problem in organizations
in providing the stability and the centralization needed
to make decisions and make sure information gets to the
top and providing the flexibility to respond to immediate
demands; and without, you know, really studying this, I
would say that what we know about Columbia is that flexibility,
at least in a couple of circumstances, really wasn't there.
That becomes interesting in thinking about the differences
in the pre-launch decision-making structure and post-launch
decision-making structure. That is, the post-launch decision-making
structure is actually designed to create that kind of flexibility
so that you could pull in people as you need it and so on.
What's ironic about it is it looks as if had there been
either a direct route for engineering concerns to get implemented
to shortcut what really little bureaucracy there seemed
to be in that process that that would have helped, that
if, you know, that could have circumvented the kind of need
for hierarchical requests for imaging. In terms of the overall
impact on NASA, I really can't say that.
ADM. GEHMAN: From my understanding, though, one of
the post-Challenger results has been a much more formal
FRR process. As you are probably aware, no more telephone
calls, it's all face-to-face, it's done at the Cape, and
you've got to be there and they're done in big rooms like
this with hundreds of people in the room with several different
layers, everybody there, and there's a whole lot of signing
that goes on. People at several layers actually sign pieces
of paper that say, of the thousands of things that I'm responsible
for, they've all been done with the exception of A, B, C,
and D, and then they have to be waived or something like
that. Then they go through a many, many hour process of
making sure that everything's been taken care of and every
waiver has been carefully analyzed and in front of lots
of high-level people. So it's very meticulous, it's very
formal, and it's an eyeball-to-eyeball commitment that my
organization has done everything my organization is supposed
to have done.
Is that the kind of an organization in which weak and mixed
signals can emerge? I mean, is that the kind of organization
which would recognize mixed and weak signals and routine
signals? Is that compatible kind of with your -- I'm still
talking Challenger -- with some of the principles you outlined
DR. VAUGHAN: This was fairly much the procedure that
existed at the time of Challenger, where every layer of
Flight Readiness Review had to sign off on it. The criticism
at the time, post Challenger, was that what was happening
was the engineers who were making the analyses and coming
forward at the Level 4, the ground level of Flight Readiness
Review, those were the people who were getting the mixed,
weak, and routine signals; but when they came together,
they had to come up with a consensus position for their
project manager to carry forward. And once they agreed,
then they began gathering the supportive data that this
was an acceptable flight risk. And as their recommendation
worked itself up through the hierarchy, the system was designed
to criticize it, to bring in people with other specializations
who could pick it apart, and the result of that was to make
them go back to the desk and sometimes to do more engineering
analysis. That engineering analysis tended always to support
the initial recommendation. So by the time it came out the
top of the process, it was something that might have been
more amorphous on a day-to-day basis was dogma and very
convincing, which is why, with a backdrop of having that
kind of information, you have people who believe in acceptable
risk, it's based on solid engineering and history, who need
to be convinced by hard data that something different is
happening this time.
The system is designed to review decisions that have been
made, that if there is a mistake in the fundamental engineering
analysis, they can criticize it, but they can't uncover
it at the other layers, which would mean that you would
need another kind of system to detect that, such as outsiders
who bring fresh eyes to a project on a regular basis. The
Aerospace Safety and Advisory Panel was very effective during
the years of Challenger, with the exception of the fact
that their charter kept them coming for visits perhaps 30
times a year. So it was impossible for them to track all
the problems; and at that point when Challenger happened,
they were not aware of the O-ring erosion and the pattern
that was going on.
ADM. GEHMAN: I'm still trying to understand the principles
here. It seems to me that in a very, very large, complex
organization like NASA is, with a very, very risky mission,
some decisions have to be taken at middle-management levels.
I mean, not every decision and not every problem can be
raised up to the top and there must be a process by which
the Level 2, Level 3, and Level 4, that the decisions are
taken, minority views are listened to, competent engineers
weigh these things, and then they take a deep breath and
say, okay, we've heard you, now we're going to move on.
Then they report up that they've done their due diligence,
you might say.
I'm struggling to find a model, an organizational model
in my head, when you've got literally thousands and thousands
of these decisions to make, that you can keep bumping them
up higher in the organization with the expectation that
people up higher in the organization are better positioned
to make engineering decisions than the engineers. I mean,
you said yourself, "Hindsight is perfect." We've got to
be really careful about hindsight, and I'm trying to figure
out what principles to apply.
We as a board are certainly skittish about making organizational
changes to a very complex organization for fear of invoking
the law of unintended consequences. So I need to understand
the principles and I'm trying to figure out a way that I
can apply your very useful analysis here and apply it to
find a way to figure out what the principles are we ought
to apply to this case. So the part that I'm hung up on right
now is how else can you resolve literally thousands of engineering
issues except in a hierarchical manner in which some manager,
he has 125 of these and he's sorted through them and he
reports to his boss that his 125 are under control. I don't
know how to do that.
DR. VAUGHAN: Well, two things. First, somehow or
other in the shuttle program, there is a process by which,
when a design doesn't predict an anomaly, it can be accepted.
That seems to me to be a critical point, that if this is
not supposed to be happening, why are we getting hundreds
of debris hits, if it wasn't supposed to happen at all.
It's certainly true that in a program where technical problems
are normal, you have to set priorities; but if there is
no design flaw predicted, then having a problem should itself
be a warning sign, not something that is taken for granted.
The idea is to spot little mistakes so that they don't turn
into big catastrophes, which means spotting them early on.
Two things. And one I'm certain that NASA -- maybe both
of them -- that NASA may be very aware of is the fact that
engineers' concerns need to be dealt with. I can understand
the requirement for hard data. But what about the more intuitive
kinds of arguments? If people feel disempowered because
they've got a hunch or an intuition and let somebody else
handle it because they feel like they're going to be chastised
for arguing on the basis of what at NASA is considered subjective
information, then they're not going to speak up. So there
need to be channels that assure that, even giving engineers
special powers if that's what's necessary.
The other is the idea of giving more clout to the safety
people to surface problems. So, for example, what if the
safety people, instead of just having oversight, were producing
data on their own, tracking problems to the project for
which they're assigned and, in fact, doing a trend analysis
to keep people's eye on the big picture so that the slippery
slope is avoided?
ADM. GEHMAN: Thank you for that.
DR. VAUGHAN: Let me add also that there are other
models of organizations that deal with risky systems, and
social scientists have been studying these. They have been,
you know, analyzing aircraft carrier flight decks and nuclear
operations and coal-mining disasters. There are all kinds
of case studies out there and people who are working in
policy to try to see what works and what doesn't work. Are
there lessons from air traffic control that can be applied
to the space shuttle program? What carries over? Is there
any evidence that NASA has been looking at other models
to see what might work with their own system?
I know that in air traffic control they use an organizational
learning model. What we find out from this comparison between
Columbia and Challenger is that NASA as an organization
did not learn from its previous mistakes and it did not
properly address all of the factors that the presidential
commission identified. So they need to reach out and get
more information and look at other models, as well.
Thinking about how you might restructure the post-launch
decision-making process so that what appears to have happened
in Columbia didn't happen again, how can that be made efficient,
may be something -- maybe it needs to look more like the
pre-launch decision process. But is there any evidence that
NASA has really played with alternative models? And my point
about organization structure is as organizations grow and
change, you have to change the structures, but don't do
it without thinking about what the consequences might be
on the ground.
DR. LOGSDON: Just a short follow-up to that. Diane,
your book came out in 1996, I think, right, and was fairly
widely reviewed. We at the board discovered in some of our
briefings from outside folks that the submarine safety program
uses your work as part of the training program for people
that worry about keeping submarines safe. Have you had any
interactions with NASA since the book came out?
DR. VAUGHAN: No. 8
DR. LOGSDON: Have you ever been invited to talk to
a NASA training program or engage in any of the things that
you just discussed might be brought to bear?
DR. VAUGHAN: No, though, in fact, as you said, the
book did get quite a lot of publicity. I heard from many
organizations that were concerned with reducing risk and
reducing error and mistake. The U.S. Forest Service called,
and I spoke to hotshots and smoke-jumpers. I went to a conference
the physicians held, looking at errors in hospitals. I was
called by people working in nuclear regulatory operations.
Regular businesses, where it wasn't risky in the sense that
human lives were at cost. Everybody called. My high school
boyfriend called. But NASA never called.
ADM. GEHMAN: Anybody want to comment on that?
GEN. BARRY: What was his name?
ADM. GEHMAN: Let me finish my thought here. Professor
Vaughan, again we're back to this organizational issue which
I'm trying to determine the principles that I can apply
from your analytical work here. If the processes we're talking
about in the case of NASA, if they didn't follow their own
rules, would that alarm you? What I mean is if there were
waivers or in-flight anomalies or systems that didn't work
the way they were supposed to work and, in the fact that
they didn't work the way they were supposed to work, somehow
started migrating its way down lower in the message category
to where it wasn't sending messages anymore and therefore
it was technically violating their own rules because they're
supposed to deal with these things, would that be a significant
alarm for you?
DR. VAUGHAN: Well, I think that one of the things
to think about here is that NASA is a system that operates
by rules; and maybe one of the ways to fix the problem is
to create rules to solve the problems. So what are the rules
when engineers need images, for example? Can't they find
a way where they have their own authority, without seeking
other authority, to get the necessary images? So I think
I read that someplace, where the harmony between the way
the organization operates and thinks in the key aspects
of the culture itself are something that you might want
to build on.
DR. WIDNALL: Actually I'm starting to frame in my
own mind that the problem is that there is, in fact, one
underlying rule and it's a powerful rule and it's not stated
and it's not stated as simply as this question of following
your own procedural rules. But let me sort of get into that.
I've certainly found your framework very helpful because
I've mused over this issue of how an organization that states
that safety is its No. 1 mission can apparently transition
from a situation where it's necessary to prove that it's
safe to fly, to one in which apparently you have to prove
that it's not safe to fly. I think what's happening is,
in fact, that engineers are following the rules but this
underlying rule is that you have to have the numbers.
DR. VAUGHAN: Right.
DR. WIDNALL: That's not the rule you stated, which
was that you should follow the procedures and resolve all
DR. VAUGHAN: This is a norm.
DR. WIDNALL: Those are these kind of rules. I'm talking
about the really basic rule that says you have to have the
numbers. So that basically means that every flight becomes
data and that concern about an anomaly is not data. So a
flight with an anomaly becomes data that says it's safe
to fly. So the accumulation of that data, of those successful
flights, puts the thumb on the scale that says it's safe
to fly; and people who have concerns about situations in
one of these uncertain situations that you talk about, they
don't have the data.
So I think it may be getting at, in some sense, changing
the rule to one that it is not okay to continue to operate
with anomalies, that the underlying rule of just having
data is not sufficient to run an organization that deals
with risky technologies. Because otherwise you're just going
to end up with a pile of data that says it's okay to fly,
and you're not likely to get much data on the other side.
ADM. GEHMAN: Is that a question?
DR. WIDNALL: That's kind of a comment.
DR. VAUGHAN: I completely agree with you. One of
the reasons I emphasized in an earlier slide that you need
to understand your culture is that it works in ways that
we don't really realize. So how many people there understand
the effect of intuition and hunch, which are absolutely
integral to good engineering, and how the kind of impression
on numbers suppress that kind of information in critical
People are disempowered from speaking up, by the very norms
of the organization. Things like language, though. For example,
the term I've read in the paper, "That's in family." That's
a real friendly way of talking about something that's not
really supposed to be happening in the first place. In nuclear
submarines, they don't talk about it as "in family"; they
talk about it as a degradation of specification requirements,
which has a negative feeling to it. These kinds of languages
which we think of as habits of mind reflect attitudes that
are invisible, but the language really shows.
So the question is, you know, how can you get back in touch
with the importance of engineering intuition and hunch in
formal decision-making. Usually it works in the informal
decision. You know, I think that's why the NASA administrators
believe that they've got a safety culture and that people
are free to express whatever they think; but when it comes
to a formal decision, they fall back into the formal rules
and that expression of concern doesn't get expressed.
Even if you take something as simple as an engineering presentation,
the fact that it's reduced to charts, which are systematic,
gets all the emotion out of it. It begins to look even more
routine. The engineer in Challenger who saw the burned grease,
the black grease, was seriously alarmed. I asked him, you
know, later, "Did they see this? What did they see? Did
they get a photograph?" He said yes. I said, "How did it
look in the photograph?" He said it did not look serious
in the photograph. So emotion is keyed to some kind of a
logic based in engineering experience, and it should be
valued and a way found to express it.
GEN. BARRY: Diane, I'm going to ask you a short question,
and then I'm going to ask a longer question, if I may. First,
the short question, focusing on organizational failure.
The Rogers Commission, did they fall short on institutional
recommendations in the aftermath of Challenger, or were
they good ones and they just weren't followed through by
DR. VAUGHAN: The Rogers Commission was very good
at identifying what they called contributing causes and
that I would call system causes. That is, they identified
safety cuts, cuts in safety personnel. They identified the
failure of NASA to respond to recommendations of the Aerospace
Safety and Advisory Panel. They identified the history of
the program and the fact that it was a design that was built
on budget compromises in the beginning. They identified
production pressures. They identified all those kinds of
outside sources that had impacted the decision-making and
that were a part of NASA's history.
In the recommendations, they didn't come forward with anything
that said give them more money, change the culture. They
weren't sociologists. They weren't social scientists and
not trained to think about how that might have actually
worked. The way it looked like it worked was in the sense
that there were pressures there and key managers, namely
Lawrence Malloy, who was the project manager for the solid
rocket booster project at that time, was the operator who
made the error. Once that happened and the key person was
identified and people changed and new people came in, then
the system problems remained.
They fixed the technology. They fixed the decision-making
structure in ways I described earlier. But the organization
didn't respond and neither did -- in keeping with my point
earlier about top leaders being responsible -- the organization
did not respond in terms of getting more money beyond what
it took at that point to fix the technical problem. They
got an initial boost, but they've been under budgetary constraints
all along. The recommendations in the volume of the presidential
commission were related strictly to internal NASA operations.
They were not directed towards policy-making decisions that
might have affected the program.
GEN. BARRY: Okay. Let me build on that a little bit
and just carry it on and see if this resonates with you.
I'm going to list off a bunch of items here and see if this
falls true with what you know to be from Challenger that
might be able to be translated over to Columbia.
First of all, you stated that with Level 4 identifying problems
and being able to try to communicate that up the institution,
the organization kind of stymied that. So I would characterize
that as needing to prove that there is a problem in the
early stages of the FRR or before flight. I think post Challenger,
you know, there has been a fix on that and, remember, the
Flight Readiness Review is supposed to prove not only launch
but also en route and on recovery. So it's the whole flight.
It seems like they've solved the problem on trying to say
is there a problem in proving it. To post launch. There's,
some would argue, an attitude that you have to prove there
is a problem. So we kind of fix it on the launch side; but
after it's launched, we kind of relegate back to maybe the
way it was prior to Challenger: Prove to me there is problem.
Now, if we try to look pre and post launch, pre-launch is
very formal, as Admiral Gehman outlined earlier. You've
even alluded to it in the book. Post-launch, it could be
argued, less formal, more decentralization, more delegation
certainly, okay, from what we see at the FRR prior to launch.
Multi-centers are involved prior to launch. I mean, they
all meet and they all sit at the same place, they're all
eyeball to eyeball. Center director is represented, program
managers. Post-launch, again decentralized, it's mostly
a JSC operation. Of course, KSC gets involved within the
landing at Kennedy.
There's a tyranny of analysis pre-launch maybe and that
is because you've got -- well, you have a long-term focus
because you've had time. But post-launch, there's a tyranny
of analysis but it's in real time because you don't have
as many hours and you've got to make decisions quicker and
all that other stuff.
The real question -- if this resonates with you at all --
could it be argued that during Columbia, NASA had a "Prove
there is no problem" prior to launch and post-launch it
was "Prove to me there is a problem" and we have this formal
and informal kind of focus. It seems to me after Challenger
we fixed the prior to launch, certainly with having people
appear in person and no VTCs or no over-the-phone. Everybody
had to be there in person. And we have maybe a problem that
we need to fix post-launch with the MMT and the decentralization
elements and maybe the delegation.
I certainly don't want to relegate it to a headquarters
level, but there are some things that need maybe to be fixed
there. So I would ask really your opinion that is there
some kind of a delineation in your mind, from what you know
to date, pre and post launch, that we might be able to provide
solid recommendations on to improve NASA?
DR. VAUGHAN: I'm wondering if the post-launch flexibility
is such that you can, in fact, have similar things going
on in two different parts of the process in which people
are not in touch. So I understand that video requests really
originated from two different points, working engineers
in two different locations, and that they didn't really
know that the other had originated a request.
It certainly seems that the mentality of proving your point
when you've got a time line like you do and it's an unprecedented
circumstance, as it was with Columbia, is wrong, of course,
in retrospect. The question you're asking is how can we
convert that into a process that prevents this from happening
No, a famous sociologist once told me when I was beginning
the analysis of the Challenger launch, "It's all these numbers.
It's all these numbers, and there are these debates about
issues. Why don't you do it like they do it in the Air Force?
You just should have a red button for stop and a green button
for go." And there's a lot to be said for simplifying a
complex system, whether it's decentralized or centralized,
so that key people can respond quickly and shortcut the
hierarchy. I don't know if that begins to answer your question.
But there maybe need to be some more rules created in the
sense that --
GEN. BARRY: And this is really stretching it but
DR. VAUGHAN: Maybe it needs to be more formal than
it is and maybe it needs to be more like the pre-launch
procedure in terms of the rigor of numbers of people from
different parts who are looking at problems that crop up
while a mission is in process instead of waiting just --
I mean, some sort of a formalized procedure where there's
a constant ongoing analysis instead of you've got worried
engineers in two different locations who are kind of independently
running around, trying to get recognized and get attention
to the problem.
MR. WALLACE: NASA's taken quite a pounding here today
but I'm wondering what we can --
DR. VAUGHAN: I thought this morning they were coming
off pretty good.
MR. WALLACE: I would just like to talk about what
we can sort of learn about what they do well -- in other
words, areas where we don't seem to have this normalization
of deviance or success-based optimism. Like BSTRA balls
and the flow liner cracks and some of those fairly recent
examples where there were serious problems detected with
the equipment, in some cases detected because of extreme
diligence by individual inspectors and really very aggressively
and thoroughly fixed.
It seems to me that part of the problem of normalization
of deviance is sometimes the level of visibility that an
issue gets. How do you sort of the bridge that gap between
those things that get enough visibility or sense of urgency
and those that somehow seem to slip below that threshold?
DR. VAUGHAN: Someone said after the book was first
published -- and then again now I've been getting a lot
of E-mails. Someone said at the time the book was published,
"I bet if you took any component part of the shuttle and
traced it back, you would find this same thing going on."
Perhaps doing a backward tracing on other parts of the shuttle
could show you two things. First, what are the circumstances
in which they're able to locate an anomaly early and fix
it so they stop it in its tracks and avoid an incremental
descent into poor judgment? Are there other circumstances
in which the same thing is happening? Can you find circumstances
where you do have the normalization of deviance going on?
It's interesting in the history of the solid rocket booster
project that there was a point at which they stood down
for maybe two months to fix a problem. How is that problem
identified? What are its characteristics? I would bet that
the more uncertain, the more complex the part and the more
amorphous the indications, the more likely it is to project
into a normalization-of-deviance problem, given the existing
culture where flying with flaws is okay in the first place.
MR. WALLACE: Well, sort of following on. Earlier
you said -- and good advice for this board -- that we should
try to see problems as they saw them at the time and not
engage in the hindsight fallacy or whatever that's called.
I mean, I'm not sure you said this; but my assumption is
that that's almost the only way you can learn to do better
prospectively. I mean, do you have any other thoughts on
that? In other words, to see the problem as they saw them
at the time, to me, is almost a step toward the discipline
of seeing the next one coming.
DR. VAUGHAN: Right. It's an experimental technology
still; and every time they launch a vehicle, they've made
changes. So they're never launching the same one, even though
it bears the same name. This is a situation in which, like
most engineering concerns where you're working with complex
technologies, you're learning by mistake. So that's why
post-flight analysis is so important. You learn by the things
that go wrong. Every once in a while you're going to have
a bad mistake.
ADM. GEHMAN: Did I understand the point that you
made both in your book and in your presentation here is
that the answer to perhaps Mr. Wallace's question lies in
the theory of strong signals? In other words, if NASA gets
a strong signal, they act on it. No problem. They very aggressively
shut the program down and go fix it. The problem is in the
weak, routine, or mixed signals. Those are the ones that
seem to bite us. Of course, there are a lot of them; and
they don't quite resonate with the organization. Is that
a good analogy?
DR. VAUGHAN: It is. The idea of a trend analysis
is that it could pick out stronger signals from lesser ones
before it becomes, you know, an enormous problem; but the
recognition of the pattern is important, bringing forth
the pattern so that the people who are making decisions
are constantly in touch with the history of decisions that
they've gone through before.
I have to say with that, though, it's important that they
have quantitative evidence to fly. Maybe the more qualitative
evidence could be brought in in other ways further up the
chain, that whereas in Flight Readiness Review, for example,
they present everything on charts and they ask -- the purpose
of Flight Readiness Review is to clean the hardware and
get it ready to go. The purpose of it is to clear up the
problems as it works its way through the Flight Readiness
Review process. What happens, as I mentioned, is that the
engineering argument tends to get tighter and tighter because
they're constantly doing the work to investigate and respond
to questions and, in a sense, defend what they've said or
find out if there are flaws.
At the time of Challenger, I read thousands of engineering
documents for all the Flight Readiness Reviews that they
had had and I didn't see anyplace in the Flight Readiness
Review process that would allow for the presentation of
simply intuitions, hunches, and concerns, where qualitative
evidence might be presented, like a clear image or even
a vague image of a piece of debris the size of a Coke cooler,
for example, rather than charts for an engineering analysis,
you know, that there ought to be room in the process for
ADM. GEHMAN: In your experience, particularly with
what I'm calling these weak signals or this muttering around
the room that the O-rings can't take freezing temperatures
but we're not really sure whether they can or cannot, I
have in my mind a model that says that it's unfair or not
reasonable to set as a standard for the organization to
act on literally hundreds of these misgivings that the tens
of thousands of people may have and that it's an unfair
standard to require the people who have these doubts to
prove that their doubt could cause the loss of the vehicle
or the crew. But I have in my mind that it's a more reasonable
standard that management should realize that the accumulation
of signals from the process are cutting into their safety
margins and that you can accumulate these things not in
a measurable way but in a subjective way, particularly in
a regime in which you have very thin safety margins to begin
with, that you should be able to reasonably determine that
you're narrowing your safety margins in a way that should
concern management. Is that a reasonable characterization
of the standard or the bar that we set here?
DR. VAUGHAN: I think that shows up in the problem
of lack of data in both of these circumstances, that there
were early warning signs and in neither case had those early
warning signs been pursued and say, "Well, the imagery is
bad. We know this is happening. We can't see exactly where
it's hitting. Why don't we get this now?"
I mean the power of the E-mail exchange was that they really
hadn't thought the possibility of failure through. There
was no plan for what needed to happen if there was, in fact,
a serious tile hit and damage to the wing, what would they
do at re-entry and what would it mean to attempt a wheels-up
landing at the landing site, and that failure to pursue
the trajectory of having a problem that's repeating. Like
if you think about cost maybe in terms of if that's a factor
in making issues a priority at NASA, which obviously it
is anyplace -- you can't fix everything -- think of the
cost if you simply don't have the data you need, which is,
I think, the most stunning thing about the comparison of
the two cases. At the time when conditions were highly uncertain,
in neither case did they have the data; and having that
background data is important.
ADM. GEHMAN: In your review of the Challenger decision,
did you personally come to the conclusion that the launch
decision would have come out differently if the Morton Thiokol
engineers' split decision -- because some of the Morton
Thiokol engineers said it was safe to launch, but they were
split on that -- and if the managers at Marshall had reported
that there was a split decision, that the FRR would have
come out differently? Did you have any evidence of that?
DR. VAUGHAN: The manager at Marshall did not know
that there was a disagreement at Thiokol. That was one of
the problems with them being in three locations. No one
ever thought to poll the delegation. So no one on the teleconference
knew really where anyone else stood. They knew what Thiokol's
final recommendation was and they assumed that Thiokol had
gone back and re-analyzed their data, seen the flaws in
it, and been convinced it was safe to fly. So the fact that
not every one was heard from was critically important.
By the same token, Thiokol engineers didn't understand that
they had support in the other places, that one of the NASA
managers who was at the Cape was really sitting there making
a list of people to call because he believed that the launch
was going to be stopped. So that was truly a problem.
Now I've lost sight of your question.
ADM. GEHMAN: The question was: In your research about
Marshall, did you come to the personal conclusion from talking
to people that the fact that the cold temperature analysis
at Morton Thiokol was a split decision, that that would
have made any difference at Marshall? I mean, did anybody
say, "If I had known that, I would have changed my mind"?
DR. VAUGHAN: Yes. However, the goal is for unanimity
and here's again where numbers count, that in the instance
where engineering opinion is divided, then they make what's
known as a management risk decision, that the managers take
over and the managers at Thiokol then, who knew that their
engineers were split, made a management decision. In retrospect,
that was the most horrendous example of failing to listen
to your technical people who said, "You know, I can't prove
it, but I know it's away from goodness in our data base."
ADM. GEHMAN: This principle that I'm following up
on here is important because we do have to be careful of
hindsight; and it may be that, even armed with what is admittedly
a minority opinion of a bad outcome, it could be that these
are judgment calls that are made in good faith with people
doing the best they can and they make a mistake. I mean,
they call it wrong. So the question is whether or not we
can indict the system, based on these incidents.
DR. VAUGHAN: I think you have to analyze -- you have
to do a social fault tree analysis and figure out what actually
happened and what went on, how is information relayed. I'm
sure that's work that's ongoing with you.
ADM. GEHMAN: That brings me to my next question --
and pardon me for monopolizing the time here. Another good
writer on this subject, who I think is Nancy Leveson, in
one of her models she suggested that we need to diagram
these decision-making systems because, just as you say,
it's not a person, it's a culture, it's an organization
that's really driving these things. Are you aware that anybody's
ever diagramed the FRR or the waiver, in-flight anomaly
disposition system? Has that ever been diagramed, to your
DR. VAUGHAN: Not that I know of. But what would be
more interesting would be to look at the more informal decision-making
processes because the rules are so strong for how the information
is addressed in Flight Readiness Review that that would
probably turn out the same every time. What you would want
to look at are the more informal processes and try to map
them and understand where the information stopped and why
MR. WALLACE: I'd like your thoughts on the concept
of whether an organization, this one, can sort of become
process-bound. You cannot fault the thoroughness of the
processes. But, I mean, is there a point at which they can
almost subvert other thinking processes, that people become
so confident in the thoroughness of the processes and the
fact that they're tested, they reach a comfort level with
processes where they become the be-all and end-all?
DR. VAUGHAN: Well, that's one of my main concerns
about NASA, that the fact that it is a very rule-guided
organization and the fact that they do believe that when
they follow all the rules that they have done their best
and have confidence. That's why the rules tend to carry
such heavy weight. Not only do they aid them with the process
but then they have a cultural effect which builds confidence.
If you're not in touch with the everyday engineering data
itself, you can lose sight of the fact that it is still
an experimental system. So it's the dark side of the organization.
The same kinds of procedures that you implement to make
it work better also can have an unanticipated consequence,
and that's why keeping in touch with all the ambiguities
in the engineering decision-making would be important.
Any other doubts and concerns? You know, by the time you
get to the top of the Flight Readiness Review process, nobody's
going to say that. One of the proposals from the presidential
commission was that an engineer accompany his project manager
at each level of the Flight Readiness Review tier, the feeling
that because engineering concerns did not get carried up
to the top prior to Challenger and in the eve-of-launch
teleconference, they thought that would be a good idea.
Rather than the engineers at Level 4 turning over all their
information to their project manager and then the project
manager carries it forward, let's integrate engineers into
the process. But can you imagine some engineer in the top
Level 1 Flight Readiness Review with 150 people, after all
that's gone on, standing up and saying, "I don't feel good
about this one"?
ADM. GEHMAN: Well, I agree with you. I agree with
you. But I would compound that with an organizational scheme
in which even though that engineer works in the engineering
department and technically doesn't work in the program office
but his position and his salary is funded by the program
office and he wouldn't exist if the program office didn't
pay him. In other words, we've wickered this thing to where
the money flows down through the projects and they send
money over to the engineering office to hire people. So
now put yourself in the position of this guy who's going
to contradict the officer who's paying his salary, and you
don't have a very comfortable formula.
DR. VAUGHAN: I understand that. I think there's a
parallel situation with safety people.
ADM. GEHMAN: Well, yes and no. There is a safety
organization in the programs and in the projects and their
positions depend upon the largesse of the project managers,
but there's also an independent safety organization.
DR. VAUGHAN: I meant in terms of rank. Like independent
authority and power based by where they come in the GS ranking
ADM. GEHMAN: Absolutely. That's a question I'm going
to ask you after General Barry and Dr. Logsdon have a chance.
DR. LOGSDON: I have a comment that's as much directed
at the board as it is at Professor Vaughan. It's just that
this discussion made me think of this line of reasoning.
We've been talking about the rigor of the pre-flight process
for readiness review, compared to a different structure
for what goes on during a mission. There's almost a symbolic
element here. The management of the launch is a Kennedy
Space Center responsibility; and the moment that the shuttle
clears the launch tower, the control over the mission shifts
to Johnson. Sean O'Keefe is trying to say that NASA is a
single organization, but he's got a long way to go to achieve
that goal. These are very proud organizations and, of those,
Johnson is the very proudest of the proud because it's one
of the only two places in the world that knows how to manage
a space flight. There are now -- what's it, '61 -- so 42
years of experience of managing humans in space.
So we're beginning to talk about maybe we can examine the
process of mission management and see whether it measures
up to some standard of high-performance organizations, and
I think that's what we have to do. But there's a lot of
received wisdom and maybe it's ossified wisdom by this point
in the process. So as we go towards that, I think we have
to make sure that we don't have unintended consequences.
So, I say, that's just a comment, not a question.
ADM. GEHMAN: Would you like to comment on his comment?
DR. VAUGHAN: Well, he directed that to the board,
ADM. GEHMAN: In the interest of time, I'll go on
to General Barry.
GEN. BARRY: I'd just like to add one more thing to
your parallel kind of discussion between Challenger and
Columbia. Could you just see if there's anything you know
of that you could add to this kind of construct? You know,
there was a lot of organizational changes here in the last
couple of years. We moved Palmdale to Kennedy. We moved
the Huntington Beach engineering support mostly to JSC but
some to KSC. And, of course, we've got the International
Space Station support going on. So there's some organizational
elements that are unique to Columbia this time; but there
are some Challenger organizational elements, too. You know,
the JSC leadership was being shared by Jesse Moore at the
time between JSC but he was also running the space flight
program as associate administrator. Also, we had an interim
administrator at the time during Challenger. Are there any
parallels that you're seeing between the organizational
aspects between Columbia and Challenger?
DR. VAUGHAN: At the administrative level?
GEN. BARRY: Well, just organizational elements that
we might be able to draw from.
DR. VAUGHAN: One, but it's cultural. It seems like
there is a gap between perceptions of risk between working
engineers and top administrators. So at the time of Challenger,
engineers were very concerned with every launch, even though
they had gone through all the rigors of the procedure; but
at the same time, the people at the top thought it was an
operational system. The parallel I see is, you know, working
engineers really familiar with what's going on and having
concerns, but decisions made that really do echo the period
of Challenger where it's okay to take citizens along for
a ride, which suggests that top-level administrators have
rather lost touch with the fact that it is an experimental
system, a message that they clearly understood post Challenger.
John mentioned symbolic meanings, and they can be really
important. It's hard to judge exactly what the effect is
of a top administrator believing that it's again safe enough
to fly people who are not trained as astronauts. Subtle
things like "faster, cheaper, better" can have an effect
on a culture, even at the same time that you're doing everything
possible to encourage safety.
Certain actions have symbolic meaning. The fact that you
have a safety representative sitting in on a Mission Management
Team or in a particular wherever they're assigned can have
symbolic meaning. Signs posted that it's safety, safety,
safety can convince that you have a safety culture; and
yet when you look at the way the organization works, you
may not have as strong a safety culture as you wished. The
safety person who is assigned to Mission Management Team
decisions, if that is the case, is in a position of not
having hands-on information and reviewing their decision
but not, in a sense, dependent upon them because they have
the leadership responsibility. So what kind of weight, you
would want to know, is that person really bringing to that
situation? Do they have the influence that they are listened
to? Do they have the data to really do anything more than
oversight at that point? How do you really put them in a
position where they can recognize a warning sign and talk
with people who are higher ranked than they are, in a definitive
way, that is convincing in a crisis situation?
ADM. GEHMAN: That leads to my question. That is,
would you be content -- let me just outline this in rough
form -- of a process to satisfy that issue. That is, that
senior management, the management who's got the ultimate
responsibility in these decisions, that they would kind
of be forced to listen to these engineering doubts because
of an organization in which you had checks and balances
among essentially coequal branches of some kind. In other
words, that the engineers were organizationally and culturally
equal to the project managers and the safety and mission
assurance people were not only -- I agree with you. I understand
exactly what you're saying. It's not good enough to just
sit at the table. You have to come to the table with some
clout and usually that clout's in the form of analysis or
data or research or else I won't sign your chit for your
money or something like that. You've got to come with something.
And my model suggests that if you did that, you would be
creating some degree of managerial chaos but, on the other
hand, you would be making sure that engineering reservations
and engineering concerns were well researched and got surfaced
independently at the right level. So you've kind of got
this trade-off between a little bit of managerial chaos,
you would have the danger of the organization not speaking
with one voice and all those kinds of things but, on the
other hand, you would satisfy the requirement that signals
would get heard.
DR. VAUGHAN: Surfaced.
ADM. GEHMAN: Does that sound reasonable?
DR. VAUGHAN: It does sound reasonable. Someone said
if every engineer had every concern, you would probably
never launch a mission; and that's probably true.
ADM. GEHMAN: Probably true.
DR. VAUGHAN: It seems in post-launch conditions where
the clock is ticking, in line with Dr. Barry's suggestion
about how could we restructure the post-launch decision
process, that it would be especially important, then, to
create that kind of an open process.
ADM. GEHMAN: Okay. Well, thank you very much, Dr.
Vaughan. You've been very patient with us. We hope we haven't
tried your patience too much as we try to understand the
very sound principles that you have exposed us to, both
in your book and in your briefing here today.
The board is sensitive about the law of unintended consequences,
and we want to be very careful that we understand more about
these managerial principles before we go writing something
down on a piece of paper that we might regret. But your
study has had an influence on this board and we're indebted
to you for coming and helping us with here today.
DR. VAUGHAN: Thank you. Thanks for having me.
(Hearing concluded at 4:38 p.m.)
Back to April 23 Transcripts