SYS511 Engineering Processes Systems Integration Case Study Paper We will be focusing on systems integration and test aspects of these case studies. Write

SYS511 Engineering Processes Systems Integration Case Study Paper We will be focusing on systems integration and test aspects of these case studies. Write a short paper highlighting your views on the case with respect to shortfalls in integration and/or test efforts. Do not forget to look at systems engineering processes (planning, requirements management, configuration management, risk analysis, etc) with respect to and how they support integration and test. Do not just regurgitate the case study back to me, I have already read it; I am looking for your views. One last thing, don’t give me a monolithic block of text, make sure I can clearly delineate the answers to the questions.

Papers should be no more than two pages. Address the following questions:

In systems engineering terms describe the fundamental problem (s) encountered.
Example: “This appears to be a case where requirements were not clearly defined, analyzed and documented. No configuration management process was implemented and interfaces were not adequately defined…”
What do you think the program did right?
Based on what you have learned in class to date, what would YOU have done differently and why?
Have you experienced any problems or successes similar to that in the paper, if so what was it?
What were your top 3 “takeaways”? Why?

We will spend some time analyzing the problem and discussing the case via discussion board. Discussion board participation is mandatory and will be part of your grade. The Role of Software in Spacecraft Accidents
Nancy G. Leveson∗
Aeronautics and Astronautics Department
Massachusetts Institute of Technology
Abstract: The first and most important step in solving any problem is understanding the
problem well enough to create effective solutions. To this end, several software-related spacecraft accidents were studied to determine common systemic factors. Although the details in
each accident were different, very similar factors related to flaws in the safety culture, the
management and organization, and technical deficiencies were identified. These factors include
complacency and discounting of software risk, diffusion of responsibility and authority, limited
communication channels and poor information flow, inadequate system and software engineering (poor or missing specifications, unnecessary complexity and software functionality, software
reuse without appropriate safety analysis, violation of basic safety engineering practices in the
digital components), inadequate review activities, ineffective system safety engineering, flawed
test and simulation environments, and inadequate human factors engineering. Each of these
factors is discussed along with some recommendations on how to eliminate them in future
projects.
1
Introduction
Software is playing an increasingly important role in aerospace systems. Is it also playing an
increasing role in accidents and, if so, what type of role? In the process of a research project
to evaluate accident models, I looked in detail at a variety of aerospace accidents that in some
way involved software.1,2 Many of the factors were in common across several of the accidents.
To prevent accidents in the future, we need to attack these problems.
The spacecraft accidents investigated were the explosion of the Ariane 5 launcher on its
maiden flight in 1996; the loss of the Mars Climate Orbiter in 1999; the destruction of the
Mars Polar Lander sometime during the entry, deployment, and landing phase in the following
year; the placing of a Milstar satellite in an incorrect and unusable orbit by the Titan IV
B-32/Centaur launch in 1999; and the loss of contact with the SOHO (SOlar Heliospheric
Observatory) spacecraft in 1998.
On the surface, the events and conditions involved in the accidents appear to be very
different. A more careful, detailed analysis of the systemic factors, however, reveals striking
similarities. Systemic factors are those that go beyond the specific technical causes, such as
a flawed O-ring design in the Space Shuttle Challenger accident, and include the reasons why
those failures or design errors were made. For Challenger, the latter include flawed decision
making, poor problem reporting, lack of trend analysis, a “silent” or ineffective safety program,
∗
Mailing Address: Professor Nancy Leveson, Room 33-313, Dept. of Aeronautics and Astronautics, MIT, 77
Massachusetts Ave., Cambridge MA 02139; Tel: 617-258-0505, email: leveson@mit.edu.
This paper has been accepted for publication in the AIAA Journal of Spacecraft and Rockets.
1
communication problems, etc. Systemic factors are those related to the overall system within
which the technical device is developed and operated.
A difficulty was encountered in that several of the accident reports implicated the software
but then, for some unknown reason, never investigated the software development process in any
depth to determine why the error was made. In some cases, it was possible to find information
about the software development problems from sources outside the official accident investigation
report. One conclusion from this observation might be that accident investigation boards must
include more software experts and must more thoroughly investigation the reasons for the
introduction of the errors and their lack of detection once introduced if we are to learn from
our mistakes and improve our processes.
The accidents are first briefly described for those unfamiliar with them, and then the common factors are identified and discussed. These factors are divided into three groups: (1)
flaws in the safety culture, (2) management and organizational problems, and (3) technical
deficiencies.
2
The Accidents
Ariane 501
On June 4, 1996, the maiden flight of the Ariane 5 launcher ended in failure. About 40 s after
initiation of the flight sequence, at an altitude of 2700 m, the launcher veered off its flight path,
broke up, and exploded. The accident report describes what they called the “primary cause”
as the complete loss of guidance and attitude information 37 s after start of the main engine
ignition sequence (30 seconds after liftoff).3 The loss of information was due to specification
and design errors in the software of the inertial reference system. The software was reused
from the Ariane 4 and included functions that were not needed for Ariane 5 but were left in for
“commonality.” In fact, these functions were useful but not required for the Ariane 4 either.
Mars Climate Orbiter (MCO)
The Mars Climate Orbiter (MCO) was launched December 11, 1998 atop a Delta II launch
vehicle. Nine and a half months after launch, in September 1999, the spacecraft was to fire its
main engine to achieve an elliptical orbit around Mars and to skim through the Mars upper
atmosphere for several weeks, in a technique called aerobraking, to move into a low circular
orbit. On September 23, 1999, the MCO was lost when it entered the Martian atmosphere in
a lower than expected trajectory. The investigation board identified what it called the “root”
cause of the accident as the failure to use metric units in the coding of a ground software file
used in the trajectory models.4 Thruster performance data were instead in English units.
Mars Polar Lander (MPL)
Like MCO, Mars Polar Lander (MPL) was part of the Mars Surveyor program. It was launched
January 3, 1999, using the same type of Delta II launch vehicle as MCO. Although the cause
of the MPL loss is unknown, the most likely scenario is that the problem occurred during the
entry, deployment, and landing (EDL) sequence when the three landing legs were to be deployed
from their stowed condition to the landed position.5,6 Each leg was fitted with a Hall Effect
magnetic sensor that generates a voltage when its leg contacts the surface of Mars. The descent
engines were to be shut down by a command initiated by the flight software when touchdown
2
was detected. The engine thrust must be terminated within 50 milliseconds after touchdown
to avoid overturning the lander. The flight software was also required to protect against a
premature touchdown signal or a failed sensor in any of the landing legs.
The touchdown sensors characteristically generate a false momentary signal at leg deployment. This behavior was understood and the flight software should have ignored it. The
software requirements did not specifically describe these events, however, and consequently the
software designers did not account for them. It is believed that the software interpreted the
spurious signals generated at leg deployment as valid touchdown events. When the sensor data
was enabled at an altitude of 40 meters, the software shut down the engines and the lander free
fell to the surface, impacting at a velocity of 22 meters per second and was destroyed.
Titan/Centaur/Milstar
On April 30, 1999, a Titan IV B-32/Centaur TC-14/Milstar-3 was launched from Cape Canaveral.
The mission was to place the Milstar satellite in geosynchronous orbit. An incorrect roll rate
filter constant zeroed the roll rate data, resulting in the loss of roll axis control and then yaw
and pitch control. The loss of attitude control caused excessive firings of the reaction control
system and subsequent hydrazine depletion. This erratic vehicle flight during the Centaur main
engine burns in turn led to an orbit apogee and perigee much lower than desired, placing the
Milstar satellite in an incorrect and unusable low elliptical final orbit instead of the intended
geosynchronous orbit.
The accident investigation board concluded that failure of the Titan IV B-32 mission was due
to an inadequate software development, testing, and quality assurance process for the Centaur
upper stage.7 That process did not detect the incorrect entry by a flight software engineer of a
roll rate filter constant into the Inertial Navigation Unit software file.
The roll rate filter itself was included early in the design phase of the first Milstar spacecraft, but the spacecraft manufacturer later determined that filtering was not required at that
frequency. A decision was made to leave the filter in place for the first and later Milstar flights
for “consistency.”
SOHO (SOlar Heliospheric Observatory)
SOHO was a joint effort between NASA and ESA to perform helioseismology and to monitor the
solar atmosphere, corona, and wind. The spacecraft completed a successful two-year primary
mission in May 1998 and then entered into its extended mission phase. After roughly two months
of nominal activity, contact with SOHO was lost June 25, 1998. The loss was preceded by a
routine calibration of the spacecraft’s three roll gyroscopes and by a momentum management
maneuver.
The flight operations team had modified the ground operations procedures as part of a
ground systems reengineering effort to reduce operations costs and streamline operations, to
minimize science downtime, and to conserve gyro life. Though some of the modifications were
made at the request of the SOHO science team, they were not necessarily driven by any specific
requirements changes. A series of errors in making the software changes along with errors in
performing the calibration and momentum management maneuver and in recovering from an
emergency safing mode led to the loss of telemetry.8 Communication with the spacecraft was
eventually restored after a gap of four months.
3
3
Flaws in the Safety Culture
The safety culture is the general attitude and approach to safety reflected by those working in an
industry. The accident reports all described various aspects of complacency and a discounting
or misunderstanding of the risks associated with software.
Success is ironically one of the progenitors of accidents when it leads to overconfidence
and cutting corners or making tradeoffs that increase risk. This phenomenon is not new, and
it is extremely difficult to counter when it enters the engineering culture in an organization.
Complacency is the root cause of most of the other accident factors described in this paper and
was exhibited in all the accidents studied.
The Mars Climate Orbiter (MCO) report noted that because JPL’s navigation of interplanetary spacecraft had worked well for 30 years, there was widespread perception that “orbiting
Mars is routine” and inadequate attention was devoted to navigation risk management and
contingency planning. The MCO investigators found that project management teams appeared
primarily focused on meeting mission cost and schedule objectives and did not adequately focus on mission risk. A recommendation common to several of the accident reports was to pay
greater attention to risk identification and management.
The official report on the MPL loss concludes that the pressure of meeting the cost and
schedule goals resulted in an environment of increasing risk in which too many corners were cut
in applying proven engineering practices and in the checks and balances necessary for mission
success. “Lack of adequate risk identification, communication, management, and mitigation
compromised mission success.”6
In the SOHO loss, overconfidence and complacency, according to the accident report, led to
inadequate testing and review of changes to ground-issued software commands to the spacecraft,
a false sense of confidence in the team’s ability to recover from a safe-hold mode (emergency sun
reacquisition) from which a recovery sequence must be commanded and executed under ground
operator control, the use of tight schedules and compressed timelines that eliminated any time
to handle potential emergencies, inadequate contingency planning, responses to emergencies
without taking the designed-in time to consider the options, etc. Protections built into the
process, such as formal reviews of critical decisions by senior management and engineering
staff, were bypassed. The functional content of an operational procedure was changed without
appropriate documentation and review of the changes.
After two previous SOHO spacecraft retreats to safe mode, the software and procedures
were not reviewed because (according to the accident report) higher priority had been assigned
to other tasks. The report concludes that the success in recovering from the previous safe mode
entries led to overconfidence by the operations team in their ability to recover and a lack of
appreciation of the risks involved in entering and recovering from the safing mode.
The Ariane 5 accident report notes that software was assumed to be correct until it was
shown to be faulty. As noted by the Ariane accident investigation board, the opposite assumption is more realistic. A similar attitude prevailed in Titan/Centaur operations. For example,
on the day of the launch, the attitude rates for the vehicle on the launch pad were not properly
sensing the earth’s rotation rate (the software was consistently reporting a zero roll rate) but
no one had the responsibility to specifically monitor that rate data or to perform a check to see
if the software attitude filters were operating correctly. In fact, there were no formal processes
to check the validity of the filter constants or to monitor attitude rates once the flight tape was
actually loaded into the Inertial Navigation Unit at the launch site. Potential hardware failures
are usually checked up to launch time, but it may have been assumed that testing removed all
software errors and no further checks were needed.
4
While management may express their concern for safety and mission risks, true priorities
are shown during resource allocation. Although budget decisions are always difficult when
resources are constrained—and budgets are almost always less than is optimal—the first things
to be cut are often system safety, system engineering, mission assurance, and operations, which
are assigned a low priority and assumed to be the least critical parts of the project.
In the Milstar satellite loss, the Titan Program Office had no permanently assigned civil
service or military personnel nor full-time support to “work” the Titan/Centaur software. They
had decided that because the software was “mature, stable, and had not experienced problems
in the past,” they could best use their limited resources available after the initial development
effort to address hardware issues. The Titan program office had cut support for monitoring the
software development and test process by 50% since 1994 and had greatly cut the number of
engineers working launch operations.
The SOHO Mission Management Plan required that the NASA Project Operations Director be responsible for programmatic matters, provide overall technical direction to the flight
operations team, and interface with the ESA technical support director. The position had been
descoped over time by NASA from a dedicated individual during launch and commissioning to
one NASA individual spending less than 10% of his time tracking SOHO operations. ESA was
to retain ownership of the spacecraft and to be responsible for its technical integrity and safety,
but they were understaffed to perform this function in other than routine situations. In both
SOHO and MCO, the operations group did not have a mission assurance manager.
Complacency can also manifest itself in a general tendency of management and decision
makers to discount unwanted evidence of risk. A culture of denial 9 arises in which any evidence
of significant risk is dismissed. In the MCO, SOHO, and Titan losses, warning signs existed
that the software was flawed, but they went unheeded.
The problems experienced with the Mars Climate Orbiter (MCO) software during the early
stages of the flight did not seem to raise any red flags. During the first four months of the MCO
mission, the ground software angular momentum desaturation (AMD) files were not used in
the orbit determination process because of multiple file format errors and incorrect spacecraft
attitude data specifications. Four months were required to fix the files. Almost immediately
(within a week) it became apparent that the files contained anomalous data that was indicating underestimation of the trajectory perturbations due to desaturation events. Despite all
these hints that there were serious problems in the software and perhaps the development process, reliance was still placed on the supposedly fixed software without extra manual checks or
alternative calculations to check the results.
Three months before the loss of the SOHO telemetry, ground software problems had triggered an emergency sun reacquisition (a safe hold mode entered when there are attitude control
anomalies) and a shortcut in the recovery from this emergency sun reacquisition led to a second
one. A resulting recommended comprehensive review of the software and procedures had not
been implemented before the accident because higher priority had been assigned to other tasks.
Engineers noticed the problems with the Titan/Centaur software after it was delivered to
the launch site and they were reported back to LMA in Denver, but nobody seemed to take
them seriously.
Some of the complacency can arise from a misunderstanding of the risks associated with
software. Throughout the accident reports, there is an emphasis on failures as the cause of
accidents and redundancy as the solution. Accidents involving software, however, are usually system accidents that result from dysfunctional interactions among components, not from
individual component failure. All these accidents (as well as almost all the software-related
accidents known to the author) resulted from the software doing something wrong rather than
5
the computer hardware or software failing to operate at all. In fact, in most cases the software
or hardware components operated according to their specifications (i.e., they did not fail), but
the combined behavior of the components led to disastrous system behavior.
All the accidents investigated for this paper displayed some aspects of system accidents.
System accidents are caused by interactive complexity and tight coupling.10 Software allows us
to build systems with a level of complexity and coupling that is beyond our ability to control;
in fact, we are building systems where the interactions among the components (often controlled
by software) cannot all be planned, understood, anticipated, or guarded against. This change
is not solely the result of using digital components, but it is made possible because of the
flexibility of software. Note that the use of redundancy only makes the problem worse—the
added complexity introduced by redundancy has resulted in accidents that otherwise might not
have occurred.
The Ariane 5 accident report notes that according to the culture of the Ariane program,
only random failures were addressed and they were primarily handled with redundancy. The
engineers designing the Ariane 5 inertial guidance system opted to shut down the computer when
an exception was raised in an unnecessary function (the alignment function after takeoff):
The reason behind this drastic action lies in the culture within the Ariane programme of only addressing random hardware failures. From this point of view,
exception—or error—handling mechanisms are designed for a random hardware
failure which can quite rationally be handled by a backup system.3
This approach ob…
Purchase answer to see full
attachment