June 27th, 2008 - VOL. 1 Issue 5

Features

[ Page 1 ] [ Page 2 ]

From the Archive: Robert Frosch on Systems Engineering

Nearly 40 years ago, Dr. Robert A. Frosch delivered a speech on the practice of systems engineering that remains relevant to this day.

ASK the Academy reaches back to “Readings in Systems Engineering,” a 1993 anthology edited by Francis T. Hoban and William M. Lawbaugh (NASA SP-6102), for a defining article on systems engineering by NASA’s fifth Administrator, Robert A. Frosch. The text below appears as printed in that volume.

A Classic Look at Systems Engineering

Editor's Note:
Before his term as NASA Administrator, Bob Frosch was an assistant secretary of the U.S. Navy in charge of research, development, test and evaluation of Navy programs. In that capacity, he delivered a controversial and well-remembered speech to the IEEE Group on Aerospace and Electronic Systems during IEEE's international convention in New York on March 26, 1969. Edited portions of that famous speech follow in an effort to preserve what is now considered a classic formulation of systems engineering as an art rather than a science.

In this presentation, I really will be discussing the application of systems engineering to development, and in particular to military systems development (with which I am most familiar). However, from reading various journals and newspapers, I suspect my re- marks are of more general applicability. I have said some of these things before, but some bear repeating and some I hope will spark new ideas.

I couple systems engineering, systems analysis and Management (with a capital "M"), because in practice they seem to be closely related terms, referring to the same constellation of systematic practices and attitudes.

• We badly lack: systems engineering of systems engineering; systems analysis of systems analysis.
• And, heaven knows, there is no: management of Management.
• Therefore, I will now preach against home, motherhood and apple pie.

To the charge that I am writing about bad systems engineering, I can only say that I am taking a pragmatic view: the thing is defined by what is done, not what is said; and if what I am describing is bad systems engineering, I can only say that I seldom see any other kind.

What I want to do is discuss briefly a series of antitheses (and perhaps an unbalanced question or two) that pit the systems world against what I believe are some aspects of the real world.

If I plot a graph versus time of what appears to be a recent rising tide of costs, cost overruns, unsatisfactory performance and unhappiness among engineers, I have reason to worry. (If this trend continues, we may have to debate whether the question "whither engineering?" is spelled with one "h" or two.) If I plot on the same graph versus time the rise in talk, directives, and use of "systems engineering," "systems analysis" and "Management," I see high correlation between the two graphs--trouble versus time and the use of systems engineering versus time. This does not prove causation, but it suggests, at least, that the "new techniques" are proving to be a poor substitute for real science and engineering; they are, at the least, not doing what they are advertised as doing, if they are indeed actually not making things worse. It could be that things would be even worse without these new techniques, but I would like to ask some questions and suggest some reasons for believing that systems engineering, systems analysis and Management, as practiced, are likely to be part of the problem, and indeed causative agents.

I believe that the fundamental difficulty is that we have all become so entranced with technique that we think entirely in terms of procedures, systems, milestone charts, PERT diagrams, reliability systems, configuration management, maintainability groups and the other minor paper tools of the "systems engineer" and manager. We have forgotten that someone must be in control and must exercise personal management, knowledge and understanding to create a system. As a result, we have developments that follow all of the rules, but fail.

I can best describe the spirit of what I have in mind by thinking of a music student who writes a concerto by consulting a checklist of the characteristics of the concerto form, being careful to see that all of the canons of the form are observed, but having no flair for the subject, as opposed to someone who just knows roughly what a concerto is like, but has a real feeling for music. The results become obvious upon hearing them. The prescription of technique cannot be a substitute for talent and capability, but that is precisely how we have tried to use technique.

Paper vs. People

My first antithesis pits the systems world of paper and arrangements against the real world of people and hardware. When paper appears in the real-world version of a system, it is generally only as an abstracted commentary. For example, in a very basic sense it really is of no consequence whether the documentation on a weapons system is good, bad or nonexistent; that is only a commentary on whether or why the people and the hardware actually work when called upon, and a tool to help them work. If the systems arrangements on paper and the documentation can help to make the stuff work, then they are of some use. If they are merely the formal satisfaction of a requirement, they are only an interference with engineering. Systems, even very large systems, are not developed by the tools of systems engineering, but only by the engineers using the tools. In looking back at my experiences in development, including watching a number of Navy developments over the past few years, it seems quite clear that in most cases where a system gets into trouble, a competent manager knows all about the problem and is well on the way to fixing it before any management systems ever indicate that it is about to happen. This happens if for no other reason than because the competent manager is watching what is going on in great detail and perceives it long before it flows through the paper system. That is to say, personal contact is faster than form-filling and the U.S. mails. A project manager who spends much time in a Management Information Center instead of roving through the places where the work is being done is always headed for catastrophe. The MIC can assist the people who are not involved in the project toward learning of after-the-fact problems, but that is roughly all that it can do, and its value even for this purpose is frequently questionable.

Blaming deficiencies in management systems for problems that exist in real unknowns, or in the deficiencies of people, is mere foolishness. In a poem called "Bagpipe Music," by Louis MacNeice, the final couplet is:

"The glass is falling hour by hour,
the glass will fall forever
But if you break the bloody glass,
you won't hold up the weather."

Linearity vs. The Real World

One of the key misassumptions in modern systems engineering and systems analysis is that the total problem can be, and frequently is, decomposed into sub-problems; the sub-problems can be solved more or less independently, and the total solution can be synthesized by combination of the sub-solutions, treating the interactions of the parts as "interfaces." The real world is, however, highly non-linear, and unless real attention is paid to this fact, the linear decomposition treatment will fail catastrophically, because the interaction terms may be as large as the sub-problems and not reducible to simple interfaces. The result may well remain decomposed.

A Classic Look at Systems Engineering

This criticism is frequently answered by the comment that problems are unmanageable unless sliced up and, therefore, the procedure is used even though we know it may be seriously in error. This is the case of the man who played in a poker game that he knew to be crooked, because it was the only game in town; or the drunk who looked for his ring under the street lamp even though he had lost it a block away in the dark--the light was better under the street light. I have some difficulty seeing that a bad analysis is really better than an informed judgment, especially since faith in the analysis (and/or the decomposed solution to the problem) is frequently, nay, usually, used as a substitute for seeking or applying any judgment at all. I am often faced with a result that seems absurd, and can even produce a quick analysis that at least makes it obvious that the solution is absurd, but am then given the answer, "Well, that's what the analysis showed."

Such a situation usually indicates room for deep criticism, either of the way in which the problem was divided up, or of peculiarities of the assumptions that drive the problem in curious and unsuspected ways, particularly through the unsuspected (by the systems person) nonlinearities of the problem. It sometimes appears that the only rational subdivision of the problem is to fractionize the blame to the point where approval is sought by default.

I would argue that careful attention to the parts of the problem that do not seem to be easily decomposable into semi-independent parts might be one very good guide to areas involving high risk, since these are likely not to be amenable to our usual rules, procedures and technologies, and hence probably will have to be approached empirically.

Serial vs. Iterative Models

Systems engineering techniques themselves contribute to disaster because they are all paper techniques and there are only two instead of N dimensions available. What we end up displaying are linear sequential measures of system progress.

The PERT diagram and the milestone chart are excellent examples. These both essentially assume that the progress of development and design consists of doing step A, then step B, then step C, etc. Anyone who has ever carried out a development or a design (as opposed to setting up a management system for doing so) is well aware of the fact that the real world proceeds by a kind of feedback iterative process that looks more like a helix than like a line. That is to say, you do A, then B, then C, then you look at C and go back and change part of A again, and that causes you to fiddle with B and perhaps bring in a B-prime that you bounce against C, and then go back to A and then jump to D, so that there has to be continual adjustment, going back and forth so that the system is adjusted to itself and to its end objectives as it changes and as the design or development proceeds. Because it is difficult to predict this process or to diagram it, or to predict its costs precisely without using competent engineers, the systems engineering procedures simply ignore the iterative, feedback nature of the real world because the process has been degraded to clerical reporting. To a large extent, this tends to constrain project managers from doing work in the real way toward doing it in a way that fits with their management tools. This is clearly nonsense.

As a specific example, doctrine says that one is to consider the "ilities," that is, maintainability, reliability, operability, etc., from the very beginning of the process. This is a vast waste of time and effort. I do not mean that one should not think about these things at the beginning, but it is certainly ridiculous to have a complete plan for the logistics of the maintenance of an object that has not yet been designed. I have seen overruns in expenditure and unnecessary effort generated by the fact that the linear sequencing of milestones had forced development of a complete maintenance and reliability plan for what was no longer the design, and had not been the design for three months. The machinery forced everyone to grind on and on because, after all, the maintenance and reliability milestones could not be missed without disaster and fear of cancellation of the project, even though the plan being worked out had nothing whatever to do with the hardware being designed.

In fact, the point at which to start serious work on configuration control, maintainability and reliability cannot be very well preplanned; it can be roughly preplanned, but it must be adjusted to be at the point at which the design means something and is likely to stay still long enough so that the redesign for the "ilities" will really make some sense. Judgment, not tools, is what is required.

[ Page 1 ] [ Page 2 ]

If you would like to forward this article to a friend click here

If you would like to receive ASK the Academy straight to your email box click here