System effectiveness

Innovation from the Latin word innovare, which means that something new is introduced into an existing system implies the concept of improvement. For an improvement to take place, people have to change the way they make decisions or make different choices.

However, it should be taken into account that, as the English 16th century theologian Richard Hooker wrote change is not made without inconvenience, even from worse to better.

Technological innovations of systems of interest often mean that the projects launched to make these systems available are becoming increasingly complex and more expensive.

Designing a system poses a multitude of challenges, creating a composite decision-making process. These kinds of decisions are dictated by cost-effectiveness considerations, taking into account the correlation between the compliance to the system total cost requirements, objectives and schedule constraints.

It should be emphasised that technological innovations characterising advanced systems require a careful balance of the whole acquisition, ownership, and retirement costs, encompassed in the life cycle cost, in order to avoid unjustified trends in cost growth.

The lifetime objective of system effectiveness, in close correlation with systems life cycle cost, needs to be taken into account from the very early stages of a project. This will allow more freedom in system requirements implementation.

The subject of life cycle cost is also remarkable for a wider range of engineering disciplines an example being those of interest to the construction industry.

Another noteworthy area of application is railway engineering, where a number of current studies are based on life cycle cost-effectiveness evaluations. Therefore, the scope of cost-effectiveness encompasses almost the totality of complex engineering projects.

One of the key objectives of systems engineering is to realise projects that can perform their mission as cost-effectively as possible, taking into consideration the whole performance cost, schedule and risk targets over the system life cycle.

Longterm success and user satisfaction rely upon demonstrated effectiveness of the total system inclusive of personnel. In all systems, failure to address longterm issues can result in failure to accomplish the objectives a poor design, unnecessary manpower burden, increased incidence of human errors, excessive costs, and negative impacts to the environment and public health and safety.

Moreover, economic penalties may include a lack of customer confidence, reduced market share and occurrence of product liability. Without this total system approach, the system as an enterprise solution will not meet optimal total performance and/or life cycle cost objectives.

Definition of systems requirements
The devil is in the cost details. This is the case for project users around the world who face the task of procuring, deploying and maintaining systems, equipment and material in a reliable condition.

Generically speaking, we refer to a system as every type of asset released as a result of a design effort.

This would also mean specific retirement procurement processes are influenced by stages at the end of the system life cycle. The total amount of these costs, for a single project, is referenced here as Life Cycle Cost (LCC).

One of the most important things to remember is that requirements problems are typically considered as the biggest contributor to cost overruns, schedule slippages and loss of capability in systems and software projects. Table 1 is a classification of typical requirements.

The life cycle of systems
According to ISO/IEC 15288  Systems and software engineering System life cycle processes,  the life cycle of a generic system is composed of the stages in a (hypothetical) sequential arrangement, as depicted in Figure 1.

The total system economic value and the life cycle cost
Daily decisions required to manage the introduction of new systems or the modification of existing systems take into consideration only those technical characteristics that are deemed significant with respect to the expected system performance.

From an economic point of view, most attention is given to initial acquisition costs, while less attention is paid to costs related to life cycle stages namely Utilisation Stage and Support Stage.

Actually, Utilisation and Support costs often play a major role in the context of the Life Cycle Cost of systems. They are markedly determined by decision making processes taking place at the inception of system lifetime.

 

As illustrated in Figure 2, the total system value derives from a balance between economic factors and design factors relevant to the system.

In professional literature, one of the earliest complete treatments of Life Cycle Cost is found in the book Design and Manage to Life Cycle Cost by B.S. Blanchard, who captured the iceberg picture (Figure 3) to represent the contrast between the acquisition costs and the ownership and retirement costs.

Two main aspects conceptually significant for the life cycle of any system are emphasised by Blanchard:

• the different visibility of Life Cycle Cost components (in accordance with the iceberg metaphor);
• the fact that it is at the early stages in an acquisition programme that the greatest gains can be realized in terms of the system Life Cycle Cost (however taking cautiously into account some downstream events often occurring during the life of the system, influencing the Life Cycle Cost itself, for example: life cycle schedule slips, changes in quantity of systems to be installed and/or in their operational conditions; significant improvement programmes).

Figures of Merit
LCC estimation process offers a number of basic elements to support decisions not only during the early life cycle stages, but in all subsequent periods.

Specific decisions are required to manage maintenance policies and to carry out trade-offs between different possible alternatives, until the system life comes to its projected end.

The following relationship: FOM = SE / LCC  introduces the helpful quantitative tool called 'Figures of Merit' (FOM), by which any system to be procured can be characterised on the basis of calculated values of System Effectiveness and Life Cycle Cost.

From a quantitative point of view, the effectiveness of a generic system is defined as a function of system performance and mission profile. It has to be considered as a goal to be taken into account from the very early stages of the life cycle, in which a larger degree of freedom is allowed in the implementation of system requirements: SE= A0 * D0 * C where: System Effectiveness (SE) is a measure of the ability of a system to achieve a set of specific mission requirements.

Operational Availability (A0) is a measure of the degree to which a system is in the operable and committable state at the start of the mission, when the mission is called for at an unknown (random) time.

Operational Dependability (D0) is a measure of the degree to which an item is operable and capable of performing its required function at any time during a specified mission profile, given system availability at the start of the mission.

Capability (C) is a measure of the ability of an item to achieve mission objectives, given the conditions during the mission.

figure 4 depicts a breakdown of System Effectiveness in its typical lower level components.

 

The first level components of System Effectiveness, Availability, Dependability and Capability are therefore measures of system ability to operate during a specified operational mission. System Effectiveness may be regarded as a collective term used to describe the system availability performance and its design factors namely reliability, maintainability and logistic support.

Figures of Merit are also dependent on Life Cycle Cost, so that LCC values, calculated for a range of solutions to be evaluated for their feasibility, will be correlated with the corresponding values of System Effectiveness.

Life cycle cost profile and summary
By using the notion of LCC, the objective of cost-effectiveness for a given system is considered from a total system life cycle perspective. All decisions made on the system design and manufacture may affect its performance, safety, reliability and maintainability, consequently determining its price and ownership costs. By performing trade-off studies on LCC the system design can be optimised from an LCC viewpoint.

Comparing performance between different systems is not a comfortable job, but comparing Life Cycle Costs is likely to be harder.

Its therefore necessary to introduce many assumptions about the systems and in some situations where realistic assumptions are hard to give indications on the variations relative to different variables have to be given as a substitute.

A sensitivity analysis will be used to input parameters that are deemed most critical. These will be varied in suitable ranges and the LCC analysis process will be repeated to appraise the corresponding results. The purpose of sensitivity analysis is to identify parameters that lead to a large variation in resulting cost. 

It is a primary purpose of LCC assessment to provide input to decisions occurring across the life cycle.

A critical requirement of LCC analysis is to assess programme affordability by identifying relevant cost drivers. Furthermore, it must be noted that at the launch of a programme, the order of magnitude of future Utilisation and Support costs can be only foreseen with a rough approximation. This introduces a significant risk factor in the assessment of total LCC, especially if comparable data cannot be derived from similar pre-existing programmes.

General Criteria for Life Cycle Cost Analysis
LCC analysis is oriented towards long-term economic factors in the life cycle of a system,  rather than trying to save money in the short term by simply purchasing systems with lower initial acquisition costs. It allows an easier overall cost visibility and a straightforward evaluation of risks.

LCC analysis is significant both for systems entering their life cycle and for existing systems requiring an identification of most significant cost factors.
The stepwise procedure typically followed in a LCC analysis is described in the subsequent sections.

A Life Cycle Cost estimate provides a comprehensive and organised account of all resources and costs required to design, develop, produce, deploy and sustain a particular system.

Conditions of Uncertainty and Risk in Life Cycle Cost Analysis. Figure 5 Risk Matrix

Risk and uncertainty are defined in the following way:

• Risk is a measure of the chance that, as a result of adverse events, a certain point estimate (specifically, a cost estimate) will be exceeded. It is typically measured on the basis of two components: (1) the probability of occurrence of adverse events, and (2) the consequences of these adverse events, should they actually manifest themselves.
• Uncertainty is the indefiniteness or variance of an event. It refers to all situations that are unpredictable, essentially due to lack of relevant information.

Frequently, the two terms risk and uncertainty are used interchangeably to mean an equal concept. Risk and uncertainty are logically correlated, for example by the assertion that uncertainty generates risk.

Basically, uncertainty is characterised by probability values, whereas risk is characterised at the same time by probability values and severity of the consequence of the adverse event.

Figure 5 depicts a typical representation of risk as a function of the two aforementioned characteristics.
Regarding the Risk Matrix, however, Prioritising Project Risks A Short Guide to Useful Techniques  by M. Hopkinson, M., P. Close, D. Hillson and S.
Ward states that the matrix can lead to prioritisation outcomes that are not appropriate. Subsequently: For example, very low probability but very high impact threats may be given lower prioritisation than would be preferred.

As the probability of an adverse event increases from 0 to 1, classification in 5 levels can be envisaged, as depicted in Table 3, showing the different possibility to apply mitigating factors to control risk.

At the same time, the severity of consequences can be categorised in five levels (Level 1-5) of growing severity. 

Biography
Massimo Pica graduated from Sapienza University in Rome, Italy, with a degree in chemical engineering. He is a former brigadier general having served in the Italian Armys Corps of Professional Engineers.

Throughout his career Massimo has built up significant experience in project management, and especially project cost management, as he participated in a number of NATO and international programmes for the design, development and production of advanced defence systems.

He is the author of Systems Lifecycle Cost-Effectiveness: The Commercial Design and Human Factors of Systems Engineering published by Gower.

---

This article first appeared in Project magazine. APM members can read all feature articles from Project magazine over recent years by accessing the Project magazine archive.

Non-members can subscribe to the UK's best-read project management magazine for as little as 56.50 per year (12 copies), which includes access to the Project magazine archive. APM members automatically receive the magazine as part of their membership:

UK: 56.50

Europe: 66.50

International: 77

View a sample issue of Project

To order yours now, click "Subscribe" below: