WO2007070962A1 - A system and method for the optimisation of complex systems - Google Patents

Abstract

A method of optimising a complex system, comprising the steps of, representing at least one quality attribute of the complex system in a graphical format, altering the complex system, and representing the at least one quality attribute of the altered complex system in a graphical format, wherein the at least one quality attributes may be compared.

Description

A SYSTEM AND METHOD FOR THE OPTIMISATION OF COMPLEX

SYSTEMS

Field of the Invention

The present invention relates to a system and method for the optimisation of complex systems. Embodiments of the invention find particular, but not exclusive, use in providing visual representation of the effect of changing a parameter in a complex system.

Background of the Invention

Computer-Based Systems are increasingly pervasive throughout society, continually increasing in complexity and cost as they are called upon to fulfil more and more complicated tasks. Unfortunately, multi-million dollar projects often fail because the systems are unreliable, inefficient, insecure and unmaintainable. Much of the failure can be attributed to incorrect assumptions about the way the system will operate once built.

Clearly, systems must be built with a range of qualities such as reliability, performance and security, in addition to their required functionality, if they are to be successful. The builders of complex systems are often faced with the challenge of meeting many potentially conflicting quality requirements simultaneously.

More particularly, the builders of complex systems must be able to understand the systems they are building, and they must be able to understand how a change to one aspect of a system has an impact on other aspects of the system.

In particular, large computer-based systems, commonly called "enterprise architectures" , can be extremely complex, and include wide area networks, hundreds of servers and terminals, hundreds of different and possibly conflicting services, such as email, Intranets, proprietary databases and software applications, PABX and other telecommunications networks, and other computer enabled services, such as security services. Moreover, such enterprise architectures may span many time zones and may simultaneously host thousands or even millions of users.

Such enterprise architectures can be staggering in their complexity, and changes to one aspect of an enterprise architecture may have wide ranging effects on the network as a whole . Moreover, enterprise architectures, although being thought of traditionally as "computer" systems, are also taking into account other off-line systems, such as the users of the computer systems. This requires enterprise architecture builders to take into account the employees of the firm, and the manner in which they interact with the components that make up the architecture.

Summary of the Invention

In a first aspect, the present invention provides a method of optimising a complex system, comprising the steps of, representing at least one quality attribute of the complex system in a graphical format, altering the complex system, and representing the at least one quality attribute of the altered complex system in a graphical format, wherein the at least one quality attributes may be compared .

The method may comprise the further step of displaying the at least one quality attributes on a common graph, wherein the relative variation in performance between the initial complex system and the altered complex system may be identified. In one embodiment, the method may include the further step of calculating the new value of the quality attribute for the altered complex system.

The method may comprise the further step of representing at least one quality attribute for a plurality of complex systems on the common graph. The method may comprise the further step of iteratively altering the complex system to produce the plurality of altered systems.

The value of the quality attribute for each of the plurality of altered systems may be calculated.

The method may comprise the further step of providing at least one indicator on the single graph, whereby, in use, the evolution of each of the complex systems may be compared . A quality space may be defined within the graphical format, the quality space defining an area of desirable values for the at least one quality attribute.

The common graph may be a star plot . The at least one indicator may be a vector. The graphical format may be one of an architecture space visualisation, a histogram or a scatterplot. The complex system may be any type of complex system, including computer systems, computer networks, organisations, enterprises. The complex system may be any type of complex system, including but not limited to computer systems, computer networks, and organisations.

In many embodiments, the complex system is modelled as an architecture, which may be one of an Enterprise Architecture, a Computer Based System Architecture, an Infrastructure Architecture, a Data Architecture, an Information Architecture, an Application Architecture, and an Organisational Architecture.

In a second aspect, the present invention provides a system for visualising changes to a complex system, comprising means to receive information pertaining to at least one quality attribute of the complex system, means to receive information pertaining to the at least one quality attribute of an altered complex system, and means to graphically represent the at least one quality attribute for the complex system and the altered complex system, wherein the graphical means displays the relative variation in performance between the complex system and the altered complex system.

In a third aspect, the present invention provides a system for optimising a complex system, comprising, means for receiving and representing at least one quality attribute of the complex system in a graphical format, means for receiving and representing the at least one quality attribute of the altered complex system in a graphical format, wherein the at least one quality attributes may be compared. In a fourth aspect, the present invention provides a computer program arranged to carry out the method steps of a first aspect of the invention.

In a fifth aspect, the present invention provides a computer medium incorporating a computer program in accordance with a fourth aspect of the invention. Detailed Description of the Drawings

Notwithstanding any other forms which may fall within the scope of the present invention, a preferred embodiment will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 is an example of how an initial architecture for a complex system is evolved, to navigate the solution space ; Figure 2 is a diagrammatic representation of the architecture space, the solution space and the quality space ;

Figure 3 is an example of an optimisation model in accordance with an embodiment of the present invention; Figure 4 is a diagrammatic representation of the manner in which the embodiment of Figure 3 operates;

Figure 5 depicts a system boundary model which depicts the manner in which the optimisation model of Figure 3 operates; Figure 6 is a picture of the architecture space visualisation as displayed by an embodiment of the present invention;

Figure 7 and 8 are histogram views of the quality attributes for the example architecture of Figure 6 ; Figure 9 is a scatterplot view of the quality attributes for the example architecture of Figure 6;

Figure 10 and 11 are star plot views of the quality attributes for the example architecture of Figure 6;

Figure 12 is a diagram of an initial example architecture;

Figure 13, 14 and 15 are star plots of quality attributes relevant to the example architecture of Figure 12 ; — o ~-

Figure 16 is an example of the solution space derived from the initial architecture of Figure 12 through utilisation of an embodiment of the present invention;

Figure 17 is a scatterplot view of quality attributes for the solution space of Figure 16;

Figure 18 is a star plot view of quality attributes for the solution space of Figure 16;

Figure 19 is a histogram view of quality attributes for the solution space of Figure 16; Figure 20 to 27 are star plot views of quality attributes for evolutions of a real life example of a complex system;

Figure 28 and 29 are histograms the values of quality attributes for evolutions of a real life example of a complex system;

Figure 30 is a scatterplot view of quality attributes for evolutions of a real life example of a complex system;

Figure 31 to 33 are star plot views of quality attributes for further evolutions of a real life example of a complex system;

Figure 34 is a scatterplot view of quality attributes for the further evolution of a real life example of a complex system;

Figure 35 and 36 are histograms the values of quality attributes for the further evolutions of a real life example of a complex system;

Figure 37 to 40 are star plot views of quality attributes for yet further evolutions of a real life example of a complex system; Figure 41 to 43 are star plot views of quality attributes for the yet further evolutions of a real life example of a complex system, including pareto fronts;

Figure 44 to 46 are histograms the values of quality attributes for the yet further evolutions of a real life example of a complex system; and

Figure 47 is a scatterplot view of quality attributes for the yet further evolution of a real life example of a complex system.

Description of a Specific Embodiment

An embodiment of the present invention provides a tool which may be used to effectively optimise complex systems by understanding the effect that changing one or more parameters of the system can have on the system as a whole. Embodiments of the tool achieve this, in part, by providing visualisation tools which allow a user to explore the effect of various changes to a system, to determine how those changes affect other performance metrics of the system.

The tool is particularly useful when optimising complex systems, as it provides a user with constant feedback. This allows the user to be confident they understand the features of the system, and hence make an educated decision based upon this knowledge. This exchange is referred to as the "dialog" between the user and the tool . In the following description, a number of terms will be utilised. The term "architecture" will be used to generally refer to the logical and physical components of a system and the relationships between them. Types of architectures that are relevant to this application include, but are not limited to; Enterprise Architectures, Computer Based System Architectures, Infrastructure Architectures, Data Architectures, Information Architectures, Application Architectures, and Organisational Architectures.

The process of making changes to an architecture to improve one or more performance characteristics (such as throughput, cost, security, reliability, etc.) will be referred to as architectural optimisation. This process may also be more generally referred to as "exploring the architecture space" . By exploring the architecture space, a set of optimal solutions may be converged upon.

Within the architecture space there is also a solution space. The solution space is the set of possible solutions (i.e. the total number of changes that can be made to the architecture) . The solution space, from a user's point of view, is tree-shaped. A user will start with an initial architecture, and then continuously perform evolutions in order to produce a subset of the solution space . It is assumed that the process can only discover a subset of the solution space, because the total solution space is considered to be infinite. However, in practical terms, only a subset of the solution space will provide acceptable solutions which meet all the functionality and performance metrics required of the system. Therefore, it is more important for the user to ascertain whether they are moving towards or away from the acceptable solution space, and not so important for the user to discover "all" solutions.

The iterative process of discovering a solution space is illustrated in Figure 1. Initially, the known solution space consists only of an initial architecture. A first set of evolutions may be applied to this initial architecture in order to produce a new set of descendent architectures. These descendent architectures can then be used as antecedent architectures for a second set of evolutions, which in turn produce a new set of - S - architectures. This process can be continued ad infinitum, until an acceptable architecture is found.

This manner of navigating the solution space is based upon evolutionary strategies. The iterative process proceeds in an arbitrary manner, and hence will only traverse an arbitrary subset of the solution space.

It will be understood that in other embodiments of the invention, more complex and robust strategies may be utilised. For example, genetic algorithms may be utilised to create new architectures.

Returning to the iterative process, the process is guided by the quality space, which shows the variation of the quality attributes over the solution space.

The process of discovering the quality space follows the process of navigating the solution space, as each architecture will have a set of quality attributes associated with it. Therefore, as each architecture in the solution space is traversed, it is possible to evaluate its quality attributes, and hence discover the quality space simultaneously.

Conceptually, if the solution space is perceived as a planar tree, the quality space may be seen as a set of quality attributes protruding orthogonally from each architecture . This is illustrated in Figure 2. The architecture space consists of both the solution space and the quality space, with the quality space attributed to each architecture in the solution space (as illustrated by the ^λbars' protruding from each architecture, which represent the architecture's quality attributes).

Evolutions can be performed on one of these architectures to produce two new architectures . The evaluation of these architectures to determine their quality attributes then allows the discovered portion of the quality space to be expanded to cover these new architectures. In this way, the discovery of the quality space closely follows the traversal of the solution space. The discovery of the quality space in concert with the traversal of the solution space provides an ever growing collection of architectures with their associated . quality attributes.

Of course, as there is an ever-growing number of possible solutions, a user must be able to discern which solution provides the best balance of attributes.

This is achieved through use of an optimisation tool, which allows a user to view in a graphical manner the suitability (or otherwise) of a particular solution against other solutions and against a number of different parameters .

In order to better describe the tool, an optimisation model will be utilised to show how an architecture is evolved and displayed using the tool . The key elements of this model are shown in Figure 3 and are described as follows:

• Initial Architecture: The optimisation starts with an initial architecture that satisfies all of the functional requirements. • Quality Requirements: The quality requirements are used as a benchmark.

• Corrector: The corrector consolidates information on the evaluated architecture into a form that provides guidance on the next step in the optimisation.

• Effector: Based upon the information received from the corrector, the effector performs evolutions to produce new candidate architectures. That is, the effector further navigates the solution space.

• Evaluator: The evaluator determines the quality attributes of the new architectures produced by the effector. That is, the evaluator discovers the quality space for the solution space navigated by the effector.

• Optimal Architectures: The outcome of this optimisation model is one or more architectures that are considered optimal. That is, they represent the best possible choices in terms of the quality requirements .

An example of the operation of this optimisation model is shown in Figure 4. Starting with an initial architecture, which is evaluated, new descendent architectures are produced by the effector. These are evaluated in turn by the evaluator, with the results of these evaluations being provided to the effector so that it can intelligently produce new architectures. A system boundary model (SBM) , describes the conceptual elements that a system is composed of. It is useful to determine what falls within the scope of the system itself, what entities the system interacts with, and what falls outside of the scope of the system. Figure 5 shows the SBM for the supporting tool.

The system has one "external" participant, the Architect. The Architect, being the central figure throughout the whole optimisation process, takes a controlling role. The system itself has the following entities:

• Solution Space: The Solution Space is navigated by the Architect . This includes providing the initial architecture and then iteratively applying evolutions to it.

• Goals: The goals represent the quality- attributes that constitute the criteria for the optimisation. These are provided by the Architect .

• Evaluator: Evaluates the architectures in the Solution Space, based upon the Goals. That is, the Evaluator discovers the quality space.

• Guidance: The results of the evaluation, which are provided to the architect using various representations .

This system boundary model reveals three user interfaces that will be required:

• Solution Space Input: Used by the architect to provide an initial architecture, and then to specify the evolutions.

• Goal Input: Used by the architect to specify the goals for the system.

• Guidance Output: Used by the architect to receive guidance from the system.

There are two key aspects to inputting the solution space. These are the representation of the architectures themselves, and performing evolutions on these architectures . The tool utilises ABACUS™, an application for evolving architectures which has been developed by Avolution Pty Ltd. Aspects of ABACUS™ are the subject of an earlier PCT Patent Application PCT/AU03/00979, and general information on the ABACUS™ product may be found at the Avolution website, http : //www. avolution. com.au.

Figure 5 shows the interface that allows for the input of the solution space. It is a simple tree view that shows the hierarchy of architectures, and the evolutions through which they were formed. Creating a new descendent architecture is simply a matter of selecting the antecedent architecture, and then evolving it. A new evolution and architecture are created, with the new architecture able to be modified to represent the changes that have come about through the evolution.

When ABACUS evaluates an architecture, the results of these evaluations are recorded as properties of the architectural elements, and of the architecture itself. The properties that are recorded on the architectures themselves may be considered the quality attributes of that architecture. Therefore, the quality goals can be defined as the values of the architecture properties, and the tool allows the architect to specify goals for the optimisation in this manner.

Additionally, the tool allows the architect to specify best-possible, worst-possible and required values for each goal .

There are two important facets to providing guidance to the architect. Firstly, there is the matter of the actual data that is to be provided. Secondly, this data must be presented in a manner that is understandable.

The guidance data provided to the architect is based upon the quality attributes of the architectures. That is, the guidance data must communicate the features of the architecture space to the architect .

The criteria used in architectural optimisation are the quality attributes of the architectures. These are the most basic criteria that can be used to compare various architectures.

In order to allow for ease of comparison of different quality attributes, the quality attributes will be normalised using a spectrum analysis. This requires that the architect specify best- and worst- possible values for each Goal so that the actual value will be represented between 0 and 100, with a value of 0 indicating the worst-possible value and a value of 100 indicating the best-possible value.

In addition to quality attributes, a Pareto-dominance ranking can be used in a similar way to other qualities as a criterion for the optimisation. The ranking method consists of finding all the non-dominated solutions in the population, and assigning these a rank of zero. A new set of non-dominated solutions is then found, which ignores all solutions that have already been assigned a rank of zero. This new set of non-dominated solutions is assigned a rank of one. This process continues until all of the solutions in the population have been given a rank.

The approach of Pareto-optimality leads to a related concept called the Pareto Front. The Pareto front is formed by the value of the objective vector for each non-dominated solution that has been found. The Pareto front is of interest because it shows the set of best possible solutions that have been discovered at any one time.

The Pareto front can be seen to advance towards an improved position because, as more solutions are explored, the solutions contained within the non-dominated set improve. Therefore, the Pareto front is useful as a way of gauging the progress of the optimisation. As long as the Pareto-front continues to improve, the optimisation is continuing to find better and better solutions. If the guidance data is to be useful to the architect, it must be presented in a manner that allows the key features of the architecture space to be quickly interpreted. However, given the large number of quality attributes, and the corresponding multitude of possible relationships between them, this task is quite a challenge .

The fact that relationships among more than three attributes will have to be considered presents a problem in terms of visualisation, because humans are only capable of visualising in three spatial dimensions.

Therefore, the visualisations made available by the tool are : • Architecture Space Visualisations;

• Histogram Views;

• Scatterplot Views ; and

• Star Plot Views (also known as a "trade-off diagram" ) . The Architecture Space Visualisation shows a three-dimensional rendering of the architecture space. It allows the architect to view the progress of the optimisation in a format that closely matches the actual concept of architecture spaces. It is similar to the representation shown in Figure 3, in that it assumes the solution space is a planar tree, and shows the quality attributes protruding orthogonally from it. An example of this view is shown in Figure 6.

The quality attributes shown in this visualisation are customisable, and the architect also has the option of showing the Pareto-ranking of each architecture as another column in the quality space.

The Histogram View shows a series of histograms, one for each goal, which show the distribution of the values of the quality attributes for all architectures. An example is shown in Figure 8, in which the distributions of the value of three quality attributes — average response time; utilisation and cost — are shown. The horizontal axis shows the range of values for the particular quality attribute being displayed in that histogram. These have been placed into class intervals labelled from one to 10. A quality attribute with a normalised value of 0-0.1 is placed in the first class interval; those with a value of 0.1-0.2 are placed in the second class interval; and so on. The vertical axis shows the number of architectures that have a value in that range for the quality attribute being considered. For example, in Figure 7, four architectures are in the fifth class interval for average response times, indicating that these four architectures had a normalised value of 0.4-0.5 for this quality attribute.

Another feature of the histogram view is the ability to highlight a set of class intervals on one histogram in a histogram view, and subsequently highlight the corresponding values architectures on the other histograms in that histogram view. An example of this is shown in Figure 8. In this example, the poor values for average response times have been highlighted. The fact that these correspond to good values for cost indicates a trade-off among these two quality attributes.

The Scatterplot View consists of a number of scatterplots, with one plot for each possible pairing of goals. This provides the architect with a comparison of each quality attribute to all the other quality attributes. An example is shown in Figure 9.

Each point in each individual scatterplot represents a single architecture. So, in Figure 9, the top-left scatterplot shows cost on the horizontal axis, and reliability on the vertical axis, and each point shows the values of these two quality attributes for each architecture . A Star Plot View (trade-off diagram) represents an architecture as a single point within a quality space that contains an axis for each of the quality attributes. The position of the architecture's point is calculated by using a weighted-vector-sum of the architecture's quality vector sum of each quality attribute, based upon the position of the axis for that quality attribute.

Evolutions are shown on the star plot as arrows, which connect the antecedent architecture to the descendent architecture.

Figure 10 shows an example. Architectures with high reliability will be pushed towards the top-left; those with high security will be pushed towards the top-right, while inexpensive architectures will be pushed towards the bottom. Architectures with either high values for all of these quality attributes, or low values for all, will remain around the centre.

However, the position of these axes is not fixed. It is possible for the architect to change them by: • Scaling: Scaling a coordinate axis changes the sensitivity of the position of a point to its value in that dimension. • Rotation: Rotating a coordinate axis makes that dimension more or less correlated to the other dimensions.

Figure 11 shows the same star plot, but with the axes having being scaled and rotated. The same architectures and evolutions can be seen, but their orientation has changed. In this example, it is now possible to see that security and cost may be inversely related to reliability.

In general, rotation and scaling of axes is a very useful way for an architect to be able to discern relationships and patterns in the architecture space . Turning now to a worked example, an initial architecture for a hypothetical complex system is shown in Figure 12.

The architecture consists of a single component of type ^λA' , connected to the extremities of the system by two connections of type ^λPipe' . This is the architecture which is to be optimised for security and reliability.

For the purposes of this worked example, simplified models of the quality attributes to be optimised against are used. These are now described.

The reliability of an architecture is defined as the probability that there will be a complete path from the start to the end of the architecture. This requires each component and connection to be rated according to its own individual reliability, that is, the probability that it will be able to be used as part of the path. This is evaluated using ABACUS'S built-in reliability calculator.

The security of an architecture is defined as the product of the probabilities that each element in the architecture will not be compromised. That is, security is the product of one minus the probability of compromise (PoC) of each element in the architecture.

The cost of an architecture is calculated simply by adding the given cost of each element used in the architecture.

In addition to components of type ^ΛA' , other components are available. These are:

• B: Functionally identical to A, but with higher reliability than A; • C: Functionally identical to A (and B) , but with higher security than A;

• Splitter: Allows the parallelisation of components when used with a 'Joiner' , by splitting a single connection into multiple connections; and • Joiner: Allows parallelisation of components when used with a ^vSplitter', by joining multiple connections into a single connection.

As shown in Figure 13, the initial architecture (the only architecture that has evolutions directed away from it, but none directed towards it) started in a state that was relatively favourable to both Cost and Security, but not to Reliability.

This can be determined from the position of the point representing the initial architecture in the star plot. This point has been 'pulled' towards the bottom-right of the star plot, in between the Cost and Security axes. This indicates that these two quality attributes are having a greater effect on the point's position than Reliability is, and hence must be of larger magnitude. However, neither Security nor Cost had enough of an effect on the point's position to pull it away from the other, and hence the point stays approximately between these two axes. This indicates that these quality attributes are roughly of the same magnitude.

It should be noted that the 'magnitude' of the quality attributes is affected by the normalisation process, and in particular, the values that have been chosen as best-possible and worst-possible for the quality attributes. This means quality attributes that have normalised values that are of "equal' magnitude could more accurately be said to be at the same position relative to their own best-possible and worst-possible values.

Figure 13 then shows that Reliability can be improved to the detriment of both Security and Cost, corresponding to the introduction of parallelism. Two such evolutions have been attempted, and in both cases, they move in the direction of the Reliability axis. This indicates that Reliability has increased, and hence is ^λpulling' the architectures in that direction. However, the evolution is also moving in such a direction that it is also moving away from the Security and Cost axes, indicating that these quality attributes are becoming relatively worse.

Alternatively, Reliability can be improved through the introduction of more reliable components. This evolution is approximately perpendicular to the Security axis, indicating that Security is not affected by this evolution. However, it has still moved away from the Cost axis, indicating that this solution is more expensive.

Finally, Security can be improved without affecting Reliability, but with less favourable cost by replacing components with more secure components. In this case, the evolution is approximately perpendicular to Reliability, but moves away from the Cost axis.

Note that these particular evolutions that use more reliable or secure components do not affect Security and Reliability respectively because only components of type ^λA' have been replaced; had components of types ^λB' and ^λC been replaced, these quality attributes would have become less favourable . Alternatively, Security can be improved without affecting Reliability, but with less favourable cost by replacing components with more secure components. Figure 14 shows that Reliability can also be improved without affecting Security by replacing components with more reliable components, but only at greater expense, that is, with less favourable cost. However, it is important to note the fact that these particular Extensibility evolutions do not affect Reliability and Security respectively because only components of type ^λA' have been replaced; had components of types ^λB' and "C been replaced, these quality attributes show the continued use of these evolutions. One interesting observation is that the earlier evolutions were larger in magnitude, that is, longer on the star plot, but successively became smaller in magnitude, that is, shorter on star plot. This indicates that the architecture is exhibiting a form of convergence, in that it reaches a point where later evolutions have a smaller effect on the quality attributes.

This pattern can be seen to continue in Figure 15, as the evolutions continue to be applied.

An observation that can be made at this point of note is that the same types of evolutions are approximately parallel. Parallel evolutions indicate that the changes in quality attributes (the quality attribute deltas) brought about by these evolutions are proportionally the same. That is, each of the quality attributes change by the same amount relative to each other.

Therefore, the fact that the same types of evolutions are parallel to each other would indicate that these evolutions are ^Λ commutative' , in that they appear to be unaffected by the order in which they are applied. That is, there is a high confidence on the effects of each type of evolution.

Figure 16 shows the solution space for this optimisation. It demonstrates the iterative nature of performing evolutions, and the way it generates new generations of architectures.

This process of optimisation has revealed the relationships among the quality attributes that were explored. In addition, the scatterplot view and histogram view also provide interesting information. In general, the information provided by these views describes information on the relationships among quality attributes from a more general perspective. Relationships between each pair of quality attributes are shown in the scatterplot view, in Figure 17.

There is a trade-off between Reliability and Security, as well as a trade-off between Reliability and Cost. However, Security and Cost are positively correlated, indicating that it is actually cheaper to get better Security. This result, which may be somewhat counter-intuitive, is a result of the security model being used to evaluate the architectures. This model assumes that security is inversely proportional to the number of elements in the system. That is, the more elements that are in the system, the worse security will be. However, more elements in the system will drive the costs up, and hence Security and Cost are positively correlated. This information is also shown on a star plot (Figure 18) . The evolutions have not been shown for the purposes of clarity. Additionally, the axes have been rotated and scaled such that Security and Cost are in approximately the same position, and Reliability is perpendicular to them. This makes the star plot into something resembling a 2D plot, with the sum of Security and Cost being the horizontal axis, and Reliability being the vertical axis. With the star plot oriented in this manner, it can be clearly seen that a trade-off exists between Reliability and both Security and Cost. Information on the spread of values for each quality attribute can be seen in the Histogram View, as shown in Figure 19.

This Histogram View demonstrates that Reliability converges onto a small range of values, indicating that^' Reliability quickly becomes insensitive to changes. Conversely, both Security and Cost exhibit a wide spread of values, indicating that they remain quite sensitive to changes. Once again, it should be noted that this spread is affected by the best-possible and worst-possible values that have been selected for each quality attribute.

Choosing values that are too far apart will result in the histogram appearing very narrow, as the variation in actual values will be small in comparison to the range being considered. However, in the case of Reliability, this is not the case, as there is actually a spread in the values, but most of them have converged.

This worked example demonstrates the utility of the optimisation tool. It has shown that by proposing an initial architecture, and then subjecting this to a variety of evolutions, a great range of candidate architectures can be generated, hence navigating the solution space. These architectures are evaluated to discover the associated quality space. This process is then visualised to allow the explored architecture space to be analysed, demonstrating its salient features.

While the worked example is useful to demonstrate the functionality of the tool in performing an exploratory architectural optimisation, it was based on an artificial model. A practical study is also provided, to determine whether optimisation utilising the tool is useful in a "real-world" system.

The system used for this case study is the system provided by Avolution Pty Ltd to demonstrate ABACUS. It is constructed around a generic enterprise called "ABC Ltd" , that should be representative of many financial services, fast-moving consumer goods, logistics companies and government organisations. Rather than simply being a model of a computer-based system in itself, this model includes all aspects of the organisation, including people, processes and technology. That is, it is an enterprise architecture.

The composition of the example enterprise is described as :

^wJt is comprised organisationally of a. head office, various state/regional offices, several branches or stores and an outsourcer site. These sites are running various applications across a multitude of services and are executing a multitude of business processes for a variety of customers".

The objective of the optimisation to be carried out on the ABC Ltd enterprise architecture is to reduce the cost of the system without sacrificing performance. This involves lowering the projected cost of the system over five (5) years while tracking a range of performance indicators for the system to ensure no significant degradation occurs. The expected outcome of this optimisation is a set of optimal solutions that quantify the trade-off between cost and performance for the ABC Ltd enterprise.

Note that there were many other quality attributes that could have been included in this optimisation. The in-built ABACUS evaluators include another three besides cost and performance, and this does not include the custom evaluators that can be used by ABACUS. However, in the interests of simplicity, this optimisation will only consider performance and cost .

In order to ensure that all aspects of the system's performance are considered, this optimisation will track a wide variety of measures. The performance measures monitored are :

• Bandwidth: The total amount of data processed by the system.

• Frequency: The total amount of offered traffic throughout the system, for all connections.

• Response (Ave) : The sum of the average response time for each connection in the system.

• Response (Min) : The sum of the minimum response time for each connection in the system. • Response (Max) : The sum of the maximum response time for each connection in the system.

• Throughput: The total amount of offered traffic actually processed by the system, for all connections . • Utilisation: The sum of the utilisation of each component in the system. This includes components that represent computer systems, as well as components that represent departments within the organisation. In addition to the cost of the system over five (5) years, these measures comprise the goals for the system.

The Star plot in Figure 20 shows the initial location of the architecture in the Quality Space. It can be seen that quality attributes of the architecture are biased towards the performance attributes, and away from cost.

Figure 21 shows this same star plot with the addition of "sticks" . These sticks show relative magnitude of each quality attribute for each architecture.

As seen in Figure 21, the initial architecture is deficient in terms of cost. That is, the architecture is too expensive based upon the given best and worst values. This is the reason why the architecture is biased towards performance, and away from cost, in the star plot. It is already known, based upon previous knowledge of ABC Ltd' s enterprise architecture, that an opportunity- exists for server consolidation. That is, the migration of applications existing on two or more servers onto a single server. Therefore, this is used as the first evolution in the optimisation. The result of this optimisation is shown in Figure 22.

This evolution has improved performance. This result may be counter-intuitive, but it is explained by the fact that there is now less blocking in the architecture due to an application being moved from a highly utilised server to a server that has less utilisation.

However, the direction of the evolution in the star plot is approximately perpendicular to the Cost axis. This indicates that cost did not change significantly in this evolution.

Given the good results of previous evolution, further server consolidation is attempted at other points within the architecture. This consolidation uses the heuristic of migrating applications to servers with low utilisation figures, rather than randomly choosing applications to migrate. Essentially, the architect uses local characteristics of the architecture to affect global ones. This observation carries important implications for the concept of optimisation. It shows that in addition to using data on the global properties of the architecture as guidance on what evolution to make, an architect will also find it useful to use local properties. This fact must be taken into account by any methodology and supporting tool, as it is clearly an important reflection on how architects design an architecture.

The results of this second server consolidation are shown in Figure 23. In this situation, the star plot is somewhat ambiguous, as the architecture has moved away from performance goals, and towards the cost goal. This may be due to either improving cost or degradation in performance. However, inspection of the actual change in quality attributes for the evolution shows that the latter is true.

A star plot which places the performance goals orthogonally to the cost goal, as exhibited in Figure 24, can be used to show that there has been a degradation of performance accompanied by a slight improvement in cost.

A third evolution is made in which another consolidation of servers occurs, the results of which are displayed in Figure 25 and 26. This evolution has had- the exact same effect as previous evolution, as exhibited by the fact that the two evolutions are parallel. However, this evolution is more pronounced than the previous one, indicating that it has changed the quality attributes by a greater magnitude. The last two evolutions made performance worse without significantly improving cost. Clearly, the tactic of server consolidation by itself is not adequate to meet the requirements of this optimisation.

Another heuristic to reduce cost is to downsize the headcount in departments that have low utilisation. Note that this evolution involves the human resources of the organisation, rather than the technical resources.

However, because the ABC Ltd model is comprehensive enough to include the entire organisation, and ABACUS is very flexible in the way it models architecture, it is still possible to enact this change. In effect, both servers and departments are considered as a type of component capable of doing processing. Starting from the descendent architecture of the first evolution, which was the last evolution to have any- significant benefit, an evolution is performed based upon this heuristic. The result of this evolution is shown in Figure 27.

This evolution has had a significant positive effect on cost, but has sacrificed performance to achieve it. Enough information is now available to begin examining broader trends to be found in the optimisation. It has been found that a better way to improve cost without degrading performance is to downsize departments, rather than consolidating servers. This is because:

• removing servers does not save as much money,- and • removing servers has a significant adverse affect on response times .

Other useful information can be gained from the other views available from the tool . The Histogram View shows that Bandwidth, Frequency and Throughput are insensitive to the changes being made, as shown in Figure 28.

This contrasts with the other goals, which display significant variation, as shown in Figure 29.

The Scatterplot View also provides information on the relationships among response time measures. It shows that all of the response time measures, that is, maximum, minimum and average response times, are strongly- correlated with each other. This can be clearly seen in Figure 30.

This information allows us to remove some goals from star plot, as they do not provide any additional useful information. The important goals to track are Utilisation, Average Response Time, and Cost, and these are the goals that will be monitored hereafter. Given the previous evolution's success at improving the cost of the system by downsizing the headcount in departments, this evolution is attempted again at a different point in the architecture, with the results shown in Figure 31.

Once again, this evolution has had a significant positive effect on cost, but has degraded performance. At this stage, it is decided that cost has been reduced to an acceptable level, so now an attempt is made to improve performance without significantly reversing the cost improvements that have been achieved thus far.

A particular department has been found that is highly utilised, but is not one of those that has just been downsized. Therefore, an evolution is tried that adds two people to this department, as shown in Figure 32.

As expected, the addition of these people has improved performance, but makes the enterprise system cost more. Additionally, an evolution is tried that adds only one person to the overloaded department, as shown in Figure 33.

Once again, this evolution gives better performance, but is more expensive. However, the magnitude of the change is smaller than it was for the previous evolution. This is intuitive, as only one person has been added, as compared to adding two people in the previous evolution.

Once again, the Scatterplot View shows some interesting relationships among the various goals, as shown in Figure 34.

Firstly, there is a very clear trade off between Utilisation and Cost. This validates the heuristic of focusing on components with high and low utilisation when trying to improve performance and cost respectively. Additionally, the relationship between average response time and cost contains two groupings. These correspond to the two different types of evolution attempted, which either targeted the enterprise's technology or the departments within the enterprise. Neither of these two groupings exhibits any significant correlation.

The Histogram View also provides some interesting information. The histograms for response times, shown in Figure 35, show that response times congregate around certain values. This indicates that the response times are not highly sensitive to the changes that have been made for this optimisation.

The histogram for Cost, shown in Figure 36, exhibits twin peaks—one for the technology-based evolutions, which have limited use in reducing overall cost, and one for the people-based evolutions, which are more useful in reducing cost ,

Until now, the Pareto front has been of little use, because there are too many goals resulting in all of the architectures being part of the Pareto front. Therefore, the number of goals is reduced to the three that have been found most useful so far: Average Response Time, Utilisation and Cost.

This provides the Pareto front shown in Figure 37. The trade-off between cost and performance is clear on this Star Plot. At this point, since it will be attempted to reduce costs even further, the best possible value for Cost must be made lower. However, this will slightly modify the views from the way they appeared before. One possible significant evolution to reduce cost involves changing the entire delivery model for the enterprise by removing the State Offices, and merging any of their functions back into the Strategy and Operations department at Head Office. This reorganises the entire architecture with a major change to its topology.

This appears to be an attractive option because the State Offices are only utilised at 5%. However, it must be ensured that this is a valid evolution for the enterprise, which still allows it to meet its required functionality, including business and regulatory requirements. Assuming that this is a valid evolution, the results are shown in Figure 38. Mot surprisingly, this evolution has resulted in a very large improvement in cost, but with a significant degradation in performance.

The degradation in performance in the previous evolution is due to the Strategy and Operations department, which took on the functions of the State

Offices, now being almost fully utilised. Therefore, one more person is added to this department, with the results shown in Figure 39.

This evolution has resulted in significantly enhanced performance, with little change in cost. The elimination of the State Offices means that the servers that used to be part of those offices can be consolidated, as they are no longer in geographically separate locations. This consolidation is first attempted by merging the two servers into one. This is shown in Figure 40.

This has resulted in lesser performance, with little improvement in cost. Therefore, another approach should be used.

Another way of consolidating these servers is via the migration of their applications onto a server at Head

Office that is only being utilised at 5%. The results of this evolution are shown in Figure 41.

This evolution has resulted in a significant - 33 -

The second reachable Quality Space was achieved through evolutions that downsized the headcounts in departments in the organisation. Allowing these types of evolutions stretched the reachable Quality Space out in the direction of improving cost. This is due to the fact that, in this particular enterprise, of the oft touted management consultant mantra that "labour is far more costly to the organisation than technology" . However, allowing downsizing of headcounts in departments significantly reduced performance when compared to evolutions that consolidated servers.

The third and final reachable Quality Space was achieved by allowing the radical evolution of eliminating state offices. This stretched out the reachable Quality Space even further in the direction of improving cost. However, it actually improved the performance of the organisation .

Therefore, this case study has shown that the subset of the Quality Space that is reachable by a certain architecture is dependent upon the types of evolutions that are allowed. To continue to enlarge the reachable Quality Space, it is necessary to use evolutions that are more 'radical' in nature. This was illustrated by the fact that the relatively minor evolution of consolidating servers only produced a small reachable Quality Space when compared with the space that was reachable when using the far more radical evolution of eliminating state offices.

This case study has also illustrated the sensitivity of quality attributes to the various types of evolutions that have been attempted. Cost has been shown to be only marginally sensitive to the consolidation of servers, with only slight improvements been made with these evolutions.

However, with evolutions that downsized headcounts in - 32 - performance gain, and thus is far more beneficial than the previous evolution.

The elimination of State Offices has lead to a Pareto front that is considerably more attractive than what was left after Evolution 7. In effect, the Pareto front has been "pushed out" . This has been illustrated in Figure 42, where the Pareto front after server consolidation, department downsizing and the removal of state offices has been shown, in addition to the final non-dominated set, which is highlighted. Although it is possible that such an evolution is not permissible for the system in real life, this optimisation has shown that it is at least worthwhile considering.

Clearly, the subset of the Quality Space that is reachable depends upon what evolutions are used. In this case study, three distinct reachable Quality Spaces are found based on the three types of evolutions that are allowed. The use of additional evolution types served to increase the size of the reachable Quality Space. As shown in Figure 43, this closely resembles the advance of the Pareto front .

The first space is reachable simply by using evolutions that consolidated servers. In terms of cost, this space was relatively deficient. This was because the cost benefit of consolidating servers is small compared to the other types of evolution. However, in terms of the performance characteristics of average response times and utilisation, this space was the best, containing the architecture on the Pareto front that exhibiting exhibits the best performance qualities. This is because server consolidation evolutions did not have a large effect in reducing performance compared to the other evolution types . departments, and subsequent evolutions that removed state offices, costs made significant improvements. These improvements are the reason for the three distinct reachable Quality Spaces that were formed. These three distinct spaces can be clearly seen in a histogram of Cost as three distinct peaks, as shown in Figure 44.

The performance quality attributes showed rather different characteristics to cost. Response times never showed any significant improvement from that of the initial architecture. However, some evolutions resulted in a drastic degradation of response times. This is shown in a histogram of response times, which has a hard upper limit, and a tail that reaches the worst possible values. This is shown in Figure 45. This indicates that the evolutions allowed in this optimisation do not significantly improve response times, but carry the possibility of severely worsening them. Utilisation, on the other hand, exhibited a hard lower limit. This is due to utilisation being limited by the number of processing components present in the architecture. This can be seen in a histogram of utilisation, as shown in Figure 46.

When these quality attributes are viewed together in a Scatterplot View, as shown in Figure 47, it can be seen that the performance attributes vary largely independently of cost.

This case study demonstrates the practicality of the exploratory architecture optimisation method, as well as the supporting tool. Using the guidance provided by the tool, it was possible to find a solution that met the goals of the optimisation, that is, to reduce costs while maintaining performance. The guidance provided by the tool was complemented with local characteristics of the architecture, such as the utilisation of specific components, demonstrating that such local characteristics are often used to affect global ones.

In addition to finding a set of optimal solutions, the optimisation process revealed many characteristics of the architecture space. This included the relationships of the quality attributes to each other, and the effects that various types of evolutions had on the quality attributes. This information provides the first steps in building a knowledge base that can be used for further guidance in future optimisations.

This is made possible through the synergistic interaction between the ABACUS product and the optimisation method and tool described herein. Utilisation of the visualisation tool allows the architect to instantly see the effect a change in one aspect of a system has on the system as a whole (or on a set of parameters that quantify the performance of the system) . This allows for sensible and practical evolution of the architecture, as the architect is able to make informed decisions on whether the change is warranted and desirable, which in turn allows for the implementation of informed and credible system changes .

That is, a tool in accordance with an embodiment of the present invention provides a large amount of information in a succinct and understandable manner, which in turn leads to informed decision making.

Claims

CLAIMS :

1. A method of optimising a complex system, comprising the steps of, representing at least one quality attribute of the complex system in a graphical format, altering the complex system, and representing the at least one quality attribute of the altered complex system in a graphical format, wherein the at least one quality attributes may be compared.

2. A method in accordance with Claim 1, comprising the further step of displaying the at least one quality attributes on a common graph, wherein the relative variation in performance between the initial complex system and the altered complex system may be identified.

3. A method in accordance with Claim 1 or Claim 2, comprising the further step of calculating the new value of the quality attribute for the altered complex system.

4. A method in accordance with Claim 2 or Claim 3 , comprising the further step of representing at least one quality attribute for a plurality of complex systems on the common graph.

5. A method in accordance with Claim 4, comprising the further step of iteratively altering the complex system to produce the plurality of altered systems.

6. A method in accordance with Claim 5, comprising the further step of calculating the value of the quality attribute for each of the plurality of altered systems .

7. A method in accordance with Claim 4 , Claim 5 or Claim 6, comprising the further step of providing at least one indicator on the single graph, whereby, in use, the evolution of each of the complex systems may be compared.

8. A method in accordance with Claim 7, comprising the further step of defining a quality space within the graphical format, the quality space defining an area of desirable values for the at least one quality attribute.

9. A method in accordance with any one of the preceding claims, comprising the further step of plotting the common graph as a star plot .

10. A method in accordance with Claim 7, Claim 8 and

Claim 9 when dependent on Claim 7 or 8, whereby the at least one indicator is a vector.

11. A method in accordance with Claim 1, comprising the further step of selecting the graphical format is one of an architecture space visualisation, a histogram or a scatterplot .

12. A method in accordance with any one of the preceding claims, wherein the complex system is modelled as an architecture.

13. A method in accordance with Claim 12, wherein the architecture is one of a Enterprise Architecture, Computer Based System Architecture, Infrastructure Architecture, Data Architecture, Information Architecture, Application Architecture, and an Organisational Architecture.

14. A method in accordance with any one of the preceding claims, where the complex system includes a computing system.

15. A method in accordance with any one of the preceding claims, where the complex system is an organisation.

16. A system for visualising changes to a complex system, comprising means to receive information pertaining to at least one quality attribute of the complex system, means to receive information pertaining to the at least one quality attribute of an altered complex system, and means to graphically represent the at least one quality attribute for the complex system and the altered complex system, wherein the graphical means displays the relative variation in performance between the complex system and the altered complex system.

17. A system for optimising a complex system, comprising, means for receiving and representing at least one quality attribute of the complex system in a graphical format, means for receiving and representing the at least one quality attribute of the altered complex system in a graphical format, wherein the at least one quality attributes may be compared.

18. A system in accordance with Claim 17, wherein the at least one quality attributes is displayed on a common graph, such that the relative variation in performance between the initial complex system and the altered complex system may be identified.

19. A system in accordance with Claim 17 or Claim 18, comprising means to calculate the new value of the quality attribute for the altered complex system.

20. A system in accordance with Claim 18 or Claim 19, wherein the quality attributes for a plurality of complex systems are displayed on the common graph.

21. A system in accordance with Claim 20, comprising means to iteratively alter the complex system to produce the plurality of altered systems.

22. A system in accordance with Claim 21, comprising calculating means to calculate the value of the quality attribute for each of the plurality of altered systems .

23. A system in accordance with Claim 20, Claim 21 or Claim 22, comprising means to display at least one indicator on the common graph, wherein the evolution of each of the complex systems may be compared.

24. A system in accordance with Claim 23, comprising means to define a quality space within the graphical format, the quality space defining an area of desirable values for the at least one quality attribute .

25. A system in accordance with any one of the preceding claims, wherein the common graph is a star plot.

26. A system in accordance with Claim 23, Claim 24 and Claim 25 when dependent on Claim 23 or 24, wherein the at least one indicator is a vector.

27. A system in accordance with Claim 17, wherein the graphical format is one of an architecture space visualisation, a histogram or a scatterplot.

28. A system in accordance with any one of the preceding claims, wherein the complex system is modelled as an architecture .

29. A system in accordance with Claim 28, wherein the architecture is one of a Enterprise Architecture, Computer Based System Architecture, Infrastructure Architecture, Data Architecture, Information Architecture, Application Architecture, and an Organisational Architecture.

30. A system in accordance with any one of the preceding claims, wherein the complex system includes a computing system.

31. A system in accordance with any one of the preceding claims, wherein the complex system is an organisation .

32. A computer program arranged to, when executed on a computing system, carry out the method steps of any one of Claims 1 to 15.

33. A computer medium incorporating a computer program in accordance with Claim 32.