WO2008132063A2

WO2008132063A2 - Method for the quantitative evaluation of modifications in a software system and the effects thereof

Info

Publication number: WO2008132063A2
Application number: PCT/EP2008/054675
Authority: WO
Inventors: Jürgen SALECKER
Original assignee: Siemens Aktiengesellschaft
Priority date: 2007-04-25
Filing date: 2008-04-17
Publication date: 2008-11-06
Also published as: DE102007026226A1; WO2008132063A3

Abstract

According to the invention, a classification of all modifications is carried out in a simple and very reliable manner, according to the criterion of the impact of the modification on the system. No information is required about the project structure, only very limited information about the software architecture. To this end, cross-impact factors are calculated for each modification, and then represented in a list created, for example, according to the variable. A test section receives very clear indications as to the order in which the tests should be advantageously carried out, and qualitative information, such as severity levels inter alia, can be take into account.

Description

description

Method for the quantitative assessment of changes in a software system and their effects

The invention relates to a method for the quantitative assessment of changes in a software system and their effects, in which a classification of the changes takes place.

In software development, there are many changes in very short periods of time. A test department now has to decide what should be intensively, or less intensively tested. Due to lack of time, most tests can not be completed. Therefore, the test department needs clear information which areas of a software product should be tested more intensively. This becomes even more important when the development department reports that a certain number of bugs have been fixed and it is up to you to decide which bug fixes should be tested first and how hard they should be tested.

According to the current state of the art, every error is assigned a so-called "Severity Level", that is, based on qualitative statements made by software architects,

Customers, or project managers made a statement about the possible severity, and therefore also about the importance of the error. However, this level does not say anything about the possible difficulties that arise in eliminating the error. Fatal errors can and often even have a very simple cause, once the solution is very often trivial or so-called lightweight errors, however, can result in a complex error correction. With every error correction, or actually with any change in a software system, there is a risk that unintentional new errors will be incorporated. In the current state of the art is trying - usually more bad than right - the respective error correction or similar changes with software quality methods such as "pair debugging", "code reviews" and "pair coding" to verify. However, these methods lead to satisfactory results only with great effort. In many cases, however, these methods are not completely feasible due to lack of time, which in turn means that new errors are often unintentionally incorporated into the system.

Another change class is e.g. the implementations of system requirements. Here is the challenge as early as possible to recognize which - even partially - implemented system requirements mean the greatest risk in terms of project success or failure.

The object underlying the application is now to provide a simple but most promising method for the quantitative assessment of changes in a software system and their effects, in which the abovementioned are completely or largely avoided.

This object is achieved by the features of claim 1. The features of the subclaims relate to advantageous embodiments of the invention and advantageous uses.

The invention will be explained in more detail with reference to an embodiment shown in the drawing.

Essentially, the invention consists in a simple and very reliable way of classifying all changes according to the criterion of their influence on the system. No information about the project structure is needed, only very limited information about the software architecture. For this purpose, the respective change impact calculated factors (cross-impact-factors) and then presented in a, for example, by the size falling list. This gives a test department very clear indications in which order the tests should advantageously be carried out, but also qualitative statements such as severity levels or the like can be taken into account. The necessary values for the formula can easily be extracted from the data of the configuration management system used. A prerequisite is that the IDs for the error flag, "Feature" IDs and similar IDs can be defined using a regular expression.

The invention calculates a change impact factor CIF (cross-impact factor) for each change-which can be identified by means of an ID, and then provides, for each ID type, a ranking of the respective changes, sorted according to falling CIF. That As many ranking lists as there are different ID types are calculated.

A change in this context is either a bug fix, a new feature, or any other identifiable change.

The calculated change impact factor CIF (cross-impact factor) indicates in a simple numerical way the impact of the change on the whole system. The higher (numerical) the factor, the more influential the changes were to the software system, which will inevitably increase the likelihood of introducing new ones. The change impact factor CIF (cross-impact-factor) is calculated according to the following formula:

n = number of changes in which the same ID (error, feature ID, or similar) was detected. c = number of different software components that have changed with respect to this change d = number of different developers who were involved in this change

This classifies the error eliminations according to their actual influence on the software system.

A further embodiment provides for inclusion of further influencing variables in the formula. Advantageous are influencing variables, such as: s = number of different development sites that contributed to this change a = number of different development departments that contributed to this change.

In a usage example is not the

Change impact factor CIF for each request ID at a given time, but considered the change in the cross-impact factor CIF for each request ID integrated over time or summed. Shows the integration of the cross-impact factor CIF for a specific request ID, e.g. monotonically increasing values, this is considered according to the invention as an indication of a particularly risky requirement. If, for a requirement, the corresponding implementations always have a relatively large impact, there is a risk that, firstly, new errors will be built into the system, and second, that the requirement may need to be re-analyzed, e.g. In terms of technical uniqueness, i. whether this requirement is defined in such a way that the implementation into an implementation can really take place and in relation to its real necessity.

If all changes are classified according to their influence then the usually limited test resources can be used much more purposefully. For fault eliminations with a high change impact factor CIF (Cross-impact-factor), the test effort should advantageously be more extensive, ie not only the elimination of the actual error should be considered, but it should be tested so that possible side effects can be detected. Again, for error eliminations with a very low change impact factor CIF (cross-impact-factor), the test effort may be limited to the scenario of the error and need not be extended further. Thus, a test appropriate to the "impact" of the change is achieved, and that exactly fits the particular change in the software system.

In the drawing, the structure of a software package is shown by way of example, wherein the software package levels CIF-Ll, ... CIFL4, wherein the level CIF-Ll the entire software package SP, the level CIF-L2 immediately underlying components Cl and C2, the level CIF-L3 subcomponents SCl and SC2 and the lowest level CIF-L4 modules MIl, .. M23 includes. The change impact factor CIF (cross-impact-factor) is calculated along the software architecture, ie typically at all levels, for each of these components and for the entire software package.

These calculations are calculated with each new baseline. As a baseline here in this context is meant the list of all necessary artifacts with their versions for a software system at any given time. A baseline is usually calculated by configuration management and provided to the development department.

The frequency of the calculation, or when there is a new one

Baseline is strongly dependent on the culture of development and also the phase in which the product development is located and means, for example, daily or weekly.

With the help of these calculated change impact factor CIF ranking lists results a very finely graduated however also at the same time very clear presentation which change had which influence in the software. For each unique ID, such as debugging ID, "feature" ID, or similar, a separate CIF is calculated, but then re-merged into a global change impact factor CIF, which includes all IDs. Depending on the maturity state of the product development, it is possible either to concentrate on specific IDs with their change impact factor CIF or to observe more the global change impact factor CIF.

Further, for each calculated change impact factor CIF, the input data, i. the names of the components, and the names developers for any more detailed analysis available. All of this information is calculated automatically from the data that usually provides a configuration management system. The only prerequisite is that every developer notes the changes related to an ID, an additional feature, error, or similar, using the configuration management system.

In this context, a change does not mean every file that has been modified for error correction, but a change can contain one or even several files. No matter how many files are needed for a developer to make a change, this change is considered a singular action, which is expressed by the parameter n in the formula.

By way of example, the following table shows a section from a typical "development baseline" of CIF ranking lists:

The table shows the respective change impact factors of three different IDs "Bugzilla", "ClearQuest" and the "Feature DB" In addition, the global change impact factor CIF (cross-impact-factor) is also shown in line 1. The numbers in the " Summary "Rows are not simple mathematical sums of values below These numbers represent the number of different" Changes "," Number of Developers "and" Number of Components "collected in this database It is very easy to see this in the above example, the focus of the changes is on the IDs captured in the "Bugzilla" database. Next is also to see clearly that the IDs in lines 3 to 6 by far the highest change impact factors. Which gives the test team clear indications regarding the test scopes to be performed.

Claims

claims

1. A method for the quantitative assessment of changes in a software system (SP) and their effects, - in which for a change with a particular change ID each a change impact factor CIF is calculated according to the formula CIF = n * c * d, where n is the number of changes in which the same change ID is detected, c is the number of different software components that have been changed with respect to this change, and d is the number of different developers involved in this change and at least a ranked list of changes sorted by the change impact factors CIF is created.

2. The method of claim 1, wherein for a change with a particular change ID each a change impact factor CIF is calculated according to the extended formula CIF = n * c * d * s * a, where s is the number of different development sites is the and a is the number of different development departments involved in this change.

3. The method of claim 1 or 2, wherein the m change impact factor CIF along the software architecture for all components (MIl, .. M23, SCl, SC2, Cl, C2) and finally for the entire software package (SP) is calculated ,

4. A method according to any one of the preceding claims, wherein a change is either debugging or introducing an additional feature.

5. Use of the method according to one of the preceding claims, in which the change in the change impact factor CIF for each request ID is integrated or summed over time and identified by a steady increase in this value particularly critical request IDs.

6. Use of the method according to one of the preceding claims, in which the test effort for the respective changes according to the respective change impact factor CIF is carried out, wherein changes with a high change impact factor are tested more extensively than changes with a low change impact factor.