EP1483704A2

EP1483704A2 - Method and system for the automatic planning of experiments

Info

Publication number: EP1483704A2
Application number: EP03704635A
Authority: EP
Inventors: Rolf Burghaus; Georg Mogk; Thomas Mrziglod; Peter HÜBL
Original assignee: Bayer AG
Current assignee: Bayer Intellectual Property GmbH
Priority date: 2002-03-01
Filing date: 2003-02-17
Publication date: 2004-12-08
Also published as: DE10209146A1; RU2004129323A; CA2477768A1; US20030237058A1; WO2003075169A2; CN1802657A; WO2003075169A8; NZ534988A; NO20044172L; US7079965B2; JP2005519394A; AU2003206912A1; IL163845A0

Abstract

The invention relates to a method for automatically planning experiments, comprising the following steps: a degree of similarity between two experiments is entered; an evaluation criterion for an individual experiment is entered; a quality parameter is determined based on the degree of similarity and the evaluation criterion; and a number of experiments are detected in which the quality parameter takes on an extreme value.

Description

Method and system for the automatic planning of experiments

The invention relates to a method for the automatic planning of experiments and a corresponding computer program product and a system for the automatic planning of experiments.

Various methods for planning experiments and trials are known from the prior art. The terms "experiment" and "experiment" are used synonymously below.

It is known from the prior art to plan tests using statistical test planning methods. Such planning methods are used, among other things, to determine an empirical process model for the relationship between the control and disturbance variables in a process and the resulting product and process properties with a minimum number of tests. Such statistical test planning can be carried out, for example, with the aid of the computer program “STAVEX” (STATISTICAL EXPERIMENTAL PLANNING WITH EXPERT SYSTEM). Another commercially available computer program for the test planning is the program “Statistica®”, StatSoft (Europe) GmbH.

In the field of statistical test planning, different types of test planning are distinguished in the prior art. In particular, a distinction is made between the classic, full-factorial method and modern Taguchi or Shainin methods.

The classic, fully factorial method is the origin of all statistical test planning methods. It is based on a comparison of all quality-related factors with one another based on the variance analysis. Numerous variants have been developed over the past decades and validated in research and development laboratories. For cost reasons, however, the modern Taguchi and Shainin processes are predominantly used.

The Shainin-DOE (DOE = Design of Experiment) is considered a suitable method for process optimization because it isolates so-called "strong" influencing variables and examines them for relevance and dependency.

The Taguchi DOE is based on previous, partial factorial, orthogonal test plans. Because of the drastic savings in trial runs by preselecting the most important influencing factors, this is a quick and relatively economical procedure for trial and process planning.

Other known statistical types of test planning are partial factorial test plans ("fractional factorial"), Plackett-Burmann test plans, central composite plans ("central composite"), box-bound test plans, D-optimal plans ("D-optimal designs"), Mixing plans, balanced block plans, Latin squares, Desperado plans (see also http://www.versuchsplanung.de).

Further test planning procedures are known from Hans Bendemer, “Optimal

Test planning ", series of German paperbacks (DTB, volume 23, and ISBN 3- 87144-278-X) and Wilhelm Kleppmann, paperback test planning," Optimize products and processes "2nd, extended edition, ISBN: 3-446-21615-4 ,

Furthermore, a method for controlling a manufacturing process via an efficient experimental design is known from US Pat. No. 6,009,379. Test points are evenly distributed on a multidimensional spherical surface in order to weight the individual manufacturing parameters evenly. The invention is based on the object of creating an improved method for the automatic planning of experiments and a corresponding computer program product and system.

The object on which the invention is based is achieved in each case with the features of the independent patent claims. Preferred embodiments of the invention are specified in the dependent claims.

The present invention makes it possible to carry out test planning for a predetermined number M of experiments by uniformly distributing the experiments in a discretized parameter space which can be restricted by constraints. Expert knowledge of the user of the corresponding test planning tool can be incorporated here, namely by defining a measure of similarity between two different experiments and by defining a measure of evaluation for individual experiments.

This allows a very high degree of flexibility in the specification of the experiments to be carried out and the question of which experiments are considered to be similar or dissimilar from a process perspective. Another advantage is that experiments that have already been carried out can also be taken into account when planning the experiment.

The use of the present invention is particularly advantageous for the test planning for data acquisition for the training of neural networks or hybrid neural networks with rigorous model components. In particular, a uniform distribution of the test planning in the relevant space can be achieved with the invention, so that a neural network or a hybrid neural network can be trained with a relatively small number of test data. This enables a very significant saving of time and costs for the execution of experiments to obtain such a database, since the number of experiments required for training can be optimized by the invention. A particular advantage compared to the test planning program known from the prior art is that, in principle, any boundary conditions for the type of experiments can be specified. In addition, by evaluating the difference between two experiments, prior knowledge - for example, structural information about a process to be modeled - can already be taken into account when planning the experiment.

Another particular advantage of the invention can be seen in the fact that the evaluation of a collective of experiments and the evaluation of individual experiments is carried out using different measures:

On the one hand, a similarity measure is defined, which represents a numerical value for the similarity or dissimilarity of two experiments. The background to this is that the experiments to be planned should be chosen so that they are as dissimilar as possible. What similarity or dissimilarity for a particular

Trial means can be defined by the user via the definition of the similarity measure.

Separately, an evaluation measure with regard to the evaluation of individual experiments can be specified by the user. A wide variety of criteria can be included in this assessment measure, such as the conditions of the test facility, costs for carrying out an experiment, the time required to carry out an experiment, etc. For example, certain parameter combinations such as high pressure and high temperature can damage the Run a test facility so that such "forbidden" parameter combinations are provided with an appropriate evaluation.

The similarity and evaluation measures defined in this way are incorporated into a quality measure which combines the similarity measure and the evaluation measure with one another. Based on the quality measure, those experiments for which the quality measure takes an extreme value are then determined according to the invention. Depending on the choice of the similarity measure and evaluation measure and the choice of the respective sign, this extreme value is a maximum or a minimum of the quality measure.

According to a preferred embodiment of the invention, the similarity measure is based on the Euclidean distance between two experiments in a parameter space. Each experiment is defined by a vector which contains the parameter values of the various test variables for this specific experiment. This parameter space is preferably discretized, ie the parameter values can only assume certain discrete values. The distance between two experiments is e.g. as the Euclidean distance of the parameter vectors of the considered experiments.

A reciprocal Euclidean distance between two experiments is preferably defined as a measure of similarity. Furthermore, an exponential function can be defined as a measure of similarity, which has the reciprocal Euclidean distance of two experiments as exponents. Further definitions of similarity measures are possible depending on the respective test scenario.

The evaluation measure can also be defined in the form of a formula. Alternatively, the evaluation measure can also be defined in the form of a table or the like.

According to a preferred embodiment of the invention, the similarity measures of all pairings of parameter vectors are calculated and added to calculate the quality measure. Furthermore, the evaluation measures are calculated for all the experiments under consideration. The summed similarity measures and the summed evaluation measures are then added or subtracted depending on the definition of the similarity and evaluation measures. This addition or subtraction then results in the quality measure which is to be minimized or maximized. According to a preferred embodiment of the invention, experiments already carried out are also used for the calculation of the similarity measure and the evaluation measure. For example, if a number N experiments have already been performed, M new experiments are searched in the discrete parameter space. The quality measure is then calculated based on the N experiments already performed and the M new experiments in order to select the M new experiments.

According to a further preferred embodiment, the extreme value of the quality measure is determined using a Monte Carlo method. Alternatively, e.g. a genetic algorithm or another suitable numerical optimization method are used.

According to a further preferred embodiment of the invention, the test planning program is connected to the system control, so that the

Parameter vectors of the experiments found can be transferred to the controller.

Preferred exemplary embodiments of the invention are explained in more detail below with reference to the drawings. Show it:

FIG. 1 shows a flow diagram of a first embodiment of the method for the automatic planning of experiments,

FIG. 2 shows a second embodiment based on Monte Carlo optimization,

Figure 3 is a block diagram of an embodiment of an inventive

Systems for automatic planning with a computer program product for carrying out the planning, FIG. 1 shows a flow chart for the automatic planning of experiments.

The experiments are to be planned in a d-dimensional parameter space.

Discretizations ^J mitj e (l..n,) ie (l ..d) are specified for the individual coordinates. Here n denotes the number of possible settings for the coordinate i.

In step 1 of the method, this d-dimensional parameter space is defined with its discretizations.

In step 2, the number M of experiments to be planned is entered.

So the experimenter has the possibility to do the number of new ones

To specify experiments.

In step 3, a similarity measure R of two experiments is entered.

For example, the similarity measure R is defined such that the more similar two experiments are, the greater the similarity measure. One possibility for defining the similarity measure R of two experiments xi and x ₂ is given below:

with a monton growing function ', ^{e.g. z = eχ} P ^(z) or ^• > ^{(z) = z} .

In step 4, an evaluation measure S of an individual experiment x, is then entered. This evaluation measure S is used to evaluate individual experiments that are compatible with discretization X. For example, in the case of S (x) = -∞, the experiment is considered to be prohibited, since it would, for example, damage the test facility. However, allowed experiments are rated with a higher value S. The function S (x) can be stored, for example, in the form of a table. In addition to the boundary conditions specified by the test facility, other criteria, such as, for example, the time and / or cost involved in carrying out a particular experiment, can also be represented using the evaluation measure S.

In general, the evaluation measure S in the form of a function or table is chosen such that the larger the more desirable an individual experiment is, the more desirable it is.

In step 5, this results in a quality measure Q, which is based on the similarity measure R and the evaluation measure S. In step 6, a minimum of the quality measure Q is determined, that is to say a selection of M experiments from the parameter space, so that Q becomes minimal and thus the quality reaches a maximum.

The test planning problem solved in step 6 is therefore formal:

Given are N existing experiments X _I , .. _N - We are looking for M new experiments X _f N + n .. _{fN +} ivφ _» which are compatible with the discretization X, for which S (x ≠ -∞ and the

> ≠ J

minimize.

In step 7, the M experiments determined in step 6 are output.

It is particularly advantageous in this embodiment of the test planning according to the invention that, on the one hand, the experiments to be carried out and existing differ from one another as much as possible and, on the other hand, that they are each individually desired if possible. This structural approach allows complex solving the most delicate test planning problems by formulating the measure for the difference between two experiments in a way that is adapted to the problem and, for example, implementing additional conditions through individual evaluation of experiments. If you have defined the measure for the difference or similarity and the individual evaluation function, ie the evaluation measure, the test planning is reduced to an optimization problem, which can be solved with mathematical aids.

Some examples of different types of test planning problems and their implementation using the method according to the invention are described below:

"Hard" constraints on individual experiments:

The invention not only allows experiments to be planned which differ as much as possible, but also an individual evaluation of experiments is carried out via the evaluation measure S. In this way, "hard" constraints on experiments to be carried out can be specified. For example, criteria can be specified which must be or must not be fulfilled in all experiments. The experiments are evenly distributed in the space restricted in this way. This possibility in particular allows that certain types of experiments are excluded from the outset. One possible way of doing this is for a "forbidden" experiment to be given an S = -∞ rating.

- processes with different "categories"

Assume that experiments are designed to design a process in which ordinary continuous values, such as pressure and temperature, can be changed. In addition, the addition of various alternative additives or systems (parts) is to be examined, for example. The similarity of two experiments should be defined in the following way Be tested: Tests with different additives / systems (parts) are fundamentally rated as extremely dissimilar, since these are not comparable. In experiments with the same additive, the similarity is given by the differences in pressure and temperature.

recipes

When planning experiments to optimize recipes, e.g. Given given starting materials, their amounts are varied experimentally. Here you can either describe a test plan using the ratios of the quantities of substance thrown in, or you can formulate the recipe directly using the weighed-in quantities and as a boundary condition you have to add up to 100% (- »hard constraint).

Processes with multiple targets / incomplete data:

Interesting questions for test planning arise when experimental data are already available but are incomplete. In other words, not all parameters were logged in individual experiments or not all target variables (eg product quality properties) were determined. For the evaluation of new experiments to be carried out, it is crucial to determine whether a similar experiment already exists.

The answer to this question depends on which target variable is to be described. If a target variable has not been determined for a given experiment, this experiment cannot be used to model the target variable not measured. When planning new experiments, it should therefore be required that they provide new information with regard to as many output variables as possible. Technically, this can be achieved by separately determining the distance to the existing experiments for all target values and then averaging over the target values. Experiment planning with previous structural knowledge:

If there is structural prior knowledge about a process, this prior knowledge influences the question of which experiments can be regarded as similar or dissimilar. In some cases, it is possible to calculate variables derived from the primary process parameters that characterize the mechanisms of the process (secondary process variables). The similarity measure R is then implemented in such a way that the secondary variables are first calculated from the primary variables in order to determine the similarity therefrom.

Optimization experiments:

In many cases (for reasons of experimental capacity alone) one will not be interested in the complete modeling of a process. One would rather like to experimentally determine an optimal operating point. In such a case, one will proceed iteratively.

In a first step, experiments are planned that allow a model to be created that can distinguish between good and bad operating conditions. Based on this model, a sequence of new experiments is planned, which specifically examine the range of desired operating states. This is realized with the help of the evaluation measure S. With the data obtained in this way, the model can be improved in the interesting parameter areas and a new series of tests can now be created with more ambitious process quality specifications. This process is repeated until the optimum state of the process has been found with the desired accuracy. A possible method for solving the optimization problem, that is, for choosing M experiments so that the quality measure Q takes an extreme value, is the Monte Carlo method.

FIG. 2 shows an embodiment with regard to the implementation of a

Monte Carlo method to solve the optimization problem.

In step 8, a number of M experiments are arbitrarily selected from the parameter space. Furthermore, a quality difference ε is initialized. In step 9, an experiment is then arbitrarily selected from these M experiments. In step 10, the quality measure Q is calculated for the test plan with this experiment.

In step 11, a coordinate of this experiment is selected arbitrarily. If the coordinate is ordinal, it is preferably increased or decreased by one step at random; if the coordinate is categorical, it will be chosen at random. This is done in step 12.

In step 13, the quality measure Q 'is calculated for the selected experiment with the varied coordinate. In step 14, the quality measures Q and

Q 'compared with each other. If Q '<Q + ε, this means that the quality of the selected experiment with the varied coordinate is not significantly worse than the quality of the unchanged selected experiment.

In this case, in step 15 the experiment selected in step 9 is replaced by this selected experiment with the varied coordinate. The quality difference ε is then reduced. If, on the other hand, the quality has deteriorated significantly, the experiment originally selected in step 9 is retained. In this case too, the quality difference ε can be reduced. Steps 9 to 14 and optionally 15 are repeated until an abort condition is reached. The value of ε is continuously reduced to zero. For example, a termination condition can be a maximum number of iterations; Another choice of a termination condition is when the quality measure Q no longer changes or does not change significantly. This way and

A minimum of the quality measure Q and thus the M experiments sought can thus be determined (cf. step 6 of FIG. 1).

FIG. 3 shows a block diagram of an embodiment of a system according to the invention. The system includes a test facility 30 for carrying out the experiments. The experimental system 30 is controlled by a controller 31. A computer 32 is connected to the controller 31 and has a computer program 33 for carrying out the test planning. The computer program 33 contains a function 34 for calculating the similarity measure and a function 35 for calculating the evaluation measure (cf. steps 3 and step 4 of the figure

!) •

In order to enable automated, cyclical operation of the overall system, the computer program 33 has an adaptation module 44 for adapting the evaluation measure 35 to the results of the experiments 39 carried out.

The termination module 45 is also provided for the cyclic operation of the system, so that the method ends when a predefined termination criterion is reached.

Furthermore, the computer program 33 contains a function 36 for calculating the quality measure based on the functions 34 and 35 (cf. step 5 of FIG. 1).

The computer program 33 also contains an image 37 of the discrete parameter space X (cf. step 1 of FIG. 1). The computer program also includes

33 a program module 38 for calculating an extreme value of the quality measure by means of the function 36, that is to say for the selection of M experiments, so that the quality measure takes on an extreme value.

The computer 32 also has a memory 39 for storing the parameter vectors from previous experiments. On this memory 39, the

Access computer program 33 for extreme value calculation with program module 38.

The computer 32 also has a user interface 40, via which a user can use a screen 41 to enter the functions 34, 35 and / or 36 as well as an adaptation scheme into the adaptation module 44 and an abort criterion into the termination module 45. The user can also specify the parameter space 37 via the user interface 40.

After the user has specified the functions 34, 35 and / or 36 and after a specification of the parameter space 37 has been specified, the experiments can be automatically planned by the computer program 33. For this purpose, the program module 38 accesses the memory 39 and the function 36 in order to optimize them, that is to say to find a number of M experiments in such a way that the quality measure takes an extreme value.

After this optimization problem has been solved, for example with the aid of a Monte Carlo method (cf. FIG. 2) or a genetic or evolutionary algorithm or by means of another mathematical optimization method, the parameter vectors of the M experiments to be carried out are available. These parameter values are transmitted to the controller 31 as a file 42, which has the form of a matrix, for example. The controller 31 makes the corresponding settings in the test installation so that the individual M experiments are carried out. The controller 31 determines the measurement results of interest from the test system 30 and combines them into a file 43 which is automatically transmitted to the computer 23. The user of the computer 32 can open the file 43 and, if necessary, analyze it using further software. For the planning of further M 'experiments, the M experiments just carried out in the

Memory 39 transferred, so that they are used in a subsequent planning for the evaluation of functions 34, 35 and / or 36.

On the basis of all the experiments 39 carried out so far, the computer program 33 can adapt the evaluation measure 35 with the aid of the adaptation module 44. With the new settings, the system plans the new test series and transmits the experiments to the control system 31. This is repeated cyclically until a predefined reduction criterion (termination module 45) is met, or the method is ended through user intervention via user interface 40.

LIST OF REFERENCE NUMBERS

Test facility 30 control 31

Computer 32

Computer program 33

Function 34

Function 35 Function 36

Settlement 37

Program module 38

Memory 39

User interface 40 screen 41

File 42

File 43

Adaptation module 44

Termination module 45

Claims

claims

1. Procedure for automatic planning of experiments with the following

steps:

Input of a similarity measure of two experiments,

Entering an evaluation measure for an individual experiment,

- determination of a quality measure based on the similarity measure and the evaluation measure,

Finding a number of experiments in which the quality measure takes an extreme value.

2. Method according to claim 1, the experiments being selectable from a discrete parameter space.

3. Method according to claim 1 or 2, the similarity measure being based on the Euclidean distance between two experiments in a parameter space.

4. Method according to Claim 1, 2 or 3, the measure of similarity being based on the reciprocal Euclidean distance between two experiments in a parameter space.

5. The method according to any one of the preceding claims 1 to 4, wherein the similarity measure is based on an exponential function with a reciprocal Euclidean distance between two experiments as exponents.

6. The method according to any one of the preceding claims 1 to 5, wherein the evaluation measure includes an evaluation with regard to the feasibility and / or feasibility of an experiment.

7. The method according to any one of the preceding claims 1 to 6, wherein the assessment measure includes an assessment of the cost or the time required to carry out an experiment.

8. The method according to any one of the preceding claims 1 to 7, wherein the evaluation measure is entered in the form of a function.

9. The method according to any one of the preceding claims 1 to 8, wherein the evaluation measure is entered in the form of a table.

10. The method according to any one of the preceding claims 1 to 9, wherein the

Quality measure is determined based on the summed similarity measures of all considered experiments and the summed assessment measures of all considered experiments.

11. The method according to claim 10, wherein a first number of experiments already carried out and a second number of experiments to be planned are considered for determining the quality measure.

12. The method according to any one of the preceding claims 1 to 11, wherein the experiments in which the quality measure takes an extreme value are found by means of a Monte Carlo method.

13. Procedure according to Claim 12 with the following steps:

Selecting a number (M) of experiments from a parameter space, Choose one of the experiments selected from the parameter space,

Calculation of the quality measure for the selected experiment,

Choosing a coordinate of the chosen experiment,

Variation of the selected coordinate within the parameter space,

- Calculation of the quality measure for the selected experiment with the varied coordinate,

Replacement of the selected experiment by the selected experiment with the varied coordinate if the quality measure for the selected experiment with the varied coordinate results in a quality that is not at least one quality difference (ε) worse than the quality measure for the selected experiment,

gradual reduction of the quality difference (ε) towards zero.

14. Method according to claim 13, wherein a number (N) of experiments already carried out are also taken into account for the calculation of the quality measures.

15. The method according to any one of the preceding claims 1 to 14, wherein for

Finding a number of experiments in which the quality measure takes an extreme value using a genetic algorithm.

16. Computer program product for carrying out a method according to one of the preceding claims 1 to 15.

17. System for the automatic planning of experiments with

Means (33, 39) for finding a number of experiments in which a quality measure takes an extreme value, the quality measure based on a similarity measure between two experiments and one

Based on the assessment of an individual experiment,

Means for outputting the found experiments (42), in which the quality measure takes an extreme value, to a control unit (31) for carrying out the experiments.

18. System according to claim 17, the means for finding being linked to means for storing (39) experiments already carried out and with means (44, 45) for automatically and cyclically carrying out the planning of experiments.