WO2003079285A2

WO2003079285A2 - Method, arrangement, computer programme with programme code means, computer programme product for the weighting of input parameters for a neuronal structure and neuronal structure

Info

Publication number: WO2003079285A2
Application number: PCT/DE2003/000756
Authority: WO
Inventors: Christoph Tietz; Hans-Georg Zimmermann
Original assignee: Siemens Aktiengesellschaft
Priority date: 2002-03-20
Filing date: 2003-03-10
Publication date: 2003-09-25
Also published as: WO2003079285A3

Abstract

The invention relates to a neuronal structure for the modelling of a dynamic system the structure of which permits an automatic weighting of parameters coming into the system which varies with time. The neuronal structure thus comprises a first neuronal partial structure, the first representation relationship of which describes a forward relationship of the dynamic system and a second neuronal partial structure the second representation relationship of which describers a retro relationship for the dynamic system. The partial systems are coupled together such that deviations between the first input parameters of the first neuronal structure and the second output parameters of the second neuronal structure may be determined, the use of which permits the weighting to be achieved.

Description

description

Method and arrangement as well as computer program with program code means and computer program product for weighting input variables for a neural structure and neuronal structure

The invention relates to a neural structure and a method and an arrangement as well as a computer program with program code means and a computer program product for weighting input variables for a neural structure.

From [1] it is known to use a neural structure, for example a neural network, to describe and model a dynamic process and its process behavior.

In general, a dynamic process is described by a state transition description, which is not visible to an observer of the dynamic process, and an initial equation, which describes observable quantities of the technical ^' dynamic process.

Such a process behavior of a dynamic process is shown in Fig. 2.

The dynamic process 200 or a dynamic system 200, in which the dynamic process runs, is subject to the influence of an external input variable u of a predeterminable dimension, an input variable u * t at a time t being designated u * t:

u _t e 5R ¹ ,

where 1 is a natural number. The input variable u * t at a time t causes one

Change in the dynamic process.

An internal state s * t (s * te 9ϊ ^m ) of predeterminable dimension m at a time t cannot be observed by an observer of the dynamic system 200.

Depending on the inner state s * t and the input variable ut, a state transition of the inner state s * t of the dynamic process is caused and the state of the dynamic process changes into a subsequent state s - ^ + i at a subsequent time t + 1 ,

The following applies:

st +1 = ^f ( ^s t ' ^u t) - ⁽ 1 ⁾

where f (.) denotes a general mapping rule.

An output variable y ^ observable by an observer of the dynamic system 200 at a time t depends on the input variable * t and the internal state s - ^ -

The output variable yj- (yj- e 9ϊ ⁿ ) is predeterminable dimension n.

The dependency of the output variable y on the input variable u * t and the internal state s- ^ of the dynamic process is given by the following general rule:

Yt = g ( ^s t) '* ⁽ 2 ⁾

where g (.) denotes a general mapping rule. To describe the dynamic system 200, a neural structure of interconnected computing elements in the form of a neural network of interconnected neurons is used in [1]. The connections between the neurons of the neural network are weighted. The weights of the neural network are summarized in a parameter vector v.

Thus, an internal state of a dynamic system, which is subject to a dynamic process, depends on the input variable u- ^ and the internal state of the previous time s- ^ and the parameter vector v according to the following rule:

^s t + l ⁼ NNv, s _t , ut), (3)

where NN (.) denotes a mapping rule specified by the neural network.

This description of the dynamic system 200 according to relation (3) is also referred to as the “forecast approach”.

Alternatively, the dynamic system can also be:

s _t = f (s _t _ι, u _t ) (1 ^Λ )

With

s _t = NN (v, s _t _ι, u _t ) (3)

describe what is known as the "consistency approach". "Forecast appraoch" and "consistency appraoch" lead to slight structural differences in the respective network structures, but are equivalent, alternatively usable forms of description for dynamic systems. [2] discloses a further neural structure for describing the dynamic system 200, a neural network referred to as a time delay recurrent neural network (TDRNN).

The known TDRNN is shown in FIG. 5 as a neural network 500 which is developed over a finite number of times (shown 5 times: t-4, t-3, t-2, t-1, t).

The neural network 500 shown in FIG. 5 has an input layer 501 with five partial input layers 521, 522,

523, 524 and 525, each of which contains a predeterminable number of input computing elements to which input variables u -4, u -3, -2 _r ut-i and u * t at predefinable times t-4, t- 3, t-2, t-1 and t, ie time series values described below with predetermined time steps, can be applied.

Input computing element, i.e. Input neurons are connected via variable connections to neurons with a predefinable number of hidden layers 505 (5 hidden layers shown).

Neurons of a first 531, a second 532, a third 533, a fourth 534 and a fifth 535 hidden layer are each connected to neurons of the first 521, the second 522, the third 523, the fourth 524 and the fifth 525 partial input layer.

The connections between the first 531, the second 532, the third 533, the fourth 534 and the fifth 535 hidden layer with the first 521, the second 522, the third 523, the fourth 524 and the fifth 525 partial input layer are respectively equal. The weights of all connections are each contained in a first connection matrix Bi.

Furthermore, the neurons of the first hidden layer 531 with their outputs are inputs of neurons of the second hidden layer 532 according to a structure given by a second connection matrix A _] _. The neurons of the second hidden layer 532 are connected with their outputs

Inputs of neurons of the third hidden layer 533 according to a given by the second connection matrix A] _

Structure connected. The outputs of the neurons of the third hidden layer 533 are connected to inputs of neurons of the fourth hidden layer 534 according to a structure given by the second connection matrix A] _. The fourth hidden layer 534 neurons are with theirs

Outputs connected to inputs of neurons of the fifth hidden layer 535 according to a structure given by the second connection matrix A] _.

In the hidden layers, the first hidden layer 531, the second hidden layer 532, the third hidden layer 533, the fourth hidden layer 534 and the fifth hidden layer 535, “internal” states or “internal” system states s -4 st- 3, ^s t-2 ^s t- _l r and st represents a dynamic process described by the TDRNN at five successive times t-4, t-3, t-2, t-1 and t.

The information in the indices in the respective layers indicates the time t-4, t-3, t-2, t-1 and t, to which the signals that can be tapped or fed at the outputs of the respective layer relate ( u -4, u -3, ^u t-2 ' ^u tl' ^u t ⁾ •

An output layer 520 has five partial output layers, a first partial output layer 541, a second partial output layer 542, a third partial output layer 543, a fourth partial output layer 544 and a fifth partial output layer

545 on. Neurons of the first partial output layer 541 are measured in accordance with an output connection matrix C _] _

Structure connected to neurons of the first hidden layer 531. Neurons of the second partial output layer 542 are e- if necessary according to the structure given by the output connection matrix C] _ with neurons of the second hidden one

Layer 532 connected. Neurons of the third partial output layer 543 are connected to neurons of the third hidden layer 533 in accordance with the output connection matrix C] _. Neurons of the fourth partial output layer 544 are connected to neurons of the fourth hidden layer 534 according to the output connection matrix C] _. Neurons of the fifth partial output layer 545 are connected to neurons of the fifth hidden layer 535 in accordance with the output connection matrix C] _. The output variables for a point in time t-4, t-3, t-2, t-1, t can be tapped at the neurons of the partial output layers 541, 542, 543, 544 and 545 (yt-4 _* Yt-3 r Yt- 2>Yt-l> Yt) ■

The principle that equivalent connection matrices in a neural network have the same values at any given time is referred to as the principle of the so-called shared weights.

The arrangement known from [2] and referred to as Time Delay Recurrent Neural Network (TDRNN) is trained in a training phase in such a way that a target variable y _{t is} determined in each case on an actual dynamic system for an input variable ut. The tuple (input variable, determined target variable) is called the training date. A large number of such training data form a training data set.

Thereby, successive tuples (ut-4, y ^ _.)

^(u t-3 r Y - 3 ⁾ ' ^(u t-2' ^ _2 ^{) of the} times (t-4, t-3, t-3, ...) of the training data set each have a predetermined time step.

The TDRNN is trained with the training data record. An overview of various training methods can also be found in [1]. It should be emphasized at this point that only the output variables yt-4 r Yt-3 r • • • Yt ^{can be} seen ^at times t-4, t-3, ..., t of the dynamic system 200. The "inner" system states s -4 ** st-3 • ■ ••> • ^s t cannot be observed.

The following cost function E is usually minimized in the training phase:

where T is a number of times taken into account.

From [5] and [6] further developments of the neural structure known from [2] and known as Time Delay Recurrent Neural Network (TDRNN) are known.

The further developments from [5] are particularly suitable for determining future states of a dynamic process, which is referred to as "overshooting".

Fig.la from [5] shows a basic structure on which the further developments known from [5] are based.

The basic structure is a neural network developed over three times t, t + 1, t + 2.

It has an input layer which contains a predeterminable number of input neurons, to which input variables ut can be applied at predeterminable times t, that is to say time series values described below with predefined time steps. The input neurons are connected via variable connections to neurons with a predefinable number of hidden layers (3 hidden layers shown).

Neurons of a first hidden layer are connected to neurons of the first input layer.

The connection between the first hidden layer and the first input layer has weights which are contained in a first connection matrix B.

Furthermore, the neurons of the first hidden layer are connected with their outputs to inputs of neurons of a second hidden layer according to a structure given by a second connection matrix A. The outputs of the neurons of the second hidden layer are connected to inputs of neurons of a third hidden layer in accordance with a structure given by the second connection matrix A.

In the hidden layers of the first hidden layer, said second hidden layer and the third hidden layer respectively "internal" conditions or "inner" system states are, s st + i and st + 2 ^is d-described dynamic process at three successive points in time t, represents t + 1 and t + 2.

The details in the indices in the respective layers each indicate the time t, t + 1, t + 2, to which the signals (u- ^) which can be tapped or supplied at the outputs of the respective layer relate.

An output layer 120 has two sub-output layers, a first sub-output layer and a second sub-output layer. Neurons of the first partial output layer are connected to neurons of the first hidden layer in accordance with a structure given by an output connection matrix C. the. Neurons of the second partial output layer are also connected to neurons of the second hidden layer in accordance with the structure given by the output connection matrix C.

The output variables can be tapped at a time t + 1, t + 2 from the neurons of the partial output layers (yt + l / yt + 2 ⁾

A further development of this basic structure from [5] is shown in Fig. 6.

Further developments of the TDRNN structure from [6], so-called Error Correction Recurrent Neural Networks (ECRNN), relate to a structurally determined error correction mechanism which is integrated as a structural component in a neural structure. 7 shows a basic structure with corresponding functional relationships of an ECRNN.

In [3] there is also an overview of the basics of neurons

Find networks and the possible applications of neural networks in the area of economics.

In many cases, dynamic processes or dynamic systems depend on a large number of external influencing variables, i.e. the external input variable u is of a very high dimension. This applies in particular to dynamic systems in the area of economics.

At a point in time t, however, only a part of the influencing variables of ut is typically relevant, which can also change over time.

When describing such dynamic systems using neural structures, the following questions are raised: What are the actually relevant external influencing factors and how does their influence on the dynamic system shift over time?

This time-variant or dynamic differentiation of important and less important influencing variables cannot be guaranteed by the above known neuronal structures. The importance of influencing variables that changes over time is not taken into account in these neural structures.

In the neural structures described above, all influencing variables are considered statically equally important. Influencing factors that are considered less important can only be completely removed from the neural structure and are therefore completely disregarded, also in terms of time.

The invention is therefore based on the object of specifying a neural structure as well as a method or an arrangement or a corresponding computer program with program code means or a corresponding computer program product which differentiates, in particular a time-variant, dynamic differentiation of influencing variables of a dynamic one Systems enables.

This object is achieved by the neural structure and the method and the arrangement as well as by the computer program with program code means and the computer program product for weighting input variables for a neural structure with the features according to the respective independent patent claim.

In the method for analyzing influencing variables of a dynamic system using a first and a second neural substructure, the mapping behavior of which each describe a dynamic behavior of the system, the first neural substructure being adapted such that its first mapping behavior describes a forward behavior of the dynamic system,

The second neural substructure is adapted such that its second mapping behavior describes a backward behavior of the dynamic system:

Deviations between first input variables of the first neuronal substructure and second output variables of the second neuronal substructure are determined, which deviations represent a measure for a weighting of the influencing variables.

The arrangement for analyzing influencing variables of a dynamic system has a first and a second neural substructure, the mapping behavior of which describes a dynamic behavior of the system,

the first neural substructure is adapted such that its first mapping behavior describes a forward behavior of the dynamic system, the second neuronal substructure is adapted such that its second mapping behavior describes a backward behavior of the dynamic system, the first and the second neural substructure are coupled to one another in such a way that deviations between first input variables of the first neuronal substructure and second output variables of the second neuronal substructure are determined, which deviations represent a measure of a weighting of the influencing variables.

The neural structure has a first and a second neural substructure, the mapping behavior of which each describe a dynamic behavior of a dynamic system,

the first neural substructure being adapted such that its first mapping behavior describes a forward behavior of the dynamic system, the second neural substructure is adapted such that its second mapping behavior describes a backward behavior of the dynamic system,

- The first and the second neural substructure being coupled to one another in such a way that deviations between first input variables of the first neuronal substructure and second output variables of the second neuronal substructure are determined.

The invention clearly represents a structural one

Extension (see Fig. La) represents a known neural structure.

Dynamic systems are usually formulated as cause-and-effect relationships (cf. comments on Fig. 2, relationships (1) to (3)), which are represented by the neuronal structures known from [1], [2] or [5] can. These cause-effect relationships are expressed in these neural structures in that an information flow generated in these neural structures moves forward in time, i.e. from the past to the future. This is called forward behavior. Causes of input variables ut at previous times (t- 2), (t-1), ... lead to (noticeable) effects in output variables yt at time (t or t + 1). The input variables ut are mapped to the output variables yt by the neural cause-effect structure.

The invention extends these neural cause-effect structures with a neural substructure which carries out an effect-cause analysis and thus prepares a causal synthesis.

With this (effect-cause) extension structure or effect-cause structure, a backward information flow, d-. H. a flow of information directed from the future into the past. Such is called backward behavior. Effects in output variables yt at time (t) "lead" or have their causes in input variables t at time (t-1), (t-2), .... Output variables y (as input variables of the extension structure) are mapped onto the input variables ut (as output variables of the extension structure) in the opposite way to the cause-effect structure.

The two structures are linked by comparing actual causes with modeled causes, which are generated using the effect-cause structure, and deriving the relevance of individual external influencing factors.

A particular advantage of the invention is that the invention enables analysis and dynamic consideration of influencing variables of a dynamic system based on their temporal relevance ("Dynamic Feature Selection").

The computer program with program code means is set up to carry out all steps according to the inventive method when the program is executed on a computer.

The computer program product with program code means stored on a machine-readable carrier is set up to carry out all steps according to the inventive method when the program is executed on a computer.

The arrangement and the computer program with program code means, set up to carry out all steps according to the inventive method when the program is executed on a computer, and the computer program product with program code means stored on a machine-readable carrier, set up all steps according to the Carrying out inventive methods when the program is executed on a computer are particularly suitable for carrying out the method according to the invention or one of its further developments explained below.

The described software solutions can also be implemented decentrally or distributed, i.e. that parts of the computer program or parts of the computer program product - also as independent partial solutions - run on different (distributed) computers or are executed by them or are stored on different storage media.

Preferred developments of the invention result from the dependent claims.

The further developments described below relate both to the method and to the arrangement, the neural structure, the computer program with program code means and the computer program product.

The invention and the further developments described below can be implemented both in software and in hardware, for example using a special electrical circuit.

Furthermore, an implementation of the invention or a further development described below is possible by means of a computer-readable storage medium on which the computer program with program code means which carries out the invention or further development is stored.

The invention or any further development described below can also be implemented by a computer program product which has a storage medium on which the computer program with program code means which carries out the invention or further development is stored. In a further development, the first and / or the second neural substructure is or are a neural network developed over several points in time, for example a TDRNN, or neural networks unfolded over several points in time, in which one or in which a temporal dimension of the described one dynamic system is developed as a spatial dimension.

The first are used to implement automatic, time-variant and dynamic weighting of the influencing variables

Input variables of the first neural substructure are weighted using the deviations.

The invention is particularly suitable for determining the dynamics of a dynamic process on which the system is based. The dynamics result from the first output variables of the first neural substructure. The higher the dynamics of the process and the number of factors influencing the dynamic system, i.e. the more complex the dynamic system, the more advantageous the invention proves, since it can take into account different, rapidly changing relevances of influencing variables.

Chemical processes are usually highly complex or highly complex dynamic processes and are influenced by many physical variables. Accordingly, the invention is particularly suitable for determining and analyzing the dynamics of a dynamic process, such as in a chemical reactor. This analysis can then be used to monitor or control the dynamic process, in particular a chemical process.

The same applies to economic or macroeconomic processes or systems, which are characterized in particular by a very large number of influencing factors, which moreover change their relevance very dynamically. Consequently the invention can be used in particular to analyze the dynamics of such systems.

In addition, the invention is particularly suitable for predicting a state of the dynamic system. The

The forecast is created using the first output variables of the first substructure.

In one development, the invention has a measuring arrangement for recording physical signals, for example an electrocardio gram (EKG), by means of which the dynamic system, in this case a human circulation, is described. These physical signals, the EKG signals, are then fed to the first neuronal substructure for analyzing the system.

In addition, the first neural substructure and the second neural substructure can be coupled such that further deviations can be formed between first output variables of the first neuronal substructure and second input variables of the second neuronal substructure.

In addition, it can make sense that the first and / or second neural substructure is / are designed as an error correction recurrent neural network (ECRNN). Fundamentals of such ECRNN are described in [6] and can be built into the neural substructures accordingly.

Exemplary embodiments of the invention are shown in figures and are explained in more detail below.

Show it

Figures la to lc sketch of a basic structure of a neural arrangement and sketches of an arrangement and an alternative arrangement according to a first embodiment example (An.: la new, lb / c correspond to slide pta_5 / 14 or 18);

FIG. 2 shows a sketch of a general description of a dynamic system

99pl348);

FIG. 3 shows a sketch of a neural arrangement with an integrated error correction mechanism according to a second exemplary embodiment (note: corresponds to slide pta_5 / 20);

FIG. 4 shows a sketch of a chemical reactor, from which quantities are measured, which are processed further with the arrangements according to the first exemplary embodiment

(Note: from old application 99pl34);

FIG. 5 shows a sketch of an arrangement of a TDRNN, which is unfolded over time with a finite number of states (note: from old application 99pl348);

FIG. 6 shows a sketch of a further development of a TDRNN suitable for the “overshooting” (note: corresponds to slide pta_5 / 7),

Figure 7 is a sketch of an ECRNN with basic functional relationships (note: corresponds to slide pta_5 / 10).

FIG. 8 shows a sketch of a neural arrangement with an integrated error correction mechanism according to a second

Exemplary embodiment (note: corresponds to film pta 5/22).

First embodiment: chemical reactor 4 shows a chemical reactor 400 which is filled with a chemical substance 401. The chemical reactor 400 comprises a stirrer 402 with which the chemical substance 401 is stirred. Further chemical substances 403 flowing into the chemical reactor 400 react for a predeterminable period in the chemical reactor 400 with the chemical substance 401 already contained in the chemical reactor 400. A substance 404 flowing out of the reactor 400 is transferred from the chemical reactor 400 derived an output.

The stirrer 402 is connected via a line to a control unit 405, with which a stirring frequency of the stirrer 402 can be set via a control signal 406.

A measuring device 407 is also provided, with which concentrations of chemical substances contained in chemical substance 401 are measured.

Measurement signals 408 are fed to a computer 409, in which

Computer 409 is digitized via an input / output interface 410 and an analog / digital converter 411 and stored in a memory 412. A processor 413, like the memory 412, is connected to the analog / digital converter 411 via a bus 414. The calculator 409 is also on the

Input / output interface 410 connected to the controller 405 of the stirrer 402 and thus the computer 409 controls the stirring frequency of the stirrer 402.

The computer 409 is also connected via the input / output interface 410 to a keyboard 415, a computer mouse 416 and a screen 417. In addition, appropriately programmed software is stored in the memory. 412, which enables the functionality described below. The chemical reactor 400 represents a dynamic, technical system 200 and is subject to a dynamic process on which the dynamic system is based.

This chemical process is highly complex and exhibits extremely dynamic process behavior, which is influenced by a large number of influencing variables with changing relevance.

The chemical reactor 400 is described by means of a status description. In this case, the input variable u is composed of an indication of the temperature prevailing in the chemical reactor 400, the pressure prevailing in the chemical reactor 400, the stirring frequency set at the time t and a large number of other variables influencing the process behavior. The input variable is therefore a high-dimensional vector.

The aim of the modeling of the chemical reactor 400 described in the following is to determine the dynamic development of the substance concentrations, in order to enable efficient generation of a predefinable target substance to be produced as the outflowing substance 404.

This is done using the arrangements or neural networks described below and shown in FIGS. 1 a to 1c.

For a simple understanding of the principles underlying the neural networks FIGS. 1b 130 and 1c 160, a basic neural structure 100 is shown in FIG.

Starting from this basic structure 100, the neural networks 130, 160 shown in FIGS. 1b and 1c are formed.

The neural networks 130 (Consistency Approach), 160 (Forecast Appraoch) shown in FIGS. 1b and 1c are all can be used alternatively. Each fulfills the task described above ("Dynamic Feature Selection") equally.

The symbols used in the representation correspond to the generally customary symbolism in the representation of neural structures, as already used in the above network descriptions.

FIG. 1 a shows the neural basic structure 100 with a first neural substructure 101, the first mapping behavior of which describes a forward behavior 103 of the dynamic process or system. First input variables 111 are mapped to first output variables 112 by the first neural substructure 101.

Furthermore, the neural basic structure 100 has a second neural substructure 102, the second mapping behavior of which describes a backward behavior 104 of the dynamic system. Second input variables 113 are mapped to second output variables 114 by the second neural substructure 101.

The substructures 101, 102 are coupled to one another in such a way that deviations 120 between the first input variables 111 of the first neuronal structure 101 and the second output variables 114 of the second neuronal structure 102 can be determined.

Using the deviations 120, weights 121 are determined with which the first input variables 111 supplied to the first substructure 101 are weighted.

Seen clearly, these weights represent a filter for the first input variables 111 of the first substructure 101. Consistency Approach (Fig.lb, 130)

1b shows a neural network 130 based on the neural basic structure 100 according to the consistency appraoch.

The neural network 130 has a first neural substructure 131 and a second neural substructure 132, each of which over several points in time t, here (t-3) to (t + 3) at the first 131 or (t) to (t- 3) in the second neural substructure 132, are unfolded recurrent networks.

The two neural networks 131, 132 each have an input neuron layer 133 and 134, a hidden neuron layer 135 and 136, and an output neuron layer 137 and 138, respectively.

The input neuron layers 133 and 134 are each connected to the hidden layers 135 and 136 via connections weighted with connection matrices B and E, respectively.

The neurons of the hidden layers 135 and 136 are in turn connected by connection matrices A and F weighted connections.

The output neuron layers 137 and 138 are each connected to the hidden layers 135 and 136 via connections weighted with connection matrices C and G, respectively.

Between the input neuron layer 133 of the first subset 131 and the hidden neuron layer 135 of the first subset 131 is ATU an intermediate layer of neurons 140, wweellcchhee wwiitthh eeiinneerr GGee ^¬ weighting at weighted states, _/ _{_.} generated, fed This intermediate neuron layer 140 is connected to the hidden layer 135 via connections weighted with connection matrices D.

The output neuron layer 138 of the second substructure 132 is further connected to the input neuron layer 133 of the first substructure 131 via connections weighted with connection matrices H.

In this output layer 138 states, so-called difference states u ^{. *} - - ^U t ^{) u} tl ^{~ u} tl ^ ' ^u t-2 ^{~~ u} t-2 ^ ^unc u, _ _o - u? _ ₃ ) (in squared form , Index d: measured state u, index t +/- i: time step [i: natural number]).

These difference states of the output layer 138 are combined in a weighting neuron 139 via a connection weighted with a weighting factor (-α).

The weighting at there is over with a

Identity matrix Id weighted connection is fed into the neurons of the weighting layer 140, as a result of which the weighted states atu,, _ there. be generated.

It is thereby achieved that the input variables supplied to the neural network 130 are weighted automatically, dynamically and according to their relevance.

The neuron links are designed in such a way that a forward-looking information flow 141, represented by states st-3, ≤t-2 / ^s tl ' ^s t etc., is generated in the hidden layer 135 of the first substructure 131. This

States extend up to time (t + 3), which is known as "overshooting" from [5].

Correspondingly, the second layer becomes in the hidden layer 136 Substructure 132 generates a backward-directed information flow 142, represented by states rt, r * t_ι, rt-2 ' ^r t-3.

The neural network 130 described is based on the following relationships:

^s t = f ti ^'a t ^u ^{tj; y} t = ^g ( ^s t) ^ ⁽⁵⁾

rt = H ^r + l '* yt) ^{; u} t = ^G ( ^r t) ' ⁽⁶⁾ where F (.), G (.) denote a general mapping rule and internal state rt internal system states

-α∑ (u _t -iu ^ _ _i ) ² at = ei (activity filter) (7

A method based on a back-propagation method, as described in [1], is used for training the neural network 130 described above.

In the method known from [1], the following cost function E is minimized in the training phase:

where T is a number of times taken into account.

For the neural network 130, the cost function is modified to:

E = ^ Σ (yt ^~ y? J + - utf → rnin, (10)

^T t = lf, g, F, G y _t - y _t ): upper minimum output error, u ^ - U): lower minimum input error. In particular, it should be emphasized that the difference states u _t - u?) Formed in the output layer 138 of the second substructure 132 are components of the cost function (10). The at

Training aimed at minimizing the cost function (10) clearly leads to the fact that "causes" of "effects" of the dynamics of the dynamic system shown are learned.

Training data for the training according to (5) are obtained from the chemical reactor 400 in the following way.

Concentrations are measured using the measuring device 407 for predetermined input variables and fed to the computer 409, digitized there and grouped as time series values in a memory together with the corresponding input variables which correspond to the measured variables.

In training, the training data are fed to the neural network 130 and the connection weights and also the weight at are adapted in the process.

The neural network 130 trained in accordance with the training method described above is used to determine chemical variables in the chemical reactor 400 in such a way that forecast variables yt + i yt + 2 yt + 3 ⁿ for an input variable at a time t-1 and an input variable at a time t e ner

Application phase determined by the neural network, which are then used as control variables after a possible processing of the determined variables as control variables 420, 421, the control means 405 for controlling the stirrer 402 or also an inflow control device 430 for controlling the inflow of further chemical substances 403 in the chemical reactor 400 can be used (see Fig. 4).

Forecast Approach (Fig.lc, 160) 1c shows the alternative neural structure 160 based on the neural basic structure 100 according to the forecast approach.

This neural network 160 is based on the following relationships:

^s t + ι = ^f ( ^s t ' ^a t ^u tJ ^{; y} = ^g ( ^s t)' ^{(5 λ)}

r _t _ι = FJr _t , y); u _t = G (r _t ), (6 ^

-α∑ (u _t -iu ^ _ _i ) ² a _t = ei. (l)

The structure of the neural network 160 according to the forecast approach is identical to that of the consistency approach 130. Two neural networks 131 and 132 developed over several points in time, one with a forward-looking 141 and one with a backward-looking 142 information flow, are about a "difference states" Layer 138, a weighting neuron 139 and a weighting layer 140 are linked to one another.

Differences between the two networks 130 and 160 arise only in the names and isolated links of states, which, however, does not impair or change the basic approach underlying the functionality and the two networks.

The training of the neural network 160 and the use of the neural network 160 in the application are carried out in accordance with the neural network 130.

2nd embodiment: rental price forecast

FIG. 3 shows a neural structure 300 in which the error described in [6] in the neural structure from FIG. Correction mechanism (ECRNN) was integrated (ECRNN Forecast Approach).

It should be noted that the error correction mechanism can equally be integrated in the consistency approach.

The neural structure 300 is used for a rental price forecast as described below.

In this case, the input variable u * t is made up of annual average information about a rental price, housing space, inflation and an unemployment rate, as well as other economic factors that influence a rental price.

In this case too, the input variable is a high-dimensional vector. A time series of the input variables, which consist of several successive vectors, knows time steps of one year each.

The aim of the modeling described below is to forecast a rental price for the following three years with respect to a current point in time t.

ECRNN Forecast Approach (Fig. 3, 300)

The neural structure 300 in FIG. 3 shows the neural structure 160 expanded by the error correction mechanism (ECRNN) based on the neural basic structure 100 according to the forecast approach.

This neural network 300 is based on the following relationships:

s-t + i = f | s _t , a _t uj, y _t - yj; yt = t) '( ^{5> Λ} ) r _t _ι = FJrt, y, u _t - u £); u _t = G (r _t ), (6)

The structure of the neural network 300 according to the ECRNN Forecast Approach is identical to that of the Forecast Approach 160 and the Consistency Approach 130. Two neural networks 131 and 132 developed over several points in time, one with a forward 141 and one with a backward 142 information flow , are linked to one another via a “difference state layer” 138, a weighting neuron 139 and a weighting layer 140.

Differences to the two networks 130 and 160 arise only in the error correction neurons, but this does not impair or change the functionality and the basic approach on which the networks are based.

The training of the neural network 300 is carried out in accordance with the neural networks 130 and 161. Further procedures for training the neural network described above are described in [4].

Some alternatives to the exemplary embodiments described above are shown below.

8 shows an alternative neuronal ERCNN structure based on the neuronal structure in FIG. 3. This alternative neural structure, like the neural structures described in the exemplary embodiments, contain the inventive principles for dynamic feature selection, so that the above explanations apply accordingly.

The arrangements described in the first exemplary embodiment can also be used to determine the dynamics of an electronic Cardio-grams (EKG) can be used. This enables indicators that indicate an increased risk of heart attack to be determined at an early stage. A time series from ECG values measured on a patient is used as the input variable.

The arrangement described in the second exemplary embodiment can also be used for forecasting macroeconomic dynamics, such as, for example, an exchange rate trend, or other economic indicators, such as, for example, a stock exchange price. In the case of such a forecast, an input variable is formed from time series of relevant macroeconomic or economic indicators, such as interest rates, currencies or inflation rates.

Possible realizations of the exemplary embodiments described above can be carried out with the program SENN, version 2.3.

The following publications are cited in this document:

[1] S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall, Second Edition, ISBN 0-13-273350-1, pp. 732-789, 1999.

[2] David E. Rumelhart et al. , Parallel Distributed Processing, Explorations in the Microstructure of Cognition, Vol. 1: Foundations, A Bradford Book, The MIT Press, Cambrigde, Massachusetts, London, England, 1987

[3] H. Rehkugler and H. G. Zimmermann, Neural Networks in Economics, Fundamentals and Financial Applications, Verlag Franz Vahlen Munich, ISBN 3-8006-1871-0, pp. 3-90, 1994.

[4] WO00 / 08599.

[5] WO00 / 55809.

[6] Zimmermann H.G., Neuneier R., Grothmann R., Modeling of Dynamic Systems by Error-Correction-Neural-Networks, in Soofe and Cao (Eds.), Forecasting Financial Data, Kluwer Verlag, ISBN 0792376803, 2002.

Claims

claims

1. Method for analyzing influencing variables of a dynamic system using a first and a second neural partial structure, the mapping behavior of which each describe a dynamic behavior of the system, the first neuronal substructure being adapted such that its first mapping behavior is a forward behavior of the dynamic Systems describes, - the second neural substructure being adapted such that its second mapping behavior describes a backward behavior of the dynamic system, a) in which deviations between first input variables of the first neuronal substructure and second output variables of the second neuronal substructure are determined, which deviations are a measure for a weighting of the influencing variables.

2. The method as claimed in claim 1, in which the first and / or the second neural substructure is a neural network which has been developed over a plurality of times / is a neural network which has been developed over a number of times.

3. The method as claimed in one of the preceding claims, in which the first input variables of the first neural substructure are weighted using the deviations.

4. The method as claimed in one of the preceding claims, used to determine a dynamic of a dynamic process on which the system is based, such that first output variables of the first substructure describe the dynamic.

5. The method as claimed in one of the preceding claims, used to predict a state of the dynamic system, in such a way that the forecast is created using first output variables of the first substructure.

6. Arrangement for the analysis of influencing variables of a dynamic system using a first and a second neural substructure, the mapping behavior of which each describe a dynamic behavior of the system,

the first neural substructure is adapted such that its first mapping behavior describes a forward behavior of the dynamic system,

- The second neural substructure is adapted in such a way that its second mapping behavior describes a backward behavior of the dynamic system, the first and the second neural substructure being coupled to one another in such a way that deviations between first input variables of the first neuronal substructure and second output variables of the second neuronal substructure which deviation is a measure of a weighting of the influencing variables.

7. Arrangement according to claim 6, with a measuring arrangement for detecting physical signals with which the dynamic system is described.

8. Arrangement according to claim 6 or 7, used to determine a dynamic of a dynamic process, which is the basis of the dynamic system, in particular a chemical process in a chemical reactor.

9. Arrangement according to claim 6 or 7, used to determine a dynamic of an electro-cardio gram.

10. Arrangement according to claim 6 or 7, used to determine an economic or macroeconomic dynamic in an economic or macroeconomic system.

11. Arrangement according to one of claims 6 to 10, used for monitoring and / or controlling the dynamic system, wherein first output variables of the first neural substructure can be used as monitoring variables and / or as control variables.

12. Computer program with program code means to carry out all steps according to claim 1, if the program is or are executed on one computer or parts of the program on several computers.

13. Computer program with program code means according to claim 12, which are stored on one or more computer-readable data carriers.

14. Computer program product with program code means stored on a machine-readable carrier, in order to carry out all the steps according to claim 1 when the program is executed on a computer.

15. Neural structure having a first and a second neural part structure, the imaging behavior of each describe a dynamic behavior of a dynamic system, - wherein the first neural partial structure is ^• adjusted such that their first imaging behavior describes a forward behavior of the dynamic system, - wherein the second neural substructure is adapted such that its second mapping behavior describes a backward behavior of the dynamic system, the first and the second neuronal substructure being coupled to one another in such a way that deviations between first input variables of the first neuronal substructure and second output variables of the second neuronal substructure be determined.

16, neural structure according to claim 15, in which the first neural substructure and the second neuronal substructure are coupled such that further deviations can be formed between first output variables of the first neuronal substructure and second input variables of the second neuronal substructure.

17. Neural structure according to claim 15 or 16, in which the first and / or second neural substructure is / are designed as an error correction recurrent neural network (ECRNN).