DE102008014126B4 - Method for computer-aided learning of a recurrent neural network - Google Patents

Abstract

A method for computer-aided learning of a recurrent neural network (RNN) with temporally successive states (s _t-2 , ..., s _{t + 4} ) in a finite time interval, wherein the states (s _t-2 , ..., s _{t + 4} ) are connected to successive inputs (u _t-3 , ..., u _t ) and outputs (y _t-2 , ..., y _{t + 4} ) of a dynamic system and wherein the states (s _t-2 , ..., s _{t + 4} ) are interconnected with each other and with the inputs and outputs via convectors, which allow for error backpropagation a time-forward and a time backward information flow, characterized in that the information flow in the recurrent neural network (RNN) before the start of the time interval and / or after the end of the time interval with the aid of at least one retro-causal network (RC1, RC2) or with the aid of at least one forward-facing and / or at least one rearward-facing connector (C1,. .., C6) wherein the at least one forward connector (C1, C2, C3) allows only a forward flow of information and wherein the at least one backward connector (C4, ...

Description

The The invention relates to a method for computer-aided learning of a recurrent patient neural network and a method for predicting the output of a dynamic system based on a trained neuronal recurrent Network and a corresponding computer program product.

By recurrent neural networks become temporally consecutive internal states of a dynamic system, where to learn the networks a finite time interval of states is considered. The internal conditions make a so-called hidden layer of the recurrent neuronal Net, and these states are with temporally successive inputs and outputs one connected to be modeled dynamic system. The recurrent Structure of the network is created by the states in the hidden layer among each other as well as with the inputs and outputs via connectors which are used for error backpropagation (English: Error Back propagation) one temporally forward and one temporally backwards Enable information flow. With the error backpropagation the modeling of the dynamic system is achieved by that the weights in the neural network are optimized to that the error between modeled output and given output is minimal.

at recurrent neural networks, the problem is that the networks cover only a finite time interval and no suitable initials internal state values for initializing the learning of the neural Network available. traditionally, becomes the result of the missing initial state values Inconsistency in the neural network neglected. But it is also one solution known in which a noise generator depending on the backpropa gierte Error generates a noise with which the uncertainty of the unknown estimated initial state becomes. However, considered the solution not far away from the beginning of the time interval Conditions. About that also not the future Behavior of the dynamic system considered.

The Holk Cruse: Neural Networks as Cybernetic Systems, 2nd and revised edition, Brains, Minds and Media, excerpt, publ. October 2006, "Table of Contents "(4 Pages) and pages 103 to 139, describes the concept of recurring neural networks and corresponding learning methods for them Networks.

In the document US 2007/0022062 A1 describes a method for predicting the behavior of a dynamic system, which approximates the future behavior based on a similarity analysis and predicts the dynamic behavior of the system by means of a causality analysis based on a causal-retro-causal neural network.

In the publication US 2004/0267684 A1 the concept of causal-retro-causal neural networks is described.

task The invention is a method for learning a neural To create network, which helps to improve the prediction accuracy This network of information flow from the past or from suitable for the future becomes.

These Task is by the independent claims solved. Further developments of the invention are defined in the dependent claims.

The inventive method to learn a recurrent neural network is characterized from that the information flow in the recurrent neural network ago Start of the time interval and / or after the beginning of the time interval with the help of at least one retro-causal network or with help from at least one forward directed and / or at least a backward connector considered becomes. A forward directed connector allows exclusively a time forward directed information flow, whereas a backward-looking connector uses only one temporally backward information flow allows.

Thus, according to the invention the past or future through appropriate retro-causal Networks or corresponding directional connectors become. In retro-causal Networks is the causality direction conversely, d. H. these nets run from the present into the past or from the future to the present.

• – eine Zustandsverteilung am Ende des Zeitintervalls gemessen wird, wobei die gemessene Zustandsverteilung über einen vorwärts gerichteten Konnektor mit dem Zustand am Ende des Zeitintervalls verbunden ist;
• – in Abhängigkeit von der gemessenen Zustandsverteilung ein Rauschen generiert wird, wobei das generierte Rauschen über einen rückwärts gerichteten Konnektor mit der gemessenen Zustandsverteilung am Ende des Zeitintervalls und über einen vorwärts gerichteten Konnektor mit dem Zustand am Anfang des Zeitintervalls verbunden ist;
• – die Fehlerverteilung am Anfang des Zeitintervalls gemessen wird, wobei die gemessene Fehlerverteilung am Anfang des Zeitintervalls über einen rückwärts gerichteten Konnektor mit dem Zustand am Anfang des Zeitintervalls verbunden ist;
• – in Abhängigkeit von der gemessenen Fehlerverteilung am Anfang des Zeitintervalls eine Fehlerverteilung am Ende des Zeitintervalls generiert wird, wobei die generierte Fehlerverteilung am Ende des Zeitintervalls über einen vorwärts gerichteten Konnektor mit der gemessenen Fehlerverteilung am Anfang des Zeitintervalls und über einen rückwärts gerichteten Konnektor mit dem Zustand am Ende des Zeitintervalls verbunden ist.

When using forward or backward connectors, a network model can be used which takes into account the effect of the unknown future or past suitably by noise with respect to the initial state and the future error. The network model is stiffened against this noise. The consideration of the information flow before the beginning of the time interval and after the end of the time interval is carried out in particular such that:

• A state distribution is measured at the end of the time interval, the measured state distribution being connected via a forward connector to the state at the end of the time interval;
• A noise is generated in dependence on the measured state distribution, wherein the generated noise is connected via a backward connector to the measured state distribution at the end of the time interval and via a forward connector to the state at the beginning of the time interval;
• The error distribution at the beginning of the time interval is measured, the measured error distribution at the beginning of the time interval being connected via a backward connector to the state at the beginning of the time interval;
• - An error distribution at the end of the time interval is generated as a function of the measured error distribution at the beginning of the time interval, wherein the generated error distribution at the end of the time interval via a forward connector with the measured error distribution at the beginning of the time interval and via a backward connector with the state connected at the end of the time interval.

Under State distribution here is the determined over a period of time Probability distribution of the states understood. With the above described structure is suitably by using the forward- or backward Connectors a stiffening of the network against the uncertainty of the states in the reached the distant future or the distant past. Preferably In this case, the measured state distribution at the end of the time interval and / or the generated error distribution at the end of the time interval each modeled by a target cluster, with its target value in particular set to zero. Likewise, preferably the measured error distribution at the beginning of the time interval and / or the generated noise at the beginning of the time interval by a Input cluster is modeled, its expected input value also is preferably set to zero.

The to be learned neural network describes in a preferred embodiment a time interval which is one or more consecutive past states, a present one State and one or more consecutive future states. When using retro-causal Networks for modeling the flow of information before the beginning of the time interval in this case preferably a first retro-causal network is used, where the first retro-causal network is of a current state extends to the state at the beginning of the time interval. Analog becomes to model the flow of information after the end of the time interval preferably a second retro-causal network is used, wherein the second retro-causal network of the state at the end of the time interval to a current one State extends. To improve the learning of the neural network, is in a preferred variant, an error in the second retro-causal Net by comparison of the current one State in the time interval with the current state of the second determined retro-causal network. This error is in the recurrent neural network back propagated. Preferably, the determined error is with the current state in the time interval over a forward directed connector, thereby providing a reflux the error directly to the current one Prevent state in the time interval.

The inventive method can for any designs used by recurrent neural networks. Especially The method can also be used to learn complex recurrent neural networks be used, such. B. so-called. DCNN networks (DCNN = Dynamical Consistent Neural Networks) or NRNN networks (NRNN = Normalized Recurrent Neural Network). These network structures are sufficient known in the art.

Preferably become the inputs and outputs with the neural network to be learned of a technical system, d. H. the neural network becomes learned with input or output data of a technical system. The technical system can be any technical device be, for example, a gas turbine or a wind turbine. Optionally, the inventive method, however, also for dynamic economic Systems can be used, for example, the recurrent neuronal Network modeling the inputs and outputs of an energy trading exchange, to predict, for example, an energy price.

In addition to the method described above for learning a recurrent neural network the invention further provides a method for predicting the outputs of a dynamic system based on inputs of the dynamic system, wherein the prediction is performed with a neural network learned with the learning method described above.

The Invention relates to this a computer program product with one on a machine-readable one carrier stored program code for performing the above inventive method, if the program runs on a computer.

embodiments The invention will be described below with reference to the attached figures described in detail.

It demonstrate:

1 to 3 schematic representations of recurrent neural networks and their learning methods according to the prior art;

4 a schematic representation of the solution of the problem according to the invention based on a first embodiment of the invention;

5 a schematic representation of a recurrent neural network based on a first embodiment of the invention;

6 a schematic representation of a recurrent neural network based on a second embodiment of the invention;

7 a schematic representation of a recurrent neural network based on a third embodiment of the invention;

8th an embodiment of a DCNN network according to the prior art;

9 a schematic representation of a DCNN network based on a fourth embodiment of the invention;

10 a schematic representation of a DCNN network based on a fifth embodiment of the invention; and

11 a schematic representation of a DCNN network based on a sixth embodiment of the invention.

1 schematically shows the basic process of learning a recurrent neural network with the so-called. Error Back Propagation Algorithm (German: error-backpropagation algorithm). In the 1 schematically a recurrent neural network with an input layer I, a hidden layer H and an output layer O is reproduced. Inputs I are fed via the input layer I to the neural network, this layer being connected to the hidden layer H. The hidden layer H is in turn connected to outputs y of the output layer O. By appropriate arrows P1 and P2, the information flow from the input layer I via the hidden layer H to the output layer O is reproduced. The input layer and the hidden layer are linked via weights W1 and the hidden layer and the output layer via weights W2. The non-linearity of the network is expressed by the function f (z) = tanh (z) in the hidden layer H and overall it follows 1 a multilayer perceptron architecture in which the output y is described as follows: y = W ₂ f (W ₁ x).

Typically, such a network is learned with the error backpropagation algorithm based on the following local solution:

Here E denotes the averaged error and E _t the error at the individual time steps t in the time interval [1, ..., T]. y _t ^d denotes output data on the basis of which the network is learned and NN (x _t , w) the corresponding output of the neural network. The weights are thus based in the neural network learned on the backpropagated error such that the averaged error E is minimal. The backpropagation of the error is indicated by the downwards arrows P3 and P4 in FIG 1 indicated.

2 clarifies the basic procedure for modeling a dynamic system with a recurrent neural network for a better understanding. The recurrent neural network is based on a finite unfolding in temporal direction, which transforms time into a spatial architecture. According to 2 the basic neural network structure NN is described by inputs u and outputs y, whereby a transformation takes place between the inputs and outputs, taking into account internal states s. In particular, at time t, the input u _{t of} the system is transformed to the internal state s _{t + 1} at this time using the internal state s _t as follows: s t + 1 = f (s t , u t )

From the internal state s _t at the time t, the output y _t at the time t also results from the following equation: y t = g (s t )

The internal state s _t at time t is thus used to determine the internal state s _{t + 1} at the subsequent time t + 1, which is indicated by the loop L. As represented by the arrow A1 and the equations (1) and (2), the equations for determining s _{t + 1} and y _t by matrices A, B and C and the parameter c can be described in a specific variant. The learning is based on the equation (3), according to which the quadratic error averaged between the calculated y _t output and the output y _t ^d in the time interval [1, ..., T] is minimized according to the record used for learning. As indicated by the arrow A2, the just explained network structure can be represented by a corresponding representation as network RNN. In the network RNN, the inputs u _t-3 , u _t-2 , u _t-1 and u _t , the internal states s _t-2 , s _t-1 , s _t , s _{t + 1} , s _{t + 2} , s _{t + 3} , s _{t + 4} and the outputs y _t-2 , y _t-1 , y _t , y _{t + 1} , y _{t + 2} , y _{t + 3} and y _{t + 4 of} the neural Network shown. Similarly, arrows A, B and C are used to calculate the individual quantities according to equations (1) and (2). In this case, the arrows represent connectors which enable both a time-forwarded and a backward-looking information flow in order to ensure error backpropagation.

A problem of the basis of 2 The recurrent neural network described is that the network only describes a finite time period, which in the embodiment according to FIG 2 the past time steps t - 2 and t - 1, the current time step t and the future time steps t + 1, t + 2, t + 3 and t + 4 includes. Due to this finite structure, a suitable internal state s _tm is missing for the initialization of the network. This problem increases the larger the dimension of the internal state space. Usually, the problem of the missing initial state is ignored and s _tm = 0 is set or the independence from the initial state is postulated.

3 shows a solution according to the prior art for determining an initial state. The solution is based on the so-called cleaning calculus, according to which the uncertainty of an unknown initial state s _{tm is} estimated in a suitable manner. This estimation is done by modeling noise which is transmitted through a corresponding connector id to the neural network RNN of the 3 is supplied. The noise is denoted by n (n = noise) and added to the original state. The noise of the initial state is calculated by adaptive Gaussian noise as a function of the residual error that results from error backpropagation. The backpropagation of the fault is in the diagram EBP of 3 indicated. According to the diagram, an input x is applied to the input layer I by correspondingly upwardly directed arrows via the hidden layer H to the output layer O, which provides a corresponding output y _t . With the aid of the target output y _t ^d according to the data set used for learning, the error is then calculated and backpropagated, whereby the original weights w between the layers are adapted accordingly and lead to the weights w ₊ . This is indicated by the corresponding mathematical formulas F1 and F2. Finally, one gets the backpropagated error ∂ at the end. Depending on this error, the corresponding standard deviation of the Gaussian noise is determined, so that it results in total as the initial stress level s _tm : s tm = 0 + AGN (0, σ (∂ t )) (4)

Here, AGN denotes the adaptive Gaussian noise and σ its standard dependent on ∂ _t deviation. With the aid of the adaptive Gaussian noise, a stiffening of the model with respect to the unknown and therefore uncertain initial state s _{tm is} achieved. This is indicated by the diagram D in 3 indicated, which illustrates the change in the uncertainty of the state s _t over time t. It can be seen here that the trajectory of the temporal evolution of the state s _t can be described as a finite volume tube, wherein an initially very large and unsafe state with a high volume is converted into a state with a small tube volume. The matrix A used thus becomes a contraction to push out the initial uncertainty from the system.

In the solution according to 3 If an initial state is calculated, however, this initial state does not sufficiently take into account the missing information flow of the past and the future. In particular, no suitable embedding of a finite temporally unfolded recursive neural network is achieved in an infinite time process. The general problem here is that in the network RNN the 2 the flow of information from the distant past and the distant future development of the system is missing, so that there is an uncertainty in the initial state of finite development.

To solve the above problem, the in 4 schematically presented procedure according to a first embodiment of the invention proposed. 4 shows a recurrent neural network RNN based on the illustration 2 taking into account, however, the long past and the distant future, as indicated by dashed arrows PA (PA = past) and FU (future = FU). In the network of 4 For example, the distribution of the initial states is estimated by a distribution of the final states of the dynamic system modeled with the neural network, the final distribution being injected as noise into the network to stiffen the neural network against the uncertainty of the initial state. The estimation of the distribution of the initial states based on the distribution of the final states is indicated by the arrow A3 in FIG 4 indicated. Moreover, the distribution of future errors is approximated by a distribution of the backpropagated errors at the beginning of the time interval (ie at time t - 2), where the distribution of future errors is injected as noise into the network to protect the network against the uncertainty of future ones To stiffen errors. The estimation of the distribution of the future errors by the distribution of the backpropagated errors is indicated by the arrow A4 in FIG 4 indicated.

5 shows the technical implementation of the basis of 4 generally described solution. The technical realization takes place here based on forward and backward connectors. It should first be noted that the original network structure RNN contains only connectors, which act in both directions, ie both directed forward in time and are directed backward in time, which over the temporal backward orientation, the error propagation is guaranteed. In contrast, the embodiment of the neural network according to 5 now, in addition to the original connectors of the network RNN, which are linked to the corresponding matrices A, B and C, also exclusively temporally forward or exclusively time-backward connectors. According to 5 Here, the connectors C1, C2 and C3 are exclusively forward-facing connectors, whereas the connectors C4, C5 and C6 are exclusively backward connectors. All connectors describe a corresponding identity map Id. The network according to 5 also contains two target clusters T1 and T2 and two input clusters I1 and I2. For the target clusters, the target value tar is set to 0. For input clusters I1 and I2, an input value of 0 is also assumed. The network structure according to 5 consists overall of an upper subnetwork N1 and a lower subnetwork N2, wherein through the target clusters T1 and T2 and the input clusters I1 and I2 and the corresponding connectors C1 to C6, a dynamic consistent learning of the recurrent neural network takes place.

According to the structure 5 is determined via the target cluster T1, which is connected via the forward connector C1 to the state s _{t + 4} at the end of the finite time interval, the distribution of the states at the end of the time interval. The value of the target cluster thus results in the value tar = 0 + s (s corresponds to the value of the measured state). Since this value deviates from the target specification of zero, the measured state distribution is interpreted as an error and fed via the backward connector C4 to the input cluster I1. The input cluster I1 thus receives as input value the state value s and generates a corresponding noise from this. The generation of the noise takes place with the aid of a noise generator, wherein noise generators are well known from the prior art and therefore will not be explained in detail at this point. The generated noise is then supplied via the forward connector C2 to the state s _t-2 at the beginning of the time interval. In addition, the error propagated backward in the network flows out of state s _t-2 and goes backward Connector C5 supplied to the further input cluster I2, which determines therefrom an error distribution. The error distribution is then fed via the forward connector C3 to the target cluster T2, whereupon again a noise is generated with a corresponding noise generator, which finally via the backward connector C6 and the state s _{t + 4} in the recurrent neural network is propagated back.

The error E results in the embodiment according to 5 here by the following calculation, where out denotes the output value and tar the target value: E = 1 2 (out - tar) 2

In summary, the upper subnetwork N1 generates an input noise depending on the final state distribution, whereas the lower subnetwork N2 measures the input uncertainty and injects an additional error flow into the neural network. The principle of the neural network according to 5 It is based on the assumption that predicting the distant future and the distant past is not possible. Due to this ignorance of the distant future and the distant past, the model is hardened by corresponding noise towards these distant times. In the embodiment of the 5 Thus, the effect of the unknown time in the future and the past on the finite modeling within a time interval is limited by a suitable noise of the initial state as well as the future error. Trajectories are no longer a sequence of points, but contracting finite volume tubes.

6 shows a second embodiment of a neural network according to the invention. In the embodiment of the 6 the initial state of the neural network is modeled by a so-called retro-causal network, which adjoins the state s _t-3 at the beginning of the time interval of the recurrent neural network RNN. The retro-causal neural network is in 6 designated RC1. With the network starting from the state s _t-3 against the causality direction with corresponding matrices A ← the internal states r _t-2 , r _t-1 and r _t determined, and the recalculated state r _t is fed to an input cluster I 'with noise. The noise is generated with a noise generator, with the noise - as in 3 Is modeled as Gaussian noise as a function of the residual error. Above the retro-causal network RC1 the corresponding equations for the description of the network RC1 are shown. This designates s ← t the internal state of the retro-causal network. s ← t corresponds to r _t . Furthermore, the equations for describing this network are reproduced in the right part above the original recurrent neural network RNN.

With the embodiment according to 6 a causal-retro-causal embedding is created for the backward formulation of the recurrent model, with the recurrent network RC1 proceeding from the input cluster I 'first going from the present into the past and finally with the adjoining recurrent neural network RNN of the Past into the present and from there into the future. In the embodiment according to 6 For example, the matrices B, C for the retro-causal network RC1 are identical to the matrices B, C of the recurrent neural network RNN, but this is not necessarily always the case. The model according to 6 has been described in combination with a recurrent network RNN, but can also be applied to NRNN (Normalized Recurrent Neural Network) and DCNN (Dynamic Consistent Neural Network) networks, which are specific embodiments of recurrent neural networks.

7 shows a third embodiment of a neural network according to the invention. In the embodiment according to 7 finite causal development is also embedded in a retro-causal network. This is done for the past already on the basis of 6 described retro-causal network RC1 used. The distant future is modeled with another retro-causal network RC2, which is calculated starting from the state back to the present, starting from the state s _{t + 3} at the end of the finite time interval. This results in the internal states r _{t + 2} , r _{t + 1} and r _t '. The network of 7 also has an exclusively forward connector C7 connected to a target cluster T '. This connector is optional and can be omitted if necessary. Overall, results from 7 a network which starts via the identity connector Id from the left with the presence in the state r _t and, via the retro-causal network RC1 against the causality direction, first runs into the past to the state s _t-3 . Subsequently, the recurrent neural network RNN is traversed in the causal direction from the past to the present (state s _t ) to the future (state s _{t + 3} ). Finally, in turn, the second retro-causal network RC2 is traversed from the future to the current state of r _t 'in retro-causal direction. Unlike the embodiment according to 5 Follow the network according to 7 the approach, to describe the finite modeling according to the network RNN as an expansion of the known present in the future and the past. The trajectories of the states are here a sequence of points. With the additional connector C7 done by the target cluster T ', a comparison of the current state s _t with the recalculated state r _t ', these states should match. The deviation of these states is in turn propagated back as an error in the network. In this way, the logical consistency s _t = r _t 'is taken into account in the information flow. Since the connector C7 is directed only forward, no error flow in the reverse direction via this connector to the state s _t .

In the foregoing, the invention has been described based on a conventional recurrent neural network RNN. However, the invention can be easily extended to DCNN networks. DCNN networks are well known in the art and enable a dynamically consistent description of a dynamic system. For a more detailed description of DCNN networks is in particular the publication DE 10 2004 059 684 B3 Reference is made to the entire contents of the disclosure by reference to the content of the present application. 8th shows the conventional structure of a DCNN network according to the prior art. It can be seen that data y _t-1 ^d and y _t ^d used in the network for learning flow into the network as inputs using the vectors V1. Furthermore, the outputs y _t-1 and y _t to be modeled are connected to the internal states s _t-1 and s _t via corresponding vectors V2. In addition, in addition to the linear part described by the matrices A, C _≦ and C _> , there is also a nonlinear part, which in 8th denoted by NL. This part is described by a corresponding tanh function.

für alle τ: y_τ=[Id 0 0]s_τ The network according to 8th is thus described by the following equations:

for all τ: y _{τ =} [Id 0 0] s _τ

For clarity, the DCNN network was the 8th only for a finite time interval consisting of the times t - 1, t and t + 1 reproduced. Typically, a DCNN network involves a much larger number of time steps.

The first, based on 5 described embodiment of the invention can be analogously also transferred to DCNN networks, as in 9 is reproduced. Here, like reference numerals designate the same components. Analogous to 5 For example, forward-facing connectors C1 through C3 and backward connectors C4 through C6 are used to model the far-future and far-past.

Analogous to the connectors C3 and C4 in 5 are also the connectors C3 and C4 in 9 Identity connectors. In contrast to the connectors C1, C2, C5 and C6, which are in 5 also represent identity connectors, these connectors are in 9 now connectors, which are described by the following matrix:

In the network of 9 Further, the target value y _t-1 ^{tar of} the output at time t-1 enters state s _t-1 .

Analogous to the network according to 5 will be in the 9 with the upper subnetwork N1 generates an input noise in dependence on the final state distribution, whereas an input uncertainty is measured by the lower subnetwork N2 and injected into the network as an additional error flow.

10 shows the transmission of the second embodiment of the invention according to 6 on a DCNN network. Analogous to 6 includes the network of 10 again a retro-causal part RC1, which for reasons of clarity consists only of a time step from the past (time t-1) to the current time t. In turn, the retro-causal part calculates a past system state back to the present, and this recalculated system state is used as the initial state of the causal network analysis. In the example of 10 In subnet RC1, the same consistency matrix C _≤ as in the remaining DCNN network DCNN is used. The system is again initialized with noise via an input cluster I '. The network according to 10 is particularly suitable when considering short time series.

11 shows a transmission of the third embodiment of the invention according to 7 on a DCNN network. Analogous to 7 The state s _{t-1 is followed by} the retro-causal network RC1, and the state s _{t + 1 of} the network DCNN is followed by the retro-causal network RC2. For clarity, both grids RC1 and RC2 contain only one time step, which recalculates from the past to the present or from the future to the present. In addition, the error between the calculated state r _t 'and s _{t is} again propagated back into the network via the target cluster T' by a corresponding connector C7. Analogous to the embodiment of the 7 is achieved by the embodiment of 11 a finite causal unfolding embedded in a retro-causal network. The network starts in the present and ends again in the present. Both the causal and the retro-causal parts use the same consistency matrices C _≤ .

By The embodiments described above become neural Network structures created, which with appropriate inputs and Outputs from known datasets be learned. The inputs and outputs represent measured values a dynamic system. It can with the method any dynamic systems, in particular economic systems and technical Systems to be modeled. The corresponding network is initially with known records learned, and then the learned net becomes a prediction used by expenses based on inputs. A scope the method according to the invention is for example the prediction of energy prices of an energy trading exchange. Another preferred Scope is the prediction the dynamic behavior of technical systems. For example can be based on setting parameters of the technical system Output parameters of the technical system are predicted. A possible Application here is a gas turbine, in which based on setting parameters, such as B. supplied Fuel quantity, the hum or the energy output of the gas turbine is calculated. Likewise, the method can be used for wind turbines where, based on wind turbine settings, such as B. the position of the rotor blades of the wind generator, the resulting energy is calculated.

Claims (15)

A method for computer-aided learning of a recurrent neural network (RNN) with temporally successive states (s _t-2 , ..., s _{t + 4} ) in a finite time interval, wherein the states (s _t-2 , ..., s _{t + 4} ) are connected to successive inputs (u _t-3 , ..., u _t ) and outputs (y _t-2 , ..., y _{t + 4} ) of a dynamic system and wherein the states (s _t-2 , ..., s _{t + 4} ) are interconnected with each other and with the inputs and outputs via convectors, which allow for error backpropagation a time-forward and a time backward information flow, characterized in that the information flow in the recurrent neural network (RNN) before the start of the time interval and / or after the end of the time interval with the aid of at least one retro-causal network (RC1, RC2) or with the help of at least ei a forward-facing connector (C1, ..., C6) is considered, wherein the at least one forward connector (C1, C2, C3) only allows a forward flow of information and wherein the at least one backward Directed connector (C4, C5, C6) allows only a time backward information flow. The method of claim 1, wherein the consideration of the information flow in the recurrent neural network (RNN) before the beginning of the time interval and after the end of the time interval is such that - a state distribution at the end of the time interval is measured, wherein the measured state distribution via a forward connector (C1) is connected to the state (s _{t + 4} ) at the end of the time interval; - Noise is generated as a function of the measured state distribution at the end of the time interval, wherein the generated noise via a backward connector (C4) with the measured state distribution at the end of the time interval and via a forward connector (C2) with the state ( s _t-2 ) is connected at the beginning of the time interval; The error distribution at the beginning of the time interval is measured, the measured error distribution at the beginning of the time interval being connected via a backward connector (C5) to the state (s _t-2 ) at the beginning of the time interval; - An error distribution at the end of the time interval is generated depending on the measured error distribution at the beginning of the time interval, wherein the generated error distribution at the end of the time interval via a forward connector (C3) with the measured error distribution at the beginning of the time interval and via a backward connector (C6) is connected to the state (s _{t + 4} ) at the end of the time interval. The method of claim 2, wherein the measured State distribution at the end of the time interval and / or the generated Error distribution at the end of the time interval by a target cluster (T1, T2) be modeled. Method according to one of the preceding claims, in the measured error distribution at the beginning of the time interval and / or the generated noise at the beginning of the time interval an input cluster (I1, I2) are modeled. Method according to one of the preceding claims, wherein the finite time interval one or more successive past states (s _t-2 , ..., s _t-1 ), a current state (s _t ) and one or more successive future states (s _{t + 1} , s _{t + 4} ). The method of claim 5, wherein the flow of information before the beginning of the time interval by a first retro-causal network (RC1) is modeled, wherein the first retro-causal network (RC1) _(t r) from a current state to the state (s _t-3 ) at the beginning of the time interval. Method according to Claim 5 or 6, in which the information flow after the end of the time interval is modeled by a second retro-causal network (RC2), the second retro-causal network (RC2) being dependent on the state at the end of the time interval (s _{t +3} ) to a current state (r _t '). Method according to Claim 7, in which an error in the second retro-causal network (RC2) is determined by comparing the current state (s _t ) in the time interval with the current state (r _t ') of the second retro-causal network (RC2), the error is back propagated into the recurrent neural network (RNN). The method of claim 8, wherein the detected error is associated with the current state (s _t ) at the time interval via a forward connector (C7). Method according to one of the preceding claims, in the recurrent neural network is a DCNN network and / or an NRNN network includes. Method according to one of the preceding claims, wherein with the recurrent neural network (RNN) the inputs (u _t-3 , ..., u _t ) and outputs (y _t-2 , ..., y _{t + 4} ) of a technical system can be modeled. The method of claim 11, wherein the technical System comprises a gas turbine and / or a wind turbine. Method according to one of Claims 1 to 10, in which the neural network (RNN) inputs (u _t-3 , ..., u _t ) and outputs (y _t-2 , ..., y _{t + 4} ) of an energy trading exchange. A method of predicting the outputs (y _t-2 , ..., y _{t + 4} ) of a dynamic system based on inputs (u _t-3 , ..., u _t ) of the dynamic system in which the prediction with a neural A network is performed, which is learned with a method according to any one of the preceding claims. Computer program product with one on a machine-readable carrier stored program code for performing a method according to one of the preceding claims, when the program runs on a computer.