EP1384198A2 - Method and assembly for the computer-assisted mapping of a plurality of temporarily variable status descriptions and method for training such an assembly - Google Patents

Abstract

The invention relates to the computer-assisted mapping of a plurality of temporarily variable status conditions. According to the invention, a first status description in a first state space is mapped onto a second status description in the second state space by mapping, and the second status description of a temporarily later state is taken into consideration during mapping. By carrying out a further mapping, the second state description is mapped back onto a third state description in the first state space.

Description

description

Method and arrangement for computer-aided mapping of several time-varying state descriptions and method for training such an arrangement

The invention relates to a method and an arrangement for the computer-assisted mapping of several time-varying status descriptions and a method for training an arrangement for the computer-assisted mapping of several time-changing status descriptions.

From [1] - it is known to use an arrangement for depicting several time-variable state descriptions to describe a dynamic process. This arrangement is realized by interconnected computing elements, using which the mapping is carried out.

In general, a dynamic process is usually described by a state transition description, which is not visible to an observer of the dynamic process, and an output equation, which describes observable quantities of the technical dynamic process.

Such a structure is shown in Fig.2a.

A dynamic system 200 is subject to the influence of an external input variable u of predeterminable dimension, an input variable ut at time t being designated u- ^:

^u t 9Ϊ-

where 1 is a natural number.

The input variable u at a time t causes one

Modification of the dynamic process that takes place in the dynamic system 200. An internal state s (st e 5R ^m ) of predeterminable dimension at a time t cannot be observed by an observer of the dynamic system 200.

Depending on the inner state st and the input variable ut, a state transition of the inner state s of the dynamic process is caused and the state of the dynamic process changes into a subsequent state s - ^ + i at a subsequent time t + 1.

The following applies:

st +1 = ^f ( ^s X ^u t) - ⁽ D

where f (.) denotes a general mapping rule.

An output variable y observable by an observer of the dynamic system 200 at a time t depends on the input variable u and the internal state st.

The output variable y (yt s SR ⁿ ) is predeterminable dimension n.

The dependency of the output variable yt on the input variable u and the internal state st of the dynamic process is given by the following general rule:

Yt = g ( ^s t ' ^u t)' (2)

where g (.) denotes a general mapping rule.

To describe the dynamic system 200, [7] an arrangement of interconnected computing elements in the form of a neural network of interconnected neurons is used. The connections between the neurons of the neural Network are weighted. The weights of the neural network are summarized in a parameter vector v.

Thus, an inner state of a dynamic system, which is subject to a dynamic process, depends on the input variable ut and the inner state of the previous point in time st and the parameter vector v in accordance with the following regulation:

st + i = NN (v, st, ut), (3)

where NN (.) denotes a mapping rule specified by the neural network.

The arrangement known from [1] and referred to as Time Delay Recurrent Neural Network (TDRNN) is trained in a training phase in such a way that for each input variable ut a target variable yt is determined on a real dynamic system. The tuple (input variable, determined target variable) is called the training date. A large number of such training data form a training data set.

The successive tuples (ut-4 _. Yf_ _Δ ) (ut-3 ry _ ₃ ), (ut-2> yf- ₂ ^{) of the points in} time (t-4, t-3, t-3, ...) of the training data set each have a predetermined time step.

The TDRNN is trained with the training data record. An overview of various training methods can also be found in [1].

It should be emphasized at this point that only the output variable y can be seen at a time t of the dynamic system 200. The "internal" system state st cannot be observed. The following cost function E is usually minimized in the training phase:

where T is a number of times taken into account.

[2] also provides an overview of the basics of neural networks and the possible uses of neural networks in the area of economics.

The known arrangements and methods have the disadvantage, in particular, that a dynamic process to be described can only be described with insufficient accuracy. This is due to the fact that the state transition description of the dynamic process can only be reproduced with insufficient accuracy with the illustrations used in these arrangements and methods.

The invention is therefore based on the problem of specifying a method and an arrangement and a method for training an arrangement for computer-aided mapping of a plurality of time-varying state descriptions, with which a state transition description of a dynamic system can be described with improved accuracy and which arrangement and which methods do not Disadvantages of the known arrangements and methods are subject.

The problems are solved by an arrangement and a method with the features according to the respective independent claim.

The method for the computer-aided mapping of several time-variable state descriptions, each with a time-variable state of a dynamic system Describing an associated point in time in a state space which dynamic system maps an input variable to an associated output variable has the following steps:

a) a first state description is mapped in a first state space to a second state description in a second state space by a first mapping, b) the second state description of a temporally earlier state is taken into account in the first mapping, c) the second state description second status description mapped to a third status description in the first status space, characterized in that d) the first status description is mapped by a third mapping to a fourth status description in the second status space, e) in the third mapping the fourth status description of a later status is taken into account and f) the fourth status description is mapped by a fourth image to the third status description, the images being adapted in such a way that the images of the first status description onto the third status description match the image de r Write the input variable to the associated output variable with a specified accuracy.

The arrangement for the computer-aided mapping of several time-variable state descriptions, each of which describes a time-changing state of a dynamic system at an associated time in a state space, which dynamic system maps an input variable to an associated output variable, has the following components: a) with a first imaging unit, which is set up in such a way that a first state description in a the first state space can be mapped by a first mapping to a second state description in a second state space, b) and the first mapping unit is set up in such a way that in the first mapping the second state description of an earlier state can be taken into account, c) with a second mapping unit that is set up in such a way that the second status description can be mapped by a second image to a third status description in the first status space, characterized in that d) the arrangement has a third imaging unit which is set up in such a way that the first status description by a third Mapping can be mapped to a fourth status description in the second status space, e) and the third mapping unit is set up such that the fourth status description of a later status can be taken into account in the third mapping, f) and the arrangement a fourth mapping Idungseinheit, which is set up in such a way that the fourth status description can be mapped to the third status description by a fourth mapping, the mapping units being set up in such a way that the mapping of the first status description to the third status description shows the mapping of the input variable to the associated output variable with a describe the specified accuracy.

The method for training an arrangement for computer-aided formation of a plurality of time-variable state descriptions, each of which describes a time-variable state of a dynamic system at an associated point in time in a state space, which dynamic system maps an input variable to an associated output variable, which arrangement has the following components : a) with a first mapping unit that is set up in such a way that a first status description in a first status area can be mapped by a first mapping to a second status description in a second status area, b) and the first mapping unit is set up in such a way that in the first mapping the second status description of an earlier status can be taken into account, c) with a second mapping unit, which is set up in such a way that the second status description can be mapped by a second mapping to a third status description in the first status space, d) with a third mapping unit, such It is set up that the first status description can be mapped by a third mapping to a fourth status description in the second status space, e) and the third mapping unit is set up in such a way that the fourth mapping describes the fourth status description of a later one Condition can be taken into account, f) with a fourth mapping unit, which is set up in such a way that the fourth description of the condition can be represented by a fourth mapping on the third description of the condition, has the following training steps:

- During the training, using at least one predetermined training data pair, which is formed from the input variable and the associated output variable, the mapping units are set up in such a way that the mapping of the first status description to the third status description, the mapping of the input variable to the associated output variable with a describe the specified accuracy. The arrangement is particularly suitable for carrying out the method according to the invention or one of its further developments explained below.

Preferred developments of the invention result from the dependent claims.

The further developments described below relate both to the method and to the arrangement.

The invention and the further developments described below can be implemented both in software and in hardware, for example using a special electrical circuit.

Furthermore, an implementation of the invention or a further development described below is possible by means of a computer-readable storage medium on which a computer program is stored which carries out the invention or further development.

The invention or any further development described below can also be implemented by a computer program product which has a storage medium on which a computer program which carries out the invention or further development is stored.

In one configuration, an imaging unit is implemented by a neuron layer composed of at least one neuron. However, an improved simulation of a dynamic system with regard to accuracy can be achieved by using several neurons in one neuron layer. In a further development, a state description is a vector of a predefinable dimension. Further training is preferably used to determine the dynamics of a dynamic process.

One embodiment has a measuring arrangement for detecting physical signals with which the dynamic process is described.

Further training is preferably used to determine the dynamics of a dynamic process which takes place in a technical system, in particular in a chemical reactor, or to determine the dynamics of an electrocardio gram, or to determine economic or macroeconomic dynamics.

Further training can also be used to monitor or control a dynamic process, in particular a chemical process.

The status descriptions can be determined from physical signals.

A further development is used in speech processing, the input variable being first speech information of a word to be spoken and / or a syllable to be spoken, and the output variable being second speech information of the word to be spoken and / or the syllable to be spoken.

In a further embodiment, the first speech information comprises a classification of the word to be spoken and / or the syllable to be spoken and / or pause information of the word to be spoken and / or the syllable to be spoken. The second speech information includes accentuation information of the word to be spoken and / or the syllable to be spoken. A realization in the field of speech processing is also possible, in which the first speech information comprises phonetic and / or structural information of the word to be spoken and / or the syllable to be spoken and / or the second speech information contains frequency information of the word to be spoken and / or the speaking syllable.

Embodiments of the invention are prepared in figures Darge ^¬ and are explained hereinafter.

Show it

Figure 1- sketch of an arrangement according to a first embodiment (KRKNN);

FIGS. 2a and 2b show a first sketch of a general description of a dynamic system and a second sketch of a description of a dynamic system which is based on a “causal-retro-causal” relationship;

Figure 3 shows an arrangement according to a second embodiment (KRKFKNN);

FIG. 4 shows a sketch of a chemical reactor, from which quantities are measured, which are processed further with the arrangement according to the first exemplary embodiment;

FIG. 5 shows a sketch of an arrangement of a TDRNN which is unfolded over time with a finite number of states;

FIG. 6 shows a sketch of a traffic control system which is modeled with the arrangement in the context of a second exemplary embodiment; Figure 7 sketch of an alternative arrangement according to a first embodiment (KRKNN with loosened connections);

Figure 8 sketch of an alternative arrangement according to a second embodiment (KRKFKNN with loosened connections);

FIG. 9 sketch of an alternative arrangement according to a first exemplary embodiment (KRKNN);

FIG. 10 sketch of a speech processing using an arrangement according to a first exemplary embodiment (KRKNN);

FIG. 11 sketch of a speech processing using an arrangement according to a second exemplary embodiment (KRKFKNN).

First embodiment: chemical reactor

4 shows a chemical reactor 400 which is filled with a chemical substance 401. The chemical reactor 400 comprises a stirrer 402 with which the chemical substance 401 is stirred. Further chemical substances 403 flowing into the chemical reactor 400 react for a predeterminable period in the chemical reactor 400 with the chemical substance 401 already contained in the chemical reactor 400. A substance 404 flowing out of the reactor 400 becomes from the chemical reactor 400 via an outlet derived.

The stirrer 402 is connected via a line to a control unit 405 with which a stirring frequency of the stirrer 402 can be set via a control signal 406. A measuring device 407 is also provided, with which concentrations of chemical substances contained in chemical substance 401 are measured.

Measurement signals 408 are fed to a computer 409, in which

Computer 409 is digitized via an input / output interface 410 and an analog / digital converter 411 and stored in a memory 412. A processor 413, like the memory 412, is connected to the analog / digital converter 411 via a bus 414. The calculator 409 is also on the

Input / output interface 410 connected to the controller 405 of the stirrer 402 and thus the computer 409 controls the stirring frequency of the stirrer 402.

The computer 409 is also connected via the input / output interface 410 to a keyboard 415, a computer mouse 416 and a screen 417.

The chemical reactor 400 as a dynamic technical system 250 is therefore subject to a dynamic process.

The chemical reactor 400 is described by means of a status description. In this case, an input variable ut of this state description is composed of an indication of the temperature prevailing in the chemical reactor 400, the pressure prevailing in the chemical reactor 400 and the stirring frequency set at time t. The input variable ut is thus a three-dimensional vector.

The aim of the modeling of the chemical reactor 400 described in the following is to determine the dynamic development of the substance concentrations, in order to enable efficient generation of a predefinable target substance to be produced as the outflowing substance 404.

This is done using the arrangement described below and shown in FIG. 1. The dynamic process on which the described reactor 400 is based and which has a so-called “causal-retro-causal relationship” is described by a state transition description, which is not visible to an observer of the dynamic process, and an output equation, the observable quantities of the technical dynamic process.

Such a structure of a dynamic system with a “causal-retro-causal relationship” is shown in FIG. 2b.

The dynamic system 250 is subject to the influence of an external input variable u of a predeterminable dimension, an input variable ut at a time t being referred to as ut:

^u tm ^J

where 1 is a natural number.

The input variable ut at a time t causes a change in the dynamic process taking place in the dynamic system 250.

In this case, an internal state of the system 250 at a time t, which internal state cannot be observed by an observer of the system 250, is composed of a first inner partial state st and a second inner partial state rt-

Depending on the first inner partial state st-i at an earlier point in time t-1 and the input variable ut, a state transition of the first inner partial state st-i of the dynamic process into a subsequent state st is caused.

The following applies: st = f! ( ^s tX ' ^u t) - ⁽⁵⁾

where f1 (.) denotes a general mapping rule.

Seen clearly, the first inner partial state st is influenced by an earlier first inner partial state st-i and the input variable ut. Such a relationship is usually referred to as “causality”.

Depending on the second inner partial state rt + i at a subsequent time t + 1 and the input variable ut, a state transition of the first inner state rt + i of the dynamic process into a subsequent state rt is caused.

The following applies:

^r t = ^f2 ( ^r t + l ' ^u t) - ⁽ 6 ⁾

where f2 (.) denotes a general mapping rule.

In this case, the second inner partial state rt is clearly influenced by a later second inner partial state rt + i, generally an expectation of a later state of the dynamic system 250, and the input variable ut. Such a connection is called “retro causality.

An output variable yt observable by an observer of the dynamic system 250 at a time t thus depends on the input variable Ut. the first inner partial state ≤ and the second inner partial state rt>

The output variable yt (yt e 9? ^N ) is predeterminable dimension n. The dependence of the output variable yt on the input variable ut, the first inner partial state st and the second inner partial state rt of the dynamic process is given by the following general rule:

where g (.) denotes a general mapping rule.

To describe the dynamic system 250 and its states, an arrangement of interconnected computing elements in the form of a neural network of interconnected neurons is used. This is shown in FIG. 1 and is referred to as the “causal-retro-causal neural network (KRKNN).

The connections between the neurons of the neural network are weighted. The weights of the neural network are summarized in a parameter vector v.

In this neural network, the first inner partial state s and the second inner partial state rt depend on the input variable u in accordance with the following regulations. the first inner partial state st-i. the second inner partial state rt + i and the parameter vectors v _s , vt, v _y ab:

s _t = NN (v _s , s _t _ι. u _t ), (8)

where NN (.) denotes a mapping rule specified by the neural network.

The KRKNN 100 according to FIG. 1 is a neural network developed over four times, t-1, t, t + 1 and t + 2. The basics of a neural network unfolded over a finite number of times are described in [1].

For easier understanding of the principles on which the KRKNN is based, FIG. 5 shows the known TDRNN as a neural network 500 that is deployed over a finite number of times.

The neural network 500 shown in FIG. 5 has an input layer 501 with three partial input layers 502, 503 and 504, each of which contains a predeterminable number of input computing elements, to which input variables ut have a predefinable time t, i.e. time series values described below can be applied.

Input computing elements, i.e. Input neurons are connected via variable connections to neurons of a predefinable number of hidden layers 505.

Neurons of a first hidden layer 506 are connected to neurons of the first partial input layer 502. Furthermore, neurons of a second hidden layer 507 are connected to neurons of the second input layer 503. Neurons of a third hidden layer 508 are connected to neurons of the third partial input layer 504.

The connections between the first partial input layer 502 and the first hidden layer 506, the second partial input layer 503 and the second hidden layer 507 and the third partial input layer 504 and the third hidden layer 508 are in each case the same. The weights of all connections are each contained in a first connection matrix B.

Neurons of a fourth hidden layer 509 are with their inputs with outputs of neurons of the first hidden layer 506 according to a through a second connection matrix A2 given given structure. Furthermore, outputs of the neurons of the fourth hidden layer 509 are connected to inputs of neurons of the second hidden layer 507 according to a structure given by a third connection matrix A ^. ^'

Further, a fifth neuron hidden layer 510 are connected to their inputs according to a given through the third connection A2 ^¬ matrix structure to outputs of neurons in the second hidden layer 507th Outputs of the neurons of the fifth hidden layer 510 are connected to inputs of neurons of the third hidden layer 508 according to a structure given by the third connection matrix A] _.

This type of connection structure is equivalent to a sixth hidden layer 511, which are connected to outputs of the neurons of the third hidden layer 508 according to a structure given by the second connection matrix A2 and according to a structure given by the third connection matrix A] _ to neurons of a seventh hidden layer 512.

Neurons of an eighth hidden layer 513 are in turn given according to one given by the first connection matrix A2

Structure connected with neurons of the seventh hidden layer 512 and via connections according to the third connection matrix A _] _ with neurons of a ninth hidden layer 514. The information in the indices in the respective layers indicates the time t, t-1, t-2, t + 1, t + 2, to which the taps at the outputs of the respective layer can be tapped or supplied Obtain signals (ut, ut-i. Ut-2) •

An output layer 520 has three sub-output layers, a first sub-output layer 521, a second sub-output layer 522 and a third sub-output layer 523. Neurons of the first partial output layer 521 are according to one connected to neurons of the third hidden layer 508 by a structure given an output connection matrix C. Neurons of the second partial output layer are also connected to neurons of the eighth hidden layer 512 in accordance with the structure given by the output connection matrix C. Neurons of the third partial output layer 523 are connected to neurons of the ninth hidden layer 514 according to the output connection matrix C. At the neurons of the partial output layers 521, 522 and 523, the output variables can be tapped for a time t, t + 1, t + 2 (yt, Yt + 1 / Yt + 2 ⁾

Based on this principle of so-called shared weights, i.e. The principle that the equivalent connection matrices in a neural network have the same values at a particular point in time is explained below, the arrangement shown in FIG. 1.

The sketches described in the following are each to be understood in such a way that each layer or each sub-layer has a predeterminable number of neurons, i.e. Computing elements.

Sub-layers of a layer each represent a system state of the dynamic system described by the arrangement. Accordingly, sub-layers of a hidden layer each represent an “inner” system state.

The respective connection matrices are of any dimension and each contain the weight values for the corresponding connections between the neurons of the respective layers.

The connections are directional and marked by arrows in FIG. 1. An arrow direction indicates a “computing direction *, in particular an imaging direction or a transformation direction. The arrangement shown in FIG. 1 has an input layer 100 with four partial input layers 101, 102, 103 and 104, each partial input layer 101, 102, 103, 104 each having time series values ut-i. ut, ut + i. t + 2 can be fed at a time t-1, t, t + 1 or t + 2.

The partial input layers 101, 102, 103, 104 of the input layer 100 are each connected via connections according to a first connection matrix A with neurons of a first hidden layer 110 to four partial layers 111, 112, 113 and 114 of the first hidden layer 110.

The partial input layers 101, 102, 103, 104 of the input layer 100 are additionally each connected via connections according to a second connection matrix B to neurons of a second hidden layer 120, each with four partial layers 121, 122, 123 and 124 of the second hidden layer 120.

The neurons of the first hidden layer 110 are each connected to neurons of an output layer 140, which in turn has four partial output layers 141, 142, 143 and 144, in accordance with a structure given by a third connection matrix C.

The neurons of the second hidden layer 120 are also connected to the neurons of the output layer 140 in accordance with a structure given by a fourth connection matrix D.

In addition, the sublayer 111 of the first hidden layer 110 is connected to the neurons of the sublayer 112 of the first hidden layer 110 via a connection according to a fifth connection matrix E. Corresponding connections also have all other sub-layers 112, 113 and 113 of the first hidden layer 110.

Clearly, all sub-layers 111, 112, 113 and 114 of the first hidden sub-layer 110 are connected to one another in accordance with their chronological sequence t-1, t, t + 1 and t + 2.

The sub-layers 121, 122, 123 and 124 of the second hidden layer 120 are connected to one another in opposite directions.

In this case, the sub-layer 124 of the second hidden layer 120 is via a connection according to a sixth

Connection matrix F connected to the neurons of the sub-layer 123 of the second hidden layer 120.

Corresponding connections also have all other sub-layers 123, 122 and 121 of the second hidden layer 120.

In this case, all sub-layers 121, 122, 123 and 124 of the second hidden sub-layer 120 are clearly connected, contrary to their chronological sequence, that is to say t + 2, t + 1, t and t-1.

According to the connections described, an “internal * system state s ^, st + i or st + 2 of the sub-layer 112, 113 or 114 of the first hidden layer is formed in each case from the associated input state ut, ut + χ or ut + 2 and the previous "inner * system state st-i, st or st-

Furthermore, an “internal * system state rt-χ, rt or rt + i of the sub-layer 121, 122 or 123 of the second hidden layer 120 is formed in accordance with the connections described. det from the associated input state ut-i. ut or ut + i and the temporally following "inner * system state ^r t ' ^r t + l ^or -' ^r t + 2-

In the partial output layers 141, 142, 143 and 144 of the output layer 140, a state from the associated “inner * system state st-i. st. st + i and st + 2 a part ^¬ layer 111, 112, 113 and 114 of the first hidden layer 110 and from the associated inner "system state * ti rt rt rt + i + 2 or a partial layer 121, 122 , 123 and 124 of the second hidden layer 120 are formed.

At an output of the first partial output layer 141 of the output layer 140, a signal can thus be tapped, which depends on the “internal * system states (st.rt).

The same applies to the partial starting layers 142, 143 and 144.

The following cost function E is minimized in the training phase of the KRKNN:

^τ

where T is a number of times taken into account.

The back propagation method is used as the training method. The training data set is obtained from the chemical reactor 400 in the following manner.

Concentrations are measured at predetermined input variables with the measuring device 407 and fed to the computer 409, digitized there and stored in a memory as time series values xt grouped together with the corresponding input variables that correspond to the measured variables.

During the training, the weight values of the respective connection matrices are adjusted. The adjustment is made in such a way that the KRKNN describes the dynamic system it simulates, in this case the chemical reactor, as precisely as possible.

The arrangement from FIG. 1 is trained using the training data set and the cost function E.

The arrangement from FIG. 1 trained according to the training method described above is used to control and monitor the chemical reactor 400. For this purpose, the input variables ut-i. ut determines a predicted output variable yt + i. This is then fed as a control variable, possibly after a possible preparation, to control means 405 for controlling stirrer 402 and control device 430 for inflow control (cf. FIG. 4).

2nd embodiment: rental price forecast

FIG. 3 shows a further development of the KRKNN shown in FIG. 1 and described in the context of the above statements.

The further developed KRKNN shown in FIG. 3, a so-called causal-retro-causal-error-correcting-neural network (KRKFKNN), is used for a rental price forecast.

In this case, the input variable ut is made up of information about a rental price, a housing offer, inflation and an unemployment rate, which information regarding a residential area to be examined is determined at the end of the year (December values). Thus the input large a four-dimensional vector. A time series of the input variables, which consist of several chronologically successive vectors, has time steps of one year each.

The aim of modeling co-pricing described below is to forecast a future rental price.

The dynamic process of rental price formation is described using the arrangement described below and shown in FIG. 3.

Components from Fig.l are provided with the same reference numerals with the same configuration.

In addition, the KRKFKNN has a second input layer 150 with four partial input layers 151, 152, 153 and 154, each partial input layer 151, 152, 153, 154 each having time series values y? _Ι. Y ^ r Y ₊ Xi 'Yt _+? ^{can be} fed at a time t-1, t, t + 1 or t + 2, respectively. The time series values y + -_ ι 'Yt' Y + ₊₁ 'Y ++ 2 ^s ^ ^nc thereby output values measured on the dynamic system.

The partial input layers 151, 152, 153, 154 of the input layer 150 are each connected to neurons of the output layer 140 via connections according to a seventh connection matrix, which is a negative identity matrix.

Thus, in the partial output layers 141, 142, 143 and 144 of the output layer, a difference state (y ₁ __ι ^_

Yt-1 ' ^(y t-yt'' ^(γ t + l ^"y t + l ^{) and (y} t + 2 ^{~ y} t + 2 ⁾ 9 ^forms -

The procedure for training the arrangement described above corresponds to the procedure for training the arrangement according to the first exemplary embodiment. 3rd embodiment: traffic modeling and traffic jam warning forecast

A third exemplary embodiment described below describes traffic modeling and is used for a traffic jam forecast.

In the third exemplary embodiment, the arrangement according to the first exemplary embodiment is used (cf. FIG. 1).

However, the third exemplary embodiment differs from the first exemplary embodiment and also from the second exemplary embodiment in that in this case the variable t originally used as a time variable is used as a location variable t.

An original description of a state at time t thus describes a state at a first location t in the third exemplary embodiment. The same applies in each case to a description of the state at a time t-1 or t + 1 or t + 2.

Furthermore, from the analog transfer of time variability to location variability, it follows that locations t-1, t, t + 1 and t + 2 are arranged in succession along a route in a predetermined direction of travel.

FIG. 6 shows a street 600 which is used by cars 601, 602, 603, 604, 605 and 606.

Conductor loops 610, 611 integrated in the street 600 receive electrical signals in a known manner and feed the electrical signals 615, 616 to a computer 620 via an input / output interface 621. In an analog / digital converter 622 connected to the input / output interface 621, the electrical signals are digitized in a time series and in a memory 623, which is connected via a bus 624 with the analog / digital converter 622 and a processor

625 is connected. Via the input / output interface 621, a traffic control system 650 is supplied with control signals 951, from which a predetermined speed specification 652 can be set in the traffic control system 650 or further information from traffic regulations which is transmitted to the drivers of the vehicles 601, 602, 603 via the traffic control system 650. 604, 605 and 606.

In this case, the following local state variables are used for traffic modeling:

- traffic flow speed v,

- vehicle density p (p = number of vehicles per kilometer

Vehicle ter -—), km

Fz - traffic flow q (q = number of vehicles per hour -, h

(q = v * p)), and

- Speed limits 952 displayed by the traffic control system 950 at a time.

The local state variables are measured as described above using the conductor loops 610, 611.

These variables (v (t), p (t), q (t)) thus represent a state of the technical system "traffic" at a specific point in time t. These variables are used to evaluate r (t) of a current one State, for example with regard to traffic flow and homogeneity. This assessment can be quantitative or qualitative.

In the context of this exemplary embodiment, the traffic dynamics are modeled in two phases:

Control signals 651 are formed from forecast variables ascertained in the application phase and are used to indicate which speed limitation is to be selected for a future period (t + 1). Alternatives to the exemplary embodiments

Some alternatives to the exemplary embodiments described above are shown below.

Alternative areas of application:

The arrangement described in the first exemplary embodiment can also be used to determine the dynamics of an electrocardio gram (EKG). This enables indicators that indicate an increased risk of heart attack to be determined at an early stage. A time series from ECG values measured on a patient is used as the input variable.

In a further alternative to the first exemplary embodiment, the arrangement according to the first exemplary embodiment is used for traffic modeling according to the third exemplary embodiment.

In this case, the variable t originally used as a time variable (in the first exemplary embodiment) is used as a location variable t as described in the context of the third exemplary embodiment.

The explanations for this in the third exemplary embodiment apply accordingly.

In a third alternative to the first exemplary embodiment, the arrangement according to the first exemplary embodiment is used in the context of speech processing (FIG. 10). The basics of such language processing are known from [3].

In this case, the arrangement (KRKNN) 1000 is used to determine an accentuation in a sentence 1010 to be accentuated. For this purpose, sentence 1010 to be accentuated is broken down into its words 1011 and these are each classified 1012 (part-of-speech tagging). The classifications 1012 are coded 1013 in each case. Each code 1013 is expanded by a pause information 1014 (phrase break information) which in each case indicates whether a pause is made after the respective word when the sentence 1010 to be accented is said.

Such coding of a sentence to be accentuated is known from [3] and [4].

A time series 1016 is formed from the extended codes 1015 of the sentence in such a way that a chronological sequence of states of the time series corresponds to the sequence of words in the sentence 1010 to be accentuated. This time series 1016 is applied to the arrangement 1000.

The arrangement now determines for each word 1011 an accentuation information 1020 (HA: main accent or strongly accented; NA: in addition to accent or slightly accentuated; KA: no accent or not accentuated), which indicates whether the respective word is spoken accented becomes.

The explanations for this in the first exemplary embodiment apply accordingly.

In an alternative, the arrangement described in the second exemplary embodiment can also be used to forecast macroeconomic dynamics, such as, for example, an exchange rate trend, or other economic indicators, such as, for example, a stock exchange price. In the case of such a forecast, an input variable is formed from time series of relevant macroeconomic or economic indicators, such as interest rates, currencies or inflation rates. In a further alternative to the second exemplary embodiment, the arrangement according to the second exemplary embodiment is used in the context of speech processing (FIG. 11). The basics of such language processing are known from [5], [6], [7] and [8].

In this case, a syllable-based speech processing, the arrangement (KRKFKNN) 1100 is used to model a frequency curve of a syllable of a word in a sentence.

Such modeling is also known from [5], [6], [7] and [8].

For this purpose, the sentence 1110 to be modeled is broken down into syllables 1111. For each syllable, a state vector 1112 is determined, which describes the syllable phonetically and structurally.

Such a state vector 1112 comprises timing information 1113, phonetic information 1114, syntax information 1115 and emphasis information 1116.

Such a state vector 1112 is described in [4].

A time series 1117 is formed from the state vectors 1112 of the syllables 1111 of the sentence 1110 to be modeled such that a chronological sequence of states of the time series 1117 corresponds to the sequence of the syllables 1111 in the sentence 1110 to be modeled. This time series 1117 is applied to the arrangement 1100.

The arrangement 1100 now determines for each syllable 1111 a parameter vector 1122 with parameters 1120, fomaxpos, fomaxalpha, lp, rp, which describe the frequency response 1121 of the respective syllable 1111. Such parameters 1120 and the description of a frequency response 1121 by these parameters 1120 are known from [5], [6], [7] and [8].

The embodiment contributes to the second embodiment gel ^¬ th accordingly.

Structural alternatives

7 shows a structural alternative to the arrangement from FIG. 1 according to the first exemplary embodiment.

Components from Fig.l are shown with the same design with the same reference numerals in Fig.7.

In contrast to the arrangement shown in FIG. 1, the connections 701, 702, 703, 704, 705, 706, 707 and 708 are disconnected or interrupted in the alternative arrangement according to FIG.

This alternative arrangement, a KRKNN with loosened connections, can be used both in a training phase and in an application phase.

The training as well as the use of the alternative arrangement are carried out in a manner analogous to that described in the first exemplary embodiment.

8 shows a structural alternative to the arrangement from FIG. 3 according to the second exemplary embodiment.

Components from FIG. 3 are shown with the same reference numerals in FIG. 8 with the same configuration.

In contrast to the arrangement shown in FIG. 3, the connections 801 in the alternative arrangement according to FIG. 802, 803, 804, 805, 806, 807, 808, 809 and 810 solved or interrupted.

This alternative arrangement, a KRKFKNN with loosened connections, can be used both in a training phase and in an application phase.

The training as well as the use of the alternative arrangement are carried out in an analogous manner to that described in the second exemplary embodiment.

It should be noted that it is possible to use the KRKNN with loosened connections only in the ..training phase and the KRKNN (without the loosened connections according to the first exemplary embodiment) in the application phase.

It is also possible to use the KRKNN with disconnected connections only in the application phase and the KRKNN (without the disconnected connections according to the first exemplary embodiment) in the training phase.

The same applies to the KRKFKNN and the KRKFKNN with loosened connections.

A further structural alternative to the arrangement according to the first exemplary embodiment is shown in FIG. 9.

The arrangement according to FIG. 9 is a KRKNN with a fixed point recurrence.

Components from Fig.l are shown with the same design with the same reference numerals in Fig.8.

In contrast to the arrangement shown in FIG. 1, additional connections 901, 902, 903 and 904 are closed in the alternative arrangement according to FIG. The additional connections 901, 902, 903 and 904 each have a connection matrix GT with weights.

This alternative arrangement can be used both in a training phase and in an application phase.

The training as well as the use of the alternative arrangement are carried out in a manner analogous to that described in the first exemplary embodiment.

Realization of a KRKNN by a SENN, version 3.1 program code

Furthermore, a possible implementation of a KRKNN is specified for the SENN program, version 3.1. The implementation comprises various sections, each of which contains a program code that is required for processing in SENN, version 3.1.

Possible implementations of the exemplary embodiments and of the alternatives described above can also be carried out using the SENN version 3.1 program.

Part 1: 75 JAP1 = FILE DATA / jap.txt

COLUMN 1 // JAP NIKKEI

APPLICATION Yield Curve Forecast JAP2 = FILE DATA / jap.txt

COLUMN 2 // JAP INDUSTRIAL PRODUCTION

MODE MONTH WEEK 7 JAP5 = FILE DATA / jap.txt

80 COLUMN 5 // JAP ANNUAL INFLATION FROM MIN TO MAX

INPUT = scalef (dmdol - dmdol (-1) TRAINING FROM MIN TO 12/31/1994 / dmdol (-1)) LAG -1

INPUT = scale ((US2 US2 (-1)

VALIDATION FROM 01/01/1992 TO 31/12/1994 85 / US2 (-1)) LAG -1

INPUT = scalef US6 US6 (-1)) LAG -1

INPUT = scalef (GER1 GERl (-l)

INPUT CLUΞTER mlp.externlO / GERlf-1)) LAG -1

90 INPUT = scale ((GER2 GER2 (-1)

BEGIN / GER2I-1)) LAG -1 dmdol = FILE DATA / inter.txt INPUT = scale (GER7 GER7 (-1)

COLUMN 1 I I DM / USDOLLAR) LAG -1 US2 = FILE DATA / USa.txt INPUT = scalef (JAP1 - JAPl (-l)

COLUMN 2 // US INDUSTRIAL PRODUCTION 95 / JAPK-1)) LAG -1

US6 = FILE DATA / USa.txt INPUT = scalef (JAP2 - JAP2 (-1)

COLUMN 6 // US ANNUAL INFLATION / JAP2 (-1)) LAG -1

GER1 = FILE DATA / ger.txt INPUT = scale (JAP5 - JAP5 (-1)

COLUMN 1 7 / GER DAX INDEX) LAG -1

GER2 = FILE DATA / ger.txt 100 END

COLUMN 2 // GER INDUSTRIAL PRODUCTION

DEVICE = FILE DATA / ger.txt

COLUMN 7 // GER ANNUAL INFLATION INPUT CLUSTER mlp.extern32

JAP1 = FILE DATA / ap.txt

COLUMN 1 // JAP NIKKEI 105 BEGIN

JAP2 = FILE DATA / ap.txt dmdol = FILE DATA / inter.txt

COLUMN 2 // JAP INDUSTRIAL PRODUCTION COLUMN 1 // DM / USDOLLAR

JAP5 = FILE DATA / ap.txt US2 = FILE DATA / usa.txt

COLUMN 5 // JAP ANNUAL INFLATION COLUMN 2 // US INDUSTRIAL PRODUCTION

110 US6 = FILE DATA / usa.txt

INPUT = scale ((dmdol - dmdol (-1) COLUMN 6 // US ANNUAL INFLATION / dmdol (-1)) GER1 = FILE DATA / ger.txt

INPUT = scale ((US2 - US2 (-1) COLUMN 1 // GER DAX INDEX / US2 (-1)) GER2 = FILE DATA / ger.txt

INPUT = scale (UΞ6 - US6 (-1) 115 COLUMN 2 // GER INDUSTRIAL PRODUCTION

GER7 = FILE DATA / ger.txt

INPUT = scale ((GER1 - GERl (-l) COLUMN ^■ 7 // GER ANNUAL INFLATION / GERK-1)) JAP1 = FILE DATA / j ap. txt

INPUT = scale ((GER2 - GER2 (-1) COLUMN 1 // JAP NIKKEI / GER2 (-1)) 120 JAP2 = FILE DATA / j ap. Txt

INPUT = scale (GER7 - GER7 (-1) COLUMN 2 // JAP INDUSTRIAL PRODUCTION

JAP5 = FILE DATA / j ap. txt

INPUT = scalef (JAP1 - JAP1 (-1) COLUMN 5 // JAP ANNUAL INFLATION / JAPl (-l))

INPUT = scalef (JAP2 - JAP2 (-1) 125 INPUT _ = scale ((dmdol - dmdol (-1)) / JAP2 (-1)) / dmdol (-1)) LAG -2

INPUT = scale (JAP5 - JAP5 (-1 INPUT = scale ((US2 - US2 (-1))) / US2 (- 1)) LAG -2

END INPUT = scale (US6 - US6 (-1)

130) LAG -2

INPUT = scalef (GER1 - GERl (-l))

/ GER1 -ι>) LAG -2 INPUT CLUSTER mlp.extern21 INPUT = scalef (GER2 - GER2 (-1))

/ GER2 -1)) LAG -2

BEGIN 135 INPUT = scalef GER7 - GER7 (-1) dmdol FILE DATA / inter.txt) LAG -2

COLUMN 1 // DM / USDOLLAR INPUT = scale ((JAP1 - JAPlf-1))

US2 = FILE DATA / usa.txt / JAP1 -1)) LAG -2

COLUMN 2 .// US INDUSTRIAL PRODUCTION INPUT = scale) (JAP2 - JAP2 (-1))

US6 = FILE DATA / usa.txt 140 / JAP2 -1)) LAG -2

COLUMN 6 // US ANNUAL INFLATION INPUT = scale (JAP5 - JAP5 (-1)

GER1 = FILE DATA / ger.txt) LAG -2

COLUMN 1 // GER DAX INDEX END

GER2 = FILE DATA / ger.txt

COLUMN 2 // GER INDUSTRIAL PRODUCTION 145 GER7 = FILE DATA / ger.txt INPUT CLUSTER mlp.extern43

COLUMN 7 // GER ANNUAL INFLATION BEGIN INPUT = scalef GER7 GER7 (-1) dmdol = FILE DATA / inter.txt 75) LAG -4

COLUMN 1 // DM / USDOLLAR INPUT = scalef (JAP1 JAP1 (-1))

US2 = FILE DATA / usa.txt / JAPl (-l)) LAG -4

COLUMN 2 // US INDUSTRIAL PRODUCTION INPUT = scalef (JAP2 JAP2 (-1))

US6 = FILE 'DATA / usa.txt / JAP2 (-1)) LAG -4

COLUMN 6 // S ANNUAL INFLATION 80 INPUT = scale) JAP5 JAP5I-1)

GER1 = FILE DATA / ger.txt) LAG -4

COLUMN 1 // GER DAX INDEX END

GER2 = FILE DATA / ger.txt

COLUMN 2 // GER INDUSTRIAL PRODUCTION

GER7 = FILE DATA / ger.txt 15 INPUT CLUSTER mlp.extern65

COLUMN 7 // GER ANNUAL INFLATION

JAP1 = FILE DATA / jap.txt BEGIN

COLUMN 1 // JAP NIKKEI dmdol = FILE DATA / inter.txt

JAP2 = FILE DATA / jap.txt COLUMN 1 // DM / USDOLLAR

COLUMN 2 // JAP INDUSTRIAL PRODUCTION 90 US2 = FILE DATA / usa.txt

JAP5 = FILE DATA / jap.txt COLUMN 2 // US INDUSTRIAL PRODUCTION

COLUMN 5 // JAP ANNUAL INFLATION US6 = FILE DATA / usa.txt

COLUMN 6 // US ANNUAL INFLATION

INPUT _ = scalef (dmdol - dmdol (-1)) GER1 = FILE DATA / ger.txt

/ dmdol (-1)) LAG -3 95 COLUMN 1 // GER DAX INDEX

INPUT = scalef (US2 - US2 (-1)) GER2 = FILE DATA / ger.txt

/ US2 (- 1)) LAG -3 COLUMN 2 // GER INDUSTRIAL PRODUCTION

INPUT = scalef US6 - US6 (-1) GER7 = FILE DATA / ger.txt

) LAG -3 COLUMN 7 // GER ANNUAL INFLATION

INPUT = scalef (GER1 - GERl (-l)) 100 JAP1 = FILE DATA / jap.txt

/ GER1 -1)> LAG -3 COLUMN 1 // JAP NIKKEI

INPUT = scale ((GER2 - GER2 (-1)) JAP2 = FILE DATA / jap.txt

/ GER2 -1)) LAG -3 COLUMN 2 // JAP INDUSTRIAL PRODUCTION

INPUT = scalef GER7 - GER7 (-1) JAP5 = FILE DATA / jap.txt

) LAG -3 105 COLUMN 5 // JAP ANNUAL INFLATION

INPUT = scalef (JAP1 - JAPl (-l))

/ JAP1 -1)) LAG -3 INPUT scale ((dmdol dmdol (-1)

INPUT = scale ((JAP2 - JAP2 (-1)) dmdol (-1)) LAG -5

/ JAP2 -1)) LAG -3 INPUT scalef (US2 US2 (-1)

INPUT = scale (JAP5 - JAP5 (-1) 110 US2 (-1)) LAG -5

) LAG -3 INPUT scale! US6 US6 (-1)

END LAG -5 INPUT scale ((GER1 GERl (-l) GERlf-1)) LAG -5

INPUT CLUSTER mlp.extern54 115 INPUT scale ((GER2 GER2 (-1) GER2 (-1)) LAG -5

BEGIN INPUT scale (GER7 GER7 (-1) dmdo 1 = FILE DATA / inter.txt LAG -5

COLUMN 1 // DM / USDOLLAR INPUT scale ((JAP1 JAPl (-l) US2 = FILE DATA / usa.txt 120 JAPlf-1)) LAG -5

COLUMN 2 // US INDUSTRIAL PRODUCTION INPUT scale ((JAP2 JAP2 (-1) US6 = FILE DATA / usa.txt JAP21-1)) LAG -5

COLUMN 6 // US ANNUAL INFLATION INPUT scale) JAP5 JAP5 (-1) GER1 = FILE DATA / ger.txt LAG -5

COLUMN 1 // GER DAX INDEX 125 END

GER2 = FILE DATA / ger.txt

COLUMN 2 // GER INDUSTRIAL PRODUCTION

GER7 = FILE DATA / ger.txt

COLUMN 7 // GER ANNUAL INFLATION INPUT CLUSTER mlp. input_auto

JAP1 = FILE DATA / jap.txt 130

COLUMN 1 // JAP NIKKEI BEGIN input_auto JAP2 = FILE DATA / jap.txt rexl = FILE DATA / return. txt

COLUMN 2 // JAP INDUSTRIAL PRODUCTION COLUMN 1

JAP5 = FILE DATA / jap.txt rex2 = FILE DATA / return. txt

COLUMN 5 // JAP ANNUAL INFLATION 135 COLUMN 2 rex3 = FILE DATA / return. txt

INPUT = scalef (dmdol - dmdol (-1)) COLUMN 3 / dmdol (-1)) LAG -4 rex4 = FILE DATA / return. txt

INPUT = scale ((US2 - US2 (-1)) COLUMN 4 / US2 (-1)) LAG -4 140 rex5 = FILE DATA / return. txt

INPUT = scale (US6 - UΞ6 (-1) COLUMN 5) LAG -4 rex6 = FILE DATA / return. txt

INPUT = scalef (GER1 - GERl (-l)) COLUMN 6 / GERlf-1)) LAG -4 rex7 = FILE DATA / return. txt

INPUT = scalef (GER2 - GER2 (-1)) 145 COLUMN 7 / GER2 (-1)) LAG -4 rexδ = FILE DATA / return. txt COLUMN 8 rex9 = FILE DATA / return. txt 75 rex9 FILE DATA / return. txt

COLUMN 9 COLUMN 9 rexlO = FILE DATA / return. txt rexlO FILE DATA / return. txt

COLUMN 10 COLUMN 10

INPUT = 1 rexl (O) / 10 80 INPUT = -1 * rexl (O) / 10

INPUT = 1 _* rex2 (0) / 10 LAG -1

INPUT = 1 _* rex3 (0) / 10 INPUT = -1 * rex2 (0) / 10

INPUT = 1 _* rex4 (0) / 10 LAG -1

INPUT = 1 _* rex5 (0) / 10 INPUT = -1 * rex3 (0) / 10

INPUT = 1 * rex6 (0) / 10 85 LAG -1

INPUT = 1 * rex7 (0) / 10 INPUT = -1 * rex4 [0) / 10

INPUT = 1 rex8 (0) / 10 LAG -1

INPUT = 1 * rex9 (0) / 10 INPUT = -1 * rex5 (0) / 10

INPUT = 1 _* rexlOfO) / I 10 LAG -1

END 90 INPUT = -1 * rex6 (0) / 10

LAG -1 INPUT = -1 * rex7 (0) / 10

INPUT CLUSTER mlp.mputO LAG -1 INPUT = -1 * rex8 (0) / 10

BEGIN inputO 95 LAG -1 rexl = FILE DATA / return. txt INPUT = -1 * rex9 (0) / 10

COLUMN 1 LAG -1 rex2 = FILE DATA / return. txt INPUT = -1 * rexlO (O) / 10

COLUMN 2 LAG -1 rex3 = FILE DATA / return. txt 100 END

COLUMN 3 rex4 = FILE DATA / return. txt

COLUMN 4 INPUT CLUSTER mlp. mput2 rex5 = FILE DATA / return. txt

COLUMN 5 105 BEGIN ιnput2 rex6 = FILE DATA / return .txt rexl = FILE DATA / return. txt

COLUMN 6 COLUMN 1 rex7 = FILE DATA / return. txt rex2 = FILE DATA / return. txt

COLUMN 7 COLUMN 2 rex8 = FILE DATA / return. txt 110 rex3 = FILE DATA / return. txt

COLUMN 8 COLUMN 3 rex9 = FILE DATA / return. txt rex4 = FILE DATA / return. txt

COLUMN 9 COLUMN 4 rexlO = FILE DATA / return. txt rex5 = FILE DATA / return .txt

COLUMN 10 115 COLUMN 5 rex6 = FILE DATA / return .txt

INPUT = -1 * rexl (O) / 10 COLUMN 6

INPUT = -1 * rex2 (0) / 10 rex7 = FILE DATA / return. txt

INPUT = -1 * rex3 (0) / 10 COLUMN 7

INPUT = -1 * rex (0) / 10 120 rex8 = FILE DATA / return. txt

INPUT = -1 rex5 (0) / 10 COLUMN 8

INPUT = -1 * rex6 (0) / 10 rex9 = FILE DATA / return. txt

INPUT = -1 * rex7 (0) / 10 COLUMN 9

INPUT = -1 _* rex8 (0) / 10 rexlO = FILE DATA / return .txt

INPUT = -1 * rex9 (0) / 10 125 COLUMN 10

INPUT = -1 * rexlO (O) / 10

END INPUT = -1 * rexl (O) / 10

LAG -2

INPUT = -1 * rex2 (0) / 10

INPUT CLUSTER mlp.inputl 130 LAG -2

INPUT = -1 * rex3 (0) / 10

BEGIN mputl LAG -2 rexl = FILE DATA / return. txt INPUT = -1 * rex4 (0) / 10 COLUMN 1 LAG -2 rex2 = FILE DATA / return. txt 135 INPUT = -1 * rex5 (0) / 10 COLUMN 2 LAG -2 rex3 = FILE DATA / return. txt INPUT = -1 * rexS (O) / 10 COLUMN 3 LAG -2 rex4 = FILE DATA / return. txt INPUT = -1 * rex7 (0) / 10 COLUMN 4 140 LAG -2 rex5 = FILE DATA / return. txt INPUT = -1 * rex8 (0) / 10 COLUMN 5 LAG -2 rex6 = FILE DATA / return. txt INPUT = -1 * rex9 (0) / 10 COLUMN 6 LAG -2 rex7 = FILE DATA / return. txt 145 INPUT = -1 * rexlO (O) / 10 COLUMN 7 LAG -2 rex8 = FILE DATA / return .txt END COLUMN 8 75 INPUT = -1 * rex2 (0) / 10

INPUT CLUSTER mlp.ιnput3 LAG -4

INPUT = -1 _* rex3 (0) / 10

BEGIN ιnput3 LAG -4 rexl = FILE DATA / return. txt INPUT = -1 rex4 (0) / 10 COLUMN 1 80 LAG -4 rex2 = FILE DATA / return. txt INPUT = -1 _* rex5 (0) / 10 COLUMN 2 LAG -4 rex3 = FILE DATA / return. txt INPUT = -1 * rex6 (0) / 10 COLUMN 3 LAG -4 rex4 = FILE DATA / return. txt 85 INPUT = -1 rex7 (0) / 10 COLUMN 4 LAG -4 rex5 = FILE DATA / return. txt INPUT = -1 * rex8 (0) / 10 COLUMN 5 LAG -4 rex6 = FILE DATA / return. txt INPUT - -1 _* rex9 (0) / 10 COLUMN 6 90 LAG -4 rex7 = FILE DATA / return. txt INPUT = -1 * rexl0 (0: 1/10 COLUMN 7 LAG -4 rex8 = FILE DATEN / rendite.txt END COLUMN 8 rex9 = FILE DATEN / rendite.txt 95 COLUMN 9 INPUT CLUSTER mlp.inputδ rexlO = FILE DATEN / return .txt COLUMN 10 BEGIN ιnput5 rexl = FILE DATA / return.txt

INPUT = -1 * rexl (0) / 10 100 COLUMN 1

LAG -3 rex2 = FILE DATA / return. txt

INPUT = -1 * rex2 (0) / 10 COLUMN 2 LAG -3 rex3 = FILE DATA / return. xt

INPUT = -1 * rex3 (0) / 10 COLUMN 3 LAG -3 105 rex4 = FILE DATA / return. txt

INPUT = -1 * rex (0) / 10 COLUMN 4 LAG -3 rex5 = FILE DATA / return. txt

INPUT = -1 * rex5 (0) / 10 COLUMN 5 LAG -3 rex6 = FILE DATA / return. txt

INPUT = -1 * rex6 (0) / 10 110 COLUMN 6 LAG -3 rex7 = FILE DATA / return. txt

INPUT = -1 * rex7 (0) / 10 COLUMN 7 LAG -3 rex8 = FILE DATA / return. txt

INPUT = -1 * rex8 (0) / 10 COLUMN 8 LAG -3 115 rex9 = FILE DATA / return. txt

INPUT = -1 * rex9 (0) / 10 COLUMN 9 LAG -3 rexlO = FILE DATA / return. txt

INPUT = -1 * rexlO (O) / 10 COLUMN 10 LAG -3

END 120 INPUT = -1 * rexl (0) / 10

LAG -5

INPUT = -1 * rex2 (0) / 10

INPUT CLUSTER mlp. mput4 LAG -5

INPUT = -1 * rex3 (0) / 10

BEGIN mput. 125 LAG -5 rexl = FILE DATA / return. txt INPUT = -1 * rex4 (0) / 10 COLUMN 1 LAG -5 rex2 = FILE DATEN / rendite .txt INPUT = -1 * rex5 (0) / 10 COLUMN 2 LAG -5 rex3 = FILE DATEN / rendite. txt 130 INPUT = -1 * rex6 (0) / 10 COLUMN 3 LAG -5 rex4 = FILE DATA / return. txt INPUT = -1 * rex7 (0) / 10 COLUMN 4 LAG -5 rex5 = FILE DATA / return. txt INPUT 1 * rex8 (0) / 10

COLUMN 5 135 LAG -5 rexδ = FILE DATA / return. txt INPUT = -1 * rex9 (0) / 10 COLUMN 6 LAG -5 rex7 = FILE DATA / return. txt INPUT = -1 * rexlO (O) / 10 COLUMN 7 LAG -5 rex8 = FILE DATA / return. txt 140 END COLUMN 8 rex9 = FILE DATA / return. txt COLUMN 9 INPUT CLUSTER mlp. mputβ rexlO = FILE DATA / return. txt

COLUMN 10 145 BEGIN mput6 rexl = FILE DATA / return. txt

INPUT -1 * rexl (0) / 10 COLUMN 1

LAG -4 rex2 = FILE DATA / return. txt 75 TARGET

COLUMN 2 END rex3 = FILE DATA / return. txt

COLUMN 3 rex4 = FILE DATA / return. txt TARGET CLUSTER mlp.past3

COLUMN 4 80 rex5 = FILE DATA / return. txt BEGIN

COLUMN 5 TARGET = 0 rex6 = FILE DATA / return. txt TARGET = 0 COLUMN 6 TARGET = 0 rex7 = FILE DATA / return. txt 85 END

COLUMN 7 rex8 = FILE DATA / return. txt

COLUMN 8 TARGET CLUSTER mlp.past2 rex9 = FILE DATA / return. txt

COLUMN 9 90 BEGIN rexlO = FILE DATA / return. txt TARGET = 0

COLUMN 10 TARGET = 0 TARGET = 0

INPUT -1 * rexl (0) / 10 END

LAG -6 95 INPUT -1 * rex2 (0) / 10 LAG -6 TARGET CLUSTER mlp.pastl INPUT -1 * rex3 (0) / 10 LAG -6 BEGIN INPUT -1 * rex4 (0) / 10 100 TARGET = 0 LAG -6 TARGET = 0 INPUT -1 * rex5 (0) / 10 TARGET = 0 LAG -6 END INPUT -1 * rexβ (O) / 10 LAG -6 105 INPUT -1 * rex7 (0) / 10 TARGET CLUSTER mlp .present LAG -6 INPUT -1 * rex8 (0) / 10 BEGIN LAG -6 TARGET = 0 INPUT -1 * rex9 (0) / 10 110 TARGET = 0 LAG -6 TARGET = 0

INPUT = -1 * rexlO (O) / 10 END LAG -6

END

115 TARGET CLUSTER mlp.futurel

BEGIN

TARGET CLUSTER mlp. bottleneck TARGET = 0 TARGET = 0

BEGIN 120 TARGET = 0

TARGET = 0 END TARGET = 0 TARGET = 0 END TARGET CLUSTER mlp.future2

125

TARGET CLUSTER mlp.pastδ BEGIN

TARGET = 0

BEGIN TARGET = 0

TARGET = 0 TARGET = 0

TARGET = 0 130 END

TARGET = 0

END

TARGET CLUSTER mlp.future3

TARGET CLUSTER mlp.pastδ 135 BEGIN

TARGET = 0

BEGIN TARGET = 0

TARGET = 0 TARGET = 0 TARGET = 0 END TARGET = 0 140

END

TARGET CLUSTER mlp.future4 TARGET CLUSTER mlp.past4 BEGIN 145 TARGET = 0

BEGIN TARGET = 0

TARGET = 0 TARGET = 0 TARGET = 0 END 75 rexδ FILE DATA / return. txt

COLUMN 8

TARGET CLUSTER mlp.futureδ rex9 FILE DATA / return. txt COLUMN 9

BEGIN rexlO FILE DATA / return. txt

TARGET = 0 80 COLUMN 10

TARGET = 0 TARGET = 0 TARGET rexl (l) / 10 END TARGET rex2 (l) / 10 TARGET rex3 (l) / 10

85 TARGET rex4 (l) / 10

TARGET CLUSTER mlp. futureδ TARGET rex5 (1) / 10 TARGET rex6 (l) / 10

BEGIN TARGET reχ7 (l) / 10

TARGET = 0 TARGET rex8 (l) / 10

TARGET = 0 90 TARGET rex9 (l) / 10 TARGET = 0 TARGET rexl0 (l) / 10 END END

TARGET CLUSTER mlp. output_auto 95 TARGET CLUSTER mlp.fmal2

BEGIN BEGIN fmal2 rexl = FILE DATEN / rendite. txt rexl = FILE DATA / return. txt COLUMN 1 COLUMN 1 rei <2 = FILE DATA / return. txt 100 rex2 = FILE DATA / return. txt COLUMN 2 COLUMN 2 rex3 = FILE DATA / return. txt rex3 = FILE DATA / return. txt COLUMN 3 COLUMN 3 rex4 = FILE DATA / return. txt rex4 = FILE DATA / return. txt COLUMN 4 105 COLUMN 4 rex5 = FILE DATA / return. txt rex5 = FILE DATA / return. txt COLUMN 5 COLUMN 5 rexδ = FILE DATA / endite. txt rex6 = FILE DATA / return. txt COLUMN 6 COLUMN 6 rex7 = FILE DATA / return. txt 110 rex7 = FILE DATA / return. txt COLUMN 7 COLUMN 7 rex8 = FILE DATA / return. txt rexβ = FILE DATA / return .txt COLUMN 8 COLUMN rex9 = FILE DATA / return. txt rex9 = FILE DATA / return. txt COLUMN 9 115 COLUMN 9 rexlO = FILE DATA / return. txt rexlO FILE DATA / return. txt COLUMN 10 COLUMN 10

TARGET 1 * rexl (O) / 10 TARGET = rexl (2) / 10 TARGET 1 * rex2 (0) / 10 120 TARGET = rex2 (2) / 10 TARGET 1 * rex3 (0) / 10 TARGET = rex3 (2) / 10 TARGET 1 * rex (0) / 10 TARGET = rex (2) / 10 TARGET 1 * rex5 (0) / 10 TARGET = rex5 (2) / 10 TARGET 1 * rex6 (0) / 10 TARGET = rex6 (2 ) / 10 TARGET 1 * rex7 (0) / 10 125 TARGET = rex7 (2) / 10 TARGET 1 * rex8 (0) / 10 TARGET = rex8 (2) / 10 TARGET 1 * rex9 (0) / 10 TARGET = rex9 (2) / 10 TARGET 1 * rexlO (O) / 10 TARGET = rexl0 (2) / 10 END END

130

TARGET CLUSTER mlp. f all TARGET CLUSTER mlp.fιnal3 BEGIN finall BEGIN fmal3 rexl FILE DATA / endite. txt 135 rexl FILE DATE / return. txt

COLUMN 1 COLUMN 1 rex2 = FILE DATA / return. txt rex2 = FILE DATA / return. txt

COLUMN 2 COLUMN 2 rex3 = FILE DATA / return. txt rex3 = FILE DATA / return. txt

COLUMN 3 140 COLUMN 3 rex4 = FILE DATA / return. txt rex4 FILE DATA / return. txt

COLUMN 4 COLUMN 4 rex5 FILE DATA / return. txt rex5 = FILE DATA / return. txt

COLUMN 5 COLUMN 5 rex6 = FILE DATA / return. txt 145 rex6 = FILE DATA / return. txt

COLUMN 6 COLUMN 6 rex7 FILE DATA / endite. txt rex7 = FILE DATA / return. txt

COLUMN 7 COLUMN 7 rex8 FILE DATA / return. txt 75 rexδ FILE DATA / return. txt

COLUMN 8 COLUMN 8 rex9 FILE DATA / return. txt rex9 FILE DATA / return. txt

COLUMN 9 COLUMN 9 rexlO FILE DATA / return. txt rexlO FILE DATA / return. txt

COLUMN 10 80 COLUMN 10

TARGET rexl (3) / 10 TARGET rexl (51/10 TARGET rex2 (3) / 10 TARGET rex2 (5) / 10 TARGET rex3 (3) / 10 TARGET rex3 (5) / 10 TARGET rex4 (3) / 10 85 TARGET rex4 (5) / 10 TARGET rex5 (3) / 10 TARGET rex5 (5) / 10 TARGET rex6 (3) / 10 TARGET rexδ (5) / 10 TARGET rex7 (3) / 10 TARGET rex7 (5) / 10 TARGET rex8 (3) / 10 TARGET rex8 (5) / 10 TARGET rex9 (3) / 10 90 TARGET rex9 (5) / 10 TARGET rexl0 (3) / 10 TARGET rexl0 (5) / 10

END END

TARGET CLUSTER mlp.final4 95 TARGET CLUSTER mlp.finalδ

BEGIN final4 BEGIN final6 rexl = FILE DATA / return. txt rexl = FILE DATA / return. txt COLUMN 1 COLUMN 1 rex2 FILE DATA / return. xt 100 rex2 = FILE DATA / return. txt

COLUMN 2 COLUMN 2 rex3 = FILE DATA / return. txt rex3 = FILE DATA / return. txt

COLUMN 3 COLUMN 3 rex4 = FILE DATA / return. txt rex4 = FILE DATA / return. txt

COLUMN 4 105 COLUMN 4 rex5 = FILE DATA / return. txt rex5 = FILE DATA / return. txt

COLUMN 5 COLUMN 5 rexδ = FILE DATA / return. txt rexδ = FILE DATA / return. txt COLUMN 6 COLUMN 6 rex7 = FILE DATA / return. txt 110 rex7 = FILE DATA / return. txt

COLUMN 7 COLUMN 7 rex8 = FILE DATA / return. txt rex8 = FILE DATA / return. txt

COLUMN 8 COLUMN 8 rex9 = FILE DATA / return. txt rex9 = FILE DATA / return. txt

COLUMN 9 115 COLUMN 9 rexlO = FILE DATA / return. txt rexlO = FILE DATA / return. txt

COLUMN 10 COLUMN 10

TARGET = rexl (4) / 10 TARGET = rexl (6) / 10

TARGET = rex2 (4) / 10 120 TARGET = rex2 (6) / 10

TARGET = rex3 (4) / 10 TARGET = rex3 (6) / 10

TARGET = rex. () / 10 TARGET = rex4 (6) / 10

TARGET = rex5 (4) / 10 TARGET = rex5 (6) / 10

TARGET = rexδ (4) / 10 TARGET = rex6 (6) / 10

TARGET = rex7 (4) / 10 125 TARGET = rex7 (6) / 10

TARGET = rex8 (4) / 10 TARGET = rex8 (6) / 10

TARGET = rex9 (4) / 10 TARGET = rex9 (6) / 10

TARGET = rexl0 (4! 1/10 TARGET = rexlθ (δ) / 10

END END

130

Part 2:

TARGET CLUSTER mlp.finalδ

BpNet {BEGIN final5 Globals {rexl = FILE DATA / return. txt 135 WtPenalty (

COLUMN 1 be NoPenalty rex2 = FILE DATA / return. txt Weigend {

COLUMN 2 Lambda {0} rex3 = FILE DATA / return. txt AutoAdapt {T}

COLUMN 3 140 wO {1} rex4 = FILE DATA / return. txt DeltaLambda {le-06}

COLUMN 4 ReducFac {0.9} rex5 = FILE DATA / return. txt gamma {0.9}

COLUMN 5 DesiredError {0) rexδ = FILE DATA / return. txt 145}

COLUMN 6 tDecay {rex7 = FILE DATA / return. txt lambda {0.005)

COLUMN 7 AutoAdapt {F) AdaptTime {10} 75 Divergence {0.1 EpsOb ₃ {0.001) MinEpoch; 3 1 [5} Whether ₃ Set {Training}} EpsilonFac {1)}

) PruneAlg {

ExtWtDecay {80 be FixPrune

Lambda {0.001} FixPrune {AutoAdapt (F} Perc_0 {0, .1} AdaptTime {10} Perc_l {0, .1} EpsOb ₃ {0.001} Perc_2 {0. .1} Ob] Set {Training} 85 Perc 3 { 0, .1} EpsilonFac {1}}

} EpsiPrune • [

Finnoff {DeltaEps {0. 05}

AutoAdapt {T} StartEps {0. 05} Lambda {0} 90 MaxEps {1} DeltaLambda {le-06} ReuseEps {F} RedueFae {0.9}} Gamma {0.9}} DesxredError {0} Tracer {} 95 Active {F }} Set {Validation}

ErrorFunc {File {trace} let LnCosh}

Ixl {Active {F}

Parameter {0.05} 100 Randomize {0}} PruningSet {Tram. + Valid. LnCosh {Method {S-Pruning}

Parameter {2}}

} StopControl {parametπcalEntropy {105 EpochLimit {Parameter {le-06} Active {T}

MaxEpoch {10000}}

AnyΞave {MovingExpAverage {fιle_name f.Globals.dat 110 Active {F}

} MaxLength {4}

AnyLoad {Training {F} file name f.Globals.dat Validation {T}

} Generalization {F}

ASCII {T} 115 Decay {0. 9}

LearnCtrl {CheckOb] ectiveFct {be Ξtochastic Active {F} Stochastic {MaxLength {4}

PatternSelection {120 Training {F} Let Permute Validation {T} ExpRandom {Generalization {F}

Lambda {2}}} CheckDelta {Segmentation {125 Active {F}

OutputNode {-1} Divergence {0.1} ExpectedCutOff {0.5}! PercentageForGroupB {0.2}} EtaCtrl {} 130 Mode {tPruneCtrl {be EtaSchedule PruneSchedule {EtaSchedule {be FixSchedule SwitchTime {10} FixΞchedule {ReductFactor {0.95} Lιmιt_0 {10} 135} Lιmιt_l {10} Fuzzl 0.3} Lιmιt_3 {10} MaxDelta20b ₃ {0.3} RepeatLast {T) MaxEtaChange {0.02)

} 140 MinEta {0.001}

DynSchedule {MaxEta {0.1} MaxLength {4} Smoother {1} M imumRuns {Training {F} Validation {T 145 Active {F} Generalizatlon}

} LearnAlgo {

DivSchedule {be VarioEta VaπoEta {75}

M Calls {50} Segmentation {} OutputNode {-1} MomentumBackProp {ExpectedCutOff {0.5}

Alpha {0. 05} PercentageForGroupB {0.2}} 80} Quickprop {}

Decay {0.05} tPruneCtrl (

Mu {2} Tracer {} Active {F}} 85 Set {Validation} AnySave {File {trace} fιle_name {f. Stochastic. dat}

} Active {F} AnyLoad {Randomize {0} fιle_name {f.Stochastic.dat 90 PruningSet {Tram. + Valid. }

} Method {S-Prunmg}

BatchSize {15}} Eta {0.005} LearnAlgo {DeπvEps {0} sei Con] Gradient} 95 VaπoEta {TrueBatch {MinCalls {200}

PatternSelection {} be Sequential MomentumBackProp {ExpRandom {Alpha {0.05}

Lambda {2} 100}} Quickprop {Segmentation {Decay {0.05}

OutputNode {-1} Mu {2} ExpectedCutOff {0.5}} PercentageForGroupB {0.2} 105 Low-Memory-BFGS {} Limit {2}

WtPruneCtrl {Tracer {AnySave {

Active {F} 110 fιle_name {f.LineSearch.dat 1

Set {Validation}}

File {trace} AnyLoad {

} fιle_name {f.LineSearch.dat]

Active {F}}

Randomize {0} 115 EtaNull {1}

PruningSet {Train. + Valιd. MaxSteps {10}

Method {S-Pruning} LS_Precιsιon {0.5}} TrustRegion {T} EtaCtrl {DerivEps {0}

Active {F} 120 BatchSize {2147483647}

}}

LearnAlgo {GeneticWeightSelect {be VaπoEta PatternSelection {VarioEta {be Sequential

MinCalls {200} 125 ExpRandom {} Lambda {2}

MomentumBackProp {} Alpha {0.05} Segmentation {

} OutputNode {-1} Quickprop {130 ExpectedCutOff {0.5}

Decay {0.05} PercentageForGroupB {0.2}

Mu {2}

}

LearnAlgo {

AnySave {135 be VarioEta fιle_name {f.TrueBatch.dat VarioEta {

} MinCalls {200} AnyLoad {fιle_name {f. TrueBatch. dat MomentumBackProp} 140 Alpha {0.05}

Eta {0. 05}} DerivEps {0}}} Ob] FctTracer {LmeΞearch {Active {F}

PatternSelection {145 File {ob] Func} Let Sequential} ExpRandom {SearchControl {Lambda {2} SearchStrategy { be HillClimberControl 75 HillCli berControl {% InιtιalAlιve {0.95} InputModification {InheritWeights {T} be None Beta {0.1} AdaptiveUnifor Noise {MutationType {DistributedMac- 80 NoiseEta {1} roMutation DampmgFactor {1}

MaxTπalε {50}}} AdaptiveGaussNoise {PBILControl {NoiseEta {1}

% ImtιalAlιve {0. 95} 85 DampmgFactor {1}

InheritWeights {T}}

Beta {0.1} FixedUmformNoise {

Alpha {0.1} SetNoiseLevel {

PopulationSize {40} NewNoiseLevel {0}} 90} PopulationControl {} pCrossover {1} FixedGaussNoise {

CrossoverType {SimpleCrosso- SetNoiseLevel {ver} NewNoiseLevel {0}

Scalmg {T} 95

ScalmgFactor {2}

Sharing

ΞharmgFactor {0. 05] 1 SaveNoiseLevel {

PopulationSize {50} Filename {noise_level.dat} mm. % InιtιalAlιve {0,. 01 100} max. % InιtιalAlιve {0.. 1 LoadNoiseLevel {

Filename {noise_level.dat}} pMutation {0} SaveManipulatorData {} 105 Filename {mputManip.dat}

Whether] ectιveFunctιonWeιghts {}% Alιve {0.6} LoadManipulatorData {E (TS) {0.2} Filename {inputManip.dat} Improvement (TS) {0}} E (VS) {1} 110 Norm {NoNorm} Improvement (VS) { 0} (E (TS) -E (VS)) / max (E (TS), E (VS)) {rnlp. putO {} ActFunction {

LipComplexity {0} be OptComplexity {2} 115 plogistic {testVal (dead) -testVal (alιve) {0 parameter 0.5}

}

AnySave {ptanh {fιle_name {parameter 0.5} f.GeneticWeightSelect.dat} 120}} id {

AnyLoad {parameter 0.5 fιle_name {} f.GeneticWeightSelect.dat}) 125 InputModification {

Eta {0.05} be None DerivEps {0} AdaptiveUniformNoise {BatchSize {5} NoiseEta {1} SminEpochsForFitnessTest {2} DampmgFactor {1} DmaxEpochsForFitnessTest {3} 130} SelectWeights {T} AdaptiveGaussNoise {TΞ} NoiseError {1 0.005} DampmgFactor {1}}}} 135 FixedUmformNoise {CCMenu {SetNoiseLevel {

Clusters {NewNoiseLevel {0} mlp.ιnput_auto {} ActFunction {} be ld 140 FixedGaussNoise {plogistic {SetNoiseLevel {parameter 0.5 NewNoiseLevel {0}} ptanh {}

Parameters {0.5 145}} SaveNoiseLevel {pid {Filename {noise level.dat

Parameter {0.5 LoadNoiseLevel {75 AdaptiveUniformNoise {

Filename {noise_level. dat ^' NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {1

Filename {mputManip. dat} AdaptiveGaussNoise {

} 80 NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMan p .dat)}

} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {

} 85 NewNoiseLevel {0 ^' mlp.inputl {ActFunction {be ld FixedGaussNoise {plogistic {SetNoiseLevel {

Parameter {0.5} 90 NewNoiseLevel} 1 ptanh {}

Parameters {0.5}} SaveNoiseLevel {pid {95 Filename {noise_level. dat}

Parameters {0.5}}} LoadNoiseLevel {} Filename {noise_level. dat}

InputModification {} be None 100 SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip. dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {

} Filename {mputMamp. dat}

AdaptiveGaussNoise {105} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}

} mlp. mput3 {

FixedUmformNoise {ActFunction {SetNoiseLevel {110 be ld

NewNoiseLevel {0} plogistic {parameter 0.5}

FixedGaussNoise {ptanh {SetNoiseLevel {115 parameter 0.5

NewNoiseLevel {0}} pid {

Parameter 0.5

SaveNoiseLevel {120}

Filename {noise_level.dat InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat NoiseEta {1}} 125 DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {mputManip.dat} 130}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0] mlp.ιnput2 {ActFunction {135 be ld FixedGaussNoise {plogistic {SetNoiseLevel {

Parameter {0.5} NewNoiseLevel {0}}} ptanh {140

Parameter {0.5}} SaveNoiseLevel (pid {Filename {noise_level.dat}

Parameter {0.5}}

145 LoadNoiseLevel {

Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData { Filename {mputMamp. dat} 75 AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputManip .dat}}

} FixedUmformNoise {

Norm {NoNorm} 80 SetNoiseLevel {} NewNoiseLevel {0} mlp.ιnput4 {ActFunction {be id FixedGaussNoise {plogistic {85 SetNoiseLevel {

Parameter {0.5} NewNoiseLevel {0}}} ptanh {}

Parameters {0.5}}} 90 SaveNoiseLevel {pid {Filename {noise_level.dat}

Parameters {0.5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {95} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {

} 100 Filename {mputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}

} mlp.inputδ {

FixedUmformNoise {105 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {

Parameter {0.5}}

FixedGaussNoise {110 ptanh {SetNoiseLevel {parameter {0.5} NewNoiseLevel}} p d {

Parameter {0.5}

115}

SaveNoiseLevel {}

Filename {noise_level.dat] InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat] 120 NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {mputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {125 DampmgFactor {1}

Filename {mputManip.dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {) NewNoiseLevel {0} mlp.ιnput5 {130 ActFunction {be id FixedGaussNoise {plogistic {SetNoiseLevel {

Parameter {0.5} NewNoiseLevel {0} 135 ptanh {

Parameter {0.5}

} SaveNoiseLevel {pid {Filename {noise_level.dat}

Parameters {0.5} 140}} LoadNoiseLevel {} Filename (noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {145 Filename {mputManip.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {

Filename {mputMamp.dat} 75 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.extern65 {1 ActFunction {) be id 80 FixedGaussNoise {plogistic {SetNoiseLevel {

Parameter {0.5} NewNoiseLevel {0}

) ptanh {}

Parameters {0.5} 85} SaveNoiseLevel {pid {Filename {noise_level. dat}

Parameter {0.5}} LoadNoiseLevel {

90 Filename {noise_level. dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp. dat} NoiseEta {1}} DampmgFactor {1} 95 LoadMampulatorData {1 Filename {mputMamp. dat}

AdaptiveGaussNoise {NoiseEta {1} Norm NoNorm DampmgFactor {1}}] 100 mlp.extern43 {

FixedUmformNoise {ActFunction {SetNoiseLevel {let d

NewNoiseLevel {0} plogistic {} parameter 0. 5} 105}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter 0. 5}

NewNoiseLevel {0}} pid {

110 parameters {0.5

SaveNoiseLevel {}

Filename {noise_level.dat InputModification {} be None LoadNoiseLevel {115 AdaptiveUniformNoise {

Filename {noise_level.dat NoiseEta {1}} Damp gFactor {1} SaveManipulatorData {}

Filename {mputManip.dat} AdaptiveGaussNoise (} 120 NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {mputManip.dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} 125 NewNoiseLevel {0] mlp.extern54 {ActFunction {be id FixedGaussNoise {plogistic {SetNoiseLevel {

Parameter {0.5} 130 NewNoiseLevel {0}} ptanh {

Parameters {0.5}} SaveNoiseLevel {pid {135 Filename {noise_level.dat}

Parameters {0.5})} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {} be None 140 SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {145} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}} mlp.extern32 { ActFunction { ^'} 75 {was id FixedGaussNoise plogistic {{SetNoiseLevel

Parameter {0.5 NewNoiseLevel {0}

}} ptanh {80}

Parameter {0.5

} SaveNoiseLevel {pid {Filename {noise_level. dat}

Parameter {0.5}

85 LoadNoiseLevel {

Filename {noise_level. dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp. dat} NoiseEta {1} 90} DampmgFactor {1} LoadMampulatorData {

} Filename {mputManip. dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1} 95}

} mlp.externlO {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {-} 100 parameters {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}

105 pid {

Parameter {0. 5}

SaveNoiseLevel {

Filename {noise_level.dat} InputModification {

} 110 be None

LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}

} DampmgFactor {1) SaveManipulatorData {}

Filename {inputMamp.dat) 115 AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} FixedUmformNoise {

Norm {NoNorm} 120 SetNoiseLevel {} NewNoiseLevel {0} mlp.extern21 {} ActFunction {} be id FixedGaussNoise {plogistic {125 SetNoiseLevel {

Parameter {0.5} NewNoiseLevel {0}}} ptanh {

Parameters {0.5}} 130 SaveNoiseLevel {pid {Filename {noise_level.dat}

Parameters {0.5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {135} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} 140 Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp. output_auto {

FixedUmformNoise {145 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {

Parameter {0.5} 75 parameters {2} ptanh {}

Parameter {0.5} parametricalEntropy {} Parameter {le-06} p d {}

Parameter {0.5} 80}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0} be none Weightmg {1 1 1 1 1 1 1 1 1 1}

Ixl i 85}

Parameter {0.05} mlp. final- {} ActFunction {LnCosh {be id

Parameter {2} plogistic {} 90 parameter {0.5} parametricalEntropy {parameter {le-06} ptanh {} parameter 0.5}}

Norm {NoNorm} 95 pid {ToleranceFlag {F} Parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0}} Weightmg {1 1 1 1 1 1 1 1 1 1}}

ErrorFunc {mlp.finalδ {100 sei none ActFunction {Ixl {sei id parameter {0.05} plogistic {} parameter 0.5 LnCosh {} 105 parameter {2} ptanh {}

Parameter 0.5 parametricalEntropy {parameter {le-06} pid {}

Parameter 0.5 110}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 sei none Weightmg {1 1 1 1 1 1 1 1 1 Ixl {115}

Parameter {0.05} mlp.fιnal3 {} ActFunction {LnCosh {be id

Parameter {2} plogistic {} 120 parameter {0.5} parametricalEntropy {parameter {le-06} ptanh {} parameter 0.5}}

Norm {NoNorm} 125 pid {ToleranceFlag {F} Parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 1}} ErrorFunc {mlp.finalδ {130 none ActFunction {Ixl {be id parameter {0.05} plogistic {}

Parameter {0.5} LnCosh {} 135 parameter {2} ptanh {}

Parameter {0.5} parametricalEntropy {} Parameter {le-06} pid {}

Parameter {0.5} 140}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0} be none Weightmg {1 1 1 1 1 1 1 1 1 1} Ixl {145}

Parameter {0.05} mlp.fmal2 {} ActFunction {LnCosh {let id plogistic {75

Parameter {0.5} LnCosh {

} Parameter {2} ptanh {}

Parameter {0.5} parametricalEntropy} 80 parameters {le-06 pid {}

Parameter {0.5}}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {85 Tolerance {0 0 0} be none Weightmg {1 1 1} Ixl {}

Parameter {0.05} mlp.futureδ {} ActFunction {LnCosh {90 let tanh

Parameter {2} plogistic {

} Parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} 95 parameter {0.5}})

Norm {NoNorm} pid {ToleranceFlag {F} Parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 1 100}} ErrorFunc {mlp.finall {sei none ActFunction {Ixl {let id parameter {0.05} plogistic {105}

Parameter {0.5} LnCosh {

Parameter {2} ptanh {} parameter 0.5} parametricalEntropy

110 parameters {le-06 pid {

Parameter 0.5

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {115 Tolerance {0 0 0} be none Weightmg {1 1 1} Ixl {}

Parameter {0.05} mlp.future5 {} ActFunction {LnCosh {120 let tanh

Parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} 125 parameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} Parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 1 130}} ErrorFunc {mlp. bottleneck {sei none ActFunction {Ixl {sei tanh parameter {0.05} plogistic {135}

Parameter {0.5} LnCosh {} Parameter {2} ptanh {}

Parameters {0.5} parametricalEntropy {} 140 parameters {le-06} p d {

Parameter {0.5}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {145 Tolerance {0 0 0} be none Weightmg {1 1 1} Ixl {}

Parameter {0.05} mlp.future4 { ActFunction {75] ^χ l {sei tanh parameter {0.05} plogistic {}

Parameter {0.5} LnCosh {} Parameter {2} ptanh {80}

Parameter {0.5} parametricalEntropy {

Parameter {le-06} pid {

Parameter 0.5} 85 Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be none Weightmg {1 1 1} Ixl {}

Parameter 0.05} 90 mlp.futurel {ActFunction {

LnCosh {be tanh

Parameter {2} plogistic {

} Parameter {0.5} parametricalEntropy {95} parameter {le-06} ptanh {

} Parameters {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} 100 parameters {0.5} Tolerance {0 0 0} Weightmg {1 1 1}

ErrorFunc {mlp.future3 {sei none ActFunction {105 Ixl {sei tanh parameter {0.05} plogistic {} parameter 0.5 LnCosh {

Parameter {2} ptanh {110} Parameter 0.5 parametricalEntropy {Parameter {le-06} pid {}

Parameter {0.}

115 Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be none Weightmg {1 1 1} Ixl {}

Parameters {0.05} 120 mlp.present {} ActFunction {LnCosh {sei tanh

Parameter {2} plogistic {} Parameter {0.5} parametricalEntropy 125} Parameter {le-06 ptanh {} Parameter {0.5}}}

Norm {NoNorm} Pid {ToleranceFlag {F} 130 parameters {0.5} Tolerance {0 0 0}} Weightmg {1 1 1}}} ErrorFunc {mlp.future2 {sei LnCosh ActFunction {135 Ixl {sei tanh parameter {0.05} plogistic {}

Parameter {0.5} LnCosh {} Parameter {2} ptanh {140}

Parameter {0.5} parametricalEntropy {} Parameter {le-06} pid {}

Parameter {0.5}}

145 Norm {NoNorm} ToleranceFlag {F}

ErrorFunc Tolerance {0 0 0} be none Weightmg (1 1 1) } 75 ErrorFunc {mlp.pastl {be LnCosh ActFunction {1 | xv 11 / t be tanh parameter {0.05} plogistic {} parameter 0.5 80 LnCosh {

Parameter {2} ptanh {}

Parameter {0.5} parametricalEntropy {} Parameter {le-06} pid {85}

Parameter {0.5}}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be LnCosh 90 Weightmg {1 1 1} Ixl {}

Parameter {0.05} mlp.past4 {} ActFunction {LnCosh {let tanh

Parameter {2} 95 plogistic {

} Parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

} Parameter {0.5}} ^" 100}

Norm {NoNorm} pid {ToleranceFlag {F} Parameter {0.5} Tolerance {0 0 0}} Weight g {1 1 1}}

105 ErrorFunc {mlp. past2 {be LnCosh ActFunction {Ixl {be tanh parameter {0.05} plogistic {} parameter 0.5 110 LnCosh {

} Parameter {2} ptanh {} parameter 0.5 parametricalEntropy {parameter {le-06} pid {115}

Parameter {0.5}}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} let LnCosh 120 Weightmg {1 1 1} Ixl {}

Parameter {0.05} mlp.pastS {} ActFunction {LnCosh {let tanh

Parameter {2} 125 plogistic {} parameter {0.5} parametricalEntropy} parameter {le-06 ptanh {

Parameter {0.5}

130}

Norm {NoNorm pid {ToleranceFlag {F Parameter {0.5} Tolerance {0 0 0 Weightmg {1 1 1

} 135 ErrorFunc {mlp. past3 {be LnCosh ActFunction {Ixl {be tanh parameter {0.05} plogistic {}

Parameter {0. 140 LnCosh {} parameter {2} ptanh {}

Parameter {0. parametricalEntropy {

Parameter {le-06} pid {145

Parameter 0. 5

Norm {NoNorm} ToleranceFlag {F} Tolerance {0 0 0} 75 Weightmg {1 1 1}}} Norm {NoNorm} mlp.pastδ {} ActFunction {mlp.state32 {sei tanh 80 ActFunction {plogistic {sei tanh

Parameter {0.5} plogistic {} parameter {0.5 ptanh {}

Parameter {0.5} 85 ptanh {} parameter {0.5 pid {}

Parameter {0.5} pid {} parameter {0.5} 90}

ErrorFunc {} be LnCosh Norm {NoNorm} Ixl {}

Parameters {0.05} mlp.state21 {} 95 ActFunction {LnCosh {let tanh

Parameter {2} plogistic {} parameter {0.5 parametricalEntropy {} "parameter {le-06} 100 ptanh {} parameter {0.5}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5 Tolerance {0 0 0} 105 Weightmg {1 1 1}}} Norm {NoNorm} mlp.state65 {} ActFunction {mlp.statelO {sei tanh 110 ActFunction {plogistic {se tanh parameter {0.5} plogistic {} parameter {0.5 ptanh {} parameter {0.5} 115 ptanh {} parameter {0.5 Pid {} parameter {0.5} pid {parameter {0.5

120}

Norm {NoNorm}}} Norm {NoNorm} mlp.state54 {} ActFunction {mlp.stateOl {sei tanh 125 ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5 ptanh {} parameter {0.5} 130 ptanh { } parameter {0.5 Pid {} parameter {0.5} pid {parameter {0.5

135

Norm {NoNorm}} Norm {NoNorm} mlp.state43 {} ActFunction {mlp.statel2 {sei tanh 140 ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5 ptanh {parameter {0.5} 145 ptanh {} parameter 0.5 pid {} parameter {0. 5} pid { parameter {0.5} 75 pid {parameter {0.5}

}}

Norm NoNorm}}

Norm {NoNorm} mlp.state23 {80} ActFunction {mlp.back54 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5 plogistic {} 85 parameter {0.5} ptanh {} parameter {0.5 ptanh {

Parameter {0.5} pid {} parameter {0.5} 90 pid {} parameter {0.5}}

Norm {NoNorm}} Norm {NoNorm} mlp.state34 {95} ActFunction {mlp.back43 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {

} 100 parameters {0. 5} ptanh {} parameter {0.5} ptanh {} parameter {0. pid {} parameter {0.5} 105 pid {parameter {o.

}

Norm NoNorm

} Norm {NoNorm} mlp.state45 {110} ActFunction {mlp.back32 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5 plogistic {} 115 parameter {0.5} ptanh {} parameter 0.5 ptanh {parameter {0.5} pid {} parameter {0.5} 120 pid {parameter {0.5}

}} Norm NoNorm}

} Norm {NoNorm} mlp.stateδδ {125} ActFunction {mlp.back21 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5 plogistic {} 130 parameter {0.5} ptanh {} parameter {0.5 ptanh {} parameter 0.5 pid {parameter 0.5 135 pid {parameter {0.5

Norm {NoNorm}} Norm {NoNorm mlp.back65 {140} ActFunction {mlp.backlO {sei tanh ActFunction {plogistic {sei tanh parameter {0.5 plogistic {} 145 parameter 0.5 ptanh {parameter {0. 5 ptanh {parameter 0.5 75 pid {SaveWeightsLocal {parameter {0. 5 Filename {std}}

Alive {T}

Norm {NoNorm 80 WtFreeze {T} AllowPrumng {F} EtaModifier {1}

Connectors {Penalty {NoPenalty} mlp.bottleneck-> output_auto {} WeightWatcher {85 mlp.bιas-> fιnal5 {Active {F} LoadWeightsLocal {MaxWeight {1} Filename {std} MinWeight {0}}} SaveWeightsLocal {

LoadWeightsLocal {90 Filename {std} Filename {std}}

} Alive {T}

SaveWeightsLocal {WtFreeze {T} Filename {std} AllowPrumng {F}

} 95 EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {T}} AllowPrumng {F} mlp. future4-> fmal4 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 100 Filename {std}

}} mlp.bιas-> output_auto {SaveWeightsLocal {WeightWatcher {Filename {std}

Active {F}}

MaxWeight {1} 105 Alive {T}

MinWeight {0} WtFreeze {T}} AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 110} SaveWeightsLocal {mlp.bιas-> fιnal4 {

Filename {std} LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {T} 115 SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {T} mlp. futureδ-> fmal6 {120 AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp. future3-> fmal3 {

Filename {std} 125 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 130} Penalty {NoPenalty} Alive {T}} WtFreeze {T} mlp.bιas-> fmalδ {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 135 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> fιnal3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 140} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 145 WtFreeze {T} mlp. future5-> fιnal5 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty} 75 MaxWeight {1} mlp.future2-> fmal2 {MinWeight {0} LoadWeightsLocal {} Filename {std} LoadWeightsLocal {

} Filename {std}

SaveWeightsLocal {80} Filename {std} SaveWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} Alive {F} AllowPrumng {F} 85 WtFreeze {T} EtaModifier {1} AllowPrumng {F} Penalty {NoPenalty} EtaModifier {1}

} Penalty {NoPenalty} mlp.bιas-> fmal2 {} LoadWeightsLocal {90 mlp.state56-> future6 {Filename {std} WeightWatcher {

} Active {F}

SaveWeightsLocal {MaxWeight {1} Filename {std} MinWeight {0}

} 95}

Alive {T} LoadWeightsLocal {

WtFreeze {T} Filename {std}

AllowPrumng {F}}

EtaModifier {1} SaveWeightsLocal {

Penalty {NoPenalty} 100 Filename {std}}} mlp. futurel-> fmall {Alive {T}

LoadWeightsLocal {WtFreeze {F} Filename {std} AllowPrumng {F}

} 105 EtaModifier {1}

SaveWeightsLocal {Penalty {NoPenalty} Filename {std}}

} mlp.bιas-> futureδ {

Alive {T} LoadWeightsLocal {WtFreeze {T} 110 Filename {std} AllowPrumng {F}} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.bιas-> fιnall {115 Alive {T} LoadWeightsLocal { WtFreeze {T}

Filename {std} AllowPrumng {F}} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty}

Filename {std} 120}} mlp. state45-> future5 {

Alive {T} LoadWeightsLocal {WtFreeze {T} Filename {std} AllowPrumng {F}} EtaModifier {1} 125 SaveWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.ιnput_auto-> bottleneck Alive {T} WeightWatcher {WtFreeze { FF}

Active {F} 130 AllowPrumng [F}

MaxWeight {1} EtaModifier 1}

MinWeight {0} Penalty {NoPenalty}}} LoadWeightsLocal {mlp.bιas-> future5 {

Filename {std} 135 LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {F} 140} WtFreeze {T} Alive {T} AllowPrumng {F} WtFreeze {T} EtaModifier {1} AllowPrumng {F} Penalty {NoPenalty} EtaModifier {1}} 145 Penalty {NoPenalty} mlp.bιas-> bottleneck {} WeightWatcher {mlp. state34-> future4 {

Active {F} LoadWeightsLocal { Filename {std} 75 Penalty {NoPenalty}

SaveWeightsLocal {mlp.state01-> futurel {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 80} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 1 Penalty {NoPenalty} Alive {T}} 85 WtFreeze {F} mlp.bιas-> future4 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {90 mlp.bιas-> futurel {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} 95 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {T} mlp. state23-> future3 {AllowPrumng {F} LoadWeightsLocal {100 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.mputO-> present {

Filename {std} LoadWeightsLocal {} 105 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 110 Alive {T}} WtFreeze {T} mlp.bιas-> future3 {AllowPrumng {F} LoadWeightsLocal {EtaModifier (1}

Filename {std} Penalty {NoPenalty}} 115} SaveWeightsLocal {mlp. statelO-> present {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} 120 SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.statel2-> future2 {125 AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> present {

Filename {std} 130 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 135} Penalty {NoPenalty} Alive {T} 1 WtFreeze {T} mlp.bιas-> future2 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 140 Penalty {NoPenalty}}} SaveWeightsLocal {mlp. mputl-> pastl {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 145} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} Alive {T} 75 Penalty {NoPenalty} WtFreeze {T}} AllowPrumng {F} mlp.bιas-> paεt2 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 80} mlp.state21-> pastl {SaveWeightsLocal {LoadWeightsLocal {Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {85 WtFreeze {T}

Filename {std} AllowPrumng {F}

} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F}} AllowPrumng {F} 90 mlp.back21-> past2 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.bιas-> pastl {SaveWeightsLocal { LoadWeightsLocal {95 Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPrumng {F}

} - 100 EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {T}} AllowPrumng {F} mlp.mput3-> past3 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 105 Filename {std}

}} mlp.backlO-> pastl {SaveWeightsLocal {WeightWatcher {Filename {std}

Active {F}}

MaxWeight {1} 110 Alive {T}

MinWeight {0} WtFreeze {T}} AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 115} SaveWeightsLocal {mlp.state43-> past3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 120 SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}) Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.mput2-> past2 {125 AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> past3 {

Filename {std} 130 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 135} Penalty {NoPenalty} Alive {T}} WtFreeze {T} mlp. state32-> past2 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 140 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.back32-> past3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 145} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} Alive {T} 75 WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} 80 WtFreeze {F} mlp.mput4-> past4 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}}}

SaveWeightsLocal {85 mlp.bιas-> past5 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} 90 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} WtFreeze {T} mlp.state54-> past4 {AllowPrumng {F}

LoadWeightsLocal {95 EtaModifier {1}

Filename {std} Penalty {NoPenalty}

SaveWeightsLocal {mlp.back54-> past5 {Filename {std} LoadWeightsLocal {

) "100 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 105 Alive {T}

} WtFreeze {F} mlp.bιas-> past4 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}} 110}

SaveWeightsLocal {mlp. mputδ-> pastδ {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} 115 SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} WtFreeze {T} mlp.back43-> past4 {120 AllowPrumng {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.bιas-> past6 {Filename {std} 125 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 130} Penalty {NoPenalty} Alive {T}

} WtFreeze {T} mlp.mput5-> past5 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1} Filename {std} 135 Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.back65-> past6 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 140} WtFreeze {T} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} 145 WtFreeze {F} mlp.state65-> past5 {AllowPrumng {F}

LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty} 75 mlp. extern65-> state65 {Alive {T} WeightWatcher {WtFreeze {F} Active {F} AllowPruning {F} MaxWeight {1} EtaModifier {1} MinWeight {0} 80 Penalty {NoPenalty}

}}

LoadWeightsLocal {mlp.extern43-> state43 {

Filename {std} LoadWeightsLocal {} Filename {std}

SaveWeightsLocal {85}

Filename {std} SaveWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} 90 WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.past6-> state65 { } WeightWatcher {95 mlp.past4-> state43 {

Active {F} LoadWeightsLocal {

MaxWeight {1} Filename {std}

MinWeight {0}}} SaveWeightsLocal {LoadWeightsLocal {100 Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {WtFreeze {F

Filename {std} AllowPruning [F}} 105 EtaModifier {1}

Alive {T} Penalty {NoPenalty WtFreeze {F}} AllowPruning {F} mlp. state54-> state43 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 110 Filename {std}}} mlp. extern54-> state54 {SaveWeightsLocal {LoadWeightsLocal {Filename {std}

Filename {std}}} 115 Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty WtFreeze {F} 120} AllowPruning {F} mlp. extern32-> state32 EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.past5-> state54 {125 SaveWeightsLocal {LoadWeightsLocal {Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} 130 AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty WtFreeze {F}} AllowPruning {F} mlp.past3-> state32 {EtaModifier {1} 135 LoadWeightsLocal {Penalty {NoPenalty} Filename {std} mlp. state65-> state54 SaveWeightsLocal {WeightWatcher {Filename {std} Active {F} 140} MaxWeight {1} Alive {T} MinWeight {0} WtFreeze {F}

} AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename td} 145 Penalty {NoPenalty}}} ΞaveWeightsLocal {mlp.state43-> state32 {

Filename {std} LoadWeightsLocal { Filename {std} 75 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state21-> statel0 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 80} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} 85 WtFreeze {F} mlp.extern21-> state21 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}}}

SaveWeightsLocal {90 mlp.present-> state01 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 95 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.past2-> state21 {AllowPruning {F}

LoadWeightsLocal {100 EtaModifier {1}

Filename {std} Penalty {NoPenalty}

SaveWeightsLocal {mlp.statel0-> state01 {

Filename {std} LoadWeightsLocal {} 105 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 110 Alive {T}} WtFreeze {F} mlp.state32-> state21 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 115} SaveWeightsLocal {mlp.state01-> statel2 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 120 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.externl0-> statel0 {125 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.statel2-> state23 {

Filename {std} 130 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 135} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.pastl-> statelO {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 140 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state23-> state34 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 145} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} Alive {T} 75} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} 80 Penalty {NoPenalty} mlp.state34-> state45 {} LoadWeightsLocal {mlp.back43-> back54 {

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {85}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} 90 WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.state45-> state56 { LoadWeightsLocal {95 mlp.past3-> back43

Filename {std} LoadWeightsLocal} Filename {std SaveWeightsLocal {}

Filename {std} SaveWeightsLocal} ^" 100 Filename std

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F 1 Penalty {NoPenalty} 105 EtaModifier {1} Penalty {NoPenalty mlp.past5-> back65 {} WeightWatcher {mlp.back32-> back43 {

Active {F} LoadWeightsLocal

MaxWeight {1} 110 Filename {std}

MinWeight {0}}} SaveWeightsLocal LoadWeightsLocal {Filename {std

Filename {std}} 115 Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F)} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F} 120} AllowPruning {F} mlp.past2 ~> back32 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.back54-> back65 {125 SaveWeightsLocal {WeightWatcher {Filename {std}

Active {F}}

MaxWeight {1} Alive {T}

MinWeight {0} WtFreeze {F}} 130 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.back21-> back32 {Filename {std} 135 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 140} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.past4-> back54 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} 145 Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.pastl-> back21 {Filename {std} LoadWeightsLocal { Filename {std} Lambda {0}} AutoAdapt {T} SaveWeightsLocal {wO {1}

Filename {std} DeltaLambda {le-06}} 75 ReducFac {0.9}

Alive {T} Gamma {0.9} WtFreeze {F} DesiredError {0} AllowPruning {F}} EtaModifier {1} WtDecay {Penalty {NoPenalty 80 Lambda {0.005}} AutoAdapt {F} mlp.backl0-> back21 {AdaptTime {10 } LoadWeightsLocal {EpsOb] {0.001}

Filename {std} Ob] Set {Training}} 85 EpsilonFac {1} SaveWeightsLocal {}

Filename {std} ExtWtDecay {} Lambda {0.001}

Alive {T} AutoAdapt {F} WtFreeze {F} 90 AdaptTime {10} AllowPruning {F} EpsOb] {0.001} EtaModifier {1} Ob] Set {Training} Penalty {NoPenalty EpsilonFac {1}}} mlp.present-> backlO {95 Finnoff {LoadWeightsLocal {AutoAdapt {T}

Filename {std} Lambda {0}} DeltaLambda {le-06} SaveWeightsLocal {ReducFac {0.9}

Filename {std} 100 Gamma {0.9}} DesiredError {0}

Alive {T} WtFreeze {F} AllowPruning {F} ErrorFunc {EtaModifier {1} 105 let LnCosh Penalty {NoPenalty Ixl {parameter {0.05}

AnySave {LnCosh {file name {f.CCMenu.dat} 110 parameters {2}

AnyLoad {parametricalEntropy {file name f.CCMenu.dat} parameter {le-06}}

115

RecPar {AnySave {decay_c {1} file name f.Globals.dat delta_t {0.1} epsilon {0.01} AnyLoad {max_ιter {1} 120 file name {f.Globals.dat} show {F}}

Reset_Errors {F} ASCII {T}}} TestRun {LearnCtrl {

Filename {Test} 125 is Stochastic

Part.Transformed {F} Stochastic {} PatternSelection {Online {be permute

Filename {Online.dat} ExpRandom {

130 lambda {2}} segmentation {

OutputNode {-1} ExpectedCutOff {0.5}

135 PercentageForGroupB {0.2}

Part 3: }

WtPruneCtrl {PruneSchedule {

BpNet {140 be FixSchedule

Globals {FixSchedule {WtPenalty {Lιmιt_0 {10 be NoPenalty Lιmιt_l {10 Weigend {Limit 2 {10 Lιmιt_3 {10} 75 MaxDelta20b] {0.3} RepeatLast {T} MaxEtaChange {0.02}

} MmEta {0.001}

DynSchedule {MaxEta {0.1} MaxLength {4} Smoother {1} M imumRuns {0} 80 Training {F}} Validation {T} Active F} Generalization {F}

} LearnAlgo {

DivSchedule {85 is VarioEta

Divergence {0.1} VarioEta {MinEpochs {5} MinCalls {50

}}

MomentumBackProp {

PruneAlg {90 Alpha {0.05} be FixPrune}

FixPrune {Quickprop {

Perc_0 {0. .1} Decay {0.05}

Perc_l {0. .1} Mu {2}

Perc_2 {0, .1} 95}

Perc 3 {0 .1}}} AnySave {EpsiPrune | r file name {f.Stochastic.dat}

DeltaEps' {0. 05}}

StartEps {0. .05} 100 AnyLoad {

MaxEps {1} file name {f.Stochastic.dat}

ReuseEps {F}}} BatchSize {1} Eta {0.01}

Tracer {105 DerivEps {0}

Active {F}}

Set {Validation} TrueBatch {

File {trace} PatternSelection {} be sequential

Active {F} 110 ExpRandom {Randomize {0} Lambda {2} PruningSet {Train. + Valιd. }} Method {S-Prunmg} Segmentation {} OutputNode {-1}

ΞtopControl {115 ExpectedCutOff 0.5} EpochLi it {PercentageForGroupB {0. 2}

Active {T}}

MaxEpoch {10000}}} WtPruneCtrl {Mov gExpAverage {120 Tracer {

Active {F} Active {F}

MaxLength {4} Set {Validation}

Training {F} File {trace}

Validation {T}}

Generalization {F} 125 Active {F}

Decay {0.9} Randomize {0}} PruningSet {Tram. + Valid. } CheckOb] ectιveFct {Method {Ξ-Prumng}

Active {F} 1

MaxLength {4} 130 EtaCtrl {

Training {F} Active {F}

Validation {T}}

Generalization {F} LearnAlgo {be VarioEta

CheckDelta {135 VarioEta {Active {F} MinCalls {200} Divergence {0.1} MomentumBackProp {

Alpha {0.05}

EtaCtrl {140}

Mode {Quickprop {be EtaSchedule Decay {0.05}

EtaSchedule {Mu {2}

ΞwitchTime {10}}

ReductFactor {0. 95 145}

) AnySave {FuzzCtrl {file name {f.TrueBatch.dat}

MaxDeltaOb]) (0.3} AnyLoad {75} fιle_name {f.TrueBatch.dat} MomentumBackProp {} Alpha {0.05}

Eta {0.05}} DerivEps {0}}} 80 Ob] FctTracer {LmeSearch {Active {F}

PatternSelection {File {ob] Func} Let Sequential} ExpRandom {SearchControl {

Lambda {2} 85 SearchStrategy {} be HillClimberControl Segmentation {HillClimberControl {

OutputNode {-1}% InιtιalAlιve {0.95} ExpectedCutOff {0.5} InheritWeights {T} PercentageForGroupB {0.2} 90 Beta {0.1}

MutationType {DistπbutedMacroMutation

WtPruneCtrl {MaxTnals {50} Tracer {}

Active {F} 95 PBILControl {Set {Validation}% ImtιalAlιve {0. 95} File {trace} InheritWeights {T}} Beta {0. 1}

Active {F} Alpha {0.1} Randomize {0} 100 PopulationSize {40} PruningSet {Tram. + Valid. } Method {Ξ-Prumng} PopulationControl {} pCrossover {1}

LearnAlgo {CrossoverType {SimpleCrosso- sei Con] Gradient 105 VarioEta {Scalmg {T}

MinCalls {200} ScalingFactor {2}

} Sharing {T} MomentumBackProp {SharmgFactor {0. .05]

Alpha {0.05} 110 PopulationSize {50}} mm. % InιtιalAlιve {0. .01} Quickprop {max. % ImtιalAlιve {0. ■ 1}

Decay {0.05}}

Mu {2} 1} 115 pMutation {0} low memory BFGS {

Limit {2} Ob] ectiveFunctionWeights {

% Alιve {0.6}

E (TS) {0.2}

AnySave {120 Improvement (TS) {0} file name {f.LineSearch.dat E (VS) {1}

Improvement (VS) {0}

AnyLoad {(E (TS) -E (VS)) / max (E (TS), E (VS)) {0 file name f.LineSearch.dat}}

125 LipComplexity {0}

EtaNull {1} OptComplexity {2} MaxSteps {10} testVal (dead) -testVal (alive) {0} LS_Precιsιon {0.5}} TrustRegion {T} AnySave {DerivEps {0} 130 file name {BatchSize {2147483647} f. GeneticWeightSelect. dat}}}

GeneticWeightSelect {AnyLoad {PatternSelection {fιle_name {be Sequential 135 f. GeneticWeightSelect. dat} ExpRandom {}

Lambda {2} Eta {0. 05}} DerivEps {0} Segmentation {BatchSize {5}

OutputNode {-1} 140 ümmEpochsForFitnessTest ExpectedCutOff {0.5} ffmaxEpochsForFitnessTest {3} PercentageForGroupB {0. SelectWeights {T}} SelectNodes {T}} maxGrowthOfValError {0. 005}

LearnAlgo {145 be VarioEta VarioEta {CCMenu {

MinCalls {200} Clusters { mlp. mput0_auto {75} ActFunction {} be the FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel} 80} ptanh {parameter {0.5}} SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5} 85}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {90 Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {

} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} 95 Norm {NoNorm} DampmgFactor {1}}

} mlp.mputδ {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id NewNoiseLevel {0 100 plogistic {parameter {0.5}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0 105} pid {

} parameter {0.5}

SaveNoiseLevel {

Filename {noise_level.dat} 110 InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} Damp gFactor {1} SaveManipulatorData {115}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} 120 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp. mput6 {} ActFunction {} let id 125 FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}} ptanh {} parameter {0.5} 130} SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5}} LoadNoiseLevel {

135 Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} 140 LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} 145 mlp. mput4 {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic { parameter {0.5} 75 NewNoiseLevel {0}} ptanh {} parameter {0.5}} SaveNoiseLevel {pid {80 Filename {noise__level.dat} parameter {0.5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {} be None 85 SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat}

NoiseEta {1}}

DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat} AdaptiveGaussNoise {90}

NoiseEta {1} Norm {NoNorm}

DampmgFactor {1}}} mlp.ιnput2 {FixedUmformNoise {ActFunction {

SetNoiseLevel {95 be id

NewNoiseLevel {0 plogistic {parameter {0.5}}

FixedGaussNoise {ptanh {~ SetNoiseLevel {100 parameters {0.5}

NewNoiseLevel {0}}} pid {} parameter {0.5}}} SaveNoiseLevel {105}

Filename {noise_level.dat} InputModification {

} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}

} 110 DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {

} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat} 115}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.mput3 {} ActFunction {120} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}}} ptanh {125 parameters {0.5}} SaveNoiseLevel {pid { Filename {noise_level.dat} parameter {0.5}}} 130 LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1} 135} DampmgFactor {1} LoadMampulatorData {

} Filename {inputMamp.dat} AdaptiveGaussNoise {}

NoiseEta {1} Norm {NoNorm}

DampmgFactor {1} 140} mlp.inputl {FixedUmformNoise {ActFunction {

SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {

} 145 parameters 0.5}}} FixedGaussNoise {ptanh {

SetNoiseLevel {parameter 0.5 } 75 SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0. 5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {80} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} 85 Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp.externalδ {

FixedUmformNoise {90 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0 J plogistic {} parameter {0.5}}}

FixedGaussNoise {95 ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0]}

} pid {

} parameter {0.5}

100

SaveNoiseLevel {

Filename {noise_level.dat} InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} 105 NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {110 DampmgFactor {1}

Filename {inputMamp.dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0) mlp.mputO {115} ActFunction {} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0)} 120 ptanh {} parameter {0.5}} SaveNoiseLevel {pid { Filename {noise_level.dat} parameter {0.5} 125} LoadNoiseLevel {

Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {

AdaptiveUniformNoise {130 Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} 135 Norm {NoNorm} DampmgFactor {1}}} mlp. externalδ {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} 140 plogistic {} parameter {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0} 145}

} pid {

} parameter {0.5}} 75 Filename {noise_level.dat;

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} 80 LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} 85 mlp. external3 {

FixedUmformNoise {ActFunction {SetNoiseLevel {be ld

NewNoiseLevel {0} plogistic {parameter {0.5}

90}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}} pid {} 95 parameters {0.5}}} SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {

} be none

LoadNoiseLevel {100 AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {} 105 NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}

} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {

} 110 NewNoiseLevel {0} mlp. external4 {ActFunction {be ld FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0. 5} 115 NewNoiseLevel {0} ptanh {parameter 0.5}

SaveNoiseLevel {pid {120 Filename {noise_level.dat parameter {0.5} LoadNoiseLevel {

Filename {noise_level.dat j

InputModification {} be None 125 SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {130} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}} mlp. external2 {

FixedUmformNoise {ActFunction {SetNoiseLevel {135 be ld

NewNoiseLevel {0} plogistic {} parameter 0.5}

FixedGaussNoise {ptanh {SetNoiseLevel {140 parameter 0.5}

NewNoiseLevel {0}} pid {parameter {0.5

SaveNoiseLevel {145

Filename {noise__level.dat InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise { NoiseEta {1} 75} DampmgFactor {ι 1 LoadMampulatorData {

} Filename {inputMamp.dat}

AdaptiveGaussNoise {}

NoiseEta {1} Norm {NoNorm}

DampmgFactor i 1 80} mlp.externalO {FixedUmformNoise {ActFunction {

SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {

} parameter {0.5}}} FixedGaussNoise {ptanh {

SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}

} 90 pid {parameter {0.5}}

SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {

} 95 be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} 100 AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} FixedUmformNoise {

Norm {NoNorm} 105 SetNoiseLevel {} NewNoiseLevel {0} mlp. externall {ActFunction {be id FixedGaussNoise {plogistic {110 SetNoiseLevel {parameter {0.5} NewNoiseLevel {0} ptanh {parameter {0.5}} 115 SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5}}} LoadNoiseLevel {} Filename { noise_level.dat}

InputModification {120} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} 125 Filename {inputMamp.dat}

AdaptiveGaussNoise {NoiseEta {1} Norm {NoNorm DampmgFactor {1}}} mlp. autoassoc {

FixedUmformNoise {130 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {} parameter 0.5}}}

FixedGaussNoise {135 ptanh {SetNoiseLevel {parameter 0.5}

NewNoiseLevel {0}} pid {

} parameter {0.5}

140}

SaveNoiseLevel {}

Filename {noise_level.dat ErrorFunc {} be LnCosh LoadNoiseLevel {Ixl {

Filename {noise_level.dat 145 parameters {0.05}}} SaveManipulatorData {LnCosh {

Filename {inputMamp.dat} parameter {2} 75 parameters {0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5}}}

Norm {NoNorm} 80 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0}} Weightmg {1 1 1 1 1 1 1 1 1 1}}} ErrorFunc {mlp. futurel_target {85 be LnCosh ActFunction {Ixl {be id parameter {0.05} plogistic {} parameter {0.5} LnCosh {} 90 parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5} 95}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0} let LnCosh Weightmg {1 1 1 1 1 1 1 1 1 1} Ixl {100} parameter {0.05} mlp. future4_target {} ActFunction {LnCosh {be id parameter {2} plogistic {} 105 parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} 110 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0 Weightmg {1 1 1 1 1 1 1 1 1 1} ErrorFunc {mlp. future2_target {115 be LnCosh ActFunction {Ixl {be id parameter {0.05} plogistic {} parameter {0.5} LnCosh {} 120 parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5} 125}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0 set LnCosh Weightmg {1 1 1 1 1 1 1 1 1 1 Ixl {130} parameter {0.05} mlp.futureδ target {} ActFunction {LnCosh {set id parameter { 2} plogistic {} 135 parameters {0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5}}

Norm {NoNorm} 140 pid {ToleranceFlag {F parameter {0.5} Tolerance {0 0 0 0 0) Weightmg {1 1 1 1 1}} ErrorFunc {mlp. future3_target {145 be LnCosh ActFunction {Ixl {be id parameter {0.05} plogistic { LnCosh {75 be tanh parameter {2} plogistic {

} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

} 80 parameters {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weighting {1 1 1 1 1 1 1 1 1 1} 85

} ErrorFunc {mlp.future6_target {be LnCosh ActFunction {Ixl {be id parameter {0.05} plogistic {90} parameter {0.5} LnCosh {

} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} 95 parameter {le-06} pid {1 parameter {0.5}}} norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {100 Tolerance {0 0 0 0 0} be LnCosh Weighting {1 1 1 1 1} Ixl {1 parameter {0.05} mlp.pastδ {

} ActFunction {

LnCosh {105 be tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} 110 parameter {0.5}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5 Tolerance {0 0 0 0 0 0 0 0 0 0} Weighting {1 1 1 1 1 1 1 1 1 1 115}} ErrorFunc {mlp.auto_zero {sei LnCosh ActFunction { Ixl {sei tanh parameter 0.05 plogistic {120} parameter {0.5} LnCosh {} parameter 2} ptanh {parameter {0.5} parametricalEntropy {} 125 parameter {le-06} pid {} parameter {0.5}}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {130 Tolerance {0 0 0 0 0} let LnCosh Weighting {1 1 1 1 1} Ixl {} parameter {0.05} mlp.past4 {} ActFunction {LnCosh {135 let tanh parameter {2} plogistic {} parameter {0.5 } parametricalEntropy {} parameter {le-06} ptanh {

140 parameters {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0} Weighting {1 1 1 1 1 145}

} ErrorFunc {mlp.pastδ {be LnCosh ActFunction {Ixl { parameter {0.05} 75 mlp.pastl {} ActFunction {LnCosh {be tanh parameter {2} plogistic {} parameter {0. .5} parametricalEntropy {80 parameter {le-06} ptanh {parameter {0. • 5}}

Norm {NoNorm} pid {ToleranceFlag {F} 85 parameters {0. • 5} Tolerance {0 0 0 0 0}} Weighting {1 1 1 1 1}}} ErrorFunc {mlp.past3 {be LnCosh ActFunction {90 Ixl { be tanh parameter {0, .05 plogistic {) parameter {0.5} LnCosh {} parameter {2} ptanh {95 parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5}) 7 100 norm { NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0} be LnCosh Weightmg {1 1 1 1 1} Ixl {} parameter {0.05} 105 mlp.present {} ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {110} parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} 115 parameter {0.5} Tolerance {0 0 0 0 0 Weight g {1 1 1 1 1} ErrorFunc {mlp.past2 {sei LnCosh ActFunction {120 Ixl {sei tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {} parameter {2} ptanh {125} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5}}

130 ToleranceFlag {NoNorm}

ErrorFunc {Tolerance {0 0 0 0 0} be LnCosh Weighting {1 1 1 1 1} Ixl {} parameter {0.05} 135 mlp.futurel {} ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {140} parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} 145 parameters {0.5} Tolerance {0 0 0 0 0} Weight g {1 1 1 1 1}} ErrorFunc { be none 75 Weightmg 1 1 1 1 1} Ixl {parameter {0.05} mlp.future4 {

} ActFunction {

LnCosh {be tanh parameter {2} 80 plogistic {

} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

10} parameters {0.5}} 85}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0}}

15 Weightmg {1 1 1 1 1}}

90 ErrorFunc {mlp.future2 {sei none ActFunction {Ixl {sei tanh parameter {0.05}

20 plogistic {} parameter 0.5 95 LnCosh {parameter {2} ptanh {} parameter 0.5} parametricalEntropy {

25 parameters {le-06} pid {100} parameters {0.5}}} Norm {NoNorm}} ToleranceFlag {F}

30 ErrorFunc {Tolerance {0 0 0 0 0} be none 105 Weightmg {1 1 1 1 1} Ixl {} parameter {0.05} mlp.futureδ {

} ActFunction {

35 LnCosh {be tanh parameter {2} 110 plogistic {

1 parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

40} parameters {0.5}} 115}

Norm {NoNorm} pid {ToleranceFlag {F} parameter (0.5} Tolerance {0 0 0 0 0}}

45 Weightmg {1 1 1 1 1}}

120 ErrorFunc {p.future3 {sei none ActFunction {Ixl {sei tanh parameter {0.05}

50 plogistic {} parameters {: 0. • 5} 125 LnCosh {

J parameter {2} ptanh {} parameter {: 0. .5} parametricalEntropy {

55) parameters {le-06} pid {130 parameters | : o, .5}} Norm {NoNorm}

} ToleranceFlag {F}

60 ErrorFunc {Tolerance {0 0 0 0 0} be none 135 Weightmg {1 1 1 1 1} Ixl {} parameter | [o .05} mlp.futureδ {

} ActFunction {

65 LnCosh {be tanh parameter i [2} 140 plogistic {parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

70 parameters {0.5}

145

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5} Tolerance {0 0 0 0 0} } 75}

ErrorFunc {} set none Norm {NoNorm} Ixl {} parameter {0.05} mlp.state21 {} 80 ActFunction {LnCosh {set tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy} parameter {le-06 85 ptanh {1 parameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 90} Weightmg {1 1 1 1 1}

} Norm {NoNorm} mlp.state65 {} ActFunction {mlp.statelO {sei tanh 95 ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5} ptanh {} parameter {0.5} 100 ptanh {} parameter {0.5 } pid {} parameter {0.5} pid {parameter {0.5}

105

Norm {NoNorm}}} Norm NoNorm mlp.state54 {ActFunction {mlp.stateOl {sei tanh 110 ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter 0.5} ptanh {parameter {0.5} 115 ptanh {} parameter {0.5 pid {} parameter {0.5} pd {} parameter {0.5} 120

Norm {NoNorm}} Norm {NoNorm} mlp.state43 {} ActFunction {mlp.statel2 {sei tanh 125 ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5} ptanh {} parameter {0.5} 130 ptanh { } parameter {0.5} pid {} parameter {0.5} pd {} parameter {0.5}} 135}

Norm {NoNorm}}} Norm {NoNorm} mlp.state32 {} ActFunction {mlp.state23 {sei tanh 140 ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5} ptanh {} parameter {0.5} 145 ptanh {} parameter {0.5} pid {1 parameter {0.5} pid { parameter {0. 5} 75 MinWeight {0}} LoadWeightsLocal {

Norm {NoNorm} Filename {std}

}} mlp.state34 {80 SaveWeightsLocal {ActFunction {Filename {std} sei tanh} plogistic {Alive {T} parameter {0.5} WtFreeze {F}} 85 AllowPruning {F} ptanh {EtaModifier {1} parameter {0.5} Penalty { NoPenalty}}} pid {mlp. futurel-> futurel_target {parameter {0.5} 90 LoadWeightsLocal {

Filename {std}}

Norm {NoNorm} SaveWeightsLocal {} Filename {std} mlp.state45 {95} ActFunction {Alive {F} sei tanh WtFreeze {F} plogistic {AllowPruning {F} parameter {0.5} EtaModifier {1}

1 100 Penalty {NoPenalty} ptanh {} parameter {0.5} mlp.bιas-> futurel_target {} LoadWeightsLocal {pid {Filename {std} parameter {0.5} 105}} SaveWeightsLocal {} Filename {std}

Norm {NoNorm}}} Alive {F} mlp.stateδδ {110 WtFreeze {F} ActFunction {AllowPruning {F} sei tanh EtaModifier {1} plogistic {Penalty {NoPenalty} parameter {0.5}}} 115 mlp. uture2-> future2_target {ptanh {LoadWeightsLocal {parameter {0.5} Filename {std}}} pid {SaveWeightsLocal {parameter {0.5} 120 Filename {std}}

Alive {F}

Norm {NoNorm} WtFreeze {F}} AllowPrumng {F}} 125 EtaModifier {1} Connectors {Penalty {NoPenalty} mlp. auto_zero-> autoassoc} WeightWatcher {mlp.bιas-> future2_target {Active {F} LoadWeightsLocal {MaxWeight {1} 130 Filename {std} MinWeight {0}}} SaveWeightsLocal {

LoadWeightsLocal {Filename {std} Filename {std}}} 135 Alive {F}

SaveWeightsLocal {WtFreeze {F} Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F} 140} AllowPruning {F} mlp. future3-> future3_target {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.bιas-> autoassoc {145 SaveWeightsLocal {WeightWatcher {Filename {std} Active {F}} MaxWeight {1} Alive {F} WtFreeze {F} 75 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> future3_target {80 AllowPruning {F} LoadWeightsLocal {EtaModifier {1 }

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> future6_target {

Filename {std} 85 LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 90} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp. future4-> future4_target {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 95 Penalty {NoPenalty}}} SaveWeightsLocal {mlp. ιnput0_auto-> auto_zero

Filename {std} WeightWatcher {} Active {F}

Alive {F} 100 MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 105} mlp.bιas-> future4_target {SaveWeightsLocal {LoadWeightsLocal Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {110 WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} 115 mlp.bιas-> auto_zero {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} Active {F}} MaxWeight {1} mlp. future5-> future5_target {MinWeight {0} LoadWeightsLocal {120}

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} 125 Filename {std}

Alive {F}} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPrumng {F} Penalty {NoPenalty} 130 EtaModifier {1}} Penalty {NoPenalty} mlp.bιas-> future5_target { } LoadWeightsLocal {mlp. ιnput6-> pastδ {

Filename {std} LoadWeightsLocal {} 135 Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 140 Alive {F} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp. futureδ-> future6_target {145} LoadWeightsLocal {mlp.bιas-> past6 {

Filename {std} LoadWeightsLocal {

Filename {std} 75 Alive

SaveWeightsLocal {WtFreeze {F} Filename {std} AllowPrumng {F}

} EtaModifier {1}

Alive {F} Penalty {NoPenalty}

WtFreeze {F} 80}

AllowPruning {F} mlp.bιas-> past4 {

EtaModifier {1} LoadWeightsLocal {

Penalty {NoPenalty} Filename {std}}} mlp.mput5-> past5 {85 SaveWeightsLocal {

LoadWeightsLocal {Filename {std} Filename {std}}

} Alive {F}

SaveWeightsLocal {WtFreeze {F} Filename {std} 90 AllowPruning {F}

} EtaModifier {1}

Alive {F} Penalty {NoPenalty}

WtFreeze {F}}

AllowPruning {F} mlp. mput3-> past3 {

EtaModifier {1} 95 LoadWeightsLocal {

Penalty {NoPenalty} Filename {std}}} mlp.state65-> past5 {SaveWeightsLocal {

WeightWatcher {Filename {std} Active {F} 100} MaxWeight {1} Alive {F} MinWeight {0} WtFreeze {F}

} AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} 105 Penalty {NoPenalty}

SaveWeightsLocal {mlp.state43-> past3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 110} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 115 WtFreeze {F} mlp.bιas-> past5 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} SaveWeightsLocal {120 mlp.bιas-> past3 {

Filename {std} LoadWeightsLocal {} Filename {std)

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.mput4-> past4 {AllowPruning {F} LoadWeightsLocal {130 EtaModifier {1}

Filename {std} Penalty {NoPenalty

SaveWeightsLocal {mlp. mput2-> past2 {Filename {std} LoadWeightsLocal {

} 135 Filename {std}

Alive {F}}

WtFreeze {F} SaveWeightsLocal {

AllowPruning {F} Filename {std}

EtaModifier {1}}

Penalty {NoPenalty 140 Alive {F}} WtFreeze {F} mlp.state54-> past4 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

} 145}

SaveWeightsLocal {mlp.state32-> past2 {Filename {std} LoadWeightsLocal {

Filename {std} 75}

SaveWeightsLocal {mlp.statelO-> present {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 80 SaveWeightsLocal {AllowPruning {F} Filename std EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> past2 {85 AllowPruning {F} LoadWeightsLocal { EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> present {

Filename {std} 90 LoadWeightsLocal {1 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 95} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.mputl-> pastl {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 100 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state01-> futurel {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 105} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 110 WtFreeze {F} mlp. state21-> pastl {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {115 mlp.bιas-> futurel {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 120 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> pastl {AllowPruning {F} LoadWeightsLocal {125 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.statel2-> future2 {

Filename {std} LoadWeightsLocal {} 130 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 135 Alive {F}} WtFreeze {F} mlp mput0-> present {AllowPruning {F} LoadWeightsLocal { EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 140} SaveWeightsLocal {mlp.bιas-> future2 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 145 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F} WtFreeze {F} 75 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp. state23-> future3 {80 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state56-> future6 {

Filename {std} 85 LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 90} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp.bιas-> future3 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 95 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> futureδ {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 100} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 105 WtFreeze {F} mlp. state34-> future4 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {110 mlp.external6-> state65 j

Filename {std} WeightWatcher {} Active {F}

Alive {F} MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F} 115} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.bιas-> future4 {SaveWeightsLocal {LoadWeightsLocal {120 Filename {std}

Filename {std}}) Alive {F} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} 125 EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} mlp.past6-> state65 {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} 130 Active {F}} MaxWeight {1} mlp. state45-> future5 {MinWeight {0} LoadWeightsLocal {}

Filename {std} LoadWeightsLocal {} 135 Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 140 Alive {F} AllowPrumng {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.bιas-> future5 { 145} LoadWeightsLocal {mlp.external5-> state54 (

Filename {std} LoadWeightsLocal {

Filename {std} 75 Alive

SaveWeightsLocal {WtFreeze {F} Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F} 80} AllowPruning {F} mlp.external3-> state32 {EtaModifier {1} LoadWeightsLocal (Penalty {NoPenalty} Filename {std}}} mlp.past5-> state54 {85 SaveWeightsLocal {LoadWeightsLocal {Filename {std} Filename {std}}} Alive {F}

SaveWeightsLocal {WtFreeze {F} Filename {std} 90 AllowPruning {F}} EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} mlp.past3-> state32 {EtaModifier {1} 95 LoadWeightsLocal {Penalty {NoPenalty} Filename {std} mlp. state65-> state54 SaveWeightsLocal {WeightWatcher {Filename {std} Active {F} 100} MaxWeight {1} Alive {F} MinWeight {0} WtFreeze {F}} AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} 105 Penalty {NoPenalty

SaveWeightsLocal {mlp.state43-> state32 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 110} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 115 WtFreeze {F} mlp. external4-> state43 • AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {120 mlp.external2-> state21 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.past4-> state43 {AllowPruning {F} LoadWeightsLocal {130 EtaModifier {1}

Filename {std} Penalty {NoPenalty}

SaveWeightsLocal {mlp.past2-> state21 {Filename {std} LoadWeightsLocal {

} 135 Filename {std}

Alive {F}}

WtFreeze {F} SaveWeightsLocal {

AllowPruning {F} Filename {std}

EtaModifier {1}}

Penalty {NoPenalty] 140 Alive {F}} WtFreeze {F} mlp. state54-> state43 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty

} 145}

SaveWeightsLocal {mlp.state32-> state21 {Filename {std} LoadWeightsLocal {

Filename {std} 75

SaveWeightsLocal {mlp.statel0-> state01 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 80 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.externall-> statelO {85 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state01-> statel2 {

Filename {std} 90 LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 95} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.pastl-> statelO {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 100 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.statel2-> state23 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 105} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}

} 110 WtFreeze {F} mlp.state21-> statel0 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {115 mlp.state23-> state34 {

Filename {std} LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal (AllowPruning {F} 120 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp.external0-> state01 {AllowPrumng {F) LoadWeightsLocal {125 EtaModifier {1} Filename {std} Penalty {NoPenalty}

SaveWeightsLocal {mlp.state34-> state45 {Filename {std} LoadWeightsLocal {} 130 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 135 Alive {F}} WtFreeze {F} mlp.present-> state01 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}} 140}

SaveWeightsLocal {mlp. state45-> state56 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 145 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive F} WtFreeze {F} INPUT mput3; AllowPruning {F} INPUT mput4; EtaModifier {1} INPUT mput5; Penalty {NoPenalty INPUT mputδ;

75

EXTERNAL external 65;

AnySave {EXTERN extern54; file name {f.CCMenu.dat} EXTERN extern43;

} EXTERN extern32;

AnyLoad {80 EXTERN extern21; file name {f.CCMenu.dat} EXTERN externlO;

}} TestRun {AUTO output_auto;

Filename {Test} 85

Part. Transformed {F} AUTO fmal6; } AUTO inal5; Online {AUTO fmal4;

Filename {Onlme.dat} AUTO fmal3;

90 AUTO fmal2;

AUTO finall;

ZERO bottleneck;

95 ZERO future 6;

Part 4: ZERO futureδ;

ZERO future4;

ZERO future3; net {ZERO future2; const no STATE = 6; 100 ZERO futurel;

ZERO present;

ZERO pastl; cluster INPUT (EQUIVALENT, ZERO past2;

IN ); ZERO past3; cluster EXTERNAL (EQUIVALENT, 105 ZERO past4;

IN ); ZERO past5; cluster STATE (EQUIVALENT, ZERO pastβ;

DIM (no_ _STATE), HID); cluster BACK (EQUIVALENT, STATE stateδδ;

DIMfnr _^ _STATE), HID); 110 STATE state54; cluster ZERO (EQUIVALENT, STATE state43;

OUT); STATE state32; cluster AUTO (EQUIVALENT, STATE state21;

OUT); STATE statelO;

115 STATE stateOl; connect STATE_STATE (STATE -> STATE statel2;

STATE; // A STATE state23; connect ZERO STATE (ZERO -> STATE state34;

STATE; // B STATE state45; connect STATE_ZERO (STATE -> 120 STATE state56;

ZERO; // C connect INPUT_ZERO (INPUT -> BACK back65;

ZERO; // D BACK back54 connect ZERO_AUTO (ZERO -> BACK back43

AUTOMOBILE ; // E 125 BACK back32 connect EXTERN_STATE (EXTERN -> BACK back21

STATE; // E BACK backlO connect BIAS_ZERO (blas ->

ZERO; // - connect BIAS_AUTO (blas -> 130

AUTOMOBILE ; connect BACK_BACK (BACK -> connect (mput_auto -> bottleneck

BACK; // F, INPUT_ZERO); // D connect BACK_ZERO (BACK -> connect (bottleneck -> out-

ZERO; // G 135 put_auto, ZERO_AUTO); // E connect ZERO_BACK (ZERO ->

BACK; // H connect (blas -> bottleneck, BIAS_ZERO); connect (blow -> out-

INPUT mput_auto; 140 put_auto, BIAS AUTO);

INPUT mputO; // -

INPUT inputl;

INPUT mput2; 75 connect inputδ -> past6

INPUT_ZERO); // D connect stateOl -> statel2, connect input5 -> pastδ STATE_STATE); // A

INPUT_ZERO); // D connect stateOl -> futurel, connect input4 -> past4 80 STATE_ZERO); // C

INPUT_ZERO); // D connect futurel -> finall, connect input3 -> past3 ZERO_AUTO); // E

INPUT_ZERO); // D connect input2 -> past2 connect statel2 -> state23,

INPUT_ZERO); // D 85 STATE_ΞTATE); // A connect inputl -> pastl connect statel2 -> future2,

INPUT_ZERO); // D STATE_ZERO); // C connect inputO -> present connect future2 -> final2,

INPUT_ZERO); // D ZERO_AUTO); // E

90 connect externδδ -> state65 connect state23 -> state34,

EXTERN_STATE); STATE_STATE); // A connect extern54 -> state54 connect state23 -> future3,

EXTERN_STATE); STATE_ZERO); // C connect extern ₄ 3 -> state43 95 connect future3 -> final3,

EXTERN_STATE); ZERO_AUTO); // E connect extern32 -> state32

EXTERN_STATE); connect state34 -> state45, connect extern21 -> state21 STATE_STATE); // A

EXTERN_ΞTATE); - 100 connect state34 -> future4, connect externlO -> statelO ΞTATE_ZERO); // C

EXTERN_STATE); connect future4 -> final4,

ZERO_AUTO); // E connect pastδ -> state65 105 connect state45 -> state56,

ZERO_STATE); // B STATE_ΞTATE); // A connect state45 -> futureδ, connect state65 -> pastδ STATE_ZERO); // C

ΞTATE_ZERO); // C connect futureδ -> finalδ, connect past5 -> state54 110 ZERO_AUTO); // E

ZERO_STATE); // B connect stateδδ -> state54 connect state56 -> futureδ,

STATE_ΞTATE); // A STATE_ZERO); // C connect futureδ -> finalδ, connect state54 -> past4 115 ZERO_AUTO); // E

STATE_ZERO); // C connect past4 -> state43

ZERO_STATE); // B connect blas -> pastδ, connect state54 -> state43 BIAS_ZERO);

STATE_STATE); // A 120 connect bias -> past5, BIAS_ZERO); connect state43 -> past3 connect blas -> past4,

STATE_ZERO); // C BIAS_ZERO); connect past3 -> state32 connect bias -> past3, ZERO_STATE); // B 125 BIAS_ZERO); connect state43 -> state32 connect bias -> past2,

STATE_STATE); // A BIAS_ZERO); connect bias -> pastl, connect state32 -> past2 BIAS_ZERO);

STATE_ZERO); // C 130 connect bias -> present, connect past2 -> state21 BIAS_ZERO);

ZERO_STATE); // B connect bias -> futurel, connect state32 -> state21 BIAS_ZERO);

ΞTATE_STATE); // A connect bias -> future2,

135 BIAΞ_ZERO); connect state21 -> pastl connect bias -> future3,

STATE_ZERO); // C BIAS_ZERO); connect pastl -> statelO connect bias -> future4,

ZERO_STATE); // B BIAS_ZERO); connect state21 -> statelO 140 connect bias -> futureδ,

STATE_STATE); // A BIAS_ZERO); connect bias -> futureδ, connect statelO -> present BIAS_ZERO);

STATE_ZERO); // C connect present -> stateOl 145 connect bias -> finall,

ZERO_STATE); // B BIAS_AUTO); connect statelO -> stateOl connect bias -> final2,

STATE_ΞTATE); // A BIAS AUTO); connect bias -> final3, AdaptTime {10}

BIAS_AUTO); EpsObj {0.001} connect bias -> final4, ObjSet {Training}

BIAS_AUTO); 75 EpsilonFac {1} connect bias -> finalδ,}

BIAS_AUTO); • ExtWtDecay {connect bias -> finalδ, Lambda {0.001}

BIAS_AUTO); AutoAdapt {F}

80 AdaptTime {10} EpsObj {0.001} connect present -> backlO, ObjSet {Training}

ZERO_BACK); // H EpsilonFac {1}} connect backlO -> pastl, 85 Finnoff {

BACK_ZERO); / / G AutoAdapt {T} connect pastl -> back21, Lambda {0}

ZERO_BACK); // H DeltaLambda {le-06} connect backlO -> back21, ReducFac {0.9}

BACK_BACK); / / F 90 Gamma {0.9} DesiredError {0} connect back21 -> past2,}

BACK_ZERO); // G} connect past2 -> back32, ErrorFunc {

ZERO_BACK); // H 95 be LnCosh connect back21 -> back32, Ixl {

BACK_BACK J ^" ; IIF parameter {0.05}} connect back32 -> past3, LnCosh {

BACK_ZERO); // G 100 parameters {2} connect 1 past3 -> back43,}

ZERO_BACK); // H parametricalEntropy {connect 1 back32 -> back43, parameter {le-06}

BACK_BACK); // F}

105} connect 1 back43 -> past4, AnySave {

BACK_ZERO); // G file name {f.Globals.dat} connect i past4 -> back54,}

ZERO_BACK); // H AnyLoad {connect 1 back43 -> back54, 110 file name {f.Globals.dat}

BACK_BACK); // F}

ASCII {T} connect (back54 -> pastδ,}

BACK_ZERO); // G LearnCtrl {connect (past5 -> back65, 115 be Stochastic

ZERO_BACK); // H Stochastic {connect (back54 -> back65, PatternSelection {

BACK_BACK); // F be Permute ExpRandom {connect (back65 -> pastδ, 120 Lambda {2}

BACK_ZERO); // G}

} mlp; Segmentation {

OutputNode {-1} ExpectedCutOff {0.5}

125 PercentageForGroupB {0.2}}

Part 5: WtPruneCtrl {PruneSchedule {

BpNet {130 be FixSchedule

Globals {FixSchedule {WtPenalty {Limit_0 {10} be NoPenalty Limit_l {10} Weigend {Limit_2 {10}

Lambda {0} 135 Limit_3 {10) AutoAdapt {T} RepeatLast {T} wO {1}}

DeltaLambda {le-06 DynSchedule {ReducFac {0.9} MaxLength {4} Gamma {0.9} 140 MinimumRuns {0} DesiredError {0} Training {F}} Validation {T} WtDecay {Generalization {F}

Lambda {0.005}} AutoAdapt {F} 145 DivΞchedule { Divergence {0.1} 75 VarioEta {MinEpochs {5} MinCalls {50}} MomentumBackProp {

PruneAlg {Alpha {0.05} be FixPrune 80}

FixPrune {Quickprop {

Perc_0 {0. .1} Decay {0.05}

Perc_l {0. .1} Mu {2}

Perc_2 {0. .1}}

Perc_3 {0. .1} 85} AnySave {

EpsiPrune {file name {f.Stochastic.dat}

DeltaEps {0.05}

StartEps {0., 05 AnyLoad {

MaxEps {1} 90 fιle_name {f.Stochastic.dat}

ReuseEps {F}}

} BatchSize {15}

1 Eta {0.005}

Tracer {DerivEps {0}

Active {F} 95}

Set {Validation TrueBatch {

File {trace} PatternSelection {be sequential

Active {F} ExpRandom {Randomize {0} 100 Lambda {2} PruningSet {Train. + Valιd. }} Method {S-Prumng} Segmentation {} OutputNode {-1}

StopControl {ExpectedCutOff {0.5} EpochLimit {105 PercentageForGroupB {0.2} Active {T} MaxEpoch {10000}

} WtPruneCtrl {MovingExpAverage {Tracer {

Active {F} 110 Active {F}

MaxLength {4} Set {Validation}

Training {F} File {trace}

Validation {T}}

Generalization {F} Active {F}

Decay {0.9} 115 Randomize {0}

} PruningSet {Train. + Valιd. }

CheckOb] ectιveFct {Method {Ξ-Prumng} Active {F}} MaxLength {4} EtaCtrl {Training {F} 120 Active {F} Validation {T}} Generalization {F} LearnAlgo {} be VarioEta

CheckDelta {VarioEta {Active {F} 125 MinCalls {200} Divergence {0.1}}} MomentumBackProp {} Alpha {0.05}

EtaCtrl {} Mode {130 Quickprop {be EtaSchedule Decay {0.05} EtaSchedule {Mu {2}

SwitchTime {10} 1 ReductFactor {0.95}}

} 135 AnySave {

FuzzCtrl {fιle_name {f.TrueBatch.dat]

MaxDeltaOb] {0.3}} MaxDelta20b] {0.3} AnyLoad {MaxEtaChange 0. 02 file name {f.TrueBatch.dat] MinEta {0.001 140} MaxEta {0.1} Eta {0.05} Ξmoother {1} DerivEps {0}} LmeSearch {

Active {F} 145 PatternSelection {

} be sequential

LearnAlgo {ExpRandom {be VarioEta Lambda {2} } 75 be HillClimberControl

Segmentation {HillClimberControl {

OutputNode {-1}% InιtιalAlιve {0.95} ExpectedCutOff {0.5} InheritWeights {T} PercentageForGroupB {0.2 Beta {0.1} 80 MutationType {DistπbutedMacroMutation}

WtPruneCtrl {MaxTrials {50} Tracer {

Active {F} PBILControl {

Set {Validation} 85% ImtιalAlιve {0. 95}

File {trace} InheritWeights {T}} Beta {0.1}

Active {F} Alpha {0.1} Randomize {0} PopulationSize {40} PruningSet {Tram. + Valid. } 90} Method {Ξ-Prumng} PopulationControl {} pCrossover {1}

LearnAlgo {CrossoverType {SimpleCross- be Con] Gradient ver} VarioEta {95 Scalmg {T}

MinCalls {200} ScalmgFactor {2}} Sharing {T} MomentumBackProp {ΞharmgFactor {0.05}

Alpha {0.05} PopulationSize {50}} - 100 mm. % ImtιalAlιve {0.01} Quickprop {max. ImtιalAlιve {0.1}

Decay {0.05}

Mu {2}} pMutation {0} low memory BFGS {105}

Limit {2} whether] ectιveFunctιonWeιghts {}% Alιve {0.6}

E (TS) {0.2}

AnySave {Improvement (TΞ) {0} file name f.LineSearch.dat 110 E (VΞ) {1}

} Improvement (VS) {0}

AnyLoad {(E (TS) -E (VS)) / max (E (TS), E (VS)) {0 file name f. LineSearch.dat}

LipComplexity {0}

EtaNull {1} 115 OptComplexity {2} MaxSteps {10} testVal (dead) -testVal (alive) {0} LS_Precιsιon {0.5}} TrustRegion {T} AnySave {DerivEps {0} fιle_name {BatchSize {2147483647} 120 f.GeneticWeightSelect .dat}}}

GeneticWeightSelect {AnyLoad {PatternSelection {file name {be Sequential f.GeneticWeightSelect.dat} ExpRandom {125}

Lambda {2} Eta {0.05}) DerivEps {0} Segmentation {BatchSize {5}

OutputNode {-1} # mmEpochsForFιtnessTeεt {2} ExpectedCutOff {0.5} 130 SmaxEpochsForFitnessTest {3} PercentageForGroupB {0.2 ΞelectWeights {T}} SelectNodes {T}} maxGrowthOfValError {0.005}

LearnAlgo {be VarioEta 135 VarioEta {CCMenu {

MinCalls {200} Clusters {} mlp. ιnput_auto {MomentumBackProp {ActFunction {

Alpha {0.05} 140 let id} plogistic {} parameter {0.5

Ob] FctTracer {} Active {F} ptanh {File {ob] Func} 145 parameters {0.5}

SearchControl {pid {ΞearchStrategy {parameter 0.5 } 75 LoadNoiseLevel {

Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1} 80} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1} 85}} mlp.inputl {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {} 90 parameters {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}

} 95 pid {

} parameter {0.5}}

SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {} - 100 be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise__level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} 105 AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} FixedUmformNoise {

Norm {NoNorm} 110 SetNoiseLevel {} NewNoiseLevel {0} mlp.mputO {} ActFunction {} be id FixedGaussNoise {plogistic {115 SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}}} ptanh {parameter {0.5}} 120 SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {125} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} 130 Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp.ιnput2 {

FixedUmformNoise {135 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {parameter {0.5}}

FixedGaussNoise {140 ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}} pid {parameter {0.5}

145}

SaveNoiseLevel {}

Filename {noise level.dat InputModification {be None AdaptiveUniformNoise {75 Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {

} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} 80 Norm {NoNorm} DampmgFactor {1}}

} mlp. mput4 {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0] 85 plogistic {parameter {0.5}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0 90}

} pid {

} parameter {0.5}}

SaveNoiseLevel {1

Filename {noise_level.dat [95 InputModification {

} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat ^' NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {100}

Filename {inputMamp.dat} AdaptiveGaussNoise {

} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} 105 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.ιnput3 {} ActFunction {} be ld 110 FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}

}} ptanh {parameter {0.5} 115} SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5}} LoadNoiseLevel {

120 Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} 125 LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm) DampmgFactor {1}}} 130 mlp.inputδ {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {} parameter {0.5}} 135}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}

} pid {

} 140 parameters {0.5}}

SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {} be None LoadNoiseLevel {145 AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData { AdaptiveGaussNoise {75

NoiseEta {1} Norm {NoNorm}

DampmgFactor {1}}} mlp.extern65 {FixedUmformNoise {ActFunction {

SetNoiseLevel {80 be ld

NewNoiseLevel {0} plogistic {

} parameter {0.5}}} FixedGaussNoise {ptanh {

SetNoiseLevel {85 parameters {0.5}

NewNoiseLevel {0}} pid {parameter {0.5}}

SaveNoiseLevel {90}

Filename {noise_level.dat} InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} 95 DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {mputManip.dat} 100}} FixedUmformNoise {Norm {NoNorm} SetNoiseLevel {

NewNoiseLevel {0] mlp. mput6 {} ActFunction {105} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter 0.5} NewNoiseLevel {0] ptanh {110} parameter 0.5} SaveNoiseLevel {pid {Filename {noise_level.dat parameter 0.5}}

115 LoadNoiseLevel {

Filename {noise_level.dat

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1} 120} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1} 125}} mlp.extern54 {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {} 130 parameters {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}} 135 pid {parameter {0.5}}

SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {} 140 be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} 145 AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Tilename {inputMamp.dat}} FixedUmformNoise {75 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {} parameter {0.5}}

FixedGaussNoise {80 ptanh {SetNoiseLevel {parameter 0.5}

NewNoiseLevel {0} pid {parameter {0.5}

85}

SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} 90 NoiseEta {1}

} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {

} NoiseEta {1} LoadMampulatorData {95 DampmgFactor {1}

Filename {inputMamp.dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.extern43 {100} ActFunction {} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}} 105 ptanh {} parameter {0.5}} SaveNoiseLevel {pid { Filename {noise_level. dat] parameter {0.5} 110}} LoadNoiseLevel {} Filename {noise_level. dat]

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {115 Filename {inputMamp. dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {mputManip .dat}

AdaptiveGaussNoise {} NoiseEta {1} 120 Norm {NoNorm} DampmgFactor {1}}} mlp.extern21 {

FixedUmformNoise {ActFunction {SetNoiseLevel {be ld

NewNoiseLevel {0} 125 plogistic {} parameter {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0} 130

} pid {

} parameters {0.5

SaveNoiseLevel {

Filename {noise_level.dat} 135 InputModification {} be None LoadNoiseLevel {AdaptiveUni ormNoise

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {140}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} 145 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {C mlp.extern32 { 75

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5 NewNoiseLevel {0 pid

80 parameters 0.5}

SaveNoiseLevel {}

Filename {noise_level.dat} ErrorFunc {} be LnCosh LoadNoiseLevel {85 Ixl {

Filename {noise_level.dat} parameter {0.05}}} SaveManipulatorData {LnCosh {

Filename {inputMamp.dat} parameter {2}} 90} LoadMampulatorData {parametricalEntropy {

Filename {mputManip.dat} parameter {le-06}}}

Norm {NoNorm}}} 95 Norm {NoNorm} mlp.externlO {ToleranceFlag {F} ActFunction {Tolerance {0 0 0 0 0 0 0 0 0 0} be id Weightmg {1 1 1 1 1 1 1 1 1 1} plogistic {} parameter {0.5} X00 mlp. finalδ {} ActFunction {ptanh {be id parameter {0.5} plogistic {} parameter {0.5} pid {105} parameter {0.5} ptanh {parameter {0.5}}

InputModification {pid {be None 110 parameter {0.5} AdaptiveUniformNoise {} NoiseEta {1} DampmgFactor {1} ErrorFunc {} be LnCosh

AdaptiveGaussNoise {115 Ixl {NoiseEta {1} parameter 0.05 DampmgFactor {1}} LnCosh {

FixedUmformNoise {parameter 2} SetNoiseLevel {120}

NewNoiseLevel {0} parametricalEntropy {parameter {le-06}}

FixedGaussNoise {SetNoiseLevel {125 Norm {NoNorm}

NewNoiseLevel {0 ToleranceFlag {F}

Tolerance {0 0 0 0 0 0 0 0 0 0

} Weightmg {1 1 1 1 1 1 1 1 1 1 l

SaveNoiseLevel {130 mlp.fmal5 {

Filename {noise_level.dat] ActFunction {} be id LoadNoiseLevel {plogistic {

Filename {noise_level.dat] parameter {0., 5}} 135 SaveManipulatorData {ptanh {

Filename {inputMamp.dat} parameter 0.5}}} LoadMampulatorData {pid {

Filename {mputManip.dat} 140 parameters 0.5}}

Norm {NoNorm}} ErrorFunc {mlp.output_auto {be LnCosh ActFunction {145 Ixl {be ld parameter {0.05 plogistic {} parameter {0.5} LnCosh { parameter {2} 75 plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5}} 80}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 1}} 85 ErrorFunc {mlp.fιnal4 {be LnCosh ActFunction {Ixl {be id parameter {0.05} plogistic {} parameter {0.5} 90 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {parameter {le-06} pid {95} parameter 0.5}

Norm {NoNorm}

} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0 let LnCosh 100 Weightmg {1 1 1 1 1 1 1 1 1 1 Ixl {} parameter {0.05} mlp. finall {} ActFunction {LnCosh {be id parameter {2} 105 plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5}} 110}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0 Weighting {1 1 1 1 1 1 1 1 1 1

115 ErrorFunc {mlp.fιnal3 {be LnCosh

ActFunction {Ixl {be id parameter {0.05} plogistic {} parameter {0. .5} 120 LnCosh {l parameter {2} ptanh {} parameter {0, .5} parametricalEntropy {parameter {le-06} pid {125 parameter {0. .5}} norm {NoNorm}

} ToleranceFlag {F} ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0 let LnCosh 130 Weightmg {1 1 1 1 1 1 1 1 1 1

Ix] {} parameters {0.05 mlp. bottleneck {t ActFunction {LnCosh {sei tanh parameter {2} 135 plogistic {parameter {0.5} parametricalEntropy} parameter {le-06 ptanh {parameter {0.5}

140}

Norm {NoNorm} pid {ToleranceFlag {parameter {0.5}

Tolerance {0 0} Weightmg {1 1}

} 145 ErrorFunc {mlp.fιnal2 {be none ActFunction Ixl {be id parameter {0.05} 75 ActFunction {

LnCosh {be tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} 80 ptanh {parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5}

Tolerance {0 0 0} 85}

Weighting {1 1 1}}

ErrorFunc {mlp. futureδ {be none

ActFunction {Ixl {sei tanh 90 parameters {0.05} plogistic {} parameters {0., 5 LnCosh {

1 parameter {2} ptanh {} parameter {0., 5 95 parametricalEntropy {

) parameter {le-06} pid {} parameter {0., 5}} norm {NoNorm}

} ~ 100 ToleranceFlag {F} ErrorFunc {Tolerance {0 0 0} be none Weightmg {1 1 1}

Ixl {} parameter {0. .05 mlp.future3 {

105 ActFunction {

LnCosh {be tanh parameter {2} plogistic {parameter {0.5} parametricalEntropy {} parameter {le-06} 110 ptanh {parameter {0.5}}

Norm {NoNorm pid {

ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0} 115} Weightmg {1 1 1}}

ErrorFunc {mlp.futureδ {sei none ActFunction {Ixl {sei tanh 120 parameter {0.05} plogistic {} parameter {0. 5} LnCosh {l parameter {2} ptanh {} parameter {0. ■ 5} 125 parametricalEntropy {parameter { le-06} pid {} parameter {0. 5}}} norm {NoNorm}

1 130 ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be none Weightmg {1 1 1}

Ixl {} parameter {0. 05} mlp.future2 {

135 ActFunction {

LnCosh {be tanh parameter {2} plogistic {

} parameter {0.5} parametricalEntropy {} parameter {le-06} 140 ptanh {parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F parameter {0.5} Tolerance {0 0 0 145 Weightmg {1 1 1

} ErrorFunc {mlp.future4 {be none l ^χ | {75} parameter {0.05} mlp.pastl {} ActFunction {LnCosh {sei tanh parameter {2} plogistic {} 80 parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5}}}

Norm {NoNorm} 85 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1}}} ErrorFunc {mlp.futurel {90 sei LnCosh ActFunction {Ixl {sei tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {} 95 parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5} 100}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} sei none Weighting {1 1 1} Ixl {105} parameter {0.05} mlp.past2 {} ActFunction {LnCosh {sei tanh parameter {2} plogistic {} 110 parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} 115 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1}}} ErrorFunc {mlp. present {120 be LnCosh ActFunction {Ixl {be tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {} 125 parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {parameter { 0.5} 130

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be LnCosh Weighting {1 1 1} Ixl {135} parameter {0.05} mlp.past3 { ^■ } ActFunction {LnCosh {sei tanh parameter {2} plogistic {} 140 parameter {0.5} parametricalEntropy { } parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} 145 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0} Weighting {1 1 1} ErrorFunc {75 Tolerance {0 0} be LnCosh Weightmg {1 1} Ixl {} parameter {0.05} mlp.past6 {} ActFunction {LnCosh {80 be tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter { le-06} ptanh {} 85 arameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1} 90}

} ErrorFunc {mlp.past4 {be LnCosh ActFunction {Ixl {be tanh parameter {0.05} plogistic {95} parameter {0.5} LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} 100 parameter {le-06 ) pid {} parameter {0.5}}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {105 Tolerance {0 0 0} be LnCosh Weightmg {1 1 1}

Ix] {} parameter 0.05} mlp.state65 {ActFunction {

LnCosh {110 be tanh parameter 2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

115 parameters {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1} 120}} Norm {NoNorm} mlp.past5 {} ActFunction {mlp.state54 {sei tanh ActFunction {plogistic {125 let tanh parameter {0.5} plogistic {parameter {0.5} ptanh {} parameter {0, .5} ptanh {

} 130 parameters {0.5} pid {} parameters {0, .5} pid {

} parameter {0.5}}

ErrorFunc {135} be LnCosh Norm {NoNorm}

Ixl {} parameter {0., 05 mlp.state43 {ActFunction {

LnCosh {140 let tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {

145 parameters {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} 75 parameters {0.5}

Norm {NoNorm}} Norm {NoNorm} mlp.state32 {} ActFunction {80 mlp.state23 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5 plogistic {} parameter {0.5} ptanh {85} parameter {0.5 ptanh {} parameter {0.5} pid {} parameter {0.5 pid {

90 parameters {0.5}}

Norm {NoNorm}}} Norm {NoNorm} mlp.state21 {} ActFunction {95 mlp.state34 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5 plogistic {} parameter {0.5} ptanh {100} parameter {0.5 ptanh {parameter {0.5} pid {} parameter 0.5 pid {

105 parameters {0.5}

Norm {NoNorm}} Norm {NoNorm} mlp. statelO {} ActFunction {110 mlp.state45 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5 ptanh {115} parameter {0.5) ptanh {} parameter {0.5 id {} parameter {0.5} pid {} 120 parameters {0.5}

Norm {NoNorm}} Norm {NoNorm} mlp. stateOl {} ActFunction {125 mlp.state56 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5} ptanh {130} parameter {0.5} ptanh {} parameter {0.5} pid {} parameter {0.5 } pid {

135 parameters {0.5}}

Norm {NoNorm}}} Norm {NoNorm} mlp.statel2 {} ActFunction {140 mlp.backδδ {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} parameter {0.5} ptanh {145} parameter {0.5} ptanh {} parameter {0.5} pid {} pid {75 parameter 0.5 pid {} parameter {0.5}}

Norm {NoNorm} 80 Norm {NoNorm} mlp.back54 {} ActFunction {} sei tanh Connectors {plogistic {mlp.bottleneck-> output_auto {parameter 0.5} 85 WeightWatcher {} Active {F} ptanh {MaxWeight {1} parameter 0.5 MinWeight {0}}} pid {90 LoadWeightsLocal {parameter 0.5 Filename {std}} SaveWeightsLocal {

Norm NoNorm Filename {std}

95} mlp.back43 {Alive {T} ActFunction {WtFreeze {F} sei tanh AllowPruning {F} plogistic {EtaModifier {1} parameter 0.5} 100 Penalty {NoPenalty}}} ptanh {mlp.bιas-> output_auto {parameter 0.5 WeightWatcher {} Active {F} pid {105 MaxWeight {1} parameter 0.5 MinWeight {0}} LoadWeightsLocal {

Norm {NoNorm Filename {std}} 110} mlp.back32 {SaveWeightsLocal {ActFunction {Filename {std} sei tanh} plogistic {Alive {T} parameter 0.5 115 WtFreeze {F}} AllowPruning {F} ptanh {EtaModifier {1} parameter ■ 5} Penalty {NoPenalty}}} pid {120 mlp. futureδ-> fιnal6 {parameter .5} LoadWeightsLocal {} Filename {std}}

Norm NoNorm SaveWeightsLocal {

} 125 Filename (std} mlp.back21 {} ActFunction {Alive {F} sei tanh WtFreeze {F} plogistic {AllowPrumng {F} parameter 0.5 130 EtaModifier {1}} Penalty {NoPenalty} ptanh {} parameter 0.5 mlp.bιas- > fιnalδ {} LoadWeightsLocal {pid {135 Filename {std} parameter 0.5}} SaveWeightsLocal {} Filename {std}

Norm {NoNorm}} 140 Alive {F} mlp. backlO {WtFreeze {F} ActFunction {AllowPruning {F} be tanh EtaModifier {1} plogistic {Penalty {NoPenalty} parameter 0.5 145}} mlp. future5-> fmal5 {ptanh {LoadWeightsLocal {parameter 0.5 Filename {std} 75

SaveWeightsLocal {mlp. future2-> fmal2 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 80 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> fmal5 {85 AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

}} SaveWeightsLocal {mlp. bιas-> fmal2 {

Filename {std} 90 LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 95} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp. future4-> fmal4 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 100 Penalty {NoPenalty}}} SaveWeightsLocal {mlp. futurel-> fmall {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 105} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 110 WtFreeze {F} mlp.bιas-> fmal4 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {115 mlp. bιas-> fmall {

Filename {std} LoadWeightsLocal {} Filename {std)

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 120 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp. future3-> fιnal3 {AllowPruning {F} LoadWeightsLocal {125 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp. ιnput_auto-> bottleneck

Filename {std} WeightWatcher {} 130 Active {F}

Alive {F} MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 135 Filename {std}}} mlp.bιas-> fιnal3 {SaveWeightsLocal {LoadWeightsLocal {Filename { hours }

Filename {std}}} 140 Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F} 145} AllowPruning {F} mlp.bιas-> bottleneck {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} Active {F} MaxWeight {1} 75 Filename {std}

MinWeight {0}}} SaveWeightsLocal {LoadWeightsLocal {Filename {std}

Filename {std}}} 80 Alive {F} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPrumng {F}

} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F} 85} AllowPruning {F} mlp.bιas-> future4 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty] Filename {std}}} mlp. state56-> futureδ {90 SaveWeightsLocal {WeightWatcher {Filename {std} Active {F}} MaxWeight {1} Alive {F} MinWeight {0} WtFreeze {F}

) 95 AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.state23-> future3 {Filename {std} 100 LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 105} Penalty {NoPenalty] Alive {F}

} WtFreeze {F} mlp.bιas-> future6 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} 110 Penalty {NoPenalty}

SaveWeightsLocal {mlp.bιas-> future3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 115} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 120 WtFreeze {F} mlp. state45-> uture5 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {125 mlp. statel2-> future2 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 130 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> future5 {AllowPruning {F} LoadWeightsLocal {135 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> future2 {

Filename {std} LoadWeightsLocal {

} 140 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 145 Alive {F}

} WtFreeze {F} mlp.state34-> future4 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Penalty {NoPenalty} 75 Alive {F}} WtFreeze {F} mlp.state01-> futurel {AllowPrumng {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

} 80}

SaveWeightsLocal {mlp.state21-> pastl {Filename {std} LoadWeightsLocal {

} Filename {std}

Alive {F}}

WtFreeze {F} 85 SaveWeightsLocal {

AllowPruning {F} Filename {std}

EtaModifier {1}}

Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> futurel {90 AllowPrumng {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.bιas-> pastl {Filename {std} 95 LoadWeightsLocal {

} Filename {std}

Alive {F}}

WtFreeze {F} SaveWeightsLocal {

AllowPruning {F} Filename {std}

EtaModifier {1} 100}

Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp. mputO-> present {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} 105 Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.backlO-> pastl {Filename {std} WeightWatcher {

} Active {F}

Alive {F} 110 MaxWeight {1}

WtFreeze {F} MinWeight {0}

AllowPruning {F}}

EtaModifier {1} LoadWeightsLocal {

Penalty {NoPenalty} Filename {std}} 115} mlp. statelO-> present {SaveWeightsLocal {

LoadWeightsLocal {Filename {std} Filename {std}}

} Alive {F}

SaveWeightsLocal {120 WtFreeze {F} Filename {std} AllowPruning {F}

} EtaModifier {1}

Alive {F} Penalty {NoPenalty}

WtFreeze {F}}

AllowPruning {F} 125 mlp. ιnput2-> past2 {

EtaModifier {1} LoadWeightsLocal {

Penalty {NoPenalty} Filename {std}}} mlp.bιas-> present {SaveWeightsLocal {

LoadWeightsLocal {130 Filename {std} Filename {std}}

} Alive {F}

SaveWeightsLocal {WtFreeze {F} Filename {std} AllowPruning {F}

} 135 EtaModifier {1}

Alive {F} Penalty {NoPenalty}

WtFreeze {F}}

AllowPruning {F} mlp.state32-> past2 {

EtaModifier {1} LoadWeightsLocal {

Penalty {NoPenalty} 140 Filename {std}}} mlp. mputl-> pastl {SaveWeightsLocal {

LoadWeightsLocal {Filename {std} Filename {std}}

} 145 Alive {F}

SaveWeightsLocal {WtFreeze {F} Filename {std} AllowPruning {F} EtaModifier {1} Penalty {NoPenalty} 75 Alive {F}} WtFreeze {F} mlp. bιas-> past2 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

} 80}

SaveWeightsLocal {mlp. mput ₄ -> past4 {Filename {std} LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {F} 85 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.back21-> past2 {90 AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 1 SaveWeightsLocal {mlp. state54-> past4 {

Filename {std} 95 LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} Eta ^" Modιfιer {1} 100} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp. ιnput3-> past3 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 105 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> past4 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 110} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} 115 WtFreeze {F} mlp. state43-> past3 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {120 mlp.back43-> past4 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.bιas-> past3 {AllowPruning {F} LoadWeightsLocal {130 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp. mput5-> past5 {

Filename {std} LoadWeightsLocal {} 135 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std) EtaModifier {1}} Penalty {NoPenalty} 140 Alive {F}} WtFreeze {F} mlp.back32-> past3 {AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 145} SaveWeightsLocal {mlp.state65-> past5 {

Filename {std} LoadWeightsLocal {

Filename {std} ιoo

75

SaveWeightsLocal {mlp. extern65-> state65 {

Filename {std} WeightWatcher {} Active {F}

Alive {F} MaxWeight {1} WtFreeze {F} 80 MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.bιas-> past5 {85 SaveWeightsLocal {LoadWeightsLocal { Filename {std}

Filename {std}}} Alive {F} SaveWeightsLocal {WtFreeze {F}

Filename {std} 90 AllowPruning {F}} EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F}} AllowPrumng {F} mlp.past6-> state65 {EtaModifier {1} 95 WeightWatcher {Penalty {NoPenalty} Active {F}} MaxWeight {1} mlp.back54-> past5 {MinWeight {0} LoadWeightsLocal {}

Filename {std} 100 LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {F} 105} WtFreeze {F} Alive {F} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} 110 Penalty {NoPenalty} mlp. ιnputδ-> pastδ {} LoadWeightsLocal {mlp.extern54-> state54 {

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {115}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} Alive {F} AllowPruning {F} 120 WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.bιas-> pastδ { } LoadWeightsLocal {125 mlp.past5-> state54 {

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {1

Filename {std} SaveWeightsLocal {} 130 Filename {std}

Alive {F}} WtFreeze {F} Alive {F} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} 135 EtaModifier {1}} Penalty {NoPenalty} mlp.back65-> pastδ { } LoadWeightsLocal {mlp. state65-> state54 {

Filename {std} WeightWatcher {} 140 Active {F} SaveWeightsLocal {MaxWeight {1}

Filename {std} MinWeight {0}}}

Alive {F} LoadWeightsLocal {WtFreeze {F} 145 Filename {std} AllowPruning {F}} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty} Filename {std} } 75 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 80 Alive {F}

} WtFreeze {F} mlp. extern43-> state43 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}} 85}

SaveWeightsLocal {mlp.extern21-> state21 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} 90 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp.past4-> state43 {95 AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.past2-> state21 {Filename {std} 100 LoadWeightsLocal {} Filename {std}

Alive {F} 1 WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 105} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp. state54-> state43 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} 110 Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.state32-> state21 {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F} 115} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}

} 120 WtFreeze {F} mlp. extern32-> state32 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty} 1}

SaveWeightsLocal {125 mlp.externlO-> statelO {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} 130 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp.past3-> state32 {AllowPruning {F} LoadWeightsLocal {135 EtaModifier {1} Filename {std} Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.pastl-> statelO {Filename {std} LoadWeightsLocal {} 140 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 145 Alive {F}

} WtFreeze {F} mlp.state43-> state32 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Penalty {NoPenalty} 75 Alive {F}} WtFreeze {F} mlp.εtate21-> statel0 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}} 80}

SaveWeightsLocal {mlp.state34-> state45 {Filename {εtd} LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {F} 85 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp.present-> state01 {90 AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.state45-> state56 {Filename {std} 95 LoadWeightsLocal {

} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 100} Penalty {NoPenalty} Alive {F}

} WtFreeze {F} mlp.statel0-> state01 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} 105 Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.past5-> back65 {

Filename {std} WeightWatcher {} Active {F}

Alive {F} 110 MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 115} mlp.state01-> statel2 {SaveWeightsLocal {LoadWeightsLocal { Filename {std}

Filename {std}}} Alive {F} SaveWeightsLocal {120 WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {F} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} 125 mlp.back54-> back65 {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} Active {F}} MaxWeight {1} mlp.statel2-> state23 {MinWeight {0} LoadWeightsLocal {130}

Filename {std} LoadWeightsLocal {} Filename {εtd} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} 135 Filename {std}

Alive {F}} WtFreeze {F} Alive {F} AllowPrumng {F} WtFreeze {F} EtaModifier {1} AllowPrumng {F} Penalty {NoPenalty} 140 EtaModifier {1}} Penalty {NoPenalty} mlp.state23-> state34 { } LoadWeightsLocal {mlp.past4-> back54 {

Filename {std} LoadWeightsLocal {} 145 Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {

Filename {std} } 75 Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 80 Alive {F}

} WtFreeze {F} mlp.back43-> back54 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty

} 85}

SaveWeightsLocal {mlp.backl0-> back21 {Filename {std} LoadWeightsLocal {

} Filename {εtd}

Alive {F}} WtFreeze {F} 90 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.past3-> back43 {95 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} SaveWeightsLocal {mlp.present-> backlO {

Filename {std} 100 LoadWeightsLocal {} Filename {std}

Alive {F}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 105} Penalty {NoPenalty} Alive {F}} WtFreeze {F} mlp.back32-> back43 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 110 Penalty {NoPenalty

SaveWeightsLocal {Filename {std} AnySave {

} file name {f.CCMenu.dat

Alive {F} 115}

WtFreeze {F} AnyLoad {

AllowPruning {F} file name {f.CCMenu.dat

EtaModifier {1}

Penalty {NoPenalty} 120 RecPar \ [mlp.past2-> back32 {decay_ c 1: 1}

LoadWeightsLocal {delta ^" t (: 0. .9 Filename {std} epsilon {: o .01

} max_ιter {50

SaveWeightsLocal {125 how \ [T} Filename {std} Reεet Errors {

Alive {F} TestRun {WtFreeze {F} Filename {Test} AllowPruning {F} 130 Part .Transformed {F} EtaModifier {1}} Penalty {NoPenalty Online {

} Filename {Onlme. dat [mlp.back21-> back32 {

LoadWeightsLocal {

Filename {std} 135

SaveWeightsLocal {Part 6: Filename {std}}

Alive {F} WtFreeze {F} BpNet {AllowPruning {F} 140 Globals EtaModifier {1} WtPenalty {Penalty {NoPenalty} be NoPenalty Weigend {mlp.pastl-> back21 {Lambda {0 LoadWeightsLocal {145 AutoAdapt {T} wO {1} 75

DeltaLambda {le-06} DynSchedule {ReducFac {0.9} MaxLength {4} Gamma {0.9} MimmumRuns {0} DesiredError {0} Training {F}

} 80 Validation {T}

WtDecay {Generalization {F}

Lambda {0.005}} AutoAdapt {F} DivSchedule {AdaptTime {10} Divergence {0.1 EpsOb] {0.001} 85 MinEpochs {5} Ob] Set {Training}} EpsilonFac {1}}

} PruneAlg {

ExtWtDecay {be FixPrune

Lambda {0.001} 90 FixPrune {AutoAdapt {F} Perc_0 {0. 1} AdaptTime {10} Perc L {0.1} EpsOb] {0.001} Perc_2 {0. 1} Ob] Set {Training} Perc 3 {0.1} EpsilonFac { 1} 95}

} EpsiPrune {

Finnoff {DeltaEps {05}

AutoAdapt {T} StartEps {05} Lambda {0} MaxEps {1 DeltaLambda {le-06} 100 ReuseEps {ReducFac {0.9} Gamma {0.9}} DesiredError {0} Tracer {

Active {F}

105 Set {Validation}

ErrorFunc {File {trace} let LnCosh} Ixl {Active {F} parameter {0.05} Randomize {0}} 110 PruningSet {Train. + Valιd. } LnCosh {Method {Ξ-Prunmg} parameter {2}}} StopControl {parametricalEntropy {EpochLimit {parameter {le-06} 115 Active {T}

MaxEpoch {10000}}

AnySave {MovmgExpAverage {file name {f.Globals.dat} Active {F}} 120 MaxLength {4} AnyLoad {Training {F} file name {f.Globals.dat} Validation {T}} Generalization {F}

ASCII {T} Decay {0.9}} 125}

LearnCtrl {CheckOb] ectιveFct {be Stochastic Active {F} Stochastic {MaxLength {4}

PatternSelection {Training {F} Let Permute 130 Validation {T} ExpRandom {Generalization {F}

Lambda {2}}} CheckDelta {Segmentation {Active {F}

OutputNode {-1} 135 Divergence {0.1} ExpectedCutOff {0.5} PercentageForGroupB {0.2}} EtaCtrl {} Mode {WtPruneCtrl {140 be EtaSchedule

PruneSchedule {EtaSchedule {be FixSchedule SwitchTime {10} FixSchedule {ReductFactor {0.95} Lιmιt_0 {10}} Lιmιt_l {10} 145 FuzzCtrl {Lιmιt_2 {10} MaxDeltaOb] {0.3} Lιmιt_3 {10b] Max} Max MaxEtaChange {0.02} MmEta {0. 001 75 MaxEta {0.1} Eta {0.05} Smoother {1} DerivEps {0}

LineSearch {

Active {F} 80 PatternSelection {

} εel sequential

LearnAlgo {ExpRandom {be VarioEta Lambda {2} VarioEta {}

MinCalls {50} 85 segmentation {

} OutputNode {-1 MomentumBackProp {ExpectedCutOff {0.5

Alpha {0.05} PercentageForGroupB {0.2}} Quickprop {90}

Decay {0.05} WtPruneCtrl {

MU {2} Tracer {} Active {F}} Set {Validation} AnySave {95 File {trace} fιle_name {f.Stochastic.dat}}

} Active {F} AnyLoad {Randomize {0} fιle_name {f.Stochastic.dat} PruningSet {Tram. + Valid. }

} 100 Method {S-Prumng}

BatchSize {15}} Eta {0.005} LearnAlgo {DerivEps {0} let Con] gradient

} VarioEta {

TrueBatch {105 MinCalls {200}

PatternSelection {} be Sequential MomentumBackProp {ExpRandom {Alpha {0.05} Lambda {2}}

1 110 Quickprop {

Segmentation {Decay {0.05}

OutputNode {-1} Mu {2} ExpectedCutOff {0.5}} PercentageForGroupB {0.2} Low-Memory-BFGS {

115 limit {2}

WtPruneCtrl {Tracer {AnySave {

Active {F} file name f. LmeSearch.dat Set {Validation} 120} File {trace} AnyLoad {} file name f.LineSearch.dat

Active {F} Randomize {0} EtaNull {1} PruningSet {Train. + Valιd. } 125 MaxSteps {10} Method {S-Pruning} LS_Precιsιon {0.5}} TrustRegion {T} EtaCtrl {DerivEps {0}

Active {F} BatchSize {2147483647}

130}

LearnAlgo {GeneticWeightΞelect {be VarioEta PatternSelection {VarioEta {be Sequential MinCalls {200 ExpRandom {

135 lambda {2}

MomentumBackProp {} Alpha {0.05} Segmentation {

} OutputNode {-1} Quickprop {ExpectedCutOff {0.5

Decay {0.05} 140 PercentageForGroupB {0.2

Mu {2}}}

LearnAlgo {

AnySave {be VarioEta file name {f.TrueBatch.dat 145 VarioEta {

MinCalls {200)

AnyLoad {} file name f.TrueBatch.dat MomentumBackProp { Alpha {0.05 75 εel id

} plogistic {parameter {0.5}

Ob] FctTracer {} Active {F} ptanh {

File {ob] Func} 80 parameters {0.5}

}

SearchControl {pid {ΞearchStrategy {parameter 0.5} be HillClimberControl HillClimberControl {85

% InιtιalAlιve {0.95} InputModification {InheritWeights {T} be None Beta {0.1} AdaptiveUniformNoise {

MutationType {DlstπbutedMac- NoiseEta {1} roMutation} 90 DampmgFactor {1}

MaxTrials {50}}} AdaptiveGaussNoise {PBILControl {NoiseEta {1}

% InιtιalAlιve {0.95} DampmgFactor {1} InheritWeights {T} 95}

Beta {0.1} FixedUmformNoise {Alpha {0.1} SetNoiseLevel {PopulationSize {40} NewNoiεeLevel {0}}}

PopulationControl {100} pCrossover {1} FixedGaussNoise {CrossoverType {SimpleCrosso- SetNoiseLevel {ver} NewNoiseLevel {0}

Ξcalmg {T}

ScalingFactor {2} 105

Sharing {T}}

SharmgFactor {0.05} SaveNoiseLevel {

PopulationSize {50} Filename {noise_level.dat} mιn.% ImtιalAlιve {0.01}} max.% ImtιalAlιve {0.1} 110 LoadNoiseLevel {

Filename {noise_level.dat}} pMutation {0} SaveManipulatorData {} Filename {inputMamp.dat} Ob] ectιveFunctιonWeιghts {115}

% Alιve {0.6} LoadMampulatorData {

E (TS) {0.2} Filename {inputMamp.dat}

Improvement (TS) {0}}

E (VS) {1} norm {NoNorm}

Improvement (VS) {0} 120}

(E (TS) -E (VS)) / max (E (TS), E (VS)) {0 mlp.mputO {ActFunction {

LipComplexity {0} id OptComplexity {2} plogistic {testVal (dead) -testVal (alive) {0} 125 parameters {0.5}

AnySave {ptanh {fιle_name {parameter {0.5} f.GeneticWeightSelect.dat}}

} 130 pid {

AnyLoad {parameter {0.5} f le name {} f.GeneticWeightSelect.dat}}

} InputModification {

Eta {0.05} 135 be None

DerivEps {0} AdaptiveUniformNoise {

BatchSize {5} NoiseEta {1}

DmmEpochsForFitnessTest {2] DampmgFactor {1}

SmaxEpochsForFitnessTest {3]}

SelectWeights {T} 140 AdaptiveGausεNoiεe {

SelectNodeε {T} NoiseEta {1} maxGrowthOfValError {0.005} DampmgFactor {1}} FixedUmformNoise {

CCMenu {145 SetNoiεeLevel {Clusters {NewNoiseLevel {0} mlp.ιnput_auto {} ActFunction { FixedGaussNoise {75 ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}} pid {parameter {0.5}

80}

SaveNoiseLevel {}

Filename {noise_level. dat} InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level. dat} 85 NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {mputManip. dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {90 DampmgFactor {1}

Filename {mputManip. dat}}} FixedUmformNoiεe {

Norm {NoNorm} SetNoiεeLevel {} NewNoiseLevel {0} mlp.inputl {95} ActFunction {} set id FixedGausεNoiεe {plogistic {SetNoiseLevel {parameter 0.5} NewNoiseLevel {0}} 100} ptanh {parameter 0.5}} SaveNoiseLevel {pid {Filename noise_level.dat} parameter 0.5} 105}

} LoadNoiseLevel {

} Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {110 Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp.dat}

AdaptiveGausεNoiεe {} NoiεeEta {1} 115 Norm {NoNorm} DampmgFactor {1}}} mlp. mput3 {

FixedU formNoiεe {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} 120 plogistic {} parameter {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0} 125}} pid {parameter {0.5}

SaveNoiseLevel {

Filename {noise_level.dat) 130 InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat] NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {135}

Filename {mputManip.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} 140 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0 mlp.mput2 {ActFunction {be ld 145 FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0. 5} NewNoiseLevel 75 parameters {0.5}

SaveNoiseLevel {

Filename {noise_level. dat} InputModification {} be None LoadNoiseLevel {80 AdaptiveUniformNoise {

Filename {noise_level. dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {mputManip. dat} AdaptiveGaussNoise {} 85 NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp. dat}}} FixedUmformNoise {Norm {NoNorm} SetNoiseLevel {

90 NewNoiseLevel {0} mlp. mput 4 {} ActFunction {} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter 0.5} 95 NewNoiseLevel {0}

} ptanh {} parameter 0.5

SaveNoiseLevel {pid {100 Filename {noise_level. dat} parameter {0.5}}} LoadNoiseLevel {} Filename {noise_level. dat}

InputModification {} be None 105 SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip. dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {inputMamp. dat}

AdaptiveGaussNoise {110} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp. mputδ {

FixedUmformNoise {ActFunction {SetNoiseLevel {115 be id

NewNoiseLevel {0} plogistic {parameter {0.5}}

FixedGaussNoise {ptanh {SetNoiseLevel {120 parameter {0.5}

NewNoiseLevel {0} pid {

} parameter {0.5}}

SaveNoiseLevel {125}

Filename {noise_level.dat} InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} 130 DampmgFactor {1} SaveManipulatorData {}

Filename {mputManip.dat} AdaptiveGaussNoiεe {} NoiεeEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {mputManip.dat} 135}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.ιnput5 {ActFunction {140 set id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}} ptanh {145} arameter {0.5}} SaveNoiseLevel {pid {Filename { noise level.dat 75 be None

LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level. dat] NoiseEta {1}

} DampmgFactor {1} SaveManipulatorData {}

Filename {mputManip .dat} 80 AdaptiveGaussNoise {

} NoiεeEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp. dat}}} FixedUmformNoise {

Norm {NoNorm} 85 SetNoiseLevel {} NewNoiseLevel {0] mlp.externδδ {ActFunction {} be id FixedGaussNoise {plogistic {90 SetNoiseLevel {parameter {0.5} NewNoiseLevel 0}} ptanh {} parameter {0.5}}} 95 SaveNoiseLevel {pid { Filename {noise_level.dat} parameter {0.5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {100} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip. dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} 105 Filename {putManip. dat}

AdaptiveGausεNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp.extern43 {

FixedUmformNoise {110 ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {parameter {0.5}}

FixedGaussNoiεe {115 ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}} pid {parameter {0.5}

120

SaveNoiseLevel {

Filename {noise_level.dat] InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat] 125 NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {130 DampmgFactor {1}

Filename {inputMamp.dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.extern54 {135} ActFunction {} be fixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}} 140} ptanh {} parameter {0.5}}} SaveNoiseLevel { pid {Filename {noise_level.dat} parameter {0. 5} 145} LoadNoiseLevel {

Filename {noise_level.dat}

InputModification} SaveManipulatorData {75

Filename {mputManip.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp. dat}}} 80 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.extern32 {} ActFunction {} let id 85 FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiεeLevel {0}} ptanh {} parameter {0.5} 90} SaveNoiseLevel {pid { Filename {noise_level.dat} parameter {0.5}} LoadNoiseLevel {

95 Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip.dat} NoiseEta {1}} DampmgFactor {1} 100 LoadMampulatorData {} Filename {mputManip.dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} 105 mlp.externlO {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiεeLevel {0 plogistic {parameter {0.5}

110}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0} pid {

115 parameters {0.5}

SaveNoiseLevel {

Filename {noise_level.dat} InputModification {1 be None LoadNoiseLevel {120 AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {mputManip.dat} AdaptiveGausεNoise {} 125 NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {putManip.dat}}} FixedU formNoise {

Norm {NoNorm} SetNoiseLevel {} 130 NewNoiseLevel {0} mlp.extern21 {} ActFunction {} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} 135 NewNoiεeLevel {0}} ptanh {parameter {0.5}} SaveNoiseLevel {pid {140 Filename {noise_level.dat} parameter {0.5}}} LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {} Let None 145 SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData { Filename {mputManip.dat} 75 parameters {0.5}}}

Norm {NoNorm}}} ErrorFunc {mlp.output_auto {sei LnCosh ActFunction {80 Ixl {sei id parameter {0.05} plogistic {} parameter {0.5} LnCosh {parameter {2} ptanh {85} parameter 0.5} parametricalEntropy {} parameter { le-06} pid {1 parameter 0.5}}} 90 norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0} be none Weightmg {1 1 1 1 1 1 1 1 1 1} Ixl {} parameter {0.05} 95 mlp.fmal4 {

} ActFunction {LnCosh {be id parameter {2} plogistic {} parameter {0.5} p ^' arametncalEntropy I 100} parameter {le-06] ptanh {parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} 105 parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weight g {1 1 1 1 1 1 1 1 1 1}} ErrorFunc {mlp. inalδ {be LnCosh ActFunction {110 Ixl {be id parameter {0.05} plogistic {} parameter {0.5} LnCosh {} parameter {2} ptanh {115} parameter {0.5} parametricalEntropy]} parameter {le-06] pid {parameter { 0.5}} 120 Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0 0 0 0 0 0 0 0 let LnCosh Weighting {1 1 1 1 1 1 1 1 1 1

Ixl {parameter {0.05} 125 mlp.fmal3 {} ActFunction {LnCosh {εel id parameter {2} plogistic {} parameter {0.5 parametricalEntropy {130} parameter {le-06} ptanh {} parameter {0.5}

Norm {NoNorm} pid {ToleranceFlag {F} 135 parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 1}} ErrorFunc {mlp.fιnal5 {be LnCosh ActFunction {140 Ixl {be id parameter {0.05} plogistic {} parameter {0.5} LnCosh {} parameter {2} ptanh {145} parameter {0.5} parametricalEntropy} parameter {le-06 pid { } 75

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weight g {1 1 1 1 1 1 1 1 1 1}} 80 ErrorFunc {mlp.fιnal2 {se none ActFunction {Ixl {sei id parameter {0.05} plogistic {} parameter {0.5} 85 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {90} parameter {0.5} }

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} let LnCosh 95 Weightmg {1 1 1} Ixl {} parameter {0.05} mlp. futureδ {} ActFunction {LnCosh {sei tanh parameter {2} 100 plogistic {} parameter 0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5} 105}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5} Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 1} 110 ErrorFunc {mlp. finall {be none ActFunction {Ixl {be id parameter {0.05} plogistic {} parameter {0.5} 115 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {120} parameter {0.5}}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be LnCosh 125 Weightmg {1 1 1} Ixl {} parameter {0.05} mlp.future5 {} ActFunction {LnCosh {sei tanh parameter {2} 130 plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} parameter {0.5}} 135}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0 Weighting {1 1 1 1 1 1 1 1 1 1} 140 ErrorFunc {mlp. bottleneck {sei none ActFunction {Ixl {εel tanh parameter {0.05} plogistic {} parameter {0.5} 145 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy \ parameter {le-06} 75 ptanh {

} parameter 0.5}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5 Tolerance {0 0 0} 80} Weighting {1 1 1}}

} ErrorFunc {mlp.future4 {sei none ActFunction {Ixl {sei tanh 85 parameter {0.05} plogistic {} parameter {0.5} LnCosh {} parameter 2} ptanh {parameter {0.5} 90 parametricalEntropy {} parameter {le-06} pid {} parameter {0.5}}} Norm {NoNorm}} 95 ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be none Weightmg {1 1 1} Ixl {} parameter {0.05} mlp.futurel {} ^" 100 ActFunction {LnCosh {be tanh parameter {2} plogistic {

} parameter {0.5} parametricalEntropy {} parameter {le-06] 105 ptanh {

} parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5} Tolerance {0 0 0} 110} Weightmg {1 1 1}}

} ErrorFunc {mlp.future3 {sei none ActFunction {Ixl {sei tanh 115 parameter 0.05} plogistic {} parameter {0.5} LnCosh {} parameter {2} ptanh {} parameter {0.5} 120 parametricalEntropy {} parameter {le-06} pid {parameter {0.5}} Norm {NoNorm}} 125 ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be none Weighting {1 1 1} Ixl {} parameter {0.05} mlp. present {} 130 ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} 135 ptanh {} parameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0} 140} Weighting {1 1 1}}

} ErrorFunc {mlp.future2 {be LnCosh ActFunction {Ixl {be tanh 145 parameters {0.05] plogistic {parameter {0.5} LnCosh {parameter 2} 75 parameters {0.5} parametricalEntropy} parameters {le-06 ptanh {} parameters {0.5}}}

Norm {NoNorm} 80 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weightmg {1 1 1}}} ErrorFunc {mlp.paεtl {85 sei LnCosh ActFunction {Ixl {sei tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {} 90 parameter {2} ptanh {} parameter {0.5} parametricalEntropy {parameter {le-06} pid {} parameter 0.5 95}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be LnCosh Weightmg {1 1 1} Ixl {100} parameter 0.05 mlp.past4 {ActFunction {

LnCosh {be tanh parameter {2} plogistic {} 105 parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} 110 pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weightmg {1 1 1}}} ErrorFunc {mlp.past2 {115 let LnCosh ActFunction {Ixl {let tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {} 120 parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5} 125}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} let LnCosh Weightmg {1 1 1} Ixl {130 parameter 0.05 mlp.pastδ {ActFunction {

LnCosh {be tanh parameter 2} plogistic {

135 parameters {0. ■ 5} parametricalEntropy {ϊ parameter {le-06} ptanh {parameter {0, .5}

Norm {NoNorm} 140 pid {ToleranceFlag {F parameter {0. .5}

Tolerance {0 0} Weightmg {1 1 1 l

} ErrorFunc {mlp.past3 {145 εel LnCosh ActFunction {Ixl {sei tanh parameter {0. .05} plogistic { LnCosh {75 be tanh parameter {2} plogistic {} parameter 0.5} parametricalEntropy {} parameter {le-06} ptanh {

80 parameters 0.5}}

Norm {NoNorm} pid {ToleranceFlag {F parameter 0.5} Tolerance {0 0 0} Weight g {1 1 1 85}

Norm {NoNorm mlp.past6 {} ActFunction {mlp.state32 {εel tanh ActFunction {plogistic {90 εel tanh parameter 0.5} plogistic {parameter 0.5 ptanh {} parameter 0.5} ptanh {

95 parameter 0.5 pid {} parameter {0.5} Pid {} parameter 0.5}}}

ErrorFunc {100} be LnCosh norm {NoNorm I {1 parameter {0.05} mlp.state21 {} ActFunction {LnCosh {105 be tanh parameter {2} plogistic {} parameter 0.5 parametricalEntropy {} parameter {le-06} ptanh {} 110 parameter 0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5 Tolerance {0 0 0}} Weightmg {1 1 1} 115}} Norm {NoNorm mlp.state65 {} ActFunction {mlp. statelO {sei tanh ActFunction {plogistic {120 εel tanh parameter {0.5} plogiεtic {} parameter 0.5 ptanh {} parameter {0.5} ptanh {} 125 parameter 0.5 pid {} parameter {0.5} pid {} parameter 0.5}

Norm {NoNorm} 130} Norm NoNorm mlp.state54 {} ActFunction {mlp.εtateOl {sei tanh ActFunction {plogistic {135 sei tanh parameter {0.5} plogistic {} parameter 0.5 ptanh {} parameter {0.5} ptanh {} 140 parameter 0.5 pid {} parameter {0.5} pid {parameter 0.5

Norm {NoNorm} 145} Norm NoNorm mlp.state43 {}

ActFunction {mlp.statel2 { ActFunction {75 mlp.back65 {sei tanh ActFunction {plogistic {sei tanh parameter 0. 5} plogiεtic {} parameter 0.5} ptanh {80} parameter 0. 5} ptanh {) parameter 0.5) pid {} parameter 0. 5 pid {

85 parameters 0.5

Norm {NoNorm} Norm {NoNorm mlp.state23 {} ActFunction {90 mlp.back54 {sei tanh ActFunction {plogistic {sei tanh parameter 0. 5 plogistic {} parameter 0.5 ptanh {95} parameter 0. 5 ptanh {} parameter 0.5} pid {} parameter 0.5 pid {F 100 parameter 0.5}}

Norm {NoNorm} Norm {NoNorm mlp.εtate34 {} ActFunction {105 mlp.back43 {sei tanh ActFunction {plogistic {sei tanh parameter 0. 5 plogistic {} parameter 0.5 ptanh {110} parameter 0. 5 ptanh {} parameter 0.5} pid {} parameter 0.5 pid {} 115 parameter 0.5}

Norm {NoNorm}} Norm NoNorm mlp.state45 {ActFunction {120 mlp.back32 {εel tanh ActFunction {plogiεtic {sei tanh parameter 0. 5} plogistic {} parameter 0.5} ptanh {125} parameter 0. 5} ptanh {} parameter 0.5 pid {} parameter 0. 5} pid {

130 parameters 0.5}

Norm {NoNorm} Norm {NoNorm mlp.stateδδ {} ActFunction {135 mlp.back21 {sei tanh ActFunction {plogistic {sei tanh parameter 0. 5 plogistic {} parameter 0.5} ptanh {140} parameter 0. 5 ptanh {} parameter 0.5 } pid {} parameter 0. 5 pid {

145 parameters 0.5}

Norm {NoNorm

Norm {NoNorm 75 Alive {T} mlp.backlO {WtFreeze {T} ActFunction {AllowPruning {F} be tanh EtaModifier {1} plogistic {Penalty {NoPenalty} parameter {0.5} 80}} mlp. future5-> fmal5 {ptanh {LoadWeightsLocal {parameter {0.5} Filename {std}}} pid {85 SaveWeightsLocal {parameter {0.5} Filename {std}

} Alive {T} Norm {NoNorm WtFreeze {T}

90 AllowPruning {F} EtaModifier {1}

Connectors {Penalty {NoPenalty} mlp. bottleneck-> output_auto} WeightWatcher {mlp.bιas-> fmal5 {

Active {F} 95 LoadWeightsLocal {MaxWeight {1} Filename {std} MinWeight {0}}} SaveWeightsLocal {LoadWeightsLocal {Filename {std}

Filename {std} 100}} Alive {T}

SaveWeightsLocal {WtFreeze {T} Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {F} 105 Penalty {NoPenalty} WtFreeze {T}} AllowPruning {F} mlp. future4-> fmal4 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}

} 110} mlp.bιas-> output_auto {SaveWeightsLocal {WeightWatcher {Filename {std} Active {F}} MaxWeight {1} Alive {T}

MinWeight {0} 115 WtFreeze {T}} AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}}

SaveWeightsLocal {120 mlp.bιas-> fmal4 {Filename {std} LoadWeightsLocal} Filename {std

Alive {F}} WtFreeze {T} SaveWeightsLocal

AllowPruning {F} 125 Filename {std EtaModifier {1}} Penalty {NoPenalty} Alive {T} WtFreeze {T} mlp. futureδ-> fmalδ {AllowPruning {F LoadWeightsLocal {130 EtaModifier {1 Filename {std} Penalty {NoPenalty}}

SaveWeightsLocal {mlp. future3-> fmal3 {Filename {std} LoadWeightsLocal

} 135 Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}}

Penalty {NoPenalty} 140 Alive {T}} WtFreeze {T} mlp.bιas-> fmalδ {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

} 145}

SaveWeightsLocal {mlp.bιas-> fmal3 {Filename {std} LoadWeightsLocal {Filename {std} 75 Alive

SaveWeightsLocal {WtFreeze {T}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {T} 80} AllowPruning {F} mlp.bιas-> bottleneck {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} Active {F}} MaxWeight {1} mlp.future2-> fmal2 {85 MinWeight {0} LoadWeightsLocal {}

Filename {std} LoadWeightsLocal {

} Filename {std} SaveWeightsLocal {}

Filename {std} 90 SaveWeightεLocal {} Filename {std}

Alive {T}} WtFreeze {T} Alive {F} AllowPruning {F} WtFreeze {T} EtaModifier {1} 95 AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.bιas-> fιnal2 { } LoadWeightsLocal {mlp.state56-> futureδ {

Filename {εtd} 100 WeightWatcher {} Active {F} SaveWeightsLocal {MaxWeight {1}

Filename {std} MinWeight {0}}}

Alive {T} 105 LoadWeightsLocal {WtFreeze {T} Filename {std} AllowPruning {F}} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty} Filename {std}

} 110} mlp. futurel-> fmall {Alive {T} LoadWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1} SaveWeightsLocal {115 Penalty {NoPenalty}

Filename {εtd}}} mlp.bιas-> future6 {

Alive {T} LoadWeightεLocal {WtFreeze {T} Filename {std} AllowPruning {F} 120} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp.bιas-> fmall {Alive {T} LoadWeightsLocal {125 WtFreeze {T}

Filename {std} AllowPrumng {F}} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty}

Filename {std}}} 130 mlp.state45-> future5 {

Alive {T} LoadWeightsLocal {WtFreeze {T} Filename {std} AllowPrumng {F}} EtaModifier {1} SaveWeightsLocal {Penalty {NoPenalty} 135 Filename {std}}} mlp mput_auto-> bottleneck {Alive {T} WeightWatcher {WtFreeze { F}

Active {F} AllowPruning {F}

MaxWeight {1} 140 EtaModifier {1}

MinWeight {0} Penalty {NoPenalty}}} LoadWeightsLocal {mlp.bιas-> future5 {

Filename {std} LoadWeightεLocal {} 145 Filename {std} SaveWeightsLocal {}

Filename {εtd} SaveWeightsLocal {

Filename {εtd} } 75 Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 80 Alive {T}} WtFreeze {T} mlp.state34-> future4 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 85} SaveWeightsLocal {mlp.state01-> futurel {

Filename {std} LoadWeightsLocal {

} Filename {εtd}

Alive {T}} WtFreeze {F} 90 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.bιas-> future4 {95 AllowPruning {F} LoadWeightεLocal {EtaModifier {1}

Filename {εtd} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> futurel {

Filename {std} 100 LoadWeightεLocal {} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 105 1 Penalty {NoPenalty} Alive {T}} WtFreeze {T} mlp. state23-> future3 {AllowPruning {F} LoadWeightεLocal {EtaModifier {1}

Filename {εtd} 110 Penalty {NoPenalty}

SaveWeightsLocal {mlp.ιnputO-> present {Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 115} WtFreeze {F} SaveWeightsLocal (AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty] Alive {T}

} 120 WtFreeze {T} mlp.bιas-> future3 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty

}} SaveWeightsLocal {125 mlp.statelO-> present {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPruning {F} 130 Filename {std} EtaModifier {1}} Penalty {NoPenalty] Alive {T}} WtFreeze {F} mlp.statel2-> future2 {AllowPruning {F} LoadWeightsLocal {135 EtaModifier {1}

Filename {std} Penalty {NoPenalty

SaveWeightsLocal {mlp.bιas-> present {Filename {std} LoadWeightsLocal | } 140 Filename {std]

Alive {T}} WtFreeze {F} SaveWeightsLocal | AllowPruning {F} Filename {std] EtaModifier {1}} Penalty {NoPenalty} 145 Alive {T}

} WtFreeze {T} mlp.bιas-> future2 {AllowPruning {F] LoadWeightsLocal {EtaModifier {1} Penalty {NoPenalty} 75 Filename {std}}} mlp. mputl-> pastl {SaveWeightsLocal {LoadWeightsLocal {Filename {εtd}

Filename {std}}} 80 Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {T} 85} AllowPruning {F} mlp.bιaε-> paεt2 {EtaModifier {1} LoadWeightεLocal {Penalty {NoPenalty} Filename {std}}} mlp.state21-> paεtl {90 SaveWeightεLocal {LoadWeightaLocal {Filename {std}

Filename {εtd}}} Alive {T} SaveWeightsLocal {WtFreeze {T}

Filename {std} 95 AllowPruning {F}

} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} mlp.back21-> past2 {EtaModifier {1} 100 LoadWeightsLocal {Penalty {NoPenalty} Filename {εtd}}} mlp.bιas-> pastl {SaveWeightsLocal { LoadWeightsLocal {Filename {std}

Filename {std} 105}} Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} 110 Penalty {NoPenalty} WtFreeze {T}} AllowPruning {F} mlp.ιnput3-> past3 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 115} mlp.backlO-> pastl {SaveWeightsLocal {WeightWatcher {Filename {std}

Active {F})

MaxWeight {1} Alive {T}

MinWeight {0} 120 WtFreeze {T}} AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {125 mlp.state43-> past3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 130 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.mput2-> past2 {AllowPruning {F} LoadWeightsLocal {135 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} ΞaveWeightεLocal {mlp.bιas-> past3 {

Filename {std} LoadWeightsLocal {} 140 Filename {std}

Alive {T}} WtFreeze {T} SaveWeightεLocal {AllowPruning {F} Filename {εtd} EtaModifier {1}} Penalty {NoPenalty} 145 Alive {T}} WtFreeze {T} mlp.state32-> past2 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Penalty {NoPenalty} 75 Alive {T}} WtFreeze {T} mlp.back32-> past3 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

} 80}

SaveWeightsLocal {mlp. state65-> past5 {Filename {std} LoadWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {F} 85 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.ιnput4-> past4 {90 AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.bιas-> past5 {Filename {std} 95 LoadWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPruning {F} Filename {std} Eta ^" Modιfιer {1} 100} Penalty {NoPenalty} Alive {T}

} WtFreeze {T} mlp.state54-> paεt4 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1}

Filename {std} 105 Penalty {NoPenalty}

}} SaveWeightsLocal {mlp.back54-> past5 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 110} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 115 WtFreeze {F} mlp.bιas-> past4 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {120 mlp. mput6-> pastδ {

Filename {std} LoadWeightsLocal {} Filename {εtd)

Alive {T}} WtFreeze {T} SaveWeightsLocal {AllowPruning {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {T} mlp.back43-> past4 {AllowPruning {F} LoadWeightsLocal {130 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> pastδ {

Filename {std} LoadWeightsLocal {} 135 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 140 Alive {T}} WtFreeze {T} mlp.mput5-> past5 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 145} SaveWeightsLocal {mlp.back65-> past6 {

Filename {std} LoadWeightsLocal {Filename {std} 75 Active {F}

SaveWeightsLocal {MaxWeight {1} Filename {εtd} MinWeight {0}}}

Alive {T} LoadWeightεLocal {WtFreeze {F} 80 Filename {std} AllowPruning {F}} EtaModifier {1} ΞaveWeightεLocal {Penalty {NoPenalty} Filename {std}}} mlp. extern65-> state65 {85 Alive {T} WeightWatcher {WtFreeze {F} Active {F} AllowPrumng {F} MaxWeight {1} EtaModifier {1} MinWeight {0} Penalty {NoPenalty}

} 90} LoadWeightsLocal {mlp. extern43-> state43 {

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} 95 SaveWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} 100 AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}

} Penalty {NoPenalty} mlp.past6-> state65 {} WeightWatcher {mlp.past4-> state43 {Active {F} 105 LoadWeightsLocal {MaxWeight {1} Filename {std} MinWeight {0}}

} SaveWeightsLocal {

LoadWeightsLocal {Filename {std}

Filename {std} 110}} Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} 115 Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} mlp. state54-> state43 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 120} mlp. extern54-> state54 {SaveWeightsLocal {LoadWeightsLocal {Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {125 WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} 130 mlp. extern32-> state32 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {εtd}}} mlp.past5-> state54 {SaveWeightsLocal {LoadWeightsLocal {135 Filename {std}

Filename {std}}} Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPru ng {F}} 140 EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} mlp.past3-> state32 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 145 Filename {std}

}} mlp.state65-> state54 {SaveWeightsLocal {WeightWatcher {Filename {std} } 75 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 80 Alive {T}

} WtFreeze {F} mlp.εtate43-> εtate32 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

} 85}

SaveWeightsLocal {mlp.state21-> statel0 {Filename {std} LoadWeightsLocal {

Filename {std}

Alive {T}} WtFreeze {F} 90 SaveWeightsLocal {AllowPruning {F Filename {std} EtaModifier {1} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp. extern21-> state 21 {95 AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Filename {std} Penalty {NoPenalty}

}}

SaveWeightsLocal mlp.present-> state01 {Filename {std 100 LoadWeightεLocal {

Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {εtd} EtaModifier {1} 105} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.past2-> εtate21 {AllowPruning {F}

LoadWeightaLocal {EtaModifier {1}

Filename {std} 110 Penalty {NoPenalty}}}

SaveWeightsLocal {mlp.εtatel0-> state01 {

Filename {εtd} LoadWeightsLocal {

} Filename {std}

Alive {T} 115} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 120 WtFreeze {F} mlp.state32-> state21 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {εtd} Penalty {NoPenalty}}} SaveWeightsLocal {125 mlp.state01-> statel2 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 130 Filename {std} EtaModifier {1} 1 Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.externl0-> statelO {AllowPruning {F} LoadWeightsLocal {135 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.statel2-> state23 {

Filename {std} LoadWeightsLocal {} 140 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 145 Alive {T}

} WtFreeze {F} mlp.pastl-> statelO {AllowPruning {F} LoadWeightsLocal {EtaModifier {1} Penalty {NoPenalty} 75 EtaModifier {1}} Penalty {NoPenalty} mlp.state23-> state34 {} LoadWeightsLocal {mlp.paεt4-> back54 {

Filename {std} LoadWeightsLocal {} 80 Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 85 Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.state34-> state45 { 90} LoadWeightsLocal {mlp.back43-> back54 {

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} 95 SaveWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} 100 AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}

} Penalty {NoPenalty} mlp. state45-> state56 {}

LoadWeightsLocal {mlp.past3-> back43 {Filename {std} 105 LoadWeightsLocal {

} Filename {std}

SaveWeightsLocal {} Filename {std} SaveWeightsLocal {

} Filename {std}

Alive {T} 110}

WtFreeze {F} Alive {T}

AllowPruning {F} WtFreeze {F}

EtaModifier {1} AllowPruning {F}

Penalty {NoPenalty} EtaModifier {1}} 115 Penalty {NoPenalty} mlp.past5-> back65 {}

WeightWatcher {mlp.back32-> back43 {Active {F} LoadWeightsLocal {MaxWeight {1} Filename {std} MinWeight {0} 120}

} SaveWeightsLocal {

LoadWeightsLocal {Filename {std} Filename {std}}

} Alive {T}

SaveWeightsLocal {125 WtFreeze {F} Filename {std} AllowPruning {F}

} EtaModifier {1}

Alive {T} Penalty {NoPenalty}

WtFreeze {F}}

AllowPruning {F} 130 mlp.past2-> back32 {

EtaModifier {1} LoadWeightsLocal {

Penalty {NoPenalty} Filename {std}}} mlp.back54-> back65 {SaveWeightsLocal {

WeightWatcher {135 Filename {εtd} Active {F}} MaxWeight {1} Alive {T} MinWeight {0} WtFreeze {F}

} AllowPruning {F}

LoadWeightsLocal {140 EtaModifier {1} Filename {std} Penalty {NoPenalty}

}}

SaveWeightsLocal {mlp.back21-> back32 {Filename {std} LoadWeightεLocal {

} 145 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} Alive {T} Part 7: WtFreeze {F} AllowPruning {F} EtaModifier {1} Penalty {NoPenalty} 75 BpNet {

} Globais {mlp.pastl-> back21 {WtPenalty {LoadWeightsLocal {be NoPenalty Filename {std} Weigend {} 80 Lambda {0}

SaveWeightsLocal {AutoAdapt {T} Filename {std} wO {1}

} DeltaLambda. {1 .e-06}

Alive {T} ReducFac {0.9} WtFreeze {F} 85 Gamma {OS ¹ } AllowPruning {F} DeεiredError {0} EtaModifier {1}} Penalty {NoPenalty} WtDecay {} Lambda {0. 005} mlp.backl0-> back21 {90 AutoAdapt {F} LoadWeightsLocal {AdaptTime {10}

Filename {std} EpsOb] {0. 001}} Ob] Set {Training} SaveWeightsLocal {EpsilonFac {1}

Filename {std} 95}} ExtWtDecay {

Alive {T} Lambda {0. 001} WtFreeze {F} AutoAdapt {F} AllowPruning {F} AdaptTime {10} EtaModifier {1} 100 EpsOb] {0. 001} Penalty {NoPenalty} Ob] Set {Traim .ng} } EpsilonFac {1} mlp.present-> backlO {LoadWeightsLocal {Fmnoff {

Filename {std} 105 AutoAdapt {T}} Lambda {0} SaveWeightsLocal {DeltaLambda {le-06

Filename {std} ReducFac {0.9}} Gamma {0.9}

Alive {T} 110 DesiredError {0} WtFreeze {F}} AllowPrumng {F}) EtaModifier {1} ErrorFunc {Penalty {NoPenalty} εel LnCosh

115 Ixl {parameter {0.05}

AnySave {} fιle_name f.CCMenu.dat LnCosh {

} parameter {2}

AnyLoad {120} file name f.CCMenu.dat parametricalEntropy {parameter {le-06}

} RecPar {decay_c {1} 125 AnySave {delta_t {0.1} file name f.Globals.dat epsilon {0.01}} max_ιter {1} AnyLoad {εhow {F} fιle_name f.Globals.dat

Reεet_Errors {F} 130}} ASCII {T} TestRun {

Filename {Test} LearnCtrl {

Part.Transformed {F} Let Stochastic} 135 Stochastic {Online {PatternSelection {

Filename {Online.dat} be Permute ExpRandom {

Lambda {2}

140} segmentation {

OutputNode {-1} ExpectedCutOff {0.5 PercentageForGroupB {0.2 75

EtaCtrl {fashion {

WtPruneCtrl {be EtaSchedule

PruneSchedule {EtaSchedule {be FixSchedule 80 SwitchTime {10}

FixSchedule {ReductFactor {0.95

Lιmιt_0 {10}}

Lιmιt_l {10} FuzzCtrl {

Limit 2 {10} MaxDeltaOb] {0.3}

Lιmιt_3 {10} 85 MaxDelta 20b] {0. .3}

RepeatLast {T MaxEtaChange {0, .02}

MmEta {0.001}

DynSchedule {MaxEta {0.1}

MaxLength {. 1} Smoother {1}

MimmumRuns (: o 90

Training

Validation {T Active {F}

Generalization F}} i LearnAlgo {DivSchedule {95 be VarioEta

Divergence {0. 1} VarioEta {

MinEpochs {5} MinCalls {50}}}

} MomentumBackProp {PruneAlg {100 Alpha {0.05} be FixPrune}

FixPrune {Quickprop {

Perc 0 {0.1} Decay {0.05}

Perc 1 {0.1} Mu {2}

Perc_2 {0.1} 105

Perc_3 {0.1}

AnySave {

EpsiPrune {file name {f.Stochastic.dat}

DeltaEps {0.05}

StartEps {0.05 110 AnyLoad {

MaxEps {1} file name {f.Stochastic.dat}

ReuseEps {F}}} BatchSize {15}

) Eta {0.002}

Tracer {115 DerivEps {0}

Active {F}}

Set {Validation TrueBatch {

File {trace} PatternSelection {

) be sequential

Active {F} 120 ExpRandom {Randomize {0} Lambda {2} PruningSet {Tram. + Valid. } Method {S-Prunmg} Segmentation {} OutputNode {-1}

StopControl {125 ExpectedCutOff {0.5} EpochLimit {PercentageForGroupB {0.2}

Active {T}

MaxEpoch {10000}} WtPruneCtrl {Mov gExpAverage {130 Tracer {

Active {F} Active {F}

MaxLength {4} Set {Validation}

Training {F} File {trace}

Validation {T}}

Generalization {F} 135 Active {F}

Decay {0.9} Randomize {0}} PruningSet {Train. + Valιd. CheckOb] ectiveFct {Method {S-Prunmg}

Active {F}}

MaxLength {4} 140 EtaCtrl {

Training {F} Active {F}

Validation {T}}

Generalization {F} LearnAlgo {} be VarioEta CheckDelta {145 VarioEta {

Active {F} MinCalls {200}

Divergence {0.1}} MomentumBackProp { Alpha {0. 05} 75 Segmentation {} OutputNode {-1} Quickprop {ExpectedCutOff {0.5}

Decay {0.05} PercentageForGroupB {0.2}

Mu {2}}

80}

LearnAlgo {

AnySave {be VarioEta file name f. TrueBatch .dat VarioEta {

MinCalls {200}

AnyLoad {85} file name {f. TrueBatch .dat ^' . MomentumBackProp {

} Alpha {0.05}

Eta {0. 05}} DerivEps {0}}} 90 Ob] FctTracer {L eSearch {Active {F}

PatternSelection {File {ob] Func} Let Sequential} ExpRandom {SearchControl {Lambda {2} 95 ΞearchΞtrategy {

} be HillClimberControl

Segmentation {HillClimberControl {

OutputNode {-1}% InιtιalAlιve {0.95} ExpectedCutOff {0.5} InheritWeights {T} PercentageForGroupB {0.2 ^' 100 Beta {0.1}

} MutationType {DistributedMac-} roMutation} WtPruneCtrl {MaxTrials {50}

Tracer

Active {F} 105 PBILControl {

Set {Validation}% ImtιalAlιve {0.95}

File {trace} InheritWeights {T}

} Beta {0.1}

Active {F} Alpha {0.1}

Randomize {0} 110 PopulationSize {40}

PruningSet {Tram. + Valid. }}

Method {S-Prumng} PopulationControl {

} pCrosεover {1}

LearnAlgo {CroεsoverType {SimpleCrosεo- sei Con] Gradient 115 ver} VarioEta {Scalmg {T}

MinCalls {200} ScalingFactor {2}} Sharing {T}

MomentumBackProp {SharingFactor {0.05} Alpha {0.05} 120 PopulationSize {50}

} min. % ImtιalAlιve {0.01} Quickprop {max. % ImtιalAlιve {0.1}

Decay {0.05}

Mu {2}} 125 pMutation {0} low memory BFGS {}

Limit 1 2} Whether] ectiveFunctionWeights {

% Alιve {0.6}

E (TS) {0.2}

AnySave {130 Improvement (TS) {0} file name {f.LineSearch.dat} E (VS) {1}} Improvement (VS) {0} AnyLoad {(E (TS) -E (VS)) / max ( E (TS), E (VS)) {fιle_name {f.LineSearch.dat}}} 135 LipComplexity {0}

EtaNull {1} OptComplexity {2} MaxSteps {10} testVal (dead) -teεtVal (alive) {0 LS_Precιsιon {0.5}} TruεtRegion {T} AnySave {DeπvEpε {0} 140 fιle_name {BatchSize {2147483647e f.Gelectetic }}}

GeneticWeightSelect {AnyLoad {PatternSelection {file name {be Sequential 145 f.GeneticWeightSelect.dat} ExpRandom {} Lambda {2} Eta {0.05} DerivEps {0} BatchSize {5} 75 NoiseEta {1} KminEpochεForFitnessTest {{22} DampmgFactor {1} # maxEpochsForFιtnesεTest {3} SelectWeightε {T} AdaptiveGaussNoise {SelectNodeε {T} NoiseEta {1} maxGrowthOfVorE}}

CCMenu {SetNoiseLevel {Cluεters {NewNoiseLevel {0} mlp. mput_auto {85} ActFunction {} be ld FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}

} 90 ptanh {} parameter 0.5}

SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5} 95}

LoadNoiseLevel {

Filename {noise_level. dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {100 Filename {mputManip. dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {mputManip. dat}

AdaptiveGaussNoiεe {} NoiseEta {1} 105 Norm {NoNorm} DampmgFactor {1}}} mlp.inputl {

FixedU formNoise {ActFunction {SetNoiseLevel {be ld

NewNoiseLevel {0] 110 plogistic {parameter {0.5}}

FixedGaussNoise {ptanh {

SetNoiseLevel {parameter {0.5}

NewNoiseLevel 115} pid {parameter {0.5}}

SaveNoiseLevel {}

Filename {noise_level.dat} 120 InputModification {} be None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {125}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {inputMamp.dat}}} 130 FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} lp.mputO {} ActFunction {} let id 135 FixedGaussNoise {plogiεtic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}} ptanh {parameter {0.5} 140} SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5}} LoadNoiseLevel {

145 Filename {noise_level.dat}

InputModification {} εel None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip.dat} 75 NoiseEta {1}

LoadMampulatorData {DampmgFactor {1}

Filename {mputManip. dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} 80 NewNoiseLevel {0} mlp.mput2 {ActFunction {be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} 85 NewNoiseLevel {0}}} ptanh {} parameter {0.5}} SaveNoiseLevel {pid {90 Filename {noise__level .dat] parameter {0.5}} 1 LoadNoiseLevel {} Filename {noise_level. dat]

InputModification {} be None 95 SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp. dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {mputManip .dat}

AdaptiveGaussNoiεe {100} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp. mput4 {

FixedUmformNoise {ActFunction {SetNoiseLevel {105 be id

NewNoiseLevel {0} plogiεtic {} parameter {0.5}}}

FixedGaussNoise {ptanh {SetNoiseLevel {110 parameter {0.5}

NewNoiseLevel {0}}} pid {parameter {0.5}}

SaveNoiseLevel {115}

Filename {noise_level.dat] InputModification {

} εel None LoadNoiseLevel {AdaptiveU formNoiεe {

Filename {noise_level.dat] NoiseEta {1}} 120 DampmgFactor {1} SaveManipulatorData {}

Filename {inputMamp.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {DampmgFactor {1}

Filename {mputManip.dat} 125}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.mput3 {ActFunction {130 set id FixedGausεNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel {0}} ptanh {135 parameter {0.5}} SaveNoiseLevel {pid {Filename {noise_level .dat] parameter {0.5}}

140 LoadNoiseLevel {

Filename {noise_level.dat

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip.dat} NoiseEta {1} 145} DampmgFactor {1} LoadMampulatorData {} Filename {mputManip.dat} AdaptiveGaussNoise { Norm {NoNorm} 75 SetNoiseLevel {} NewNoiseLevel {0} mlp.mput5 {ActFunction {} set id FixedGaussNoise {plogistic {80 SetNoiseLevel {parameter {0.5} NewNoiseLevel 0}} ptanh {} parameter {0.5}} 85 SaveNoiseLevel {pid {Filename {noise_level. dat} parameter {0.5}}} LoadNoiseLevel {} Filename {noise_level. dat}

InputModification {90} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip .dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} 95 Filename {inputMa p. dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp.extern65 {

FixedUmformNoise {100 ActFunction {SetNoiseLevel {be id

NewNoiεeLevel {0} plogistic {} parameter {0.5}}}

FixedGaussNoise {105 ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}} pid {} parameter {0.5}} 110} SaveNoiseLevel {}

Filename {noise_level.dat] InputModification {

} εel None LoadNoiseLevel {AdaptiveUniformNoise {

Filename {noise_level.dat] 115 NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {putManip.dat} AdaptiveGaussNoise {} NoiseEta {1} LoadMampulatorData {120 DampmgFactor {1}

Filename {mputManip.dat}}} FixedUmformNoise {

Norm {NoNorm} SetNoiseLevel {} NewNoiseLevel {0} mlp.mputδ {125 ActFunction {sei id FixedGaussNoise {plogistic {SetNoiseLevel {arameter {0.5} NewNoiseLevel {0}} 130} ptanh {} parameter {0.5}} SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5} 135} LoadNoiseLevel {

Filename {noise_level.dat}

InputModification {} be None SaveManipulatorData {

AdaptiveUni ormNoise 140 Filename {mputManip.dat} NoiseEta {1}} Damp gFactor {1} LoadMampulatorData {

} Filename {putManip.dat}

AdaptiveGausεNoise {} NoiseEta {1} 145 Norm {NoNorm} DampmgFactor {1}}

} mlp.extern54 {

FixedUmformNoise {ActFunction { be id 75 FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0.5} NewNoiseLevel} ptanh {parameter {0.5} 80

} SaveNoiseLevel {pid {Filename {noise_level. dat} parameter {0.5}} LoadNoiseLevel {

} ^} 85 Filename {noise_level. dat}

InputModification {} be None SaveManipulatorData {AdaptiveUniformNoise {Filename {inputMamp. dat} NoiseEta {1}}

DampmgFactor {1} 90 LoadMampulatorData {

1 Filename {mputManip .dat}

AdaptiveGaussNoise {} NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}

} 95 mlp.extern32 {

FixedUmformNoise {ActFunction {SetNoiseLevel {be id

NewNoiseLevel {0} plogistic {parameter {0.5}

; ^} 100}

FixedGaussNoise {ptanh {SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}} pid {

} '105 parameters {0.5}

}}

SaveNoiseLevel {}

Filename {noise_level.dat} InputModification {} be None LoadNoiseLevel {110 AdaptiveUniformNoise {

Filename {noise_level.dat} NoiseEta {1}} DampmgFactor {1} SaveManipulatorData {}

Filename {mputManip.dat} AdaptiveGaussNoise {

} 115 NoiseEta {1}

LoadMampulatorData {DampmgFactor {1}

Filename {mputManip. dat}}

FixedUmformNoise {

Norm {NoNorm SetNoiseLevel {

120 NewNoiseLevel {0} mlp. extern43 {} ActFunction {} be id FixedGaussNoise {plogistic {SetNoiseLevel {parameter {0. 5} 12 5 NewNoiseLevel {0}

} ptanh {parameter {0. 5}

} SaveNoiseLevel {pid {1 3 0 Filename {noise_level.dat} parameter {0. 5}} LoadNoiseLevel {

Filename {noise_level.dat}

InputModification {} be None 13 5 SaveManipulatorData {

AdaptiveUniformNoise {Filename {inputMamp.dat} NoiseEta {1}} DampmgFactor {1} LoadMampulatorData {} Filename {mputManip.dat}

AdaptiveGaussNoise {1 4 0}

NoiseEta {1} Norm {NoNorm} DampmgFactor {1}}} mlp.extern21 {FixedUmformNoise {ActFunction {

SetNoiseLevel {1 4 5 be id

NewNoiseLevel {0} plogistic {parameter {0.5} ptanh {75 parameter {0.5}} SaveNoiseLevel {pid {Filename {noise_level.dat} parameter {0.5}}} 80 LoadNoiseLevel {} Filename {noise_level.dat}

InputModification {} εel None SaveManipulatorData {AdaptiveUniformNoise {Filename {mputManip.dat}

NoiseEta {1} 85}

DampmgFactor {1} LoadMampulatorData {} Filename {mputManip.dat} AdaptiveGaussNoise {}

NoiseEta {1} Norm {NoNorm}

DampmgFactor {1} 90}} mlp.output_auto {FixedUmformNoise {ActFunction {

SetNoiseLevel {εel id

NewNoiεeLevel {0} plogistic {

} 95 parameters {0.5}}} FixedGaussNoise {ptanh {

SetNoiseLevel {parameter {0.5}

NewNoiseLevel {0}}

100 pid {

} parameter {0.5}}

SaveNoiseLevel {}

Filename {noise_level.dat} ErrorFunc {} 105 let LnCosh LoadNoiseLevel {Ixl {

Filename {noise_level.dat} parameter {0.05}

}} SaveManipulatorData {LnCosh {

Filename {mputManip.dat} 110 parameters {2}}} LoadMampulatorData {parametricalEntropy {

Filename {mputManip.dat} parameter {le-06}}}

Norm {NoNorm} 115}} Norm {NoNorm} mlp.externlO {ToleranceFlag {F} ActFunction {Tolerance {0 0 0 0 0 0 0 0 0 0} εel ld Weighting {6 6 6 6 6 6 6 6 6 6} plogiεtic {120} parameter {0.5} mlp.fmalδ {} ActFunction {ptanh {be id parameter {0.5} plogistic {} 125 parameter {0.5} pid {} parameter {0.5} ptanh {} parameter {0.5}}}

InputModification {130 pid {εel None parameter {0.5} AdaptiveUmformNoiεe {} NoiseEta {1}} DampmgFactor {1} ErrorFunc {} 135 be LnCosh

AdaptiveGaussNoise {Ixl {NoiseEta {1} parameter {0.05} DampmgFactor {1}}} LnCosh {

FixedUmformNoise {140 parameter {2} SetNoiseLevel {}

NewNoiseLevel {0} parametricalEntropy {} parameter {le-06}}

FixedGaussNoise {145 SetNoiseLevel {Norm {NoNorm}

NewNoiseLevel {0} ToleranceFlag {F}

Tolerance {0 0 0 0 0 0 0 0 0 0} Weightmg {1 1 1 1 1 1 1 1 1 75}} ErrorFunc {mlp.fmalδ {εel LnCoεh ActFunction {Ixl {sei id arameter {0.05 plogistic {80} parameter {0.5} LnCoεh {} parameter {2} ptanh {parameter {0.5} parametricalEntropy {} 85 parameters {le-06} pid {parameter {0.5}

Norm {NoNorm}

ToleranceFlag {F}

ErrorFunc {90 Tolerance {0 0 0 0 0 0 0 0 0 0 let LnCosh Weighting {1 1 1 1 1 1 1 1 1 1 Ixl {parameter {0.05} mlp.fmal2 {} ActFunction {LnCosh {95 be id parameter {2 } plogistic {parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh { ^~ ) 100 parameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0 Weightmg {1 1 1 1 1 1 1 1 1 1 105} ErrorFunc {mlp.fmal4 {be LnCosh ActFunction { Ixl {be id parameter {0.05} plogistic {110} parameter {0.5} LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} 115 parameter {le-06} pid {} parameter {0.5}}} norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {120 Tolerance {0 0 0 0 0 0 0 0 0 0 let LnCosh Weightmg {1 1 1 1 1 1 1 1 1 1 Ixl {} parameter {0.05} mlp. finall {} ActFunction {LnCoεh {125 εel id parameter {2} plogistic {} arameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {} 130 parameters {0.5}

Norm {NoNorm} pid {ToleranceFlag {F} parameter 0.5} Tolerance {0 0 0 0 0 0 0 0 0 0}} Weightmg {1 1 1 1 1 1 1 1 1 1} 135}} ErrorFunc {mlp.fmal3 {se LnCosh ActFunction {Ixl {be d parameter 0.05} plogistic {140 parameter {0.5} LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy

} 145 parameters {le-06 pid {} parameter {0.5}} norm {NoNorm} ToleranceFlag {F} 75 parameters {0.5} Tolerance {0 0 0 0 0 0 0 0 0 0}} Weighting {1 1 1 1 1 1 1 1 1 1}}} ErrorFunc {mlp. bottleneck {sei none ActFunction {80 Ixl {sei tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {

} parameter {2} ptanh {85} parameter 0.5 parametricalEntropy {parameter {le-06}

Pid {} parameter {0.5}}} 90 Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} sei none Weighting {1 1 1} Ixl {parameter {0.05} 95 mlp.future4 {} ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter 0.5} parametricalEntropy {100} parameter { le-06} ptanh {} parameter 0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} 105 parameter 0.5 Tolerance {0 0 0} Weighting {1 1 1}} ErrorFunc {mlp.futureδ {sei none ActFunction {110 Ixl {sei tanh parameter {0.05} plogistic {} parameter { 0.5} LnCosh {} parameter {2} ptanh {115} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {} parameter {0.5}}} 120 Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} ael none Weighting {1 1 1} Ixl {} parameter {0.05} 125 mlp.future3 {} ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {130} parameter {le-06} ptanh {parameter {0.5}}

Norm {NoNorm} pid {ToleranceFlag {F} 135 parameters {0.5} Tolerance {0 0 0}} Weighting {1 1 1}}} ErrorFunc {mlp.futureδ {sei none ActFunction {140 Ixl {sei tanh parameter {0.05} plogistic {} parameter {0.5} LnCosh {} parameter {2} ptanh {145} parameter {0.5} parametricalEntropy {} parameter {le-06} pid { } 75

Norm {NoNorm} pid {ToleranceFlag {F} Parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1}}} 80 ErrorFunc {mlp.future2 {εel LnCosh ActFunction {Ixl {sei tanh parameter {0.05} plogistic { } parameter {0.5} 85 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} pid {90} parameter {0.5}}} Norm {NoNorm}} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} sei none 95 Weighting {1 1 1} Ixl {parameter {0.05} mlp.pastl {} ActFunction {LnCosh {sei tanh parameter {2} 100 plogistic {} parameter 0.5 parametricalEntropy {parameter {le- 06} ptanh {parameter 0.5

105

Norm {NoNorm} id {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1}}} 110 ErrorFunc {mlp.futurel {sei LnCosh ActFunction {Ixl {sei tanh parameter {0.05} plogistic { } parameter {0.5} 115 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy {} parameter {le-06} Pid {. 120 parameters {0.5}

Norm {NoNorm} ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} sei none 125 Weighting {1 1 1} Ixl {} parameter {0.05} mlp.past2 {} ActFunction {LnCoεh {sei tanh parameter {2} 130 plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} ptanh {parameter {0.5}

135}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0}} Weighting {1 1 1}}} 140 ErrorFunc {mlp. present {be LnCosh ActFunction {Ixl {be tanh parameter {0.05} plogistic {} parameter {0.5} 145 LnCosh {} parameter {2} ptanh {} parameter {0.5} parametricalEntropy j parameter {le-06} 75 ptanh { ^} parameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0} 80} Weightmg {1 1 1}}} ErrorFunc {mlp.past3 {sei LnCosh ActFunction {Ixl {sei tanh 85 parameter {0.05} plogistic {} parameter {0.5} LnCosh {} parameter {2} ptanh {} parameter {0.5} 90 parametricalEntropy {} parameter {le-06} pid {} parameter {0.5}}} norm {NoNorm}} 95 ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be LnCosh Weightmg {1 1 1} Ixl {} parameter {0.05} mlp.past6 {} 100 ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} 105 ptanh {} parameter {0.5}}}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0} 110} Weight g {1 1 1}}} ErrorFunc {mlp.past4 {sei LnCosh ActFunction {Ixl {sei tanh 115 parameter {0.05} plogistic {}

Parameter {0.5} LnCosh {} parameter {2} ptanh {} parameter {0.5} 120 parametricalEntropy {} parameter {le-06} pid {} parameter {0.5}}} Norm {NoNorm}} 125 ToleranceFlag {F}

ErrorFunc {Tolerance {0 0 0} be LnCosh Weightmg {1 1 1} Ixl {} parameter {0.05} mlp.state65 {} 130 ActFunction {LnCosh {sei tanh parameter {2} plogistic {} parameter {0.5} parametricalEntropy {} parameter {le-06} 135 ptanh {parameter {0.5}

Norm {NoNorm} pid {ToleranceFlag {F} parameter {0.5} Tolerance {0 0 0} 140} Weight g {1 1 1}}} Norm {NoNorm} mlp.pastδ {} ActFunction {mlp.state54 {sei tanh 145 ActFunction {plogistic {be tanh parameter {0.5 plogistic {parameter {0.5} 75 parameters {0.5} ptanh {parameter {0.5} ptanh {} parameter 0.5} pid {parameter {0.5} 80 pid {} parameters {0.5}}

Norm {NoNorm}} Norm NoNorm rnlp. state43 {85 ActFunction {mlp.statel2 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} 90 parameter 0.5} ptanh {parameter {0.5} ptanh {} parameter {0.5} pid {} parameter {0.5} 95 pid {} parameter {0.5}}}

Norm {NoNorm}}} Norm {NoNorm} mlp.state32 {100} ActFunction {mlp.state23 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} 105 parameters {0.5} ptanh {1 parameter {0.5} ptanh {} parameter {0.5} pid {} parameter {0.5} 110 pid {parameter {0.5}}

Norm {NoNorm}}} Norm {NoNorm} mlp.state21 {115} ActFunction {mlp.state34 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogiatic {} 120 parameters {0.5} ptanh {} parameter {0.5} ptanh {} parameter {0.5} pid {) parameter {0.5} 125 pid {parameter {0.5}

Norm {NoNorm}} Norm {NoNorm} mlp.εtatelO {130} ActFunction {mlp.state45 {sei tanh ActFunction {plogistic {sei tanh parameter {0.5} plogistic {} 135 parameters {0.5} ptanh {} parameter {0.5} ptanh { } parameter {0.5} pid {} parameter {0.5} 140 pid {parameter {0.5}

Norm {NoNorm} Norm {NoNorm} mlp. stateOl {145} ActFunction {mlp.state56 {be tanh ActFunction {plogistic {εel tanh plogistic {75 be tanh parameter 0.5} plogistic {} parameter {0.5} ptanh {} parameter 0.5 ptanh {} 80 parameter {0.5} pid {} parameter 0.5} pid {} parameter {0.5}}

Norm {NoNorm 85} Norm {NoNorm} mlp.back65 {} ActFunction {mlp. backlO {sei tanh ActFunction {plogistic {90 sei tanh parameter 0.5 plogistic {} parameter {0.5} ptanh {} parameter 0.5 ptanh {} 95 parameter {0.5} pid {} parameter 0.5} pid {} parameter {0.5}

Norm NoNorm 100}

Norm {NoNorm} mlp.back54 {ActFunction {sei tanh Connectorε {plogistic {105 mlp.bottleneck-> output_auto {parameter 0.5 WeightWatcher {} Active {F} ptanh {MaxWeight {1} parameter 0.5} MinWeight {0}} 110} pid {LoadWeightεLocal {parameter 0.5 Filename {εtd}} SaveWeightsLocal {

Norm NoNorm 115 Filename {std}} mlp.back43 {Alive {T} ActFunction {WtFreeze {F} sei tanh AllowPruning {F} plogistic {120 EtaModifier {1} parameter 0.5 Penalty {NoPenalty}}} ptanh {mlp.bιas-> output_auto {parameter 0.5} WeightWatcher {} 125 Active {F} pid {MaxWeight {1} parameter 0.5 MinWeight {0}}}} LoadWeightsLocal {

Norm {NoNorm 130 Filename {std}}} mlp.back32 {SaveWeightsLocal {ActFunction {Filename {std} sei tanh} plogistic {135 Alive {T} arameter 0.5} WtFreeze {F}} AllowPruning {F} ptanh {EtaModifier {1} parameter 0.5} Penalty {NoPenalty}} 140} pid {mlp. future6-> fmal6 {parameter 0.5 LoadWeightsLocal {

Filename {std}}

Norm {NoNorm 145 SaveWeightsLocal {} Filename {std} mlp.back21 {

ActFunction {Alive T} WtFreeze {F} 75 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.bιas-> fιnal6 {80 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> fmal3 {

Filename {std} 85 LoadWeightεLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1} 90) Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp. future5-> fιnal5 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 95 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.future2-> fιnal2 {

Filename {std} LoadWeightεLocal {} Filename {std}

Alive {T} 100} WtFreeze {F} SaveWeightεLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 105 WtFreeze {F} mlp.bιas-> fmal5 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {110 mlp.bιas-> fιnal2 {

Filename {std} LoadWeightsLocal {) Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 115 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp. future4-> fmal4 {AllowPruning {F) LoadWeightsLocal {120 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp. futurel-> fιnall {

Filename {std} LoadWeightsLocal {} 125 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 130 Alive {T}} WtFreeze {F} mlp.bιas-> fιnal4 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 135} SaveWeightsLocal {mlp.bιas-> fιnall {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 140 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp. future3-> fmal3 {145 AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} mlp.mput_auto-> bottleneck {75 Alive {T} WeightWatcher {WtFreeze {F}

Active {F} AllowPruning {F}

MaxWeight {1} EtaModifier {1}

MinWeight {0} Penalty {NoPenalty}} 80} LoadWeightsLocal {mlp.bιas-> future5 {

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} 85 SaveWeightεLocal {} Filename {std J

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} 90 AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.bιas-> bottleneck { } WeightWatcher {mlp.state34-> future4 {

Active {F} 95 LoadWeightsLocal {

MaxWeight {1} Filename {std}

MinWeight {0}}} SaveWeightεLocal {LoadWeightsLocal {Filename {std}

Filename {std} 100}} Alive {T} SaveWeightεLocal {WtFreeze {F}

Filename {std} AllowPrumng {F}} EtaModifier {1}

Alive {T} 105 Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} mlp.bιas-> future4 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 110} mlp.state56-> future6 {SaveWeightsLocal {WeightWatcher {Filename {std)

Active {F}}

MaxWeight {1} Alive {T}

MinWeight {0} 115 WtFreeze {F}} AllowPrumng {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {120 mlp.εtate23-> future3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.bιas-> future6 {AllowPruning {F} LoadWeightsLocal {130 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> future3 {

Filename {std} LoadWeightsLocal {} 135 Filename {std}

Alive {T}} WtFreeze {F SaveWeightsLocal {AllowPruning {F Filename {std} EtaModifier {1}}

Penalty {NoPenalty} 140 Alive {T}

} WtFreeze {F} mlp. εtate45-> future5 {AllowPruning {F} LoadWeightεLocal {EtaModifier {1}

Filename {εtd} Penalty {NoPenalty}} 145} SaveWeightεLocal {mlp.statel2-> future2 {

Filename {std} LoadWeightsLocal {

Filename {std} 75}

SaveWeightsLocal {mlp.bιaε-> present {

Filename {std} LoadWeightεLocal {} Filename {std}

Alive {T}} WtFreeze {F} 80 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.bιas-> future2 {85 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.ιnputl-> pastl {

Filename {std} 90 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightεLocal {AllowPruning {F} Filename {std} EtaModifier {1} 95} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.state01-> futurel {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 100 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state21-> pastl {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 105} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 1 Penalty {NoPenalty} Alive {T}} 110 WtFreeze {F} mlp.bιas-> futurel {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {115 mlp.bιas-> pastl {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 120 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp. mputO-> present {AllowPruning {F} LoadWeightsLocal {125 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.backlO-> pastl {

Filename {std} WeightWatcher {} 130 Active {F}

Alive {T} MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 135 Filename {std}}} mlp.statelO-> present {SaveWeightsLocal {LoadWeightsLocal {Filename { hours }

Filename {std}}} 140 Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F} 145} AllowPruning {F} mlp. mput2-> past2 {EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std} 75}

SaveWeightsLocal {mlp.bιas-> past3 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 80 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.state32-> past2 {85 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.back32-> past3 {

Filename {std} 90 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 95} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.bιaε-> past2 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {εtd} 100 Penalty {NoPenalty}}} SaveWeightsLocal {mlp. mput4-> paεt4 {

Filename {εtd} LoadWeightsLocal {1 Filename {std}

Alive {T} 105} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 110 WtFreeze {F} mlp.back21-> past2 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {115 mlp. state54-> paεt4 {

Filename {std} LoadWeightεLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 120 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.ιnput3-> past3 {AllowPruning {F} LoadWeightsLocal {125 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.bιas-> past4 {

Filename {std} LoadWeightsLocal {} 130 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning (F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} 135 Alive {T}} WtFreeze {F} mlp.state43-> past3 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {εtd} Penalty {NoPenalty}} 140} SaveWeightsLocal {mlp.back43-> past4 {

Filename {εtd} LoadWeightsLocal {} Filename {std}

Alive {T} WtFreeze {F} 145 SaveWeightεLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T} WtFreeze {F} 75 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.ιnput5-> past5 {80 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.back65-> paεt6 {

Filename {std} 85 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 90} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp. state65-> past5 {AllowPrumng {F} LoadWeightεLocal {EtaModifier {1}

Filename {εtd} 95 Penalty {NoPenalty}}} SaveWeightsLocal {mlp. extern65-> state65 {

Filename {std} WeightWatcher {} Active {F}

Alive {T} - 100 MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}} 105} mlp.bιaε-> paεt5 {SaveWeightsLocal {LoadWeightεLocal {Filename {std}

Filename {εtd}}} Alive {T} SaveWeightεLocal {110 WtFreeze {F}

Filename {εtd} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F}} AllowPruning {F} 115 mlp.past6-> state65 {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} Active {F}} MaxWeight {1} mlp.back54-> past5 {MinWeight {0} LoadWeightsLocal {120}

Filename {std} LoadWeightsLocal {} Filename {std} SaveWeightsLocal {}

Filename {std} SaveWeightsLocal {} 125 Filename {std}

Alive {T}} WtFreeze {F} Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} 130 EtaModifier {1}} Penalty {NoPenalty} mlp.mput6-> past6 { } LoadWeightsLocal {mlp. extern54-> state54 {

Filename {εtd} LoadWeightsLocal {} 135 Filename {std} SaveWeightεLocal {}

Filename {std} SaveWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 140 Alive {T} AllowPruning {F} WtFreeze {F} EtaModifier {1} AllowPruning {F} Penalty {NoPenalty} EtaModifier {1}} Penalty {NoPenalty} mlp.bιas-> past6 { 145} LoadWeightsLocal {mlp.past5-> state54 {

Filename {std} LoadWeightsLocal {} Filename {std} 75 Alive {T}

SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty} WtFreeze {F} 80} AllowPruning {F} mlp.past3-> state32 {EtaModifier {I} LoadWeightsLocal {Penalty {NoPenalty} Filename {std}}} mlp. state65-> state54 {85 SaveWeightsLocal {WeightWatcher {Filename {std}

Active {F}}

MaxWeight {1} Alive {T}

MinWeight {0} WtFreeze {F}} 90 AllowPruning {F} LoadWeightεLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightεLocal {mlp.Ξtate43-> state32 {

Filename {εtd} 95 LoadWeightεLocal {} Filename {εtd}

Alive {T}} WtFreeze {F} ΞaveWeightεLocal {AllowPruning {F} Filename {std} EtaModifier {1} 100} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp. extern43-> εtate43 {AllowPrumng {F} LoadWeightεLocal {EtaModifier {1}

Filename {std} 105 Penalty {NoPenalty}}} SaveWeightsLocal {mlp. extern21-> state21 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 110} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 115 WtFreeze {F} mlp.past4-> state43 {AllowPruning {F} LoadWeightεLocal {EtaModifier {1}

Filename {εtd} Penalty {NoPenalty}}} SaveWeightaLocal {120 mlp.past2-> state21 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPrumng {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.state54-> state43 {AllowPruning {F} LoadWeightsLocal {130 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state32-> state21 {

Filename {std} LoadWeightsLocal {} 135 Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}> Penalty {NoPenalty} 140 Alive {T}} WtFreeze {F} mlp.extern32-> state32 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 145} SaveWeightsLocal {mlp. externl0-> statel0 {

Filename {std} LoadWeightsLocal {

Filename {std} 75}

SaveWeightsLocal {mlp.statel2-> state23 {

Filename {εtd} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} 80 SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.pastl-> statelO {85 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state23-> εtate34 {

Filename {std} 90 LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightεLocal {AllowPruning {F} Filename {εtd} EtaModifier {1} 95} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.εtate21-> statel0 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} 100 Penalty {NoPenalty}}} SaveWeightsLocal {mlp.state34-> state45 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 105} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 110 WtFreeze {F} mlp.present-> state01 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {115 mlp. state45-> state56 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive [T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 120 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.statel0-> state01 {AllowPruning {F} LoadWeightsLocal {125 EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.past5-> back65 {

Filename {std} WeightWatcher {} 130 Active {F}

Alive {T} MaxWeight {1} WtFreeze {F} MinWeight {0} AllowPruning {F}} EtaModifier {1} LoadWeightsLocal {Penalty {NoPenalty} 135 Filename {std} mlp.state01-> statel2 {SaveWeightεLocal {LoadWeightsLocal {Filename {st }

Filename {std}}} 140 Alive {T} SaveWeightsLocal {WtFreeze {F}

Filename {std} AllowPruning {F}} EtaModifier {1}

Alive {T} Penalty {NoPenalty WtFreeze {F} 145} AllowPrumng {F} mlp.back54-> back65 {EtaModifier {1} WeightWatcher {Penalty {NoPenalty} Active {F} MaxWeight {1} 75 Alive {T} MinWeight {0} WtFreeze {F}

} AllowPruning {F}

LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}} 80}

SaveWeightsLocal {mlp.back21-> back32 {

Filename {std} LoadWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {F} 85 SaveWeightsLocal {AllowPrumng {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} WtFreeze {F} mlp.past4-> back54 {90 AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {mlp.pastl-> back21 {

Filename {std} 95 LoadWeightsLocal {

} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1} 100} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.back43-> back54 {AllowPruning {F}

LoadWeightsLocal {EtaModifier {1}

Filename {std} 105 Penalty {NoPenalty}

}} SaveWeightsLocal {mlp.backl0-> back21 {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T} 110} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}} 115 WtFreeze {F} mlp.past3-> back43 {AllowPruning {F} LoadWeightsLocal {EtaModifier {1}

Filename {std} Penalty {NoPenalty}}} SaveWeightsLocal {120 mlp.present-> backlO {

Filename {std} LoadWeightsLocal {} Filename {std}

Alive {T}} WtFreeze {F} SaveWeightsLocal {AllowPruning {F} 125 Filename {std} EtaModifier {1}} Penalty {NoPenalty} Alive {T}

} WtFreeze {F} mlp.back32-> back43 {AllowPruning {F} LoadWeightεLocal {130 EtaModifier {1} Filename {εtd} Penalty {NoPenalty}}

SaveWeightsLocal {Filename {std} AnySave {} 135 file name f.CCMenu.dat

Alive {T} WtFreeze {F} AnyLoad {AllowPruning {F} file name {f.CCMenu.dat EtaModifier {1}} Penalty {NoPenalty} 140}} RecPar {mlp.past2-> back32 {decay_c {1} LoadWeightsLocal {delta_t {0.1} Filename {std} epsilon {0.01}

} 145 max_ιter {1}

SaveWeightsLocal {show {F} Filename {std} Reset_Errors {F}} TestRun {

Filename {Test}

Part.Transformed {F}} Online {

Filename {Online.dat}

The following publications are cited in this document:

[1] S. Hayken, Neural Networks: A Comprehensive Foundation, Mc Millan College Publishing Company, Second Edition, ISBN 0-13-273350-1, pp. 732-789, 1999.

[2] H. Rehkugler and H. G. Zimmermann, Neural Networks in Economics, Fundamentals and Financial Applications, Verlag Franz Vahlen Munich, ISBN 3-8006-1871-0, pp. 3-90, 1994;

[3] J. Hirschberg, Pitch accent in context: predicting intonational prominence fro text, Artificial Intelligence 63, pp. 305-340, Elsevier, 1993;

[4] K. Ross et al., Prediction of abstract prosodic labeis for speech synthesis, Computer Speech and Language, 10,

Pp. 155-185, 1996;

[5] R. Haury et al., Optimization of a Neural Network for Pitch Contour Generation, ICASSP, Seattle, 1998;

[6] C. Traber, F0 generation with a database of natural F0 patterns and with a neural network, G. Bailly and C. Beanit eds. , Talking Machines: Theories, Models and Applications, Elsevier, 1992;

[7] E. Heuft et al. , Parametric Description of FO-Contours in a Prosodic Database, Proc. ICPHS, Vol. 2, pp. 378-381, 1995;

[8] C. Erdem, Topology optimization of a neural network for the generation of FO courses by integrating different codes, conference proceedings ESSV, Cottbus, 2000.

Claims

claims

1. A method for computer-aided mapping of several time-varying state descriptions, each of which describes a time-changing state of a dynamic system at a corresponding point in time in a state space, which dynamic system maps an input variable to an associated output variable, with the following steps

a) a first state description is mapped in a first state space to a second state description in a second state space, b) the first state takes into account the second state description of an earlier state, c) it is represented by a second Mapping the second status description mapped to a third status description in the first status space, characterized in that d) the first status description is mapped by a third mapping to a fourth status description in the second status space, e) in the third mapping the fourth status description of a later one State is taken into account and f) the fourth state description is mapped by a fourth image to the third state description, the images being adapted such that the images of the first state description onto the third state description are mapped describe the input variable to the associated output variable with a specified accuracy.

2. The method of claim 1, wherein the plurality of time-varying state descriptions describe a dynamic process that can be described by an economic key figure.

ÜO cυ) t \ _ oo Cπ o Cπ

Describe the output variable to the associated output variable with a specified accuracy.

4. Arrangement according to claim 3, in which at least some of the imaging units are artificial neurons.

5. Arrangement according to claim 3 or 4, in which a time-varying state description. Vector is a predefinable dimension.

6. Arrangement according to claims 1 to 5, with a measuring arrangement for detecting physical signals with which the dynamic system is described.

7. Arrangement according to claim 6, used to determine a dynamic of an electro-cardio gram.

8. Arrangement according to one of claims 3 to 7, used in speech processing, wherein the input variable is a first speech information of a word to be spoken and / or a syllable to be spoken and the output variable is a second speech information of the word to be spoken and / or to be spoken Is syllable.

9. Arrangement according to claim 8, wherein the first speech information comprises a classification of the word to be spoken and / or the syllable to be spoken and / or pause information of the word to be spoken and / or the syllable to be spoken and / or the second speech information comprises accenting information of the word to be spoken and / or the syllable to be spoken.

10. The arrangement according to claim 9, wherein the first speech information is a phonetic and / or structural information of the word to be spoken and / or to ro Cπ o Cπ o Cπ Cπ

Hi φ α Ω tr PP f cn tr cn P ¹ rt P-

"*" * 'tr d rt φ rt P ¹ Φ d Φ tr cn d P cn: Cn Hi H cn <rt 3 d cn P. rt 3 cn N rt 3 H- tr d tsi P- Φ rt 3 P- rt. d Ω rt O

Ω P- Φ P- rt cn d Ω H Φ P- Ω Φ p- cn d P d d cn H φ P- Φ rt tr N d l-i N

P Φ rt rt P- Ω P f P- rt rt tr φ rt rt rt d cn cn rt cn rt rt 3 l-i rt φ d 3 d

H li LP tr cn 1-5 rt H P- "« _* LP cn rt rt Φ P- φ φ Φ li P φ rt P- Φ tr H cn Φ rt P- Φ φ rt p- Φ PP Φ P- Φ rt Φ d P- P- d Hi rt ω

P- φ cn P- P φ P- P- Φ cn P- P- Φ cn P- Φ tr P- ddd cn P- _^ P- d tr PO P- Xf tr rt dr. P- tr φ tr rt d tr rt d P- φ φ Hi rt dd Φ φ d tr Φ dg Φ d _^ S tr OH dg Φ ü Φ d φ d li P- d P- d ι-ig Φ d p- cn tS! Φ Z φ P- cn dd Φ φ ii Φ d li Φ d LQ tr Φ li Ω

L _^ Q tr cn d ι-i LP tr LQ tr cn HP P- cn P "c_ Φ Φ P- d Φ tr

P- P <! rt LQ p- P- P P. P- PN Φ cn dd rt P Φ Ω P- 3 ^ d P- Φ cn P- _" • φ rt P- tn H P- cn z NH rt 3 φ P cn ι- ϊ Pf dd N φ dd cn φ φ H rt d cn P- d p. cn φ φ Φ " _* d cn cn Φ LP tS! P dd li φ d H P- Φ d rt d P- P- Φ tsi P. rt P d 3 d 3 Φ d P. rt d. d P. rt d rt rt H g Z cn Φ Φ cn gd P- P- LP N * 1 d ιQ P- Φ φ r. g Φ LQ P- Φ Φ ιQ p- Φ cn φ H p- d LP rt cn Φ P_ d li φ cn P- 3 φ d 3 φ d P- rt tr P- P do cn P Ω 3 H φ Ui

P g rt tr PP Ω Φ P- rt d φ g li LP d tr l_l- Φ P cn iP P- tr <i N rt P- N tr φ [JJ φ tr N g tr d P "Φ 3 li Φ φ tr P- X) d - tr P- tr φ Φ P "z tr H o" ι-i tr z. Φ tr d O: cn dz H d li Φ tr

P- Φ P- P- d Φ P- cn tr ω P- φ tr Hi g d tsi H P- d Go ι-i φ Φ P- φ d Φ

P ¹ HP "rt d P- rt P- rt P" p- P- H ddd cn d Φ P ω P- li d Ω N rt gd rt α Φ * Φ rt: tr iQ cn H rt LP d ^ <Φ LP tr P-! tr Φ d P- LQ Φ tr d tr Φ P. tr P- cn rt Ω Φ d P 3 cn cn H Φ d

P dd Ω tr cn d P tsi d P d φ φ P tr d Hi d rt φ d Hi Hi ι-i LQ tr P- φ H ddti H tsi d li P- d ^' tsi iQ o Hi tr Φ φ N P - OP

CSI cn P- tsi ω ιQ d d LQ φ d d Φ d cn Φ 3 P- Φ d Φ H cn

P- d Φ cn P. dd P- rt cn cn P- cn cn dd pf cn P- cn φ iQ Φ cn cn d P- φ d 3 cn cn cn p- xs d tr cn cn P φ rt cn rt φ L φ tr d rt P- Φ P- Ω Φ rt li P rt rt rt d (D: d φ rt rt d P- P rt P P- tsi P- Φ Φ P ddd tr N dt / _ rt

P r rt LP P- P d d d d d. rt cn d Φ H d P- P- P- d

P d Φ Φ rt d P cn tr P pf cn P- Ω Φ φ Φ Φ tSJ Ω & P "O dd α P- ii d tr φ cn d cn φ rt φ pf H cn P- N P- Φ Φ pf o tr d

Hi cn rt Φ P- P. cn Hi Φ P- H Hi tr P- P Φ ι-i cn tr rt ^ d tr P- P- li Φ tr " _^ d φ Φ ι- {cn rt P φ rt d NH Φ rt φ * _^ o LQ Φ d rt <i. O φ HP φ O ^• * d Φ cn ^• * z P P- Φ cn 3 Φ d φ (- "φ dg

P- cn P. tsi <JP d P- tr 3 P- Ω cn φ H tr Ω Ό tr 3 P- li d cn φ Ω P- d P- H 3 d H P. _^ d Pf P- rt dgf P - o O: Ω p: d Hi H r φ cn φ rt φ Φ P- Φ H P- tr rt d H φ d ι- (s: N Pf d iQ PN

H rt H P- Φ Φ Φ Φ Φ Φ P tr φ Φ P- Φ d cn d

H Φ P. P rt Φ <tr P. P- H P- P- H- d LQ LQ Φ N cn P-

P- P- φ d Φ P- P- d H tr d: tsi d P- tr Φ rt Φ Ω φ Φ li d rt cn φ rt tr H d φ d Φ P- d Φ Ω d LP dd H Φ tr pr HH • Xf rt d P cn NP li PH rt d HW cn Φ dd P d> Φ O: p: P- ι-i N

Φ d H d Φ rt P rt LP P cn rt H Φ d ιQ H d cn li d Ω li Φ s: ιQ rt tr cn H Φ H Φ H P- P P- P- LQ rt P cn P- r φ Ω tsi φ rt P- d rt rt Ω d Ω d P- d LQ LQ φ Φ Ω pf P- d. Φ ι-i P Ω tsi H tsi d Pf tr Φ P d Φ Hi P ^ <φ li li tr φ rt ω d P- d: d pf d Ω Φ d H φ rt cn rt 3 tr P- Z ddd H dd Φ rt H d Ω rt cn pf P- cn Ω P- P- tr Φ tr Φ d φ iQ P P- tS3 Φ

P Ω LQ cn Φ rt d rt tr d LQ Φ rt N P- P- iQ P- cn 3 tsi Ω d li Φ ω d tr Φ cn tr rt P Φ LQ P ι tr cn zd Φ cn LQ P- φ tr cn iQ d xf

H P- φ d P- Φ d Φ φ P Ω P- φ φ ι-i rt H ω P- Φ rt φ ι-i cn Φ P- Ω 1 P- d H P- r. H Pf cn P- tr 3 P- O: Ω rt d P 1 S! P tr P- Ω tr cn cn Φ P- cn d P- H rt rt P Ω & α pf 1 d O Ω φ d r 1 rt tr Ω σ Φ Ω Φ Φ H tr Φ φ tSJ H f

1 Φ 1 φ tr φ tr P- d 1 d cn 1 1

1 1 1 1 1 1 1

Cπ