DE102011075337A1 - Method for controlling system, involves carrying out action to control system by obtaining control information from data of system, where another action or operation is determined according to data to control system - Google Patents

Abstract

The method involves carrying out an action (110) to control a system by obtaining a control information from the data (111) of the system. Another action (113) or operation is determined according to the data to control the system. The two actions are intended to explore the components of the state space of the system, where the components include technical facility, manufacturing plant, wind turbine and gas turbine. The two actions comprise a setting of the operating parameters of the system. An independent claim is included for an apparatus for controlling a system by a processing unit.

Description

The invention relates to a method and a device for controlling a system.

There are increasing efforts to develop wind turbines for the production of renewable energy in different environments, e.g. to build on the coast or off the coast. Depending on geographical conditions, the wind has different characteristics that significantly affect the operation and efficiency of the wind turbine. However, while the wind turbine limits the power it provides at high wind speeds, it is driven by lower wind speeds most of the time, i. The power provided is usually well below the maximum possible limit of the wind turbine.

In order to optimize the power provided by the wind turbine, its operating parameters can be adjusted. However, it is disadvantageous that e.g. local conditions, e.g. Turbulence, shear winds, temperature fluctuations, moisture, dust, ice, etc. are not considered suitable. This effect is exacerbated by the use of several wind turbines in the form of so-called wind farms, as the wind turbines themselves influence the wind speeds.

The object of the invention is to avoid the above-mentioned disadvantages and in particular to provide an efficient solution for setting or for operation (at least) of a wind turbine.

This object is achieved according to the features of the independent claims. Further developments of the invention will become apparent from the dependent claims.

• – bei dem die Anlage mittels einer ersten Aktion angesteuert wird,
• – bei dem Daten der Anlage ermittelt werden als Folge der Ansteuerung mit der ersten Aktion,
• – bei dem anhand der Daten eine zweite Aktion ermittelt wird und die Anlage mit der zweiten Aktion angesteuert wird,
• – bei dem die erste Aktion und/oder die zweite Aktion bestimmt wird, um Teile eines Zustandsraums der Anlage zu explorieren.

To solve the problem, a method is specified for controlling a system,

• - in which the system is controlled by a first action,
• - in which data of the plant are determined as a result of the control with the first action,
• In which a second action is determined on the basis of the data and the system is activated with the second action,
• - In which the first action and / or the second action is determined to explore parts of a state space of the plant.

Determining the parts of the state space may be a systematic and / or unsystematic (e.g., random or pseudorandom) exploration of the state space (or part of the state space).

Thus, it is possible to increase and thus improve the knowledge base (data) for modeling through targeted actions. This knowledge base can be used for modeling and further controlling the plant. As a result, the efficiency of the plant can be effectively increased.

The present solution is particularly advantageous when the system and the model associated with it have temporal dynamics, stochastic properties, non-linear effects and / or a high dimension.

A further development is that the first action and / or the second action is an action for which no data is yet available or for which the present data is obsolete.

Thus, using the actions systematically, unsystematically, stochastically, or with mixed strategies, the state space can be explored, i. preset operating parameters are set and associated data are determined.

Another development is that the first action and the second action includes a setting of operating parameters of the plant.

• – ein technisches System,
• – eine Automatisierungsanlage,
• – eine Fertigungsanlage,
• – eine Produktionsanlage,
• – eine Maschine,
• – eine Windturbine,
• – eine Gasturbine,
• – eine Strömungsmaschine,
• – ein Energienetz,
• – ein Stromverteilungsnetz,
• – ein Lastverteilungsnetz,
• – ein Kommunikationsnetz.

In particular, it is a development that the system comprises at least one of the following components:

• - a technical system,
• An automation system,
• - a manufacturing plant,
• - a production plant,
• - a machine,
• - a wind turbine,
• A gas turbine,
• A turbomachine,
• - a power grid,
• A power distribution network,
• - a load distribution network,
• - a communication network.

It is also a further development that the determined data comprise measured values, information and / or observations of the plant.

In particular, different sensors (e.g., temperature, humidity, air pressure, speed, acceleration, etc.) may be provided to obtain data from the plant.

Furthermore, it is a development that the data is stored together with the associated first action.

For example, the data may be stored in a database, e.g. collected over a predetermined period of time. This database can be a knowledge base for a learning algorithm (s.u.).

As part of an additional training, the data is stored together with the associated first action.

Thus, the database may have records that each contain an action along with the associated data.

A further development is that the second action is determined on the basis of the data by determining a state, in particular a Markov state, by means of a state estimation.

For example, the Markov state can be determined by means of an MPEN method ( See "Markov Decision's Process Extraction Network", for example: S. Duell, A. Hans, and S. Udluft: The Markov Decision Process Extraction Network. In Proc. of the European Symposium on Artificial Neural Networks, 2010 ) be determined. In particular, the state estimation may determine a Markov state based on the data of the plant stored in the database.

An embodiment is that an optimized action is determined as the second action on the basis of a control strategy.

The state estimation thus provides a state that allows a selection from a variety of possible actions based on the control strategy. This action can then be used as a second action to control the system.

An alternative embodiment is that, based on data of the plant by means of a learning algorithm, a model of the plant is determined or modified and based on the model, the state estimation is performed.

The model can correspond to a picture of the system. Based on the data records stored in the database (see above), the model can be iteratively refined or changed. For example, this may respond flexibly to changes in the plant, or it may take into account efficiency gains that have become apparent through exploration (e.g., a stepwise systematic or (pseudo) random exploration) of the state space.

Thus, the model can be used as a predictive model to determine a Markov condition. Here, a dynamics of the system can be taken into account, the state estimation approaches the state space of the Markov states.

• – ein NFQ-Verfahren ( "Neural Fitted Q Iteration", siehe: M. Riedmiller: Neural Fitted Q Iteration – First Experiences with a Data Efficient Neural Reinforcement Learning Method. In Proc. of the European Conf. on Machine Learning, 2005 ),
• – ein RCNN ( "Recurrent Control Neural Network", siehe: A.M. Schaefer, S. Udluft, and H.-G. Zimmermann. A Recurrent Control Neural Network for Data Efficient Reinforcement Learning. In Proc. of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 2007 ; oder A. M. Schäfer, D. Schneegaß, V. Sterzing, and S. Udluft. A Neural Reinforcement Learning Approach to Gas Turbine Control. International Joint Conference on Neural Networks, 2007 ) und/oder
• – ein PGNRR-Verfahren ( "Policy Gradient Neural Rewards Regression", siehe: D. Schneegaß, S. Udluft, and Th. Martinetz. Improving Optimality of Neural Rewards Regression for Data-Efficient Batch Near-Optimal Policy Identification. In Proc. of the International Conf. on Artificial Neural Networks, 2007 )

eingesetzt werden. As a learning algorithm, for example

• - an NFQ procedure ( "Neural Fitted Q Iteration", see: M. Riedmiller: Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method. In Proc. of the European Conf. on Machine Learning, 2005 )
• An RCNN ( "Recurrent Control Neural Network", see: AM Schaefer, S. Udluft, and H.-G. Zimmermann. A Recurrent Neural Network for Data Efficient Reinforcement Learning. In Proc. of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 2007 ; or AM Schäfer, D. Schneegass, V. Sterzing, and S. Udluft. A Neural Reinforcement Learning Approach to Gas Turbine Control. International Joint Conference on Neural Networks, 2007 ) and or
• - a PGNRR procedure ( "Policy Gradient Neural Rewards Regression", see: D. Schneegass, S. Udluft, and Th. Martinetz. Improving Optimality of Neural Rewards Regression for Data-Efficient Batch Near-Optimal Policy Identification. In Proc. of the International Conf. on Artificial Neural Networks, 2007 )

be used.

A next embodiment is that the model of the plant is determined by means of a neural network, a tree structure and / or Gaussian processes.

It is also an embodiment that based on data of the system by means of the learning algorithm and by means of the model of the system, the control strategy for selecting the second action based on the state is set or modified.

Such a generator can be understood as a generator for state estimation. Also, the generator may serve to create or modify a reinforcement learning strategy.

In this case, it is advantageous that the proposed approach is applicable to continuous state spaces. For example, a Markov state may be mapped to at least one action by means of the control strategy, the control strategy itself being a result of the learning algorithm.

A development consists in that the system can be controlled in a first operating mode and in a second operating mode, the first operating mode characterizing normal operation of the system and the second operating mode designating operation of the system for generating the data.

In particular, a switchover between the operating modes can take place. The two operating modes can be activated alternately, randomly or regularly. Also, based on a predetermined scheme, an activation of the operating modes can take place. Alternatively, dynamic activation of the modes of operation may occur, e.g. caused by an improvement in the efficiency of the plant in the first operating mode. Thus, it is also possible by way of example that the second operating mode is rarely activated, if the system is already running sufficiently efficiently or by additional measurements no (or little more) increase in efficiency can be achieved.

An additional embodiment is that in the second operating mode, the system is controlled with a first or second action, which serves the targeted procurement of not yet existing data or current data.

Thus, in the second operating mode, the state space of the plant can be explored specifically, i. A behavior of the system can be determined by means of the data depending on the set operating parameters of the system (ie the actions performed). Targeted procurement of non-existent data on the state space of the plant is in particular dependent on new actions being taken and on the data being determined as to whether this will increase the efficiency or deteriorate the efficiency of the installation. The step-by-step exploration of the state space of the plant can serve to model the plant and thus improve the state estimation as well as the control strategy as a result of the learning algorithm.

The exploration (also called exploration) of the state space can also take place under consideration of the control strategy. In this case, different variants for the exploration of the state space are possible: For example, a data density can be increased or a proximity (in the state space) between already determined data (and, if applicable, its efficiency) can be taken into account in order to obtain comparable or completely different data in the state space State space to determine.

• – die Anlage mittels einer ersten Aktion ansteuerbar ist,
• – Daten der Anlage ermittelbar sind als Folge der Ansteuerung mit der ersten Aktion,
• – anhand der Daten eine zweite Aktion ermittelbar ist und die Anlage mit der zweiten Aktion ansteuerbar ist,
• – die erste Aktion und/oder die zweite Aktion bestimmbar ist, um Teile eines Zustandsraums der Anlage zu ermitteln.

The above object is also achieved by a device for controlling a system with a processing unit, which is set up such that

• - The system is controllable by means of a first action,
• - data of the system can be determined as a result of the control with the first action,
• A second action can be determined on the basis of the data and the installation can be activated with the second action,
• - The first action and / or the second action is determinable to determine parts of a state space of the plant.

In particular, the processing unit may be a processor unit and / or an at least partially hardwired or logic circuit arrangement, which is set up, for example, such that the method can be carried out as described herein. Said processing unit may be or include any type of processor or computer or computer with correspondingly necessary peripherals (memory, input / output interfaces, input / output devices, etc.).

The above explanations regarding the method apply to the device accordingly. The device may be implemented in one component or distributed in several components. In particular, a portion of the device may also be connected via a network interface (e.g., the Internet). For example, the database discussed above may be a database in a network.

According to a development, the device serves to control at least one wind turbine.

Embodiments of the invention are illustrated and explained below with reference to the drawings.

Show it:

1 to illustrate an (autonomous) learning approach for controlling a wind turbine;

2 a schematic arrangement with several wind turbines, which can be controlled via a common control unit.

It is proposed, in particular, to drive a system which can be set by means of operating parameters efficiently. The installation may be any technical system, e.g. having electrical and / or mechanical components. The plant may e.g. a technical system, an automation system, a production plant or a machine. The plant may in particular have a wind turbine. It is also possible for the plant to have several wind turbines, e.g. in the form of a wind farm can be controlled at least partially together.

Operating conditions of the system can significantly influence a target size of the system. For example, the operating conditions may be environmental conditions, environmental influences, etc. The target size of the plant may e.g. a provided electric power, a provided product, a service provided, a quality (e.g., the product provided or the service provided) or the like. be.

In the following, a wind turbine will be discussed as an example. Accordingly, other systems are possible. Also, the plant may include a variety of wind turbines.

By adapting the operating parameters of the wind turbine, it is possible (eg for a class or group of wind turbines, which may be installed in a wind farm) to provide an optimized control strategy or to learn the control strategy on the basis of existing data and / or to take into account effects such as air density, shear winds, wind turbine start-up conditions and other factors.

• – globale Wetterbedingungen,
• – globale Windbedingungen und/oder
• – allgemeine Turbineneigenschaften.

The adjustment of the operating parameters can be done automatically. For example, a global optimization of the wind turbine can be achieved by considering a state of the turbine only global effects, eg:

• - global weather conditions,
• - global wind conditions and / or
• - general turbine characteristics.

• – lokale Windrichtungen,
• – lokal vorhandene Turbulenzen und/oder
• – lokal vorhandene Scherwinde.

A local optimization of the wind turbine may be based on a condition taking into account, for example, the following location-specific information:

• - local wind directions,
• - local turbulence and / or
• - Locally available shear winds.

• – eine Drehgeschwindigkeit des Rotors der Windturbine und
• – ein Blattanstellwinkel der Rotorblätter eingestellt werden, um die Effizienz zu verbessern. Eine von der Turbine bereitgestellte Leistung ergibt sich zu: p(t) = 0,5·ρ(t)·A·C_p(t)·ν_wind(t)³ – p_I(t), (1)

wobei

t

die Zeit,

ρ(t)

eine Luftdichte,

A

eine Rotorfläche,

C_p(t)

einen Leistungskoeffizienten,

v_wind(t)

eine Windgeschwindigkeit und

p_I(t)

Verluste begründet durch Reibung und Hilfsleistung

bezeichnen. Taking into account data from a model (eg a design model) of the wind turbine and wind profiles of different turbulence levels

• A rotational speed of the rotor of the wind turbine and
• - A blade pitch of the rotor blades are adjusted to improve the efficiency. A power provided by the turbine results in: p (t) = 0.5 · ρ (t) · A · C _p (t) · ν _wind (t) ³ - p _I (t), (1)

in which

t

the time,

ρ (t)

an air density,

A

a rotor surface,

_Cp (t)

a power coefficient,

v _wind (t)

a wind speed and

p _I (t)

Losses due to friction and auxiliary power

describe.

The power can be optimized at a given wind speed by using a power coefficient optimized by means of an aerodynamic model of the turbine and its rotor blades.

The result of such optimization can be through the function P _{el (t)} = f (ν _wind (t)) (2) which provides an optimized power set point P _el (t) for each wind speed. The wind speed can be replaced by the speed of the rotor, for example. This implies that, for a given wind speed and pitch of the rotor blades, the speed of the rotor results from a balance between the torque of the rotor caused by the wind and the torque of the generator. Using a speed n (t) of the rotor thus results in the power setpoint to: P _{el (t)} = f _sp (n (t)), (3) wherein the function f _sp provides a static mapping from a rotor speed to a power setpoint.

In order to take into account further influences on the wind turbine and thus an improvement in the efficiency of the wind turbine with regard to their provided power as a target, in particular an autonomous learning approach is proposed on the basis of which an optimization of the operating parameters of the wind turbine takes place by activating a state space of the operating parameters that are possible for the wind turbine explored or explored. The information thus obtained, in particular an influence of the operating parameters on the target size, are used to derive an optimized control strategy and to use it to operate the wind turbine.

Thus, it is also possible to use local conditions which limit the operation of the wind turbine, e.g. in the form of turbulence and shear winds influence, to take into account. In addition, additional factors, e.g. Temperature, humidity, dust, ice, etc., which also exert an effect on the operation of the wind turbine, are taken into account.

1 shows a block diagram illustrating an (autonomous) learning approach for controlling a wind turbine 101 ,

A control unit 102 leads over a unit 103 to implement an investigation strategy, an action 110 off by the operating parameters of the wind turbine 101 be adjusted or modified. Due to the action 110 become data 111 from the wind turbine 101 provided or determined. With the data 111 it can be measured values, information, observations, also in relation to other wind turbines of a wind farm. The data 111 for example, in a database 106 saved. Preferably, records are stored in the database containing the data 111 along with those of the control unit 102 include predetermined operating parameters.

Based on the data 111 can a condition 112 (eg a Markov state) of the wind turbine 101 by means of a state estimator 104 be determined. Based on the condition 112 beats a unit 105 , which implements a control strategy, an optimized or optimal action 113 the unit 103 to implement the investigation strategy.

It should be noted that the unit 103 either an action 110 for optimized operation of the wind turbine 101 or an action 110 for the best possible exploration of the state space of the wind turbine 101 can provide. In particular, the unit 103 a switch between a first operating mode "efficient operation" and a second operating mode "information acquisition".

Exemplary are in 1 the units 103 . 105 as well as the state estimator 104 as part of the control unit 102 shown. Other distributed or centralized implementations in one or more functional or actual components are possible. The control unit 102 can be the function of a controller at runtime (online control) of the wind turbine 101 perceive or provide.

The state estimator 104 can determine the Markov state, for example, by means of a so-called Markov Decision Process Extraction Network (MPEN) method. In particular, the state estimator may have a compact Markov state based on accumulated (eg, collected over a period of time, in particular multi-dimensional) data 111 the wind turbine 101 determine.

The action 110 gets to the wind turbine 101 transmitted, for example, in the form of a set of operating parameters. During the autonomous learning approach, the investigative strategy of the unit 103 be used to this action 110 to modify the state space of the wind turbine (adjustable by the operating parameters) 101 (eg to search systematically or (pseudo-) randomly).

Thus, for example, the wind turbine 101 specifically by means of possibly not optimal actions 110 operated to thereby data 111 to support the learning approach to be able to consider new operating points in the state space beyond already known operating points. Based on the data 111 or in the database 106 stored records can be a learning algorithm 107 be performed. So can a model of the wind turbine 101 be improved, for example in the form of a neural network 108 , based on the state estimator 104 can be used to provide an improved estimate of states 112 to enable. Furthermore, the learning algorithm 107 by means of a control generator 109 the one from the unit 105 modify provided control strategy (see arrow 114 ), in particular to improve, thus also that of this unit 105 selected actions 113 be optimized. The neural network 108 can both the state estimator 104 as well as the control generator 109 modify (compare connection 115 ).

The learning algorithm 107 can be provided by at least one component. The learning algorithm 107 may be executable independently (offline) and, for example, continuously or at predetermined times or time intervals based on the data of the database 106 an update of the model of the wind turbine 101 carry out.

Instead of the neural network 108 can also be a generator for the state estimator 104 be provided. Instead of the control generator 109 For example, a Reinforcement Learning ("RL") generator may be provided to the unit 105 based on determined states, provides an optimized control strategy that assigns optimized actions to given Markov states.

The learning algorithm 107 can for example be based on an NFQ method. The NFQ method is based on a conventional FFNN (Feed-Forward Neural Network), using this feedforward neural network to learn a function Q.

The neural network provides target data T _i based on input data _I , where i denotes an iteration counter. Learning can be illustrated by the following three steps:

Step 1:

i

den Iterationszähler (d.h. einen Schritt i der Iteration),

r

ein Vektor von Belohnungen (alternativ auch als Kosten realisierbar),

s

ein Vektor von Zuständen,

a

ein Vektor von Aktionen,

s'

ein Vektor von den Zuständen s nachfolgenden Zuständen,

f_reward(...)

eine Funktion der Belohnung

T₀

ein Vektor von Zieldaten des neuronalen Netzes für die erste Iteration (i = 0) und

I

die Eingangsdaten für das neuronale Netz

bezeichnen. Variables of the learning algorithm are initialized: i = 0 r = f _reward (s, a, s') T ₀ = r I = {s, a} (4) in which

i

the iteration counter (ie a step i of the iteration),

r

a vector of rewards (alternatively available as a cost),

s

a vector of states,

a

a vector of actions,

s'

a vector of the states s subsequent states,

f _reward (...)

a function of reward

T ₀

a vector of target data of the neural network for the first iteration (i = 0) and

I

the input data for the neural network

describe.

The multi-rewards vector r can be estimated based on the vector of states s and actions a and the subsequent states s', since the function of the reward f _{reward is} often not explicitly known. The input data I may be considered as an amount of the data described above 111 and their associated actions 110 available.

Step 2:

An assignment of the input data I to the target data T _i is determined by means of a function Q _i on the basis of the FFNN: T _i = Q _i (I). (5)

Step 3:

mit γ ∊ [0, 1]. The target data of the next iteration T _{i + 1} are determined as follows:

with γ ε [0, 1].

The target data of the next iteration is determined by adding the reward r and an estimated additional reward Q _i for the action a 'in the state s'. The action a 'is determined from all available actions A as the action that causes the greatest reward Q _i .

The influence of the reward Q _i in comparison to the reward r can be set by means of a correction factor γ which lies for example in an interval between 0 and 1.

Steps 2 and 3 can be carried out iteratively until the iteration counter i reaches or exceeds a predefined threshold value i _max or until a predetermined termination condition is met.

The learning algorithm can also be replaced by an algorithm of an RCNN (Recurrent Control Neural Network), using the control strategy (see unit 105 ) can provide a continuous action for an estimated state. The continuous action does not necessarily have to be determined in advance since this algorithm is a generalization between actions. It is also possible to use a PGNRR (Policy Gradient Neuronal Rewards Regression) algorithm.

Since the wind speed at the turbine itself can not be measured accurately, it is not possible to determine the target function with the required accuracy. An autonomous learning approach is therefore proposed which does not require an explicit objective function and autonomously improves the target variable of a system, in this case the output of the wind turbine, by means of a learning algorithm.

2 shows a schematic arrangement with several wind turbines 101 that have a common control unit 201 are controllable. In that regard, the related to 1 described control unit 102 for a variety of wind turbines 101 Provide actions. Also, the data from the wind turbines 101 based on the control unit 201 be stored (see functionality of in 1 shown database 106 ). A central unit 202 in the example according to 2 the functionality of in 1 shown learning algorithm 107 ready.

The present solution is suitable for a combined operation of a plant, a system or a machine, e.g. at least one wind turbine, it being possible to switch between a first operating mode "efficient operation" and a second operating mode "gathering information regarding the state space of the system". In the second mode of operation, as discussed above, data is collected in response to possible actions (i.e., adjustments) to the plant. By means of targeted activation with given actions, it is thus possible to explore the state space of the installation and thus to increase a knowledge base, by means of which an improved (for example more efficient) operation of the installation is made possible. In the first operating mode, the system is controlled based on the determined information (and possibly further specified information). The system is controlled by setting its operating parameters.

The change between the first operating mode and the second operating mode can take place according to a predetermined fixed or according to a dynamic scheme. For example, a time base for controlling the system can be subdivided into time intervals. Now periodic, irregular or random time intervals can be used for the first or for the second operating mode. It is also possible that the change in efficiency serves as a measure to modify the duration or frequency of at least one of the modes of operation: e.g. As a result of the iterative implementation of the second operating mode, an increase in efficiency is scarcely possible or even no longer possible, the second operating mode can be activated correspondingly less frequently. In the other case, the gathering of information can be forced to further increase the efficiency. Also, the second mode of operation may be activated when the system need not be operated (efficiently) (e.g., when a wind turbine need not provide power) or in a fault mode - in which case at least data on the system may be collected.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• See "Markov Decision's Process Extraction Network", for example: S. Duell, A. Hans, and S. Udluft: The Markov Decision Process Extraction Network. In Proc. of the European Symposium on Artificial Neural Networks, 2010 [0021]
• "Neural Fitted Q Iteration", see: M. Riedmiller: Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method. In Proc. of the European Conf. on Machine Learning, 2005 [0027]
• "Recurrent Control Neural Network", see: AM Schaefer, S. Udluft, and H.-G. Zimmermann. A Recurrent Neural Network for Data Efficient Reinforcement Learning. In Proc. of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 2007 [0027]
• AM Schäfer, D. Schneegass, V. Sterzing, and S. Udluft. A Neural Reinforcement Learning Approach to Gas Turbine Control. International Joint Conference on Neural Networks, 2007 [0027]
• "Policy Gradient Neural Rewards Regression", see: D. Schneegass, S. Udluft, and Th. Martinetz. Improving Optimality of Neural Rewards Regression for Data-Efficient Batch Near-Optimal Policy Identification. In Proc. of the International Conf. on Artificial Neural Networks, 2007 [0027]

Claims (15)

Method for controlling a system - in which the system is controlled by a first action, - in which data of the plant are determined as a result of the control with the first action, In which a second action is determined on the basis of the data and the system is activated with the second action, - In which the first action and / or the second action is determined to explore parts of a state space of the plant. Method according to Claim 1, in which the first action and / or the second action is an action for which no data is yet available or for which the present data is obsolete. Method according to one of the preceding claims, wherein the first action and the second action comprises an adjustment of operating parameters of the plant. Method according to one of the preceding claims, in which the installation comprises at least one of the following components: - a technical system, An automation system, - a manufacturing plant, - a production plant, - a machine, - a wind turbine, - a gas turbine A turbomachine, - a power grid, A power distribution network, - a load distribution network, - a communication network. Method according to one of the preceding claims, in which the determined data comprise measured values, information and / or observations of the system. Method according to one of the preceding claims, in which the data are stored together with the associated first action. Method according to one of the preceding claims, in which the second action is determined on the basis of the data by determining a state, in particular a Markov state, by means of a state estimation. Method according to Claim 7, in which an optimized action is determined as the second action based on the state by means of a control strategy. Method according to one of claims 7 or 8, wherein based on data of the plant by means of a learning algorithm, a model of the plant is determined or modified and based on the model, the state estimation is performed. Method according to Claim 9, in which the model of the installation is determined by means of a neural network, a tree structure and / or Gaussian processes. Method according to one of claims 9 or 10, wherein based on data of the plant by means of the learning algorithm and the model of the plant, the control strategy for selecting the second action based on the state is set or modified. Method according to one of the preceding claims, wherein the system in a first operating mode and in a second operating mode is controllable, wherein the first operating mode indicates a normal operation of the system and the second operating mode denotes an operation of the system for generating the data. The method of claim 12, wherein in the second mode of operation, the system is controlled with a first or second action, which serves the targeted procurement of not yet existing data or current data. Device for controlling a system with a processing unit, which is set up such that - The system is controllable by means of a first action, - data of the system can be determined as a result of the control with the first action, A second action can be determined on the basis of the data and the installation can be activated with the second action, - The first action and / or the second action is determinable to determine parts of a state space of the plant. Apparatus according to claim 14 for controlling at least one wind turbine.

Images