JP7384311B1 - Driving support device, driving support method and program - Google Patents

Abstract

An object of the present invention is to provide a technology that can realize highly accurate operation support even when sufficient data regarding a target plant does not exist. [Solution] An operation support device according to one aspect of the present disclosure is an operation support device for supporting the operation of a target plant, and represents the state of the plant using operation performance data of plants other than the target plant. a pre-learning unit that learns a model that receives operating data as input and outputs recommended setting values for the plant; and a fine-tuning unit that fine-tunes the model learned by the pre-learning unit using operating performance data of the target plant. and a recommended setting value calculation unit that calculates recommended setting values using the model from the operation data of the target plant each time operation data is acquired from the target plant, and recommended settings calculated by the recommended setting value calculation unit. and a proposal unit that proposes the value to the operator of the target plant. [Selection diagram] Figure 3

Description

The present disclosure relates to a driving support device, a driving support method, and a program.

In plants such as combustion furnaces, stable operation is achieved by operators changing set values while taking past conditions into account, based on measured values such as the combustion state of the furnace. This places a heavy burden on the operator, and various techniques have been proposed to support the operator's operation. For example, Patent Document 1 describes a technique that uses a model created by a neural network to output recommended operations according to the operating state of a plant.

JP2019-159675A

However, depending on the plant targeted for operation support (hereinafter referred to as the target plant), there may be insufficient data representing the measured values, set values, etc. of the state of the target plant. For this reason, for example, the technique described in Patent Document 1 may not be able to create a highly accurate model.

The present disclosure has been made in view of the above points, and provides a technology that can realize highly accurate operation support even when there is insufficient data regarding the target plant.

An operation support device according to one aspect of the present disclosure is an operation support device for supporting the operation of a target plant, and inputs operation data representing the state of the plant using operation record data of plants other than the target plant. a pre-learning unit that learns a model that outputs recommended setting values for the plant as a target plant; a fine-tuning unit that fine-tunes the model learned by the pre-learning unit using operation record data of the target plant; a recommended setting value calculating section that calculates recommended setting values using the model from the operating data of the target plant each time operating data is acquired from the plant; It has a proposal department that makes proposals to plant operators.

Provided is a technology that can realize highly accurate operation support even when sufficient data regarding a target plant does not exist.

1 is a diagram showing an example of the overall configuration of a plant control system according to the present embodiment. 1 is a diagram illustrating an example of a hardware configuration of a driving support device according to an embodiment. 1 is a diagram illustrating an example of a functional configuration of a driving support device according to an embodiment. It is a flowchart which shows an example of offline processing concerning this embodiment. 3 is a flowchart illustrating an example of online processing according to the present embodiment. FIG. 3 is a diagram (part 1) showing an example of ranking of model outputs. FIG. 7 is a diagram (part 2) showing an example of ranking of model outputs. FIG. 1 is a diagram schematically showing a waste incineration plant in one embodiment. FIG. 3 is a diagram schematically showing recommended setting values for each model in an example. FIG. 3 is a diagram schematically showing operator setting values in one embodiment. FIG. 3 is a diagram schematically showing learning of a reward function in an example. FIG. 3 is a diagram schematically showing model learning in an example. FIG. 3 is a diagram schematically showing the results of model learning in an example.

An embodiment of the present invention will be described below. In the following embodiments, an operation support device 10 that can realize highly accurate operation support even when there is insufficient data regarding the target plant (measured values of the state of the target plant, set values of the target plant) is provided. The included plant control system 1 will be explained. Note that the target plant is a plant targeted for operational support by the operational support device 10.

Hereinafter, data composed of measured values of the state of the plant will be referred to as "operating data." The operation data is composed of measured values obtained by measuring physical quantities (for example, temperature, pressure, flow rate, gas concentration, etc.) representing various states of the plant using various sensors. That is, the operation data is multivariate data composed of variables (also called "state variables") representing each physical quantity.

In addition, in the following, a set of operating data at a certain time and a setting value set in the plant by an operator when the operating data was obtained (that is, a setting value at the relevant time) will be referred to as "plant operating performance data". I will call it. For example, when the operating data at time t is x _t and the setting value set in the plant by the operator when the operating data x _t was obtained is y _t , the plant operating performance data is (x _t , y _t ) It is expressed as Note that the driving data x _t at time t may be multivariate data representing time-series data of state variables measured between time t-1 and time t.

Furthermore, in the following, since we will use plant operation performance data of plants other than the target plant, we will use p' for the target plant and p∈[P ]. Here, P is the total number of plants other than the target plant.

<Example of overall configuration of plant control system 1>
FIG. 1 shows an example of the overall configuration of a plant control system 1 according to this embodiment. As shown in FIG. 1, the plant control system 1 according to the present embodiment includes an operation support device 10, an operator terminal 20, and a target plant 30. Here, the driving support device 10 and the operator terminal 20 are communicably connected via an arbitrary communication network. Similarly, the operator terminal 20 and the target plant 30 are communicably connected via any communication network, and the target plant 30 and the operation support device 10 are communicably connected via any communication network.

The operation support device 10 learns a model (hereinafter also referred to as a "recommended setting value calculation model") that calculates recommended setting values from operating data of the target plant 30 offline. In addition, each time the operation support device 10 acquires operational data from the target plant 30 online, the operation support device 10 calculates recommended setting values using the learned recommended setting value calculation model, proposes them to the operator, and operates the recommended setting value calculation model. Re-learn the basics. Here, the recommended setting value is a setting value recommended to the operator. Further, the recommended setting value calculation model is a machine learning model that calculates and outputs recommended setting values using operating data as input. Note that online refers to a state in which the target plant 30 is in operation. On the other hand, offline refers to a state other than online, and is, for example, a state before the target plant 30 starts operating or while the operation is stopped.

The operator terminal 20 is a variety of terminals (for example, a PC (personal computer), a smartphone, a tablet terminal, a wearable device, etc.) operated by an operator of the target plant 30. The operator refers to the recommended setting values proposed by the operation support device 10 and determines the setting values to actually be set in the target plant 30, and then operates the operator terminal 20 to set the setting values in the target plant 30. do. Thereby, the operation of the target plant 30 is controlled by the set value. Note that the set value refers to a value such as a manipulated variable set to the target plant 30.

The target plants 30 are various plants that are targets of operational support. Although the target plant 30 is not limited to a specific plant, an example is a garbage incineration plant. Further, the target plant 30 is not necessarily limited to a plant, and may be a supply and demand system such as an energy management system, for example.

Note that the overall configuration of the plant control system 1 shown in FIG. 1 is an example, and other configurations may be used. For example, the driving support device 10 and the operator terminal 20 may be integrated. Further, a plurality of operator terminals 20 may exist.

<Example of hardware configuration of driving support device 10>
FIG. 2 shows an example of the hardware configuration of the driving support device 10 according to this embodiment. As shown in FIG. 2, the driving support device 10 according to the present embodiment includes an input device 101, a display device 102, an external I/F 103, a communication I/F 104, a RAM (Random Access Memory) 105, and a ROM. (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is communicably connected via a bus 109.

The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, or the like. The display device 102 is, for example, a display, a display panel, or the like. Note that the driving support device 10 does not need to include at least one of the input device 101 and the display device 102.

The external I/F 103 is an interface with an external device such as the recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

Communication I/F 104 is an interface for connecting driving support device 10 to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily holds programs and data. The ROM 106 is a nonvolatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The auxiliary storage device 107 is, for example, a nonvolatile storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores programs and data. The processor 108 is, for example, various arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

Note that the hardware configuration shown in FIG. 2 is an example, and the driving support device 10 may have other hardware configurations. For example, the driving support device 10 may include multiple auxiliary storage devices 107 and multiple processors 108, or may include various hardware other than the illustrated hardware.

<Example of functional configuration of driving support device 10>
FIG. 3 shows an example of the functional configuration of the driving support device 10 according to this embodiment. As shown in FIG. 3, the driving support device 10 according to this embodiment includes an offline processing section 201 and an online processing section 202. Each of these units is realized, for example, by one or more programs installed in the driving support device 10 causing the processor 108 or the like to execute the process. Further, the operation support device 10 according to the present embodiment includes a plant operation record storage section 203 and a target plant operation record storage section 204. Each of these storage units is realized by a storage area of a storage device such as the auxiliary storage device 107, for example. Note that at least one of the plant operation record storage unit 203 and the target plant operation record storage unit 204 may be realized by a storage area of a storage device connected to the operation support device 10 via a communication network.

The offline processing unit 201 uses plant operation record data of plants other than the target plant 30 and past plant operation record data of the target plant 30 (hereinafter also referred to as "target plant operation record data") to perform offline processing. Learn the recommended setting value calculation model. Here, the offline processing section 201 includes a preliminary learning section 211 and a fine tuning section 212. The pre-learning unit 211 uses plant operation record data of plants other than the target plant 30 to pre-learn a recommended setting value calculation model. The fine-tuning unit 212 fine-tunes the pre-trained recommended setting value calculation model using target plant operation performance data. As a result, the pre-trained recommended setting value calculation model is fine-tuned for the target plant 30, and a learned recommended setting value calculation model is obtained.

Every time operating data is acquired online from the target plant 30, the online processing unit 202 calculates recommended setting values using the operating data and the learned recommended setting value calculation model, and also calculates the recommended setting values. Retrain the calculation model. Here, the online processing section 202 includes a recommended setting value calculation section 221, a proposal section 222, a reward function learning section 223, and a model learning section 224. The recommended setting value calculation unit 221 calculates recommended setting values using the operating data acquired from the target plant 30 and the learned recommended setting value calculation model. The proposal unit 222 transmits the recommended setting values calculated by the recommended setting value calculation unit 221 to the operator terminal 20. Thereby, the recommended setting value is proposed to the operator. The reward function learning unit 223 uses target plant operation performance data obtained while online to develop a reward function that outputs a higher reward as the recommended setting value is closer to the setting value actually set by the operator in the target plant 30. learn. The model learning unit 224 uses target plant operation performance data obtained while online and the reward function learned by the reward function learning unit 223 to maximize the reward calculated by the reward function. Retrain the recommended setting value calculation model.

The plant operation record storage unit 203 stores plant operation record data of plants other than the target plant 30. These plant operation performance data are given to the operation support device 10 when offline.

The target plant operation record storage unit 204 stores target plant operation record data of the target plant 30. These target plant operation performance data are given to the operation support device 10 when offline, and are created from operation data and set values collected when online. Note that it is assumed that the target plant operation performance data given to the operation support device 10 during offline is a small amount compared to the plant operation performance data stored in the plant operation performance storage unit 203. In other words, when offline, only an insufficient amount of target plant operation performance data is provided to learn an accurate recommended setting value calculation model.

<Offline processing>
The offline processing according to this embodiment will be described below with reference to FIG. 4. Note that offline processing is executed before online processing, which will be described later.

The pre-learning unit 211 of the offline processing unit 201 pre-learns a recommended setting value calculation model using plant operation record data of plants other than the target plant 30 (step S101). For example, the pre-learning unit 211 uses a known optimization method to calculate the parameters of the recommended setting value calculation model using the following equation (1). Thereby, a recommended setting value calculation model in which this parameter is set is obtained as a pre-trained recommended setting value calculation model.

ここで、

here,

はｎ番目の推奨設定値算出モデルである。また、ｗ_ｎ（ｎ∈［Ｎ］）はｎ番目の推奨設定値算出モデルのパラメータ、Ｎは推奨設定値算出モデルの総数である。ｔ_ｐはプラントｐの時刻インデックスを表す変数、Ｔ_ｐはプラントｐのプラント運転実績データの最終時刻を表す時刻インデックスである。なお、以下では、パラメータｗ_ｎを明示せずに、ｎ番目の推奨設定値算出モデルを単にｆ^ｎと記載することもある。また、時刻インデックスを単に時刻ともいう。

is the nth recommended setting value calculation model. Further, w _n (n∈[N]) is a parameter of the n-th recommended setting value calculation model, and N is the total number of recommended setting value calculation models. t _p is a variable representing the time index of the plant p, and T _p is a time index representing the final time of the plant operation performance data of the plant p. Note that hereinafter, the n-th recommended setting value calculation model may be simply written as f ⁿ without explicitly specifying the parameter w _n . Further, the time index is also simply referred to as time.

Also,

はプラントｐの時刻ｔ_ｐにおけるプラント運転実績データに含まれる運転データである。更に、

is the operation data included in the plant operation performance data of the plant p at time _tp . Furthermore,

はプラントｐの時刻ｔ_ｐにおけるプラント運転実績データに含まれる設定値である。上記の式（１）は、対象プラント３０以外の各プラントｐの各時刻ｔ_ｐ＝１～Ｔ_ｐにおける運転データから推奨設定値算出モデルによって算出された推奨設定値とそのときの実際の設定値との誤差を最小化するように、各推奨設定値算出モデルを学習することを意味している。

is a set value included in the plant operation performance data of the plant p at time _tp . The above equation (1) is based on the recommended setting value calculated by the recommended setting value calculation model from the operating data at each time t _p =1 to T _p of each plant p other than the target plant 30, and the actual setting value at that time. This means learning each recommended setting value calculation model so as to minimize the error between.

Note that N is an integer value of 1 or more, preferably 2 or more. This is because N recommended setting values each calculated by N recommended setting value calculation models are proposed to the operator, so if N≧2, the operator can compare multiple recommended setting values and This is because the actual set value can be determined.

Next, the fine-tuning unit 212 of the offline processing unit 201 fine-tunes the pre-learned recommended setting value calculation model obtained in step S101 above using the target plant operation performance data (step S102). For example, the fine tuning unit 212 uses a known optimization method to calculate the parameters of the recommended setting value calculation model using the following equation (2). Thereby, a recommended setting value calculation model in which this parameter is set is obtained as a learned recommended setting value calculation model.

ここで、ｔ_ｐ'は対象プラント３０の時刻インデックスを表す変数、－Ｔ_ｐ' ^１は対象プラント３０の対象プラント運転実績データの最初の時刻を表す時刻インデックスである。また、

Here, t _p' is a variable representing the time index of the target plant 30, and -T _p' ¹ is a time index representing the first time of the target plant operation performance data of the target plant 30. Also,

は対象プラント３０の時刻ｔ_ｐ'における対象プラント運転実績データに含まれる運転データである。更に、

is the operation data included in the target plant operation performance data at time tp _' of the target plant 30. Furthermore,

は対象プラント３０の時刻ｔ_ｐ'における対象プラント運転実績データに含まれる設定値である。上記の式（２）は、対象プラント３０の各時刻ｔ_ｐ'＝－Ｔ_ｐ' ^１～０における運転データから推奨設定値算出モデルによって算出された推奨設定値とそのときの実際の設定値との誤差を最小化するように、各推奨設定値算出モデルを学習することを意味している。

is a setting value included in the target plant operation performance data of the target plant 30 at time tp _' . The above equation (2) is based on the recommended setting value calculated by the recommended setting value calculation model from the operating data at each time t _p' = -T _p' ¹ to 0 of the target plant 30 and the actual setting value at that time. This means that each recommended setting value calculation model is learned so as to minimize the error.

As described above, in the operation support device 10 according to the present embodiment, after pre-learning the recommended setting value calculation model using the plant operation record data of plants other than the target plant 30, the target plant operation record data of the target plant 30 is used. Fine-tune the recommended setting value calculation model using As a result, the knowledge of plants other than the target plant 30 is transferred to the recommended setting value calculation model. Even if there is, a highly accurate recommended setting value calculation model can be obtained.

<Online processing>
Online processing according to this embodiment will be described below with reference to FIG. 5. Steps S201 to S205 in FIG. 5 are repeatedly executed every time operation data is acquired from the target plant 30. Hereinafter, assuming that the online start time is t _p' = 1, target plant operation performance data from time t _p' = 1 to T _p' ² -1 is stored in the target plant operation performance storage unit 204. A case will be described in which the operation data of t _p' =T _p' ² is acquired from the target plant 30.

The recommended setting value calculation unit 221 of the online processing unit 202 calculates recommended setting values using the operating data acquired from the target plant 30 and the learned recommended setting value calculation model (step S201). That is, the recommended setting value calculation unit 221 calculates, for n=1,...,N,

によりＮ個の推奨設定値を算出する。なお、ｗ_ｎは、オフライン処理が実行された直後の場合は上記のステップＳ１０２で算出された値であり、一方で後述するステップＳ２０５が実行された後は当該ステップＳ２０５で更新された値である。

N recommended setting values are calculated. Note that w _n is the value calculated in step S102 above immediately after offline processing is executed, and on the other hand, after step S205 described below is executed, it is the value updated in step S205. .

Next, the proposal unit 222 of the online processing unit 202 transmits the N recommended setting values calculated in step S201 above to the operator terminal 20 (step S202). As a result, these N recommended setting values are proposed to the operator.

Hereinafter, it is assumed that the operator refers to the N recommended setting values and determines the setting values to be actually set in the target plant 30, and then operates the operator terminal 20 to set the setting values in the target plant 30. do. Further, it is assumed that the set value is transmitted from the operator terminal 20 to the driving support device 10 as a set value at time t _p' =T _p' ² . As a result, the set of the operating data at time t _p' = T _p' ² and the set value at time t _p' = T _p' ² is targeted as the target plant operation performance data at time t _p' = T _p' ² . It is stored in the plant operation record storage unit 204.

Next, the reward function learning unit 223 of the online processing unit 202 uses the target plant operation performance data obtained while online (that is, the target plant operation performance data from time t _p' = 1 to T _p' ² ). Then, the reward function is learned (step S203). For example, the reward function learning unit 223 uses a known optimization method to calculate the parameters of the reward function using the following equation (3). Thereby, a reward function in which this parameter is set is obtained as a learned reward function.

ここで、ｒ_θは報酬関数、θは報酬関数のパラメータである。報酬関数ｒ_θは、運転データと設定値（又は推奨設定値）とを入力として、報酬を表す値を出力する関数である。上記の式（３）により、推奨設定値が、オペレータが対象プラント３０に実際に設定した設定値と近いほど高い報酬を出力する報酬関数ｒ_θを獲得できる。

Here, r _θ is a reward function, and θ is a parameter of the reward function. The reward function r _θ is a function that receives driving data and a set value (or recommended set value) as input, and outputs a value representing the reward. Using the above equation (3), it is possible to obtain a reward function r _θ that outputs a higher reward as the recommended setting value is closer to the setting value actually set by the operator in the target plant 30.

Next, the model learning unit 224 of the online processing unit 202 uses the target plant operation performance data obtained while online (that is, the target plant operation performance data from time t _p' = 1 to T _p' ² ) and the above-mentioned The recommended setting value calculation model is retrained using the reward function learned in step S204 (step S204). For example, the model learning unit 224 sets φ _n ←w _n for n = 1, ..., N, and then uses the existing optimization method or reinforcement learning method to calculate the following equation (4 ) to calculate the parameter φ _n of the recommended setting value calculation model. As a result, a recommended setting value calculation model in which this parameter φ _n is set is obtained as a relearned recommended setting value calculation model.

上記の（４）により、ｎ＝１，・・・，Ｎに対して、報酬を最大化するような推奨設定値算出モデルｆ^ｎ（つまり、オペレータの実際の運転を模擬するような推奨設定値算出モデルｆ^ｎ）が獲得できる。

According to (4) above, for n = 1, ..., N, a recommended setting value calculation model f ⁿ that maximizes the reward (that is, a recommended setting value that simulates the actual operation of the operator A calculation model f ⁿ ) can be obtained.

Then, the model learning unit 224 of the online processing unit 202 updates w _n ←φ _n for n=1, . . . , N (step S205). Note that φ _n is the parameter calculated in step S204 above. As a result, after new operation data is acquired from the target plant 30, the processes from step S201 onwards are executed using the parameters w _n updated in this step.

As described above, in the operation support device 10 according to the present embodiment, each time operation data is acquired while online, recommended setting values are proposed to the operator, and setting values actually set in the target plant 30 are proposed to the operator. The reward function and recommended setting value calculation model are dynamically learned using the method. As a result, a recommended setting value calculation model that accurately simulates the operator's driving is obtained, and highly accurate recommended setting values can be proposed to the operator.

Note that in the online process shown in Figure 5, the reward function and recommended setting value calculation model were learned in each iteration, but for example, after a certain predetermined condition is met, the reward function and recommended setting value calculation model are not learned. Alternatively, only steps S201 and S202 may be executed. Examples of such conditions include, for example, when the number of repetitions of steps S201 to S205 exceeds a predetermined number of times (including the case of 1 time), when each parameter w _n converges, etc.

<Other examples of reward function learning methods>
Another example of the reward function learning method in step S203 of FIG. 5 will be described below.

・Other examples of reward function learning method part 1
Instead of the above equation (3), the parameters of the reward function may be calculated using the following equation (5).

ここで、σはシグモイド関数である。上記の式（５）により、より安定した学習が期待できる。

Here, σ is a sigmoid function. More stable learning can be expected with the above equation (5).

・Other example of reward function learning method part 2
At each time t _p', for each combination of two arbitrary recommended setting value calculation models, the outputs (that is, recommended setting values) of the two recommended setting value calculation models are ranked based on a certain predetermined standard, and a reward function is calculated. You can also learn.

More specifically, when comparing the outputs of any two recommended setting value calculation models at each time t _p', the recommended setting value with a higher rank is f ^good , and the recommended setting value with a lower rank is f good As ^bad , an index d for identifying the pair of f ^good and f ^bad is sequentially assigned starting from 1. Then, using {(d, f ^good , f ^bad ) | d=1,...,D} (where D is the total number of pairs of f ^good and f ^bad ), the reward is calculated by the following formula (6). Calculate the parameters of the function.

上記の式（６）により、オペレータが対象プラント３０に実際に設定した設定値を用いることなく、各時刻ｔ_ｐ'における任意の２つの推奨設定値算出モデルの順位付けのみで報酬関数を学習することができる。

According to the above equation (6), the reward function is learned only by ranking two arbitrary recommended setting value calculation models at each time _tp' , without using the setting values actually set by the operator in the target plant 30. be able to.

Here, various criteria can be adopted as the criteria for ranking, but for example, it is conceivable to rank the one with a larger recommended setting value as the higher one.

As an example, if N=3, the higher the recommended setting value, the higher the rank. At this time, (d, f ^good , f ^bad ) in the case where f ² > f ³ > f ¹ when t p _' = 1 and f ¹ > f ³ > f ² when t _p' = 2 is shown in Fig. 6. . As shown in FIG. 6, when t _p' = 1, since f ² > f ³ > f ¹ , (d, f ^good , f ^bad ) = (1, f ² , f ¹ ), (2, f ³ , f ¹ ), (3, f ² , f ³ ). Similarly, when t _p' = 2, since f ¹ > f ³ > f ² , (d, f ^good , f ^bad ) = (4, f ¹ , f ² ), (5, f ¹ , f ³ ), (6, f ³ , f ² ).

・Other example of reward function learning method part 3
In the second example of the reward function learning method described above, it may be possible to allow the output rankings of the two recommended setting value calculation models to be the same. Furthermore, only the outputs of two specific recommended setting value calculation models may be ranked at a certain time tp _' (in other words, a combination of recommended setting value calculation models that cannot be ranked at a certain time tp _' may be ).

At this time, the parameters of the reward function may be calculated using the following equation (7) using {(d, a, f ^good , f ^bad ) | d=1, . . . , D}.

ここで、ａは同一時刻ｔ_ｐ'で得られたｆ^ｇｏｏｄとｆ^ｂａｄの組の数の逆数であり、各時刻ｔ_ｐ'でｆ^ｇｏｏｄに関する報酬とｆ^ｂａｄに関する報酬との差の期待値（平均）を取るためのパラメータである。

Here, a is the reciprocal of the number of pairs of f ^good and ^{f bad} obtained at the same time t _{p' ,} and the expected value ⁽ This is a parameter for taking the average).

As an example, if N=3, the higher the recommended setting value, the higher the rank. At this time, f ² > f ³ = f ¹ at t _p' = 1, f ¹ > f ³ > f 2 at t _p' = ² , and only f ¹ and f ² are ranked at t _p' = 3. FIG. 7 shows (d, a, f ^good , f ^bad ) in the case where f ¹ >f ² and f 1 can be attached. As shown in FIG. 7, when t _p' = 1, f ² > f ³ = f ¹ , so (d, a, f ^good , f ^bad ) = (1,1/2, f ² , f ¹ ), (2,1/2, f ² , f ³ ). Similarly, when t _p' = 2, since f ¹ > f ³ > f ² , (d, a, f ^good , f ^bad ) = (3,1/3, f ¹ , f ² ), ( 4,1/3, f ¹ , f ³ ), (5,1/3, f ³ , f ² ). Furthermore, when t _p' = 3, only f ¹ and f ² can be ranked and f ¹ > f ² , so (d, a, f ^good , f ^bad ) = (6, 1, f ¹ , f ² ).

<Other examples of learning methods for recommended setting value calculation model>
Hereinafter, another example of the method for relearning the recommended setting value calculation model in step S204 of FIG. 5 will be described.

・Learning method for recommended setting value calculation model part 1
When relearning the recommended setting value calculation model, use virtual operating data and the recommendations obtained by inputting that virtual operating data into the recommended setting value calculation model, rather than actual operating data and recommended setting values. A set value may also be used. At this time, the virtual driving data may be created by, for example, sampling from the distribution of actual driving data.

・Learning method for recommended setting value calculation model part 2

The parameter φ _n of the recommended setting value calculation model may be calculated using the following equation (8) instead of the above equation (4), where x is the distribution of actual operating data.

ここで、ＫＬはカルバック・ライブラー情報量、Ｌは対数尤度である。また、β及びγは予め決められたパラメータ（ハイパーパラメータ）である。

Here, KL is the Kullback-Leibler information amount, and L is the log likelihood. Further, β and γ are predetermined parameters (hyper parameters).

<Example>
An example of this embodiment will be described below.

In this embodiment, a garbage incineration plant is assumed as the target plant 30. A schematic diagram of the waste incineration plant is shown in Figure 8. As shown in Figure 8, in a waste incineration plant, waste and air are input into a combustion furnace, and the heat generated by the combustion is converted into steam, and steam and exhaust gases such as carbon monoxide (CO) are output. Ru. Since steam is generally used for power generation, etc., it is required to increase the amount of steam generated and to stabilize it. On the other hand, in order to increase the amount of steam produced, it is necessary to increase the amount of waste input and the flow rate of air, which may lead to incomplete combustion and, as a result, higher CO concentrations. For this reason, it is necessary to appropriately control the amount of garbage input and the flow rate of air. Note that the amount of garbage input is controlled by the operating speed of equipment called a feeder, and the air flow rate is controlled by the opening/closing angle of a valve or the like. Further, the steam flow rate and the exhaust gas concentration (CO concentration) are measured by a sensor or the like.

Therefore, the state variables of the waste incineration plant are x ₁ : feeder speed, x ₂ : air flow rate, x ₃ : steam flow rate, and x ₄ : exhaust gas concentration. In this embodiment, the operation data at time t _p' represents time series data of each state variable x ₁ , x ₂ , x ₃ , x ₄ between time t _p' -1 and time t _p'. shall be. In addition, as recommended setting value calculation models, two models f ¹ and f ² are assumed, which calculate time series data of recommended setting values x ₁ and x ₂ at future times using operating data at time t _{p' as} input. do.

FIG. 9 shows the input and output of models f ¹ and f ² that were pre-trained and fine-tuned in this example. As shown in FIG. 9, the models f ¹ and f ² are a time series of recommended setting values x ₁ and x ₂ (that is, feeder speed and air flow rate) at future times using the operating data at time t _p' as input. Calculate and output data. As shown in FIG. 9, model f ¹ calculates recommended setting values that maintain the past settings, while model f ² calculates recommended settings that increase the past settings.

FIG. 10 shows the setting values set in the waste incineration plant by the operator in this embodiment. As shown in Fig. 10, the operator selects an operation that reduces both the feeder speed and the air flow rate based on his own experience, referring to the recommended setting values of model ^f1 and ^f2 . ing.

FIG. 11 shows how the reward function r _θ is learned in this example. As shown in FIG. 11, in learning the reward function r _θ , the parameter θ of the reward function is learned such that the reward is higher at the set value actually set by the operator.

FIG. 12 shows how the recommended setting value calculation model ^f1 is retrained in this embodiment. As shown in FIG. 12, the parameter φ ₁ of the model f ₁ is learned using the reward function r _θ with the initial value of the parameter φ ¹ set to w ₁ . Similarly, regarding the recommended setting value calculation model f ² , the parameter φ _{2 of} the model f 2 is learned using the reward function r _θ after setting the initial value of the parameter φ ² to w ₂ .

FIG. 13 shows the results of learning the recommended setting value calculation models f ¹ and f ² in this embodiment. As shown in Figure 13, since relearning is performed online, when operating data representing the same condition is obtained thereafter, model ^f1 can calculate recommended settings close to the settings actually set by the operator. It becomes like this. It can be seen that this allows a recommended setting value calculation model that simulates the actual operation of the operator to be obtained.

<Summary>
As described above, the operation support device 10 according to the present embodiment pre-learns the recommended setting value calculation model offline using the plant operation record data of plants other than the target plant 30, and then Fine-tune the recommended setting value calculation model using operational performance data. Further, the operation support device 10 according to the present embodiment not only proposes recommended setting values calculated by the recommended setting value calculation model to the operator online every time operation data is acquired, but also proposes recommended settings values calculated by the recommended setting value calculation model to the operator. A reward function and a recommended setting value calculation model are dynamically learned using the set setting values.

Thereby, in the operation support device 10 according to the present embodiment, even if there is only a small amount of target plant operation performance data, an accurate recommended setting value calculation model can be obtained off-line. In addition, in the operation support device 10 according to the present embodiment, since the recommended setting value calculation model is re-learned online from the setting values actually set in the target plant 30, the recommended settings that accurately simulate the operator's actual operation are performed. A value calculation model can be obtained. Furthermore, in the driving support device 10 according to the present embodiment, relearning of the recommended setting value calculation model can be stabilized by estimating the reward function online.

The present invention is not limited to the above-described embodiments specifically disclosed, and various modifications and changes, combinations with known techniques, etc. are possible without departing from the scope of the claims. .

1 Plant control system 10 Operation support device 20 Operator terminal 30 Target plant 101 Input device 102 Display device 103 External I/F
103a Recording medium 104 Communication I/F
105 RAM
106 ROM
107 Auxiliary storage device 108 Processor 109 Bus 201 Offline processing unit 202 Online processing unit 203 Plant operation record storage unit 204 Target plant operation record storage unit 211 Preliminary learning unit 212 Fine tuning unit 221 Recommended setting value calculation unit 222 Proposal unit 223 Reward function learning Section 224 Model learning section

Claims (10)

An operation support device for supporting the operation of a target plant,
a pre-learning unit that uses operating record data of plants other than the target plant to learn a model that receives operating data representing the state of the plant as input and outputs recommended setting values for the plant;
a fine-tuning unit that fine-tunes the model learned by the preliminary learning unit using operation record data of the target plant;
a recommended setting value calculation unit that calculates a recommended setting value using the model from the operation data of the target plant each time operation data is acquired from the target plant;
a proposal unit that proposes the recommended setting values calculated by the recommended setting value calculation unit to the operator of the target plant;
The operating data acquired from the target plant, the recommended setting values calculated by inputting the operational data acquired from the target plant into the model, and the settings actually set in the target plant by the operator of the target plant. Using a reward function that has been learned to output a higher reward as the recommended setting value is closer to the setting value, the model is re-run so as to maximize the sum of rewards output by the reward function. A model learning section for learning,
A driving support device with The report is generated using the operating data acquired from the target plant, the recommended setting values calculated by the recommended setting value calculation unit, and the setting values actually set in the target plant by the operator of the target plant. a reward function learning unit that learns a reward function ;
The driving support device according to claim 1, comprising: The model learning section includes:
Instead of the operation data acquired from the target plant, virtual operation data sampled from the distribution of the operation data acquired from the target plant is used, and the operation data acquired from the target plant is applied to the model. 3. The model is re-learned using the recommended setting values calculated by inputting the virtual driving data to the model instead of the recommended setting values calculated by inputting the virtual driving data to the model. Driving support equipment. The model learning section includes:
Further, the model is re-trained so as to minimize the amount of Kullback-Leibler information between the distributions of the driving data before and after re-learning the model, and to maximize the log likelihood of the model. The driving support device according to item 2. The reward function learning unit is
the difference between the reward calculated by the reward function from the operation data acquired from the target plant and the recommended setting value, and the reward calculated by the reward function from the operation data acquired from the target plant and the setting value; The driving support device according to any one of claims 2 to 4, which learns the reward function so as to maximize. The reward function learning unit is
The driving support device according to claim 5, wherein the reward function is learned so as to maximize a value obtained by taking the logarithm of the sigmoid function value of the difference. The reward function learning unit is
Ranking recommended setting values calculated by the models for a combination of two of the plurality of models,
Learning the reward function so as to maximize the difference between the reward calculated by the reward function from recommended setting values with a high ranking and the reward calculated by the reward function from recommended setting values with a low ranking. The driving support device according to any one of items 2 to 4. The reward function learning unit is
The driving support device according to claim 7, wherein the reward function is learned so as to maximize an expected value or an average of the differences. The operation support equipment to support the operation of the target plant is
a pre-learning procedure for learning a model that uses operating record data of plants other than the target plant to input operating data representing the state of the plant and outputs recommended setting values for the plant;
a fine-tuning procedure of fine-tuning the model learned by the pre-learning procedure using operating performance data of the target plant;
a recommended setting value calculation procedure of calculating a recommended setting value using the model from the operation data of the target plant each time operation data is acquired from the target plant;
a proposing step of proposing the recommended setting value calculated by the recommended setting value calculating step to the operator of the target plant;
The operating data acquired from the target plant, the recommended setting values calculated by inputting the operational data acquired from the target plant into the model, and the settings actually set in the target plant by the operator of the target plant. Using a reward function that has been learned to output a higher reward as the recommended setting value is closer to the setting value, the model is re-run so as to maximize the sum of rewards output by the reward function. A model learning procedure to be learned;
A driving assistance method that performs. Operation support equipment to support the operation of the target plant,
a pre-learning procedure for learning a model that uses operating record data of plants other than the target plant to input operating data representing the state of the plant and outputs recommended setting values for the plant;
a fine-tuning procedure of fine-tuning the model learned by the pre-learning procedure using operating performance data of the target plant;
a recommended setting value calculation procedure of calculating a recommended setting value using the model from the operation data of the target plant each time operation data is acquired from the target plant;
a proposing step of proposing the recommended setting value calculated by the recommended setting value calculating step to the operator of the target plant;
The operating data acquired from the target plant, the recommended setting values calculated by inputting the operational data acquired from the target plant into the model, and the settings actually set in the target plant by the operator of the target plant. Using a reward function that has been learned to output a higher reward as the recommended setting value is closer to the setting value, the model is re-run so as to maximize the sum of rewards output by the reward function. A model learning procedure to be learned;
A program to run.

Images