JP4546332B2 - Driving support device and driving support method - Google Patents

Description

  The present invention relates to an operation support apparatus and an operation support method that support adjustment of control parameters in a plant control apparatus.

  An example of a plant controlled using a control device is a thermal power plant. Thermal power plants are required to perform load change operation according to electric power demand, and fluctuations in process values such as steam temperature are severely restricted even during this load change operation. In particular, fluctuations in steam temperature and steam pressure at the turbine inlet can be a factor that degrades the performance of the turbine, and the limit values for these process value fluctuations are particularly severe.

  The control device of a thermal power plant is required to suppress the process value within the limit range even during such load change operation, and is required to appropriately set various operation amounts. Since the value of the operation amount depends on the value of the control parameter of the control logic mounted on the control device, it is necessary to appropriately set the value of the control parameter in order to suppress the variation of the process value.

This control parameter is generally adjusted by an operator based on his / her knowledge and experience using measured values from the plant. However, in recent years, problems such as a shortage of skilled operators have occurred, and a technique for supporting control parameter adjustment is required.
In the control of a thermal power plant, fluctuations in process values are suppressed mainly by feedback control and feedforward control. In particular, a thermal power plant has a large response delay, and feedforward control is an effective control method for suppressing process value fluctuations accompanying load changes. As a method for automatically calculating an appropriate control parameter value in the feedforward control, for example, as described in Japanese Patent Application Laid-Open No. 11-242503, the control parameter value is calculated based on the characteristic amount of the response waveform of the plant process value. It is known that a neural network is used for plant behavior estimation.

Japanese Patent Laid-Open No. 11-242503

  When controlling a controlled object in which safe operation is important as in thermal power plant control, it is not desirable for safety to operate the plant using all of the automatically calculated control parameter values as they are. For the sake of safety, the operator confirms the validity of the automatically calculated control parameter value, and the automatically calculated control parameter value is used only when the control parameter value is not far from the control parameter value considered by the operator. It is effective to follow the procedure of applying to In this case, if the operator's control system adjustment policy and the automatic calculation policy are different, the automatically calculated control parameter value is outside the operator's expected range, and the automatically calculated control parameter value is reflected in the control device. Can not. In order to avoid this, it is necessary to match the control parameter adjustment policy of the operator with the automatic calculation policy in advance.

  In recent years, the reinforcement learning theory described in “Reinforcement Learning, Sadayoshi Mikami, Masaaki Minagawa, Morikita Publishing Co., Ltd., published on December 20, 2000” has attracted attention as an autonomous learning method using computers. ing. Reinforcement learning is a learning system that autonomously acquires an action decision strategy for an agent to achieve a predetermined goal through interaction with the environment using special information such as rewards. Since this method has the feature that the reward can be set freely, it is easy to give a reward that matches the control parameter adjustment policy of the skilled operator. Therefore, the possibility that the control parameter value close to the operator's assumption can be automatically calculated by using this method increases.

  As described above, the agent tries to act on the environment by trial and error, and learns the reward obtained by the behavior as a clue. For safety reasons, it is not desirable for the agent to adjust the control parameters directly on a thermal power plant by trial and error. Therefore, the agent learns in advance for a simulator that simulates the characteristics of the plant, and uses this learning result to guide the control parameter adjustment of the plant control apparatus. However, if the simulator characteristics and the plant characteristics are different, the agent may not be able to provide appropriate control system adjustment guidance.

  An object of the present invention is to provide a driving support apparatus and a driving support method capable of providing appropriate control parameter adjustment guidance even when there is an error in the characteristics of the simulator and the characteristics of the plant used by the agent in advance learning. is there.

(1) Moreover, in order to achieve the said objective, this invention is a plant which consists of the plant data obtained from a plant, the plant model which simulated the operating characteristic of the said plant, and the control model which simulated the control circuit of the said plant A driving support apparatus having parameter correction means for correcting a control parameter value of a control circuit using a simulator data calculated using a simulator as an input, and using a deviation between a predetermined target process value and its target value as at least one evaluation index The parameter correction means includes a parameter correction method learning means for learning a direction and an amount of correction of the control parameter based on a change amount of the evaluation index value with respect to the correction result of the control parameter, a simulator, Difference in response characteristics of process values to changes in operating quantity with actual plant And a guidance means for displaying the process value of the plant or its characteristic value, or an increase or decrease amount of the control parameter value on a display screen. It is.
With this configuration, even when there is an error between the characteristics of the simulator used by the agent for the prior learning and the characteristics of the plant, appropriate control parameter adjustment guidance can be provided.

( 2 ) In the above ( 1 ), preferably, the correction means sets the correction amount of the control parameter based on a difference in response characteristics of process values when the operation amount is changed stepwise between the simulator and the actual plant. It is something to change.

( 3 ) In the above ( 1 ), preferably, the plant is a thermal power plant, and the parameter correction means is configured to generate a deviation between a power generation output, a steam temperature, a steam pressure, a pipe metal temperature, and a target value corresponding to each. The control parameter value of the control circuit is corrected using at least one of them as an evaluation index.

(4) Moreover, in order to achieve the said objective, this invention is a plant which consists of the plant data obtained from a plant, the plant model which simulated the operating characteristic of the said plant, and the control model which simulated the control circuit of the said plant A driving support method for correcting a control parameter value of a control circuit using a simulator data calculated by using a simulator as an input and using a deviation between a predetermined target process value and a target value as at least one evaluation index. The direction of the increase or decrease of the control parameter and the correction amount are learned based on the change amount of the evaluation index value with respect to the correction result of the process, and based on the difference in the response characteristic of the process value to the change in the operation amount between the simulator and the actual plant Change the correction amount of the control parameter, and the process value of the plant or its characteristic amount Or it is obtained so as to display on the display screen the increase or decrease in the control parameter value.
This method can provide appropriate control parameter adjustment guidance even when there is an error between the characteristics of the simulator used for the prior learning by the agent and the characteristics of the plant.

  According to the present invention, appropriate control parameter adjustment guidance can be provided even when there is an error between the characteristics of the simulator used by the agent for the prior learning and the characteristics of the plant.

Hereinafter, the configuration and operation of the driving support apparatus according to the embodiment of the present invention will be described with reference to FIGS. Here, the case where the present invention is applied to a thermal power plant will be described as an example, but the object can also be applied to other plants such as a nuclear power plant.
First, the configuration and operation of a control system when the operation support apparatus according to the present embodiment is applied to a plant will be described with reference to FIG.
FIG. 1 is a system block diagram showing a configuration of a control system when an operation support apparatus according to an embodiment of the present invention is applied to a plant.

  A control system that controls the plant 100 includes a control device 200, a maintenance tool 300, an operation support device 400, and an input device 700.

  The control device 200 takes in the measurement signal from the plant 100 by the external input interface 210 and calculates and generates various control command signals by the calculation unit 220 while storing the received signal in the storage unit 230 as necessary. By transmitting this control command signal to the plant 100 via the external output interface 240, the plant 100 is operated so as to obtain a desired performance. In addition, an external input device 700 including a keyboard and a mouse and an image display device 710 are connected to the control device 200 and function as an interface with the driver.

  The maintenance tool 300 has a function of generating a control logic used for calculating a control command value from a measured value from the plant 100 by the calculation unit 220. The control system designer creates a control logic drawing using the external input device 700 while referring to the control logic creation screen displayed on the image display device 710. The control logic generation unit 320 generates a control logic program using design information input from the external input device 700, and this program is stored in the control logic data 330. The arithmetic unit 220 refers to the control logic data 330 and generates a control command value.

  The control system automatic learning system 400 includes an external input interface 410, an operation determination unit 420, magnetic disks 430 and 440, a model adjustment unit 450, an external output interface 460, a simulator 500, and an agent 600. Yes. The agent 600 is a control parameter adjustment unit.

  The operation determination unit 420 is connected to the model adjustment unit 450, the simulator 500, the ON / OFF of the agent 600, and the connection points of the changeover switches 510, 530, 610, 620, 630 (a Or b) is set. Mode contents, connection relations, data contents stored in the magnetic disks 430 and 440, and operations of the model adjustment unit 450, the simulator 500, and the agent 600 will be described later.

  In FIG. 1, the control device 200, the maintenance tool 300, and the driving support device 400 are described as being directly connected by a cable, but the configuration is such that data is transmitted / received via optical communication or the Internet. In some cases. In addition, some functions of the control device, the maintenance tool, and the driving support device may be separated and placed in another place, and data may be transmitted / received via the Internet or the like.

Next, the configuration of the plant supported by the operation support apparatus according to the present embodiment will be described with reference to FIG. Here, the configuration of the thermal power plant is shown.
FIG. 2 is a system block diagram illustrating a configuration of a plant supported by the operation support apparatus according to the embodiment of the present invention. The same reference numerals as those in FIG. 1 indicate the same parts.

  In the thermal power plant 100 shown in FIG. 2, fuel such as coal and biomass and air used for fuel transfer and combustion adjustment are burned to a furnace as a heat source of a boiler 130 that heats supply water and generates steam. The fuel is supplied from 140, and fuel and air are burned in a furnace to generate high-temperature gas. The gas that has passed through the boiler 130 is sent to the exhaust gas treatment device 170, and after removing harmful substances contained therein, it is released to the atmosphere by the chimney 180.

  Further, water is circulated and supplied to the boiler 130 by a water supply pump 124. The vapor that has been heated and evaporated at the furnace water wall 131 is further heated by the gas passing through the flue portion of the boiler 130 while passing through the superheater 132, and the temperature is increased and the pressure is increased. The high-temperature and high-pressure main steam is guided to the turbine 150 through the turbine control valve 123 to drive the turbine 150. The shaft power of the turbine 150 is transmitted to the generator 160 and converted into electrical energy by the generator 160. The steam that has passed through the turbine 150 is again guided to the boiler 130 and reheated by the reheater 133. This reheated steam is also guided to the turbine 150 to drive the turbine 150 and generate electric power. The steam that has passed through the turbine 150 is cooled with cooling water and passed ascites when passing through the condenser 190, and the dust that has passed through the condenser 190 passes through the water supply pump 124 and is supplied to the boiler 130 again.

  Although not shown in FIG. 2, the superheater and the reheater may be arranged in a plurality of boilers. In some cases, a desuperheater is disposed in a path through which steam passes.

  The operating state of the plant 100 is measured by a data measuring device such as the pressure measuring device 111, the temperature measuring devices 112 and 113, the output measuring device 114, and the flow measuring device 115. Although not shown, the plant 100 is also equipped with other data measuring devices for measuring various types of process, and the data measured by these data measuring devices is sent to the external output interface 110. Via the control device 200 and the driving support device 400.

  The control command value from the control device 200 is input via the external input interface 120, controls the opening degree of the valve 121, controls the amount of air supplied to the burner 140, and opens the opening degree of the valve 122. To control the amount of fuel supplied to the burner 140, and further control the amount of water supplied by the water supply pump 124.

Next, the configuration and operation of the driving support apparatus according to the present embodiment will be described with reference to FIGS.
First, the mode of data stored in the magnetic disk 430 in the driving support apparatus according to the present embodiment will be described with reference to FIG.
FIG. 3 is an explanatory diagram showing a state of data stored in the magnetic disk in the driving support apparatus according to the embodiment of the present invention.

  As shown in FIG. 3, measurement data from the plant 100 is stored on the magnetic disk 430 together with each measurement time for each measuring instrument number. For example, the pressure value P111, temperature values T112, T113, output value E114, and flow rate value F115 measured by the steam pressure gauge 111, steam thermometer 112, 113, output measuring instrument 114, and flow rate measuring instrument 115 in FIG. , Saved with time data. The magnetic disk 440 stores the results calculated by the simulator 500.

Next, the four modes of the driving support apparatus according to the present embodiment will be described with reference to FIGS. 4 and 5.
FIG. 4 is an explanatory diagram of modes of the driving support apparatus according to the embodiment of the present invention. FIG. 5 is an explanatory diagram of the connection relationship between the model adjustment unit, the simulator, the agent ON / OFF, and the switch in each mode of the driving support device according to the embodiment of the present invention.

  The driving support device 400 has four functions (modes). The mode of the driving support device 400 is selected by the driver with reference to the screen 711 shown in FIG. 4 displayed on the image display device 710. The driving support device 400 has four modes: A) control system verification mode, B) plant model adjustment mode, C) control system adjustment guidance display mode, and D) actual machine learning mode. When the driver selects the buttons 712, 713, 714, 715 displayed on the screen 711, the mode of the driving support device 400 is switched.

  FIG. 5 is a table showing the connection between the four modes, the model adjustment unit 450, the simulator 500, the ON / OFF state of the agent 600, and the switches 510, 530, 610, 620, and 660. For example, when mode A is selected, the model adjustment unit 450 is OFF, the simulator 500 is ON, and the agent 600 is ON. The switches 510, 530, 610, 620, and 660 are connected to b, b, a, a, and a, respectively. In accordance with this table, the operation determination unit 420 generates a signal for switching the ON / OFF of the model adjustment unit 450, the simulator 500, and the agent 600 and the connection relationship of the switches 510, 530, 610, 620, and 660, and transmits the signal to each device To do. Note that the simulator 500 in the OFF state is a state in which neither of the switches 510 and 530 is connected to any of the contacts a and b. Further, the OFF state of the agent 600 is a state in which none of the switches 610, 620, and 630 is connected to any of the contacts a and b.

  Next, the function of each of the four modes will be described.

  First, the A mode (control system verification mode) will be described. A plant model 520 in FIG. 1 is a model simulating the plant 100 and is a model in which various process values are calculated when a control command value is given as an input. The control model 540 is a model that outputs a control command value when a process value is given as an input. The control model 540 is constructed based on the control logic stored in the control logic data 330, and the control parameter can be changed by a signal from the agent 600. In the control parameter A mode, the switches 510 and 530 are connected to b and b, respectively, and the plant model and the control model constitute a closed loop.

The agent 600 includes an evaluation learning unit 630, learning data 640, and a control parameter setting unit 650. In order to describe the function of the agent 600, first, “state s _t ”, “action a _t ”, and “reward r _t ” are defined.

Here, an example of the “state s _t ” used in the driving support device according to the embodiment of the present invention will be described with reference to FIG.
FIG. 6 is an explanatory diagram of “state s _t ” used in the driving support apparatus according to the embodiment of the present invention.

As shown in FIG. 6 (H), the “state s _t ” includes the output deviation shown in FIG. 6 (B), the main steam temperature deviation shown in FIG. 6 (C), and the reheat shown in FIG. 6 (D). Maximum temperature, minimum value, and time for taking the maximum and minimum values of the difference between the steam temperature deviation and the control target value such as the main steam pressure deviation shown in FIG. 6 (E) and the process value (load in FIG. 6 (A)) Time based on the change start time t0). For example, in the case of the output deviation shown in FIG. 6B, the maximum value is ΔMW ₁ , the minimum value is ΔMW ₂ , the time for taking the maximum value is t ₁₁ , and the time for taking the minimum value is t ₁₂ . In FIG. 6, but is selected first maximum value and the minimum value as a state s _t, it is also possible to include second and subsequent values in state st. Further, parameters (K _FW , t ₅₁ , t ₅₂ , K _FU , t) relating to control command values such as the feed water BIR shown in FIG. 6 (F) and the fuel BIR (BIR: preceding command signal) shown in FIG. 6 (G). _{61, t 62)} can also be included in the state _{s t} a. Control parameters such as proportional gain and integration time may also be included.

“Action a _t ” is defined as an adjustment that increases or decreases the value of the control parameter of K _FW , t ₅₁ , t ₅₂ , K _FU , t ₆₁ , and t ₆₂ by a certain amount. Proportional gain of the control system, in some cases to include an adjustment to increase or decrease the control parameters of the integration time action a _t. You may also include an adjustment for changing a plurality of control parameter values at the same time into action a _t.

"Compensation r _t" is a value that gives in response to control performance improvement degree before and after carrying out the control parameter adjustment by action a _t, is calculated using the following equation (1).

Here, α ₁ , α ₂ , α ₃ , α ₄ are weighting factors, and δMW _b , δMW _a , δMST _b , δMST _a , δRST _b , δRST _a , δMSP _b , δMSP _a are respectively control parameter adjustments. Is the maximum value of the output deviation before and after, the maximum value of the main steam temperature deviation, the maximum value of the reheat steam temperature deviation, and the maximum value of the main steam pressure deviation (subscript b is before, a is after Means). In some cases, the reward includes a value obtained by integrating the deviation over time, or a deviation of other process values (fuel flow consumption, NOx concentration, etc.).

Next, a control parameter adjustment learning method in the driving support apparatus according to the embodiment of the present invention will be described with reference to FIG.
FIG. 7 is a flowchart showing the contents of the control parameter adjustment learning method in the driving support apparatus according to the embodiment of the present invention.

  FIG. 7 shows an example in which the agent applies Q-learning theory. In addition, Sarsa, an actor-clitic method, R learning, etc. may be applied as a learning method. Here, Q (s, a) in the flowchart is a value for estimating the value of executing the action a in the state s, and this value is a value stored in the learning data 640.

  First, in step s10, the evaluation / learning unit 630 arbitrarily initializes the value of Q (s, a).

  Next, in step s20, the evaluation / learning unit 630 initializes the state s. For example, when each BIR set value is set to 0, the initial state is set.

Next, in step s30, the control parameter setting unit 650 selects the action a in the state s using a policy derived from Q (s, a). An example of a policy derived from Q (s, a) is an ε-greedy policy. This is a policy of selecting an action a that maximizes Q (s, a) with the probability represented by the following expression (2) and selecting another action a with the probability of the following expression (3).

Here, | A (s) | is the total number of actions a in the state s.

  When the state s is observed, the control parameter setting unit 650 takes the action a derived from the ε-greedy policy using the value of Q (s, a) stored in the learning data 640.

  Next, in step s40, the evaluation / learning unit 630 applies the action a (control parameter adjustment) selected in step s30 to the control model 540, executes a simulation, and based on the above-described equation (1) from the analysis result. The reward r and the state s ′ shown in FIG. 6 are observed.

  Next, in step s50, the evaluation / learning unit 630 updates the value of Q (s, a) according to the following equation (5).

Here, Q (s, a) calculated by the equation (4) is stored in the learning data 640, and when the step s50 is executed, the data is read and written in a timely manner.

  Next, in step s60, the evaluation / learning unit 630 updates the state s ′ to the state s.

  Next, in step s70, the evaluation / learning unit 630 proceeds to step s80 when the control requirement specification is achieved or when a predetermined number of actions have been executed, and otherwise returns to step s30. Steps s30 to s70 are repeated until the termination determination condition is satisfied.

  Next, in step s80, the evaluation / learning unit 630 terminates the learning when the number of repeated trials predetermined by the driver is terminated.

  In the present embodiment, only the last updated value is stored in the learning data 640. However, when the update history, the corresponding state, and the action history are stored as necessary, There is also.

  In this way, in the A mode (control system verification mode), the agent 600 learns the control parameter adjustment for the learning algorithm simulator 500 of FIG.

Here, an example of a screen displayed on the image display device 710 when the A mode (control system verification mode) is applied in the driving support device according to the embodiment of the present invention will be described with reference to FIG.
FIG. 8 is an explanatory diagram of a display example of the screen when the A mode (control system verification mode) is applied in the driving assistance apparatus according to the embodiment of the present invention.

  When the driver selects the button 712 on the screen 711 in FIG. 4, a screen 720 in FIG. 8A is displayed.

  First, input the control specifications. Next, the load change condition of the simulation executed in step s40 in FIG. 7 is set on the screen 721 in FIG. 8B, and the control request specification referred to in the termination determination 637 is displayed on the screen 722 in FIG. Set.

  Next, input analysis conditions. First, on the screen 723 in FIG. 8D, the number of repetitions of the step referred to at the end determination in step s70 and the number of trials referred to in the trial end determination in step s80 are set. Next, the weighting coefficient of Expression (1) is set using the screen 724 in FIG. Finally, a name for identifying the agent is input from the screen 725 in FIG.

  Next, returning to the screen 720 of FIG. 8A and selecting calculation start on the screen 720, the agent adjusts the control parameters according to the flowchart of FIG. 7, and automatically executes the simulation.

  As a result of this trial, if there is a trial that has achieved the control requirement specification, the screen 726 of FIG. 8G is displayed, and the driver can confirm the number of trials and the number of times that the control requirement specification has been achieved. If the control requirement specification cannot be achieved, the screen 727 of FIG. 8H is displayed, and the A mode ends.

  By using the A mode (control system verification mode) of the driving support device 400, the validity of the control system using the simulator can be automatically executed before the plant is operated. That is, when the agent cannot find a control parameter that achieves the control requirement specification, it is highly likely that the control logic stored in the control logic data 330 is defective. For this reason, the driver can verify the validity of the control system without adjusting the control parameters and executing the simulation.

  Next, the B mode (plant model adjustment mode) will be described.

  In the B mode (plant model adjustment mode), as shown in FIG. 5, the switches 510 and 520 are connected to a and a, respectively, so that the simulator 500 and the model adjustment unit 450 are connected. The model adjustment unit 450 compares the process value calculated by inputting the control command given to the actual machine to the plant model 520 and the measured value of the plant 100 so that the characteristics of the plant model and the actual machine become the same. Adjust model 450. For details of the operation of the model adjustment unit 450, for example, those described in Japanese Patent Application Laid-Open No. 10-214112 and Japanese Patent Application Laid-Open No. 2001-154705 are applied.

Here, an example of a screen displayed on the image display device 710 when the B mode (plant model adjustment mode) is applied in the driving support device according to the embodiment of the present invention will be described with reference to FIG.
FIG. 9 is an explanatory diagram of a display example of a screen when the B mode (plant model adjustment mode) is applied in the driving support apparatus according to the embodiment of the present invention.

  In FIG. 4, when a button 713 is selected, a screen 730 shown in FIG. 9A is displayed. Here, when data input is selected, a screen 731 shown in FIG. 9B is displayed for setting the data range of the plant 100 used for model adjustment. When the start / end time of data acquisition is set by the driver and the next button is selected, a screen 732 of FIG. 9C is displayed, and a trend graph of the set range is displayed.

  When the start of adjustment is selected on the screen 730 in FIG. 9A after the data input is completed, the adjustment of the plant model 520 by the model adjustment unit 450 is started, and the adjustment result is as shown in a screen 733 in FIG. 9D. Displayed, and the B mode ends.

  Next, the C mode (control system adjustment guidance mode) will be described.

  In the C mode (control system adjustment guidance mode), the control parameter adjustment guidance is displayed using the operating state s of the plant and Q (s, a) stored in the learning data 640. The control parameter setting unit 650 in this mode selects an action a that maximizes Q (s, a), not an action based on the ε-greedy policy applied in the A mode (control system verification mode). This action a can be interpreted as an optimal control parameter adjustment in the operating state of the actual machine. The contents of the control parameter adjustment are transmitted to the image display device 710 via the external output interface 460, the control parameter adjustment guidance is displayed on the image display device 710, and is provided to the driver as the control parameter adjustment guidance.

  This guidance is based on the trial results of the simulator, and may not be optimal guidance when the characteristics of the plant 100 and the plant model 520 are different. In this case, the plant model 520 is adjusted using the B mode (plant model adjustment mode), then the A mode (control system verification mode) is obtained for this simulator, and finally the C mode (control) Control system adjustment guidance may be provided to the driver using the system adjustment guidance mode).

Here, with reference to FIG. 10, an example of a screen displayed on the image display device 710 when the C mode (control system adjustment guidance display mode) is applied in the driving assistance device according to the embodiment of the present invention will be described.
FIG. 10 is an explanatory diagram of a display example of a screen when the C mode (control system adjustment guidance display mode) is applied in the driving assistance apparatus according to the embodiment of the present invention.

  When the agent selection is selected from the screen 740 shown in FIG. 10A, the agent 600 selects the agent learned in the A mode (control system verification mode) from the screen 741 in FIG. A plurality of types of agents learned by changing the weight of the reward r in the A mode (control system verification mode) can be prepared, and agents used in the C mode (control system adjustment guidance display mode) are displayed on the screen 741. You can choose. When selecting an agent on the screen, a weighting factor may be displayed corresponding to the agent. The driver can provide control parameter adjustment guidance that is close to the driver's feeling by selecting an agent having a large weighting factor for the control deviation that is important to the driver.

  When real machine data input is selected from the screen 740 shown in FIG. 10A, a section for acquiring the state of the real machine is set on the screen 742 in FIG.

  When the guidance display is selected on the screen 740 shown in FIG. 10A, the control parameter adjustment guidance is displayed as shown on the screen 743 in FIG. When the driver selects “Execute” on the screen 743, the changed control parameter value is transmitted to the maintenance tool 300, and the control parameter value in the control logic executed by the control device 200 is updated.

  Finally, the D mode (actual machine learning mode) will be described.

  In the A mode (control system verification mode), the agent performs trial and error on the simulator and learns control parameter adjustment, whereas in the D mode (actual machine learning mode), the learning is performed on the actual plant. . Further, in the C mode (control system adjustment guidance display mode), the actual machine state s and the reward r after adjusting the control parameters are not observed, whereas in the D mode (actual machine learning mode), the actual machine operating state s after adjusting the control parameters The reward r is observed, and Q (s, a) in the learning data 640 is updated using Expression (4). Therefore, the agent in the D mode (actual machine learning mode) sets and learns the control parameters for the plant 100.

Here, an example of a screen displayed on the image display device 710 when the D mode (actual machine learning mode) is applied in the driving support device according to the embodiment of the present invention will be described with reference to FIG.
FIG. 11 is an explanatory diagram of a display example of a screen when the D mode (actual machine learning mode) is applied in the driving support device according to the embodiment of the present invention.

  When condition input is selected from the screen 750 shown in FIG. 11A, the weight of the reward r in the equation (1) is set on the screen 751 in FIG. 11B, and the name of the agent is set in FIG. The screen 752 is set.

  In this embodiment, the method of updating and learning Q (s, a) for the plant 100 has been described. However, it is also possible to apply Q (s, a) learned by the plant 100 to another plant. It is.

  By using the driving support apparatus 400 described above, it is possible to provide the driver with control adjustment guidance suitable for improving the control characteristics of the plant. As a result, the burden on the operator for adjusting the control parameters can be reduced. That is, when the operation support device 400 is used, since the response characteristics of the process values of the plant and the plant model match using the measured value information of the process values of the actual plant, the parameter correction method learned for the plant simulator is This is an effective parameter correction for the plant.

Next, the configuration and operation of a driving support apparatus according to another embodiment of the present invention will be described with reference to FIGS. 12 and 13. Here, the case where the present invention is applied to a thermal power plant will be described as an example, but the object can also be applied to other plants such as a nuclear power plant.
First, the configuration and operation of the control system when the operation support apparatus according to the present embodiment is applied to a plant will be described with reference to FIG.
FIG. 12 is a system block diagram showing a configuration of a control system when a driving support apparatus according to another embodiment of the present invention is applied to a plant. The same reference numerals as those in FIG. 1 indicate the same parts.

  In this embodiment, the simulator 500 in FIG. 1 is excluded and a parameter correction device 800 is introduced. The parameter correction device 800 has a function of comparing the characteristic data of the simulator with the characteristic data of the plant and correcting the quantitative part of the control parameter guided by the agent.

  In the configuration of FIG. 1, since the simulator is mounted on the driving support device 400, the configuration of the control system automatic learning system may be complicated. Therefore, in this embodiment, the agent learns in advance with a simulator, and only this learning data 640 is installed in the control system automatic adjustment system. A parameter correction device 800 is mounted as a mechanism for correcting the control parameters in consideration of the difference between the simulator and the actual plant characteristics. This makes it possible to provide appropriate control parameter guidance without mounting a simulator on the driving support device 400.

  Hereinafter, the operation of the parameter correction apparatus 800 will be described. The parameter correction apparatus 800 extracts, for example, the result of the step response test operation as data indicating the characteristics of the plant in the data processing unit 810 and stores the data in the plant characteristic data 820. The simulator characteristic data 830 stores data indicating characteristics of the simulator, such as a step response analysis result.

Next, the operation of the characteristic comparison unit 840 used in the driving support apparatus according to the present embodiment will be described with reference to FIG.
FIG. 13 is an operation explanatory diagram of the characteristic comparison unit used in the driving support apparatus according to another embodiment of the present invention.

As shown in FIG. 13 (A), when changing stepwise the fuel flow rate, the main steam temperature in the actual machine, the variation width MST _R next shown in FIG. 13 (B), the reheat steam temperature, 13 ( The change width RST _R shown in FIG. Further, the main steam temperature in the simulator has a change width MST _S shown in FIG. 13C, and the reheat steam temperature has a change width RST _S shown in FIG. When the change width is such, φ1 and φ2 calculated by the following equations (5) and (6) are transmitted to the control parameter correction unit 850.

  Further, the result of comparing the influence on the other process values at the time of the fuel step as shown in FIG. 13A is also transmitted to the control parameter correction unit 850. In addition, the result of comparing the influence on the actual value at the water supply step and the process value of the simulator is also transmitted to the control parameter correction unit 850. The result of comparing the actual machine characteristic and the simulator characteristic with respect to the influence of the change in the operation amount on each process value is transmitted to the control parameter correcting unit 850.

  The control parameter correction unit 850 corrects the change width of the control parameter setting value presented by the agent using φ1 and φ2. For example, when the control parameter adjustment presented by the agent is for adjusting the fuel flow rate in the hope of improving the main steam temperature, a value obtained by multiplying the adjustment range of the fuel flow rate by φ1 is used as the control parameter correction unit. In 850, the calculation result is displayed on the image display device 710. Thereby, the control parameter adjustment guidance according to the actual machine characteristics can be provided to the driver, for example, in the form as shown in FIG.

  As described above, according to the present embodiment, it is possible to provide the driver with control adjustment guidance suitable for improving the control characteristics of the plant. As a result, the burden on the operator for adjusting the control parameters can be reduced.

That is, for example, when the control parameter value of the fuel flow rate is increased and the fuel flow rate supplied to the thermal power plant is increased by a certain amount, both the steam temperature of the actual plant and the simulator rise. In this way, since the qualitative characteristics of the plant and the simulator match, if the operation of increasing the fuel flow rate is effective for the simulator, the same operation is effective for the actual plant. However, the characteristics of the plant may change from time to time due to changes over time, and the increase in steam temperature between the plant and simulator may not match. Therefore, when the fuel flow rate is increased by a certain amount, the quantitative steam temperature improvement effect in the simulator and the plant is different. For this reason, even if the control parameter adjustment is effective for the simulator, it may not be effective for the plant. Therefore, the information on control parameter adjustment effective for the simulator is divided into information on the direction of increase / decrease of the control parameter value and information on the change width of the control parameter value. By correcting the simulator characteristics based on the result of comparison, information on control parameter adjustment effective for the plant is obtained. Plant characteristics and simulator characteristics are compared using the method described below. In a trial operation of a thermal power plant, for example, the fluctuation range of the steam temperature when the fuel flow rate is changed stepwise is measured to evaluate the plant characteristics. When the same test conditions are input to the simulator, the fluctuation range of the steam temperature is compared with the measurement result, and the difference in sensitivity to the steam temperature when the fuel flow rate changes is calculated. Based on this calculation result, the change width of the control parameter value is corrected. As a result, guidance for control system adjustment suitable for the plant is provided to the plant operator.

It is a system block diagram which shows the structure of the control system at the time of applying the driving assistance device by one Embodiment of this invention to a plant. It is a system block diagram which shows the structure of the plant supported by the driving assistance apparatus by one Embodiment of this invention. It is explanatory drawing which shows the mode of the data memorize | stored in the magnetic disk in the driving assistance device by one Embodiment of this invention. It is mode explanatory drawing of the driving assistance device by one Embodiment of this invention. It is explanatory drawing of the connection relationship of the model adjustment part in each mode of the driving assistance device by one Embodiment of this invention, a simulator, ON / OFF of an agent, and a switch. It is explanatory drawing of "state s _t " used with the driving assistance device by one Embodiment of this invention. It is a flowchart which shows the content of the learning method of the control parameter adjustment in the driving assistance device by one Embodiment of this invention. It is explanatory drawing of the example of a display of the screen at the time of A mode (control system verification mode) application in the driving assistance device by one Embodiment of this invention. It is explanatory drawing of the example of a display of the screen at the time of B mode (plant model adjustment mode) application in the driving assistance device by one Embodiment of this invention. It is explanatory drawing of the example of a display of the screen at the time of C mode (control system adjustment guidance display mode) application in the driving assistance device by one Embodiment of this invention. It is explanatory drawing of the example of a display of the screen at the time of D mode (real machine learning mode) application in the driving assistance device by one Embodiment of this invention. It is a system block diagram which shows the structure of the control system at the time of applying the driving assistance apparatus by other embodiment of this invention to a plant. It is operation | movement explanatory drawing of the characteristic comparison part used for the driving assistance apparatus by other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Plant 200 ... Control apparatus 300 ... Maintenance tool 400 ... Operation support apparatus 500 ... Simulator 600 ... Agent 510, 530, 610, 620, 660 ... Switch 520 ... Plant model 540 ... Control model 630 ... Evaluation learning part 640 ... Learning data 650: Control parameter setting unit

Claims (4)

The plant data obtained from the plant, the simulator data calculated using a plant simulator consisting of a plant model that simulates the operating characteristics of the plant and a control model that simulates the control circuit of the plant, and a predetermined target process value And a parameter correction unit that corrects the control parameter value of the control circuit using at least one evaluation index as a deviation from the target value thereof,
The parameter correction means includes
A parameter correction method learning means for learning the direction and the correction amount of the increase or decrease of the control parameter based on the change amount of the evaluation index value with respect to the correction result of the control parameter;
Correction means for changing the correction amount of the control parameter based on the difference in response characteristics of the process value with respect to the change in the operation amount between the simulator and the actual plant,
An operation support apparatus comprising: guidance means for displaying an increase amount or a decrease amount of the process value of the plant or a characteristic amount thereof or a control parameter value on a display screen. The driving support device according to claim 1,
The operation support apparatus, wherein the correction means changes the correction amount of the control parameter based on a difference in response characteristics of process values when the operation amount is changed stepwise between the simulator and the actual plant. The driving support device according to claim 1,
The plant is a thermal power plant,
The parameter correcting means corrects the control parameter value of the control circuit using at least one of deviations from the power generation output, steam temperature, steam pressure, pipe metal temperature and the corresponding target value as an evaluation index. Driving assistance device. The plant data obtained from the plant, the simulator data calculated using a plant simulator consisting of a plant model that simulates the operating characteristics of the plant and a control model that simulates the control circuit of the plant, and a predetermined target process value A driving support method for correcting a control parameter value of a control circuit using at least one evaluation index as a deviation between the target value and the target value,
Based on the amount of change in the evaluation index value with respect to the correction result of the control parameter, learn the direction of increase or decrease of the control parameter and the correction amount,
Change the correction amount of the control parameter based on the difference in the response characteristics of the process value to the change in the operation amount between the simulator and the actual plant,
An operation support method characterized by displaying a process value of the plant or a characteristic amount thereof, or an increase amount or a decrease amount of a control parameter value on a display screen.

Images