JP2023138425A - Machine learning apparatus and exhaust heat recovery system - Google Patents

Abstract

To optimize an exhaust gas flow rate flowing through each line of an exhaust gas line introducing an exhaust gas generated in a cement firing facility into a boiler and a bypass line bypassing the boiler so as to be joined with the exhaust gas line.SOLUTION: A machine learning apparatus mechanically learns the determination of an operation value in an exhaust heat recovery system. The machine learning apparatus comprises: an operation value output part outputting an operation value to a controller; a state observation part acquiring state values including a boiler inlet gas temperature, a steam flow rate from a boiler and a power generation output value of a steam turbine generator; a reward calculation part calculating a reward based on a prescribed energy value related to the power generation output value included in the state values acquired after the operation of at least one damper based on the operation value; and a learning part mechanically learning the determination of the operation value to be output to the controller from the present state values on based on the operation value, the state values and the reward.SELECTED DRAWING: Figure 4

Description

The present invention relates to a machine learning device and an exhaust heat recovery system used in an exhaust heat recovery system that recovers heat from exhaust gas in a cement manufacturing process.

The cement manufacturing process is broadly divided into a raw material process in which cement raw materials are dried, crushed, and mixed; a firing process in which clinker, which is an intermediate product, is fired from the raw materials; and a finishing process in which gypsum is added to the clinker and crushed to make cement. Consists of. In the firing process, the cement raw material is first preheated in a preheater (hereinafter also referred to as "PH"), then calcined in a calcining furnace, then fired in a kiln, and finally, It is cooled by a cooling cooler (hereinafter also referred to as "AQC"). BACKGROUND ART Exhaust heat recovery systems have been known that recover exhaust gas heat generated by PH or AQC and generate electricity using the recovered heat.

For example, the exhaust heat recovery system disclosed in FIG. 3 of Patent Document 1 includes an exhaust gas line extending from the high temperature section of the AQC to the chimney, which is equipped with an AQC boiler that recovers heat from the exhaust gas generated by the AQC, and a low temperature section of the AQC. and a bypass line that extends from the AQC boiler and bypasses the AQC boiler. The exhaust gas line discharges relatively high temperature (eg, 360° C. on average) exhaust gas within the AQC, and the bypass line exhausts relatively low temperature (eg, 110° C. on average) exhaust gas within the AQC. Exhaust gas discharged from the AQC through the exhaust gas line is guided to the AQC boiler. In an AQC boiler, superheated steam is generated by the heat of exhaust gas, and the superheated steam is used for power generation in a steam turbine generator. The exhaust gas whose heat has been recovered by the AQC boiler joins the exhaust gas discharged from the AQC in the bypass line, passes through the dust collector, and is discharged into the atmosphere from the chimney.

In addition to the exhaust gas line that passes through the AQC boiler and the bypass line that bypasses the AQC boiler, this exhaust heat recovery system also includes an exhaust gas line that passes through the PH boiler, and a bypass that branches off from the PH exhaust gas line and bypasses the PH boiler. It also has a line.

Patent No. 5897302

It is desirable that waste heat recovery systems such as those described above be optimized to produce as much energy as possible. As such a method, for example, adjusting the opening degree of a damper provided in a bypass line or the like may be considered so that a large amount of high-temperature exhaust gas is sent to the boiler. However, the flow rate and temperature of the exhaust gas flowing into the boiler are in a relationship where if one is increased, the other can be decreased.

For example, if a damper whose opening degree can be adjusted is installed on a bypass line that bypasses an AQC boiler, if the opening degree of the damper is reduced, the amount of exhaust gas discharged from the AQC through this bypass line will be reduced, and the exhaust gas line As a result, the amount of exhaust gas emitted from AQC increases. However, since relatively low temperature exhaust gas becomes difficult to be discharged from the AQC through the bypass line, the temperature distribution within the AQC fluctuates, and the temperature of the exhaust gas flowing into the AQC boiler through the exhaust gas line decreases. Conversely, by increasing the opening degree of the damper of the bypass line, the flow rate of the exhaust gas flowing into the AQC boiler decreases, but the temperature of the exhaust gas flowing into the AQC boiler increases. Furthermore, the amount of clinker supplied from the kiln to the AQC fluctuates from moment to moment, and as a result, the temperature distribution of exhaust gas within the AQC also fluctuates. For this reason, it has been extremely difficult to understand how to adjust the flow rate of exhaust gas flowing through the exhaust gas line and the bypass line to realize an optimal system. The need to optimize the system may also arise with exhaust gas lines passing through the PH boiler and bypass lines bypassing the PH boiler.

Therefore, the present invention provides an exhaust heat recovery system that includes an exhaust gas line that guides exhaust gas generated in cement firing equipment to a boiler, and a bypass line that bypasses the boiler and joins the exhaust gas line. The purpose is to provide a machine learning device and waste heat recovery system for optimizing flow rate.

A machine learning device according to one aspect of the present invention is a machine learning device that performs machine learning to determine operating values in an exhaust heat recovery system, and includes:
The exhaust heat recovery system includes a cement firing facility having an exhaust gas generating section that generates exhaust gas during the process of firing cement raw materials, a discharge facility that releases the exhaust gas to the atmosphere outside the cement firing facility, a boiler, and an exhaust fan. is provided with an exhaust gas line that allows a part of the exhaust gas generated in the exhaust gas generation section to pass through the boiler and guides it to the discharge equipment, and an exhaust gas line that allows the remainder of the exhaust gas generated in the exhaust gas generation section to bypass the boiler. , a bypass line leading between the boiler and the discharge equipment in the exhaust gas line; at least one damper provided in at least one of the exhaust gas line and the bypass line; and a steam turbine generator that generates electricity using steam generated in the boiler,
The machine learning device includes an operating value output unit that outputs the operating value to the controller, a temperature of exhaust gas flowing into the boiler, a flow rate of steam sent from the boiler to the steam turbine generator, and the steam turbine. a state observation unit that obtains a state value including a power generation output value of the generator; a remuneration calculation unit that calculates a remuneration based on a predetermined energy value to be used, and the operation value that is output from the current state value to the controller based on the operation value, the state value, and the remuneration. and a learning section that performs machine learning to determine.

Further, an exhaust heat recovery system according to one aspect of the present invention is the exhaust heat recovery system including the above-described machine learning device.

According to the machine learning device and exhaust heat recovery system described above, machine learning can be performed to determine operating values that generate as much energy as possible. Therefore, by operating at least one damper provided in at least one of the exhaust gas line and the bypass line based on the operating value determined from the current state of the exhaust heat recovery system using the learning results of the machine learning device, It is possible to realize a system that generates as much energy as possible from exhaust gas.

According to the present invention, an exhaust gas line and a bypass line are connected to an exhaust heat recovery system that includes an exhaust gas line that guides exhaust gas generated in cement firing equipment to a boiler, and a bypass line that bypasses the boiler and joins the exhaust gas line. We can propose a machine learning device and an exhaust heat recovery system for optimizing the flow rate of flowing exhaust gas.

FIG. 1 is a schematic configuration diagram of an exhaust heat recovery system according to an embodiment of the present invention. FIG. 2 is a block diagram of a control system in the exhaust heat recovery system. FIG. 3 is a diagram explaining the basic concept of a reinforcement learning algorithm. FIG. 4 is a functional block diagram of the machine learning device. FIG. 5 is a flowchart showing the flow of machine learning (reinforcement learning) performed by the machine learning device.

[Exhaust heat recovery system]
Hereinafter, an exhaust heat recovery system according to an embodiment of the present invention will be described with reference to the drawings. The exhaust heat recovery system 1 is a system that recovers heat from exhaust gas generated during the process of manufacturing cement in the cement manufacturing equipment 2. The exhaust heat recovery system 1 according to this embodiment includes a cement manufacturing facility 2 and a power generation facility 3.

[Cement manufacturing equipment]
As mentioned above, the cement manufacturing process consists of a raw material process, a firing process, and a finishing process. As shown in FIG. 1, the cement manufacturing facility 2 includes a raw material production facility 10 that performs a raw material process and a firing facility 20 that performs a firing process.

The raw material generation equipment 10 is equipment that generates powdered raw materials obtained by drying, pulverizing, and blending cement raw materials such as limestone and clay. The raw material production facility 10 includes a raw material mill 11, a separator 12, and a silo 13. In the raw material mill 11, limestone, clay, etc. are crushed and dried. Note that exhaust gas from a PH boiler 30, which will be described later, flows into the raw material mill 11 and is used for drying cement raw materials. The cement raw material pulverized and dried in the raw material mill 11 is discharged together with exhaust gas and flows into the separator 12 . In the separator 12, the cement raw material is separated from the exhaust gas and classified. The cement raw materials separated and classified by the separator 12 are mixed and stored in the silo 13, and then supplied from the silo 13 to the firing equipment 20. Note that, as described later, the exhaust gas within the separator 12 is sent to the dust collector 14.

The firing equipment 20 is equipment for firing clinker, which is an intermediate product, from the powder raw material obtained in the raw material production equipment 10. The firing equipment 20 includes a preheater (PH) 21, a calcining furnace 22, a rotary kiln 23, and an AQC 24.

PH21 includes multiple stages of cyclone dust collectors connected in series. In PH21, the exhaust heat from the rotary kiln 23 moves sequentially from the cyclone dust collector at the bottom to the cyclone dust collector at the top, and the cement raw material moves from the cyclone dust collector at the top to the cyclone dust collector at the bottom. Move in order towards the container. The lowermost cyclone dust collector of the PH 21 is connected to the calcining furnace 22 .

In the calcining furnace 22, the cement raw material that has passed through pH21 is calcined in an atmosphere of approximately 900°C. The calciner 22 is connected to a calciner bleed line 22a that sends exhaust heat from the AQC 24 to the calciner 22, and a fuel supply line 22b that supplies fuel and the like to the calciner 22. The outlet of the calcining furnace 22 is connected to the inlet of the rotary kiln 23.

The rotary kiln 23 is a horizontally long cylindrical rotary kiln, and is installed with a slight downward slope from the raw material inlet toward the raw material outlet. In the rotary kiln 23, the cement raw material preheated and calcined in the PH 21 and calciner 22 is calcined using exhaust heat from the AQC 24 and combustion gas from the burner 25. The outlet of the rotary kiln 23 is connected to the inlet 24a of the AQC 24.

In the AQC 24, the fired product discharged from the rotary kiln 23 at a high temperature (for example, about 1400° C.) is rapidly cooled. Specifically, the fired product dropped into the AQC 24 from the exit of the rotary kiln 23 is conveyed toward the exit 24b by a conveyor (not shown) inside the AQC 24. While being conveyed by the conveyor, the baked product is cooled by being blown with cooling air from below the conveyor. The fired product cooled in the AQC 24, that is, the clinker, exits from the outlet 24b and is then sent to the clinker silo by a clinker conveyor (not shown).

The cooling air blown onto the high-temperature fired product within the AQC 24 becomes high-temperature exhaust gas. The gas temperature within the AQC 24 forms a distribution that decreases as it approaches the outlet 24b along the conveyance direction of the fired product. For example, the gas temperature near the inlet 24a of the AQC 24 is about 1350°C, and the gas temperature near the outlet 24b of the AQC 24 is about 100°C. However, since the amount of clinker supplied to the AQC 24 varies, the temperature of the exhaust gas inside the AQC 24 fluctuates. Further, a thermometer 81 for measuring the temperature of the exhaust gas is provided inside the AQC 24 (more specifically, on the upstream side of the high temperature section 24c within the AQC 24 in the conveying direction of the baked product).

[Power generation equipment]
The power generation facility 3 includes a preheater boiler (hereinafter also referred to as "PH boiler") 30, an air quenching cooler boiler (hereinafter also referred to as "AQC boiler") 40, and a steam turbine generator 50.

(PH boiler)
The PH boiler 30 is a boiler that uses exhaust gas generated at PH21 as a heating medium. The PH boiler 30 includes a boiler body 31 having a gas inlet and a gas outlet. Inside the boiler body 31, a superheater 32 and an evaporator 33, which are heat exchangers, are installed in this order from the gas inlet to the gas outlet of the boiler body 31. Further, a steam drum 34 is attached to the boiler main body 31.

The PH boiler 30 is provided in an exhaust gas line 61 (PH exhaust gas line) extending from the PH 21 to the discharge equipment 16. Specifically, high-temperature exhaust gas from the rotary kiln 23 flows through the calciner 22 and the PH 21 in this order, and then is led to the PH boiler 30 through the exhaust gas line 61.

An exhaust fan 35 is provided downstream of the PH boiler 30 in the exhaust gas line 61 . Further, a bypass line 62 (PH bypass line) that bypasses the PH boiler 30 is connected to the exhaust gas line 61. The upstream end of the bypass line 62 is connected to the exhaust gas line 61 upstream of the PH boiler 30, and the downstream end of the bypass line 62 is connected to the exhaust gas line 61 between the PH boiler 30 and the exhaust fan 35. It is connected to the.

An inlet damper 61a is provided between the upstream end of the bypass line 62 in the exhaust gas line 61 and the PH boiler 30. The bypass line 62 is provided with a bypass damper 62a. Further, a thermometer 82 for measuring the temperature of the exhaust gas is provided upstream of the PH boiler 30 in the exhaust gas line 61, more specifically, upstream of the upstream end of the bypass line 62. Further, a thermometer 83 for measuring the temperature of the exhaust gas is provided downstream of the PH boiler 30 in the exhaust gas line 61, more specifically between the PH boiler 30 and the downstream end of the bypass line 62.

The exhaust gas after heat exchange in the PH boiler 30 and the exhaust gas flowing through the bypass line 62 are sent to the raw material mill 11 described above by the exhaust fan 35 and are used as a heat source for the raw material mill 11. The cement raw material (dried material) supplied to the raw material mill 11 and the cement raw material (dried material) discharged from the raw material mill 11 are periodically sampled, and the mass of each is measured. The moisture content (moisture percentage) of the cement raw material supplied to the raw material mill 11 can be determined based on the difference between the mass of the material to be dried and the mass of the dried material in the raw material mill 11, that is, the mass reduced in the raw material mill 11. can. In this embodiment, these cement raw material mass measuring instruments are used as a moisture measuring device 17 for measuring the moisture content of cement raw materials. However, the moisture measuring device 17 is not limited to the above. Further, a thermometer 87 is provided between the raw material mill 11 and the separator 12 in the exhaust gas line 61 to measure the temperature of the exhaust gas flowing out from the raw material mill 11.

Between the separator 12 and the discharge equipment 16 in the exhaust gas line 61, a dust collector 14 and an exhaust fan 15 are provided in this order from upstream to downstream of the exhaust gas flow. The exhaust gas used for drying the cement raw material in the raw material mill 11 is sent to the separator 12 and then to the dust collector 14. The exhaust gas sent from the separator 12 to the dust collector 14 is separated into dust by the dust collector 14, and then sent to a chimney serving as a discharge facility 16 by an exhaust fan 15 and discharged into the atmosphere.

(AQC boiler)
The AQC boiler 40 is a boiler that uses exhaust gas generated by the AQC 24 as a heating medium. The AQC boiler 40 includes a boiler body 41 having a gas inlet and a gas outlet. Inside the boiler body 41, a superheater 42, which is a heat exchanger, an evaporator 43, and a preheater (economizer) 44 are installed in this order from the gas inlet to the gas outlet. Further, a steam drum 45 is attached to the boiler main body 41.

The AQC boiler 40 is connected to an exhaust gas line 63 (exhaust gas line for AQC) extending from the high temperature section 24c of the AQC 24 to the discharge equipment 48. The high temperature portion 24c of the AQC 24 is a location where the exhaust gas temperature in the AQC 24 is relatively high (for example, about 350° C.). For example, the average temperature of the high temperature section 24c of the AQC 24 is within the allowable temperature range of the AQC boiler 40. Since the upstream end of the exhaust gas line 63 is located in the high temperature section 24c of the AQC 24, relatively high temperature exhaust gas is guided from the AQC 24 to the AQC boiler 40 through the exhaust gas line 63 and is used as a heating medium for the AQC boiler 40. .

Note that the AQC 24 is connected to the above-mentioned calciner bleed line 22a. The calcining furnace bleed line 22a is connected to the AQC 24 on the upstream side in the conveyance direction of the fired product from the connection position between the exhaust gas line 63 and the AQC 24. Therefore, exhaust gas having a temperature higher than the temperature of the exhaust gas discharged through the exhaust gas line 63 (for example, about 600° C.) is extracted from the calciner bleed line 22a.

Between the AQC boiler 40 and the discharge equipment 48 in the exhaust gas line 63, a dust collector 46 and an exhaust fan 47 are provided in this order from upstream to downstream of the exhaust gas flow. A bypass line 64 (AQC bypass line) that bypasses the AQC boiler 40 extends from the low temperature section 24d of the AQC 24. The low temperature portion 24d of the AQC 24 is a portion where the exhaust gas temperature in the AQC 24 is relatively low (for example, about 150° C.), and is located downstream of the high temperature portion 24c of the AQC 24 in the direction of conveyance of the baked product. The upstream end of the bypass line 64 is connected to the low temperature section 24d of the AQC 24, and the downstream end of the bypass line 64 is connected between the AQC boiler 40 and the dust collector 46 in the exhaust gas line 63. Since the upstream end of the bypass line 64 is located in the low temperature section 24d of the AQC 24, relatively low temperature exhaust gas is discharged from the AQC 24 through the bypass line 64.

An inlet damper 63a is provided upstream of the AQC boiler 40 in the exhaust gas line 63. The bypass line 64 is provided with a bypass damper 64a. A thermometer 84 for measuring the temperature of the exhaust gas is provided between the upstream end of the exhaust gas line 63 and the AQC boiler 40 (inlet of the boiler main body 41). A thermometer 88 is provided in the exhaust gas line 63 between the AQC boiler 40 (the outlet of the boiler main body 41) and the downstream end of the bypass line 64 to measure the temperature of the exhaust gas.

The exhaust gas flowing out from the AQC boiler 40 and the exhaust gas flowing through the bypass line 64 are sent to the dust collector 46 by the exhaust fan 47, and then sent to the chimney as the discharge equipment 16, and are released into the atmosphere.

(steam turbine generator)
Steam turbine generator 50 includes a steam turbine 51 and a generator 52. The steam turbine 51 is driven by supplied steam. The steam turbine 51 is a multi-stage steam turbine, and for example, a high pressure stage of the steam turbine 51 is supplied with high pressure steam, and a low pressure stage of the steam turbine 51 is supplied with low pressure steam. The generator 52 converts the kinetic energy of the rotating shaft of the steam turbine 51 into electrical energy. The electric power generated by the steam turbine generator 50 is used to operate auxiliary machines (for example, the exhaust fans 15, 35, 47) in the exhaust heat recovery system 1.

Steam used in the steam turbine 51 is condensed in a condenser 54 by cooling water sent from a cooling tower 53. Condensate generated in the condenser 54 is sent to the AQC boiler 40 by a pump 55. The water supplied to the AQC boiler 40 is heated in the preheater 44 by being given heat from the exhaust gas. The water heated by the preheater 44 is distributed to the steam drum 45 of the AQC boiler 40, the steam drum 34 of the PH boiler 30, and the high pressure stage flasher 56.

The water supplied to the steam drum 45 of the AQC boiler 40 is heated by the evaporator 43 and then returned to the steam drum 45 for gas-liquid separation. The steam in the steam drum 45 is heated to a saturation temperature or higher in the superheater 42 and then guided to the steam turbine 51.

The water supplied to the steam drum 34 of the PH boiler 30 is heated by the evaporator 33 and then returned to the steam drum 34 for gas-liquid separation. The steam in the steam drum 34 is heated to a saturation temperature or higher in the superheater 32 and then guided to the steam turbine 51.

Of the water heated by the preheater 44, the remaining water that was not supplied to the steam drums 34, 45 is sent to the high pressure stage flasher 56. The high-pressure stage flasher 56 separates water sent from the preheater 44 into gas and liquid. The steam generated by the high-pressure stage flasher 56 is supplied to the steam turbine 51 (more specifically, the intermediate-pressure stage of the steam turbine 51), and the remaining hot water is sent to the low-pressure stage flasher 57. The low pressure stage flasher 57 separates water sent from the high pressure stage flasher 56 into gas and liquid. The steam generated by the low-pressure stage flasher 57 is supplied to the steam turbine 51 (more specifically, the low-pressure stage of the steam turbine 51), and the remaining water is sent to the preheater 44 together with condensed water from the condenser 54.

The power generation equipment 3 also includes a steam flow meter 85 that measures the flow rate of steam sent from the PH boiler 30 to the steam turbine 51, and a steam flow meter 86 that measures the flow rate of steam sent from the AQC boiler 40 to the steam turbine 51.

(Control device)
The exhaust heat recovery system 1 according to this embodiment includes a control device 7. FIG. 2 is a block diagram of a control system in the exhaust heat recovery system 1. As shown in FIG. 2, the temperature measured by the thermometers 81, 82, 83, 84, 88, the steam flow rate measured by the steam flow meters 85, 86, and the power generation output value of the steam turbine generator 50 are controlled by the control device. Sent to 7. The exhaust fans 15, 35, 47 are provided with power consumption meters 15a, 35a, 47a that measure the power consumption values of the exhaust fans 15, 35, 47, respectively. The power consumption values measured by the power consumption meters 15a, 35a, and 47a are sent to the control device 7.

Further, the control device 7 includes a controller 71 and a machine learning device 72. The controller 71 and the machine learning device 72 each include an arithmetic processing unit such as a CPU, and a nonvolatile storage unit and a volatile storage unit. The controller 71 and the machine learning device 72 are electrically connected so that they can transmit and receive information. Note that the machine learning device 72 may be provided separately from the controller 71 and provided outside the controller 71, and in this case, the machine learning device 72 may be connected to the controller 71 via a network or the like. .

The controller 71 controls the dampers 61a, 62a, 63a, 64a and the exhaust fans 15, 35, 47. Note that the controller 71 may include a control unit that controls the dampers 61a, 62a, 63a, and 64a, and a control unit that controls the exhaust fans 15, 35, and 47. The machine learning device 72 performs machine learning to generate as much energy as possible in the exhaust heat recovery system 1. The machine learning device 72 performs machine learning to determine an operating value including at least one of the opening degrees of the dampers 61a, 62a, 63a, and 64a. That is, some of the dampers 61a, 62a, 63a, 64a and exhaust fans 15, 35, 47 that are to be controlled are controlled based on the operating values output by the machine learning device 72, and the rest are stored in the storage unit. Control is based on pre-stored programs and/or data.

In the example described below, it is assumed that the bypass dampers 62a and 64a among the control targets are controlled based on the operating value output by the machine learning device 72. In this case, for example, the inlet dampers 61a and 63a are controlled by the controller 71 to maintain a predetermined opening degree (eg, 100% opening degree) while the cement manufacturing equipment 2 is in operation. Further, the exhaust fans 15, 35, and 47 are controlled by the controller 71 so as to operate at a predetermined set rotation speed (air exhaust amount) while the cement manufacturing equipment 2 is in operation.

It is difficult to explicitly show how to adjust the opening degrees of the bypass dampers 62a and 64a to realize an optimal system based on the state of the exhaust heat recovery system 1 using a simple calculation formula, etc. It is. Therefore, the machine learning device 72 as an artificial intelligence employs a reinforcement learning algorithm that automatically learns actions to reach a goal just by giving a reward. The machine learning performed by the machine learning device 72 will be described below.

-Machine learning method-
FIG. 3 is a diagram explaining the basic concept of a reinforcement learning algorithm. As shown in Figure 3, in reinforcement learning, the agent's learning progresses through interactions between the learning agent (control device 7) and the environment to be controlled (cement manufacturing equipment 2 and power generation equipment 3). . More specifically, the following (1) to (4) are repeated during agent learning.
(1) The agent observes the state of the environment s _t at time t.
(2) The agent selects an action a _t that it can take based on observation results and past learning, and executes the action a _t .
(3) By executing the action a _t , the state of the environment s _t changes to the next state s _t+1 , and the agent receives a reward r _t+1 based on this change in state.
(4) The agent proceeds with learning based on the state s _t , the action a _t , the reward r _t+1 , and the results of past learning.

In the learning in (4) above, the agent acquires a mapping of state s _t , action a _t , and reward r _t+1 as reference information for determining the amount of reward r that can be obtained in the future. For example, if the number of states that can be taken at each time is m and the number of possible actions is n, then by repeating the action, the reward r _t+1 for the set of state s _t and action a _t is memorized. A two-dimensional array is obtained. Based on the mapping obtained above, we use a value function (evaluation function), which is a function that indicates how good the current state and behavior are, and update the value function while repeating the behavior to determine the optimal value for the state. We will learn the following behaviors.

A state-action value function Q (s _t , a _t ), which is known as one of the value functions, indicates how good an action a _t is in a certain state s _t . The state-action value function Q (s _t , a _t ) is expressed as a function using the state and action as arguments. The state-action value function Q (s _t , a _t ) is based on the reward obtained for an action in a certain state and the value of the action in a future state to which the action will transition during learning while repeating the action. will be updated. The update formula for the state action value function Q(s _t , a _t ) is defined according to the reinforcement learning algorithm. For example, in Q learning, which is one of the typical reinforcement learning algorithms, the state action value function The update formula for Q(s _t , a _t ) is defined by Equation 1 below.

Then, in selecting the action a _t in (2) above, the future reward (r _t+1 +r _t+2 +...) in the current state s _t is calculated using the value function created by past learning. Select the action a _t for which is optimal (the action a _t with the highest value in the state s _t ). In Q-learning, the action a _t for which the reward (r _t+1 +r _t+2 +...) is optimal corresponds to the action a _t for which the reward (r _t+1 +r _t+2 +...) is the maximum. obtain.

As reinforcement learning algorithms, various methods such as Q learning, SARSA method, TD learning, and AC method are well known, but any reinforcement learning algorithm may be adopted as a method applied to the present invention. Since these reinforcement learning algorithms are well known, further detailed description of each algorithm will be omitted herein.

In addition, as a method for updating the value function, the value function is updated every time the state s _t shifts to a new state s _t+1 by applying a certain action a _t to the current state s _t . It is not necessary to adopt a learning method (so-called online learning). For example, by repeatedly applying a certain action a _t to the current state s _t , the state s _t shifts to a new state s _t+1 , and these states and actions are stored as learning data. However, so-called batch learning or mini-batch learning, in which the value function is updated using accumulated learning data, may be adopted.

FIG. 4 is a functional block diagram of the machine learning device 72. The machine learning device 72 shown in FIG. 4 includes a state observation section 91, a state value storage section 92, a reward calculation section 93, a learning section 94, a learning result storage section 95, and an operation value output section 96.

The state observation unit 91 acquires various types of values regarding the states of the cement manufacturing equipment 2 and the power generation equipment 3 as state values. The state value storage section 92 stores the state values acquired by the state observation section 91 and outputs the stored state values to the reward calculation section 93 and the learning section 94.

The state values include, for example, the measured value of the thermometer 81 (the temperature of the exhaust gas on the upstream side in the AQC 24), the measured value of the thermometer 82 (the temperature of the exhaust gas flowing into the PH boiler 30), and the measured value of the thermometer 83 (the temperature of the exhaust gas flowing into the PH boiler 30). The temperature of the exhaust gas flowing out from the boiler 30), the measurement value of the thermometer 84 (the temperature of the exhaust gas flowing into the AQC boiler 40), the measurement value of the thermometer 88 (the temperature of the exhaust gas flowing out from the AQC boiler 40), the measurement value of the thermometer 87 (the temperature of the exhaust gas flowing out from the AQC boiler 40), Measured value (exhaust gas temperature at the outlet of raw material mill 11), measured value of steam flow meter 85 (steam flow rate of PH boiler 30), measured value of steam flow meter 86 (steam flow rate of AQC boiler 40), steam turbine generator 50 power generation output values, power consumption values of the power consumption meters 15a, 35a, 47a, etc. The state value may include the temperature of the exhaust gas at the inlet or outlet of the exhaust fan 15, 35, 47. The status values may include values obtained during past operations as well as values obtained during the latest operation. For example, the state value may include a value stored in the control device 7 in advance. For example, the state value may include the moisture content of the cement raw material measured by the moisture measuring device 17, the set opening degree of the inlet dampers 61a, 63a, or the exhaust fan 15, 35, 47. Each setting rotation speed (exhaust air volume) may be included. The state value may include the exhaust gas temperature at the inlet of the exhaust fan 15, 35, 47.

Note that although an example of the state value is shown in FIG. 4, the state value is not limited to this. That is, the state values stored in the state value storage section 92 may not include all of the values shown in FIG. 4, or may include values other than the values shown in FIG.

The remuneration calculation section 93 analyzes the state value acquired from the state value storage section 92 and calculates the remuneration based on preset remuneration conditions. The reward calculation unit 93 outputs the calculated reward to the learning unit 94.

Reward conditions are conditions for giving rewards in reinforcement learning. The reward conditions are set in advance in the machine learning device 72 by an operator or the like. For example, the reward conditions are set such that when it is determined that the energy generated in the exhaust heat recovery system 1 is large, a larger reward is given than when it is determined that the generated energy is small. Examples of rewards include positive rewards, negative rewards, and no reward (zero reward).

In this embodiment, the remuneration conditions are set so that a remuneration is given according to an index for evaluating the energy generated in the exhaust heat recovery system 1. Specifically, a value obtained by subtracting the power consumption value of the auxiliary equipment in the exhaust heat recovery system 1 from the power generation output value of the steam turbine generator 50 is used as the evaluation index. In other words, as the energy value generated in the exhaust heat recovery system 1, the net energy value obtained by subtracting the power consumption of the auxiliary equipment from the power generation output value is set as an evaluation index, not the power generation output value itself of the steam turbine generator 50. ing.

The reward calculation unit 93 calculates a positive reward when the energy value as an evaluation index exceeds a predetermined threshold, and calculates a negative or zero reward when the energy value is less than a predetermined threshold. For example, the reward calculation unit 93 calculates a zero reward when the energy value is lower than a predetermined first threshold and higher than another second threshold, and calculates a negative reward when the energy value is lower than the second threshold. You may. The threshold value is determined based on simulation, past driving results, etc., and is stored in advance in the storage section of the control device 7.

Here, the power consumption of all the auxiliary machines that operate using the power generated by the steam turbine generator 50 may not be used for calculating the evaluation index, that is, calculating the reward. For example, among all the auxiliary machines that operate using the electric power generated by the steam turbine generator 50, only the apparatus with relatively large power consumption may be used for calculating the reward. In this example, the power consumption of the exhaust fans 15, 35, and 47, which consume more power than other auxiliary machines, is used to calculate the reward.

The learning unit 94 performs machine learning (reinforcement learning) to determine the driving value based on the state value acquired from the state value storage unit 92 and the reward acquired from the reward calculation unit 93. More specifically, the learning unit 94 expresses the state identified from the state values acquired by the state observation unit 91 (state s _t defined by a combination of state values) and the determination of the operating value in the state using arguments. The value function is updated using the value function (state action value function Q (s _t , a _t )) so that the obtained reward is maximized.

The learning result storage unit 95 stores the results learned by the learning unit 94. Note that as a method for storing the value function as a learning result, a method using an approximation function or a method using an array are generally used. However, in addition to these methods, for example, when a state has many states, supervised learning devices such as multi-value output SVM or neural network that output values by inputting state s _t and action a _t can be used. A method using the above may also be used.

The operating value output unit 96 determines an operating value based on the result learned by the learning unit 94 and the current state value, and outputs the determined operating value to the controller 71.

In this example, the operating value is the opening degree of the bypass dampers 62a, 64a, but the operating value is not limited to this. As described above, the operating value may include at least one of the opening degrees of the dampers 61a, 62a, 63a, and 64a. For example, as shown in FIG. 4, the operating values include the openings of the inlet dampers 61a, 63a and the exhaust fans 15, 35 in addition to or instead of the openings of the bypass dampers 62a, 64a. , 47 rotation speeds may be included.

Next, the flow of machine learning (reinforcement learning) performed by the machine learning device 72 will be explained with reference to FIG.

When machine learning is started, the state observation unit 91 acquires information for specifying the environment (state s _t ) as a state value (step S01). The state value storage unit 92 stores the acquired state value (step S02). The learning unit 94 identifies the current state s _t based on the state value acquired by the state observation unit 91 (step S03). The learning unit 94 selects the action a _t based on the past learning results and the state s _t specified in step S03 (step S04). Action a _t is the determination of an operating value according to a defined state s _t . Here, for example, a plurality of driving values may be prepared as selectable actions, and the action a that provides the largest future reward r may be selected based on past learning results. The operating value output unit 96 determines the operating value to be output based on the action a _t selected in step S04, and outputs the determined operating value to the controller 71. The action a _t is executed by the controller 71 (step S05), and the cement manufacturing equipment 2 and the power generation equipment 3 are operated accordingly.

After the action a _t is executed and the state transitions, the state observation unit 91 acquires information for specifying the environment (state s _t+1 ) (step S06), and the state value storage unit 92 stores it as a state value. (step S07). At this stage, the states of the cement manufacturing equipment 2 and the power generation equipment 3 are changing according to the executed action a _t as well as the time transition from time t to time t+1. The reward calculation unit 93 calculates the reward r t+ ₁ from the state value at time t+1 based on the set reward conditions (step S08). The learning unit 94 advances machine learning based on the state s _t specified in step S03, the action a _t selected in step S04, and the reward r _t+1 calculated in step S08 (step S09). The value function used for learning is determined depending on the learning algorithm to be applied. The learning result storage unit 95 stores the results learned by the learning unit 94 (step S10), and the process returns to step S03.

As described above, the machine learning device 72 repeats machine learning (reinforcement learning). Learning by the machine learning device 72 is completed when it is confirmed that an optimal system for energy generation has been achieved (for example, when it is confirmed that a predetermined energy is generated for a predetermined period). Good too.

When actually determining the operating value of the exhaust heat recovery system 1 using the learning data for which learning has been completed, the machine learning device 72 does not perform new learning and uses the learning data at the time of learning completion as is. It is also possible to perform repeated operation. In this case, the operating value output unit 96 determines the operating value based on the result learned by the learning unit 94 (that is, the learning result stored in the learning result storage unit 95) and the current state value, and Output to controller 71.

As explained above, the exhaust heat recovery system 1 according to the present embodiment includes a machine learning device 72 (machine learning device), and the machine learning device 72 outputs an operating value to output an operating value to the controller 71. 96, the temperature of the exhaust gas flowing into the AQC boiler 40, the temperature of the exhaust gas flowing out from the AQC boiler 40, the temperature of the exhaust gas flowing into the PH boiler 30, the temperature of the exhaust gas flowing out from the PH boiler 30, and the steam turbine. A state observation unit 91 that acquires a state value including the power generation output value of the generator 50, and a power generation output value included in the state value acquired after operating the bypass dampers 62a, 64a based on the operating value. A reward calculation unit 93 calculates a reward based on a predetermined energy value, and machine learning determines an operating value to be output to the controller 71 from the current state value based on the operating value, state value, and reward. A learning section 94 is provided.

According to this embodiment, machine learning can be performed to determine operating values that generate as much energy as possible. Therefore, by operating the bypass dampers 62a and 64a provided in the bypass lines 62 and 64, respectively, based on the operating values determined from the current state of the exhaust heat recovery system 1 using the learning results of the machine learning device 72. , it is possible to realize a system that generates as much energy as possible from exhaust gas.

The learning unit 94 generates a value function (state action value function Q(s _t , a _t )) that expresses the state specified from the state value acquired by the state observation unit 91 and the determination of the driving value in the state using arguments. may be used to update the value function so that the reward is maximized.

Further, in the present embodiment, the machine learning device 72 learns how to determine the operating value for adjusting the amount of inflowing exhaust gas for both the AQC boiler 40 and the PH boiler 30. Therefore, the entire system can be optimized.

In addition, the remuneration calculation unit 93 of the machine learning device 72 uses a value obtained by subtracting the power consumption value of the auxiliary equipment in the exhaust heat recovery system 1 from the power generation output value of the steam turbine generator 50 as an evaluation index for calculating the remuneration. Use. Therefore, the net power generation efficiency can be improved.

Although the preferred embodiments of the present invention have been described above, the present invention may include modifications of the specific structure and/or functional details of the above embodiments without departing from the spirit of the present invention. . The configuration of the exhaust heat recovery system 1 described above can be modified as follows, for example.

In the embodiment described above, the machine learning device 72 learned how to determine the operating value for adjusting the amount of inflowing exhaust gas for both the AQC boiler 40 and the PH boiler 30, but the present invention is not limited thereto.

For example, in the above embodiment, the machine learning device 72 learned to determine the operating values including the opening degree of the PH bypass damper 62a and the opening degree of the AQC bypass damper 64a. Machine learning may be used to determine the operating values including the opening degree of the damper 61a and/or the opening degree of the PH bypass damper 62a. That is, machine learning may be used to determine an operating value that improves the heat recovery rate of the PH boiler 30 alone, rather than the AQC boiler 40 and the PH boiler 30 as a whole. In this case, the state value may include the temperature of the exhaust gas flowing into the PH boiler 30, the flow rate of steam sent from the PH boiler 30 to the steam turbine generator 50, and the power generation output value of the steam turbine generator 50. Further, in this case, the state value may include the temperature of the exhaust gas flowing out from the PH boiler 30.

Further, for example, the machine learning device 72 may perform machine learning to determine the operating value including the opening degree of the AQC inlet damper 63a and/or the opening degree of the AQC bypass damper 64a. That is, machine learning may be used to determine an operating value that improves the heat recovery rate of the AQC boiler 40 alone, rather than the AQC boiler 40 and the PH boiler 30 as a whole. In this case, the state value may include the temperature of the exhaust gas flowing into the AQC boiler 40, the flow rate of steam sent from the AQC boiler 40 to the steam turbine generator 50, and the power generation output value of the steam turbine generator 50. Further, in this case, the state value may include the temperature of the exhaust gas flowing out from the AQC boiler 40.

Further, in the above embodiment, the energy value as an evaluation index is the value obtained by subtracting the power consumption value of the auxiliary equipment from the power generation output value of the steam turbine generator 50. Not limited. For example, the energy value as the evaluation index may be the power generation output value of the steam turbine generator 50 itself.

1 : Exhaust heat recovery system 2 : Cement manufacturing equipment 3 : Power generation equipment 7 : Control device 15 : Exhaust fan 16 : Release equipment 20 : Firing equipment (cement firing equipment)
21: Preheater (PH)
22: Calciner 22a: Calciner bleed line 22b: Fuel supply line 23: Rotary kiln 24: Air quenching cooler (AQC)
30: PH boiler 35: Exhaust fan 40: AQC boiler 47: Exhaust fan 48: Discharge equipment 50: Steam turbine generator 61: Exhaust gas line 61a: Inlet damper 62: Bypass line 62a: Bypass damper 63: Exhaust gas line 63a: Inlet damper 64: Bypass line 64a: Bypass damper 71: Controller 72: Machine learning device (machine learning device)
91: State observation section 92: State value storage section 93: Reward calculation section 94: Learning section 95: Learning result storage section 96: Operation value output section

Claims (8)

A machine learning device that performs machine learning to determine operating values in an exhaust heat recovery system,
The waste heat recovery system is
Cement firing equipment having an exhaust gas generation section that generates exhaust gas during the process of firing cement raw materials;
a discharge facility that discharges the exhaust gas to the atmosphere outside the cement firing facility;
an exhaust gas line that is provided with a boiler and an exhaust fan, and guides a part of the exhaust gas generated in the exhaust gas generation section to the discharge equipment through the boiler;
a bypass line that bypasses the boiler and guides the remainder of the exhaust gas generated in the exhaust gas generation section between the boiler and the discharge equipment in the exhaust gas line;
at least one damper provided in at least one of the exhaust gas line and the bypass line;
a controller that operates the at least one damper based on the operating value;
a steam turbine generator that generates electricity using steam generated in the boiler,
The machine learning device includes:
an operating value output unit that outputs the operating value to the controller;
a state observation unit that acquires state values including a temperature of exhaust gas flowing into the boiler, a flow rate of steam sent from the boiler to the steam turbine generator, and a power generation output value of the steam turbine generator;
a remuneration calculation unit that calculates remuneration based on a predetermined energy value related to the power generation output value included in the state value acquired after operating the at least one damper based on the operation value;
A machine learning device comprising: a learning unit that performs machine learning to determine the operating value to be output to the controller from the current state value based on the operating value, the state value, and the reward. . The exhaust heat recovery system includes an auxiliary machine that operates using electric power generated by the steam turbine generator,
The auxiliary equipment includes at least the exhaust fan,
The energy value is a value obtained by subtracting the power consumption value of the auxiliary equipment from the power generation output value of the steam turbine generator,
The remuneration calculation unit calculates a positive remuneration when the energy value exceeds a predetermined threshold, and calculates a negative or zero remuneration when the energy value is below a predetermined threshold. Machine learning device described. The energy value is the power generation output value itself,
The remuneration calculation unit calculates a positive remuneration when the energy value exceeds a predetermined threshold, and calculates a negative or zero remuneration when the energy value is below a predetermined threshold. Machine learning device described. The cement firing equipment includes a preheater (hereinafter referred to as PH) that preheats the cement raw material, a kiln that fires the cement raw material preheated by the preheater, and an air quencher that rapidly cools the fired product from the kiln while conveying it. The exhaust gas generating section is the AQC, the upstream end of the exhaust gas line is connected to a predetermined high temperature section of the AQC, and the upstream end of the bypass line is connected to a predetermined high temperature section of the AQC. The machine learning device according to any one of claims 1 to 3, wherein the end portion is connected to a low temperature section of the AQC located downstream of the high temperature section in the conveyance direction of the fired product. The discharge equipment is an AQC discharge equipment, the boiler is an AQC boiler, the exhaust fan is an AQC exhaust fan, the exhaust gas line is an AQC exhaust gas line, and the bypass line is an AQC bypass line. and the damper is an inlet damper,
The waste heat recovery system is
PH release equipment that releases the exhaust gas to the atmosphere outside the cement firing equipment;
A PH exhaust gas line, which is provided with a PH boiler and a PH exhaust fan, and leads a part of the exhaust gas generated in the PH through the PH boiler to the PH discharge equipment;
a PH bypass line that bypasses the PH boiler and leads the remainder of the exhaust gas generated in the PH between the PH boiler and the PH discharge equipment in the exhaust gas line;
further comprising at least one PH damper that is provided in at least one of the PH exhaust gas line and the PH bypass line and whose opening degree can be changed,
The steam turbine generator generates electricity using steam generated in the AQC boiler and the PH boiler,
The machine learning device according to claim 4, wherein the operating value includes an opening degree of the at least one PH damper. The cement firing equipment includes a preheater (hereinafter referred to as PH) that preheats the cement raw material, a kiln that fires the cement raw material preheated by the preheater, and an air quencher that rapidly cools the fired product from the kiln while conveying it. the exhaust gas generating section is the PH, the upstream end of the exhaust gas line is connected to the PH, and the upstream end of the bypass line is connected to the PH. The machine learning device according to claim 1, wherein the machine learning device is connected to an upstream side of the boiler in an exhaust gas line. The learning unit is configured to maximize the reward using a value function in which the state identified from the state value acquired by the state observation unit and the determination of the operating value in the state are expressed as arguments. The machine learning device according to any one of claims 1 to 3, which updates the value function. The exhaust heat recovery system, comprising the machine learning device according to any one of claims 1 to 3.

Images