JP2024017830A - Optimal control device, optimal control method and computer program - Google Patents

Abstract

An object of the present invention is to provide an optimal control device that realizes a search for an optimal value that maximizes control performance while maintaining stability. [Solution] The optimal control device according to the embodiment includes a process evaluation value calculation unit 120 that calculates an evaluation value of an evaluation function using measured values acquired in a controlled process, and an operation amount and an evaluation value as input. A process for calculating an estimated value of the phase lag from the amount to the evaluation value. Using the phase lag estimating unit 400 and information regarding the estimated value of the phase lag and information on the evaluation value, an estimated value of the rate of change of the evaluation value with respect to the manipulated variable is calculated. An evaluation function gradient estimating unit 140 that calculates the evaluation function gradient estimating unit 140, an extreme value searching unit 170 that determines the direction and amount in which the manipulated variable should move by integrating the estimated value of the rate of change, and an operation determined by the extreme value searching unit 170. It includes a manipulated variable output unit 190 that outputs a manipulated variable based on information on the direction and amount in which the quantity should move to the controlled process. [Selection diagram] Figure 3

Description

Embodiments of the present invention relate to an optimal control device, an optimal control method, and a computer program.

In plant control such as water treatment plants and environmental plants, the state of the process changes depending on disturbances and environmental conditions. In many cases, it is necessary to determine the optimal amount of process operation.

For example, in sewage treatment and wastewater treatment processes, blower air volume, pump flow rate, and chemical input amount are adjusted according to disturbances such as the inflow volume flowing into the sewage treatment plant and the inflow load (= inflow volume x inflow water quality concentration). , it is necessary to maintain the quality of the effluent water. Furthermore, in biological wastewater treatment, etc., the optimal amount of operation changes depending on annual water temperature fluctuations, especially depending on the season.
Furthermore, in flocculant injection control and chlorine injection control in water treatment, the amount of chemicals injected to maintain the required water quality changes depending on disturbances such as the turbidity of the water intake source and the amount of solar radiation. Additionally, the amount of coagulant and chlorine injected is often greatly affected by annual water temperature fluctuations.

Further, in denitrification control in a thermal power plant or the like, it is necessary to use nitrogen oxides (NOx) discharged from the power plant as a disturbance and adjust the amount of ammonia injection for treating NOx according to the amount of NOx discharged.
In petrochemical plants and steel plants, it is necessary to control the operating variables (temperature, pressure, flow rate, etc.) necessary for oil refining and steelmaking to optimal values depending on the quality and quantity of raw materials (heavy oil, iron ore, etc.).

As mentioned above, in plant control in many industries, the state of the plant changes depending on the influence of disturbances and environmental conditions. In these plants, disturbance suppression-type control is used to keep the plant in an optimal state by changing various manipulated variables in order to deal with disturbances, and scheduling-type control is used to switch control according to environmental conditions. In many cases, adaptive control that adapts to environmental conditions is employed.

A technology called Extreme Seeking Control (ESC) has been attracting attention as a typical control method for keeping plants in optimal conditions, and is beginning to be used in industry. In extreme value control, for example, you set an evaluation function for the optimality that you want to minimize or maximize, such as the operating cost or power generation amount of a plant, and then set the extreme value (= local minimum value or local maximum value) of the evaluation value. This is a control method that searches for extreme values by constantly changing the amount of operation while measuring in real time. Extreme value control is attracting attention as a practical optimal control technology for processes that involve complex phenomena that are difficult to model using mathematical formulas (e.g., sewage treatment processes, combustion processes, petrochemical processes, etc.).

Extreme value control is calculated from information obtained by an online sensor that can directly measure the value of the evaluation function that you want to optimize, and changes the amount of operation to maintain the evaluation function value at the optimal value (minimum value or maximum value). It is an adaptive search. In other words, extreme value control is attractive because it is easy to implement in controllers such as PLCs (Programmable Logic Controllers) because it searches for optimal values while changing the manipulated variable without using complex mathematical models. It's a method.

On the other hand, extreme value control does not improve control performance (speed of convergence to extreme values, error between final value and extreme value, etc.) while maintaining stability (ability to search for extreme values). In general, it is difficult to achieve both stability and control performance (convergence speed + optimality (error from optimal value)), but it is essential to apply extreme value control to actual control objects (plants). This is an important issue.

International Publication No. 2020/241657 Japanese Patent Application Publication No. 2019-83030

Yamanaka, Osamu, et al. "Extremum Seeking Based on Approximated Sign of Gradient of Unknown Plant Maps." 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE)., 2020. B. G. B. Hunnekens, M. A. M. Haring, N. van de Wouw, and H. Nijmeijer, “A dither-free extreme-seeking control approach using1st-order least-squares fits for gradient estimation,” in 53rd Conference on Decision and Control, 2014, pp. 2679-2684. Zengin, Nursefa, and Baris Fidan. "Adaptive extremum seeking using recursive least squares." arXiv preprint arXiv:2003.03891 (2020). Onishi, Yuta, et al. "Extremum Seeking Control for Wastewater Treatment Plant with Prioritized Output Constraints." 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE)., 2020. Y. Tan, D. Nesic and I.M.Y. Mareels, “On non-local stability properties of extremum seeking control”, Automatica, vol. 42 (2006), pp. 889-903.

Embodiments of the present invention have been made in view of the above circumstances, and provide an optimal control device, an optimal control method, and a computer that realize the search for optimal values that maximize control performance while maintaining stability. The purpose is to provide programs.

The optimal control device according to the embodiment operates the manipulated variable in real time based on the manipulated variable of the controlled process and the evaluation value of the evaluation function based on the controlled variable that changes according to the manipulated variable, and performs the evaluation. An optimal control device that searches for an optimal value of a value, the process evaluation value calculation unit that calculates the evaluation value of the evaluation function using the measured value acquired in the controlled process, and the operation amount and the evaluation. a process phase lag estimator that calculates an estimated value of the phase lag from the manipulated variable to the evaluation value using the value; an evaluation function gradient estimation unit that calculates an estimated rate of change of a value with respect to the manipulated variable; and an extreme value search unit that determines the direction and amount of movement of the manipulated variable by integrating the estimated value of the change rate. , a manipulated variable output unit that outputs the manipulated variable to the controlled process based on information about the direction and amount of movement of the manipulated variable determined by the extreme value search unit.

FIG. 1 is a diagram for explaining an example of an optimal control device, an optimal control method, and a control target of a computer program according to an embodiment. FIG. 2 is a diagram for explaining the principle of extreme value (local optimum value) search in extreme value control. FIG. 3 is a block diagram schematically showing a configuration example of a real-time process optimal control system to which the optimal control device of the first embodiment is applied. FIG. 4 is a diagram schematically showing a partial configuration example of the optimal control device according to the first embodiment. FIG. 5 is a diagram schematically showing an example of the relationship between time-series data of manipulated variables and time-series data of evaluation amounts when the controlled object has dynamics. FIG. 6 is a diagram schematically showing an example of time series data of manipulated variables and time series data of evaluation values before and after phase lag compensation. FIG. 7 is a diagram for explaining an example of the relationship between control stability and control performance when the frequency of a periodic signal used for extreme value control is changed. FIG. 8 is a diagram for explaining an example of the relationship between control stability and control performance when the frequency of a periodic signal used for extreme value control is changed. FIG. 9 is a diagram for explaining an example of the relationship between control stability and control performance when the frequency of a periodic signal used for extreme value control is changed. FIG. 10 is a diagram for explaining an example of the relationship between control stability and control performance when the frequency of a periodic signal used for extreme value control is changed. FIG. 11 is a diagram for explaining an example of the effect of the optimal control device of the embodiment. FIG. 12 is a diagram for explaining an example of the effect of the optimal control device of the embodiment. FIG. 13 is a diagram for explaining an example of the effect of the optimal control device of the embodiment. FIG. 14 is a diagram for explaining another example of the effect of the optimal control device of the embodiment. FIG. 15 is a diagram for explaining another example of the effect of the optimal control device of the embodiment. FIG. 16 is a diagram for explaining another example of the effect of the optimal control device of the embodiment. FIG. 17 is a diagram for explaining another example of the effect of the optimal control device of the embodiment. FIG. 18 is a diagram schematically showing another configuration example of a part of the optimal control device of the first embodiment. FIG. 19 is a diagram schematically showing a configuration example of the optimal control device according to the second embodiment. FIG. 20 is a diagram schematically showing a partial configuration example of the optimal control device according to the second embodiment. FIG. 21 is a diagram schematically showing a modification of the configuration of the optimal control device according to the second embodiment. FIG. 22 is a diagram schematically showing a modification of a part of the configuration of the optimal control device of the second embodiment.

Hereinafter, an optimal control device, an optimal control method, and a computer program according to embodiments will be described in detail with reference to the drawings.
FIG. 1 is a diagram for explaining an example of an optimal control device, an optimal control method, and a control target of a computer program according to an embodiment.
The optimal control device of this embodiment manipulates the manipulated variable in real time based on the manipulated variable of the controlled process and the evaluation value of the evaluation function based on the controlled variable that changes according to the manipulated variable, and calculates the evaluation value. This is a device that searches for the optimal value.
Here, a coagulant (PAC) injection process in a water treatment plant (for example, a water purification plant) will be described as an example of a control target of the optimal control device of this embodiment. Note that the control target of the optimal control device of the embodiment may be any process, and is not limited to the process shown in FIG.

The water treatment plant includes, for example, a mixing basin, a flocculation basin, a settling basin, a sludge basin, a sand filtration basin, and a water purification basin.
A flocculant is injected into raw water that flows into a water treatment plant in a mixing pond, and is stirred in a flocculation pond to form solids called flocs. A portion of the treated water containing flocs is discharged to the sludge pond, and a portion is precipitated in the sedimentation basin. The treated water discharged from the floc formation pond is sent to the sand filter pond, and the filtered treated water is discharged to the water purification pond.

In the flocculant injection process of the water treatment plant, flocs are formed by the injection of flocculant (PAC). The formed flocs are moved by applying an electric field. The optimal control device of this embodiment measures the floc movement speed (PV) through image processing and adjusts the injection rate (MV) using PI control so that the movement speed follows a target value (SV) near zero. are doing.

Note that the target value SV of the moving speed PV is preferably set near zero, but the optimal value is considered to vary depending on the water temperature, inflow water quality, etc. For this reason, for example, when the optimum control device of this embodiment controls the flocculant injection process, the target value SV of the floc movement speed is adjusted by extreme value control.
The control system shown in FIG. 1 has a configuration called cascade control, and has a two-stage cascade configuration so that the target value SV of the PI control becomes the manipulated variable MV value of the extreme value control.

At this time, the evaluation function in the optimal control device of this embodiment is, for example, taking into account the chemical cost of PAC, the sludge disposal cost, and the cleaning cost, and setting the sum as the evaluation function (operating cost), and There are constraints on the turbidity. The constraint conditions can be converted into an evaluation function by, for example, the method shown in Non-Patent Document 5 or a method called a penalty function. In the following, for simplicity, it is assumed that the constraint condition is a penalty function that is incorporated into the evaluation function, and the evaluation function that incorporates this water quality constraint (constraint condition) into the above operation cost is referred to as the total cost.

FIG. 2 is a diagram for explaining the principle of extreme value (local optimum value) search in extreme value control.
In FIG. 2, the horizontal axis means the amount of operation performed by extreme value control, and in this embodiment is the target value (SV) of the moving speed. Further, in FIG. 2, the vertical axis is an evaluation function (evaluation amount) to be optimized (minimized) by extreme value control, and in this embodiment is the total cost.

It is assumed in advance that there is some kind of relationship between the manipulated variable and the evaluation function, and that they have an extreme value (local minimum value). In FIG. 2, it is assumed that there is a downwardly convex functional relationship between the manipulated variable and the evaluation function.

When extreme value control is actually performed, the value of the evaluation value for the manipulated variable at each moment during control execution can be obtained in real time, but the overall shape of the evaluation function cannot be grasped in real time and is in an unknown state. . In this situation, extreme value control is a control algorithm that searches for the extreme value in the diagram (local optimum value, the example in the diagram is a unimodal downwardly convex function, so the global minimum value). It provides:

According to Figure 2, when the manipulated variable is driven by a periodic dither signal such as a sine wave, if the manipulated variable is on the right side of the extreme value (the side where the manipulated variable increases), it is driven by the dither signal. The movement of the manipulated variable and the movement of the evaluation amount (evaluation value) acquired in real time move in synchronization, that is, they move in the same phase.

On the other hand, when the manipulated variable is driven by a periodic dither signal such as a sine wave, if the manipulated variable is on the left side of the extreme value (the side where the manipulated variable decreases), the manipulated variable driven by the dither signal The movement of the evaluation amount (evaluation value) obtained in real time is opposite to the movement of the evaluation amount (evaluation value) obtained in real time. That is, on the left side of the extreme value, the operation amount driven by the dither signal and the evaluation amount (evaluation value) acquired in real time move in opposite phases.

By using this information, the optimal control device can determine whether the manipulated variable during the control operation is on the left or right side of the extreme value. The optimal control device decreases the manipulated variable when the manipulated variable during control action is to the right of the extreme value, and increases the manipulated variable when the manipulated variable during control action is to the left of the extreme value. By doing so, it becomes possible to search for extreme values.

Furthermore, when analyzed using the formula of the dither signal-driven extreme value control algorithm, the optimal control device actually estimates the average slope at an operating point such as point A shown in Fig. 2. This gradient information is used to determine whether to decrease or increase the manipulated variable during control operation (the direction in which the manipulated variable should move) and the magnitude at which the manipulated variable should be decreased or increased (the amount that the manipulated variable should move). It is supposed to be judged.
The optimal control device of this embodiment improves the stability and control performance of the extreme value control described above.

FIG. 3 is a block diagram schematically showing a configuration example of a real-time process optimal control system to which the optimal control device of the first embodiment is applied.
The real-time process optimal control system shown in FIG. 3 includes an optimal control device and a controlled object 200.

The controlled object 200 is an arbitrary process in an arbitrary plant having a manipulated variable input U and a process output Y. In this embodiment, the controlled object 200 is, for example, a flocculant injection process of a water treatment plant shown in FIG. 1, and the manipulated variable input U is a moving speed target value (SV) in the flocculant injection process. The actual manipulated variable in the coagulant injection process is the PAC injection rate, but as mentioned earlier, this control system has a two-stage cascade configuration, so the manipulated variable adjusted by extreme value control is the PAC injection rate. This is the target value.
Furthermore, the water treatment plant is equipped with various sensors that detect the value of the output Y. The output Y of the flocculant injection process is, for example, the injection rate of the flocculant (PAC), the sludge concentration in the sludge basin, the sludge flow rate, the washing frequency of the settling basin, the sludge concentration at the outlet of the settling basin, and the sludge at the outlet of the water treatment basin. The concentration is.

The optimal control device of this embodiment is, for example, an arithmetic device including at least one processor and a memory in which a program executed by the processor is recorded. Note that the memory of the optimal control device may be provided with a storage area (storage unit) for temporarily storing data (for example, time-series data of measurement values and evaluation values) while the program is being executed. The optimal control device can implement various functions described below using software or a combination of software and hardware.

The optimal control device includes a process phase delay estimation section 400, a process measurement value acquisition section 110, a process evaluation value calculation section 120, a demodulation dither signal generation section 130, an evaluation function gradient estimation section 140, and a normalization signal generation section. 150, a gradient estimation amount normalization section 160, an extreme value search section (optimum manipulated variable adaptive adjustment section) 170, a modulation dither signal generation section 180, and a manipulated variable output section 190. Process phase lag estimating section 400 includes a phase lag parameter estimating section 300 and a phase lag estimating section 1100.

The process measurement value acquisition unit 110 acquires measurement information necessary for calculating evaluation quantities and constraint conditions from the controlled object 200.
As an example, when setting the operation cost, which is the sum of chemical costs, sludge disposal costs, and filter cleaning costs, as the evaluation function, the process measurement value acquisition unit 110 may, for example, Obtain measurement information such as concentration, sludge flow rate, and cleaning frequency of the settling basin. In addition, if a restriction is placed on the sludge concentration at the outlet of the settling tank or the water purification tank, the process measurement value acquisition unit 110 also measures the value of the sludge concentration with the restriction at a predetermined period, and converts it into time series data in a predetermined format. Save it. Furthermore, the process measurement value acquisition unit 110 can acquire and save filter cleaning history data in a predetermined format.

The process evaluation value calculation unit 120 calculates the evaluation value of a preset evaluation function in real time using the information (measurement value of the controlled process) acquired by the process measurement value acquisition unit 110.
The operational cost is defined as the sum of chemical costs (PAC costs), sludge disposal costs, and filter cleaning costs.
The drug cost can be calculated by multiplying the time series data of the injection amount of PAC by a coefficient such as the drug unit price or dilution rate, and can be obtained as time series data.
The sludge disposal cost can be calculated by calculating the amount of generated sludge by multiplying the sludge concentration in the sludge pond by the sludge flow rate, and then multiplying the amount of generated sludge by the sludge disposal unit price, and time series data can be obtained.

In addition, since filtration basin cleaning is normally performed when the filtration resistance exceeds a predetermined threshold, it is not possible to know the timing of cleaning from the time series data of filtration resistance and cleaning history data. The cleaning cost at the time of cleaning is obtained from historical data. Since cleaning is considered to be an expense for the period from the previous cleaning to the next cleaning, the cleaning expenses can be converted into time-series data by distributing the cleaning expenses evenly over the period.

However, although conversion of cleaning costs into time-series data can be easily performed on past performance data, in real time, the timing of the next cleaning is not determined, so the rate of change in the filtration resistance must be monitored. However, the next cleaning timing may be estimated and converted into time-series data of cleaning costs in real time.

The operational cost is the sum of each cost converted into time-series data. Therefore, the process evaluation value calculation unit 120 can calculate the operation cost in real time using each cost obtained by the above method.
On the other hand, water quality constraints are obtained by acquiring the turbidity at the sedimentation basin outlet and the water purification basin outlet in Figure 1 as time series data, and adding a turbidity constraint value, for example, Tlim (turbidity upper limit) = 0.8 degrees. Incorporate constraints such as the following.

In extreme value control, constraint conditions cannot be handled directly, so they can be converted so that they can be handled as evaluation functions, for example, by the method of Non-Patent Document 5. Here, it is treated as an evaluation function using the concept of penalty function, which is well known in the field of optimization. That is, for example, the water quality cost is expressed as in the following equation (1).

Wcost=max(0, a×(exp(T-Tlim)-1))…………(1)
Here, T is a turbidity measurement value, Tlim is a turbidity upper limit value, and a is a design parameter, and the parameter a is larger than zero (a>0).
The cost Wcost is a converted evaluation function. In the optimal control device of this embodiment, the turbidity at the outlet of the sedimentation tank and the turbidity at the outlet of the water purification tank are converted using equation (1), and the cost W cost for the turbidity at the outlet of the water treatment tank and the turbidity at the outlet of the water purification tank are calculated. The sum of the water quality cost and the water quality cost Wcost is the water quality cost.

Equation (1) shows that when the turbidity measurement value T is less than or equal to the turbidity upper limit value Tlim (turbidity measurement value T≦turbidity upper limit value Tlim), the cost Wcost becomes 0, and the turbidity measurement value T becomes the turbidity upper limit value Tlim. Exceeding (turbidity measurement value T>turbidity upper limit value Tlim) means that Wcost increases rapidly (exponentially), which is a type of so-called penalty function.

By adding the water quality cost to the operation cost described above, the total cost J can be defined as shown in equation (2) below.
Total cost J = Chemical cost + Sludge disposal cost + Filter cleaning cost + Water quality cost... (2)
Note that in the flocculant injection control in the optimal control device of this embodiment, the evaluation function as described above is set, but depending on the problem setting, the target plant (control target) 200 and the process evaluation value calculation unit 120 cannot be separated. There are cases.

For example, in a wind power plant, when extreme value control is applied to maximize the amount of power generation by moving the wind turbine blades in accordance with the wind direction, the evaluation function J is the amount of power generation, and the manipulated variable U is the amount of power generated by the wind turbine. This is the rotation angle of the blade. In this example, there is no need to specifically define the output Y of the plant. In such problem settings, there are cases in which the output Y and the evaluation function J are not distinguished, and the output Y is not necessarily defined and measured separately from the evaluation function J.

On the other hand, even if the target plant (control target) 200 and the process evaluation value calculation unit 120 cannot be separated, it is possible to minimize the There are cases where value control can be applied.
In either case, when the evaluation function is appropriately set, the evaluation amount that changes from moment to moment can be obtained by measuring and calculating the value of the evaluation function in real time at a predetermined control cycle.

Note that when the process evaluation value is used as a signal corresponding to the output Y(s) of the transfer function model of the following equations (6) and (7) derived by the phase lag parameter estimation unit 300 described later, the parameter estimation Before calculating the value, it is necessary to obtain the process evaluation value by the process evaluation value calculation unit 120. In this case, the process evaluation value calculation unit 120 supplies the process evaluation value to the phase delay parameter estimation unit 300.

The evaluation function gradient estimation unit 140 calculates an estimated value of the slope (rate of change) of the process evaluation value with respect to the manipulated variable.
FIG. 4 is a diagram schematically showing a partial configuration example of the optimal control device according to the first embodiment. Note that FIG. 4 shows the configuration necessary for the following explanation, and the details of other configurations are omitted.

The evaluation function gradient estimation unit 140 includes a high pass filter (HPF) 141, a low pass filter (LPF) 143, and an adder 142.
The evaluation value (time series data) output from the process evaluation value calculation unit 120 is input to the high pass filter (HPF) 141 . The high-pass filter 141 outputs components of the input evaluation value having a predetermined frequency or higher. The high-pass filter 141 forces (approximately) the extreme value to 0 (if it is assumed that the unknown extreme value (local optimum value) does not change, or even if it changes, it changes very slowly. zero). Control performance can be improved by providing the evaluation function gradient estimation unit 140 with the high-pass filter 141. Note that the high-pass filter 141 is not essential in terms of the theoretical configuration of extreme value control and may be omitted.

The adder 142 receives the value output from the high-pass filter 141 and a demodulation dither signal (information regarding the estimated value of phase delay), which will be described later. The adder 142 adds the input values and outputs the sum to the low-pass filter 143.
The low-pass filter 143 outputs a component of the signal supplied from the adder 142 that is lower than a predetermined frequency. It is known from theoretical analysis results that the periodic average value of the signal output from the low-pass filter 143 is a signal proportional to the slope of the evaluation function. It can be considered as the slope of the value (more precisely, a signal proportional to the slope). For example, the gradient estimate includes information on the direction (sign (1 or -1)) and magnitude of the gradient.

The normalized signal generator 150 generates a normalized signal. The operation of the normalization signal generation section 150 is the same as that of the regularization signal generation section described in Patent Document 1, for example, and various methods can be adopted. In the optimal control device of this embodiment, a signal for converting the estimated gradient value G(t) of the evaluation function into a signal Gn(t) that satisfies the following properties [1]-[4] is called a "normalized signal". shall be.

[1] G(t)=0⇔Gn(t)=0 (Gn(t) is also 0 only when G(t) is 0)
[2] G(t) is positive (negative) ⇔ Gn(t) is positive (negative) (G(t) and Cn(t) have the same sign)
[3] G(t)<∞⇔Gn(t)<∞ (When G(t) is finite, Gn(t) also remains finite, and zero division etc. does not occur)
[4] Gn(t) does not become Gn(t)→∞ when G(t)→∞, but approaches a certain positive finite value, that is, there exists a certain 0<k<∞, and Gn (t)→k.

The normalized signal generation section 150 can employ a known signal generation method, and can employ any signal that satisfies the above properties as the normalized signal.
For example, instead of using the gradient of the evaluation function for extreme value control, it is better to use the direction of the gradient (1 or -1) for extreme value control. Parameter adjustment becomes easier.

The shape of the evaluation function that expresses the relationship between the manipulated variable and the evaluated value is unknown, and in reality it is not necessarily a downwardly convex function as shown in Figure 2, but the slope may be extremely steep depending on the operating point. It is possible that the rate of change has increased significantly, or that it has become extremely gradual. Since information on the shape of the evaluation function cannot be known in advance during control execution, the gradient of the evaluation function is estimated in real time in extreme value control.

For example, if the estimated slope is gentle, the control can be applied strongly, but if the estimated slope is steep, there is a risk that stability will be impaired if the control is applied strongly. Therefore, when performing extreme value control based on information on estimated slopes, it is necessary to estimate and predict the steepest of the assumed slopes and adjust the extreme value control parameters. It's coming. At this time, control parameter settings must be conservatively adjusted so as not to impair stability, and as a result, it becomes difficult to improve control performance (increase convergence speed).

However, in order to perform an extreme value search, it is not necessarily necessary to estimate and know the gradient itself; if only the direction of the gradient, that is, the sign (whether the manipulated variable should be increased or decreased) can be obtained, the principle In general, it should be possible to search for extreme values. From this point of view, it is considered most appropriate to use the following equation as the normalized signal.
RS=1/|G(t)|……………………………………………………………………(3)
Here, RS means a normalized signal, and |G(t)| is the absolute value of the estimated gradient of the evaluation function. When this is used as a normalized signal, the signal G _n (t) obtained by normalizing G(t) becomes a sign function of the gradient.

However, if this function is used as is, it will become discontinuous at the origin of the plane representing the relationship between G(t) and G _n (t), and the condition [1] above will not be satisfied. Therefore, it is preferable to make the above equation (3) slightly smoother. For example, by introducing a regularization constant δ>0 as shown in the following equation, the following normalized signal can be obtained.
RS=1/(δ+|G(t)|)……………………………………………………(4)
The normalized signal generation section 150 outputs the normalized signal obtained by the above equation (4) to the gradient estimation amount normalization section 160.

The gradient estimate normalization section 160 normalizes the gradient estimate using the normalization signal generated by the normalization signal generation section 150. That is, the gradient estimate normalization unit 160 applies the normalization signal of equation (4), for example, to the gradient estimate G(t) estimated by the evaluation function gradient estimation unit 140 to obtain a normalized gradient estimate. Generate the value G _n (t). The normalized gradient estimate includes information on the direction (sign) of the gradient. The gradient estimate normalization unit 160 outputs the normalized gradient estimate G _n (t) to the extreme value search unit 170.

The extreme value search unit 170 includes an integrator 171 and a gain multiplier 172, and determines the direction and amount of movement of the manipulated variable by integrating information on the slope (estimated value of the rate of change).
The integrator 171 outputs a value obtained by integrating the estimated gradient value or the normalized estimated gradient value.
The gain multiplier 172 multiplies the value output from the integrator 171 by an integral gain K and outputs the result. Integral gain K is an important parameter that determines the convergence speed of extreme value search, but by introducing phase lag compensation, which will be described later, its value can be made larger than in conventional extreme value control, and as a result, Control performance can be improved.

Note that the control algorithm using the integrator 171 and the gain multiplier 172 is almost the same as an optimization algorithm called the gradient method, and the integral gain K corresponds to a parameter called the learning rate called the gradient method.

The modulation dither signal generation section 180 generates a modulation dither signal (second periodic signal).
The modulation dither signal generated by the modulation dither signal generation section 180 may be any periodic signal, but it needs to be a signal with the same waveform as the signal of the demodulation dither signal generation section 130, which will be described later. Further, from the viewpoint of parameter identification, even if the dither signal is a periodic signal, a dither signal such as a rectangular wave that is a combination of a plurality of frequencies is preferable to a sine wave of a single frequency. When a sine wave signal corresponding to a demodulation dither signal (formula (9)) to be described later is used, the modulation dither signal generating section 180 uses a modulation dither signal expressed by the following formula.
M(t)=A sin(ωt)…………………………………………………………(5)
Here, A is the amplitude of the dither signal and is an adjustable parameter. The modulation dither signal is an essential element in dither signal-driven extreme value search control.

The manipulated variable output section 190 includes an adder, and outputs a signal (manipulated variable command signal) that is the sum of the output of the extreme value search section 170 and the modulation dither signal to the controlled object 200.
In the optimal control device of this embodiment, by using a signal obtained by adding the modulation dither signal as the manipulated variable command signal as described above, the manipulated variable is periodically driven by the modulating dither signal, and the extreme value You can explore.

In reality, if the controlled object has dynamics, there is a time lag between the manipulated variable time series data and the evaluation value time series data due to the dynamics of the process (the manipulated variable is a sine wave). When driven, there is always a phase delay).

FIG. 5 is a diagram schematically showing an example of the relationship between time-series data of manipulated variables and time-series data of evaluation amounts when the controlled object has dynamics.
When the controlled object has dynamics, for example in Figure 5, even if the manipulated variable during the control operation is on the right side of the extreme value, the manipulated variable and the evaluated amount do not move in the same phase, but the phase with respect to the manipulated variable The evaluation quantity changes with the delay of . Similarly, even if the manipulated variable during control operation is to the left of the extreme value, the manipulated variable and the evaluated amount do not move in opposite phases, but the evaluated amount changes with a phase lag relative to the manipulated variable. .

For example, if the phase delay of the evaluated amount with respect to the manipulated variable becomes 90 degrees, when the manipulated variable during the control operation is on the right side of the extreme value, the manipulated variable and the evaluated amount will be in opposite phases, and the operation during the control operation will be in opposite phase. When the quantity is on the left side of the extreme value, the manipulated quantity and the evaluation quantity are in the same phase.
If optimization by extreme value control is performed in a state where there is a phase lag of the evaluation amount with respect to the manipulated variable as described above, there is a possibility that it will take time for the manipulated variable to converge.

In order to minimize the phase lag of the evaluation variable with respect to the manipulated variable, the controlled object needs to be a process so fast that it can be regarded as an almost static process. If the controlled object responds quickly enough and can be regarded as an approximately static process, it is possible to reduce the phase lag of the evaluation variable with respect to the manipulated variable. On the other hand, the actual control target is not necessarily a process that responds quickly enough to be considered a static process.

Focusing on the fact that the concept of fast response or slow response is a relative concept, by adjusting the frequency of the dither signal, it is possible to consider the actual controlled object to be approximately static. be. That is, since the actual response speed of the controlled object is a characteristic unique to the controlled object, it is necessary to design the frequency of the dither signal to be slow (small) so that it can be determined that the response speed is sufficiently fast. By doing so, even if the response delay due to the controlled object is given, it can be made sufficiently small when viewed in terms of phase delay. This creates a trade-off between "stability" and "control performance" in extreme value control.

Therefore, in the optimal control device of this embodiment, phase delay compensation is performed as follows.
The phase lag parameter estimating unit 300 calculates estimated values of parameters related to phase lag in order to obtain phase lag information of the process evaluation value with respect to the manipulated variable (=target moving speed value). Note that the phase lag parameter estimation unit 300 may perform parameter estimation offline or online.

In the optimal control device of this embodiment, a linear transfer function model such as the following equation (6) can be assumed as an appropriate model for defining the phase lag in the phase lag parameter estimating section 300.
Here, s means the Laplace operator, a _i ,i=0,1,2,...n, b _i ,i=0,1,2,...m are the denominators of the transfer function G(s), respectively. The coefficient of the polynomial and the numerator polynomial, exp(-Ls), is a transfer function of dead time (delay time), and L represents the delay time. Further, U(s) is a value obtained by Laplace transform of the manipulated variable, and Y(s) is a value obtained by Laplace transform of a process measurement value.

Since a process measurement value may include values of a plurality of items, the phase lag parameter estimation unit 300 may use a linear transfer function model corresponding to equation (6) for each process measurement value. Generally, the response of the control system is determined by the slowest response, so in this case, for example, a value obtained by Laplace transform of the water purification pond turbidity as Y(s) may be used as a representative value. Alternatively, after converting the process measurement value into a process evaluation value to be described later, the value obtained by Laplace transform of the process evaluation value can be set as Y(s).

By defining U(s) and Y(s), the parameters of the transfer function model in equation (6), a _i ,i=0,1,2,...n, b _i ,i=0,1, 2,...m, and L can be identified. Note that equation (6) is a general linear transfer function model for continuous time systems, but the phase lag parameter estimation unit 300 does not require static information about the process, so the static gain is set to 1 in advance. You can leave it as In this case, it is necessary to normalize the data corresponding to U(s) and Y(s) in advance so that the (static) gain from U(s) to Y(s) becomes 1. The gain can be normalized to 1 by normalizing the data corresponding to U(s) and Y(s) in advance. Furthermore, in order to set the gain to 1 in equation (6), it is necessary to set a ₀ = b ₀ in advance.

When the phase lag parameter estimator 300 performs parameter estimation off-line, it uses time series data corresponding to U(s) and Y(s) that have been accumulated in advance. These parameters can be identified by using the system identification method that has been proposed.

When the phase lag parameter estimator 300 performs parameter estimation online, these parameters can be identified by, for example, using the concept of an adaptive observer known for adaptive control. For example, when the controlled object 200 is greatly excited (driven) by a disturbance, it is possible to use the adaptive observer as is.

Note that when applying a system identification method, an adaptive observer, etc., it is common to use a discrete transfer function model obtained by discretizing equation (6), rather than directly using equation (6). Furthermore, when applying these theories, since the conditions for correctly identifying parameters (identifiable conditions) are known, it is necessary to use data that satisfies the identifiability conditions.
Specifically, in the case of offline operation, it is preferable to save in advance data in which the manipulated variable (=moving speed target value SV) is changed using a signal called an M-sequence (MLS).
As described above, the phase lag parameter estimation unit 300 can systematically identify the coefficients of the transfer function model based on system identification theory and adaptive control theory.

In the field of system identification and adaptive control, it is widely known that as the number of parameters to be identified increases, the identifiability conditions for correctly identifying the parameters become stricter. Generally, in the field of process control such as coagulant injection control, it is often not desirable to apply too large a disturbance to a plant, and in reality, only a step response test is often possible. This becomes a strong constraint that cannot be ignored when dealing with an actual control target, and due to these realistic constraints, step response tests and PID control are not used in actual sites without system identification or adaptive control. There is an aspect that simple control such as is preferred.

In the phase lag parameter estimating unit 300, it is often not practical to use the form of equation (6) in a general system, and a more practical method is to adopt the model of equation (7), for example. can.
The model of equation (7) above is a special case of equation (6), and is a first-order lag + dead time model. Equation (7) only includes two parameters, the delay time L and the time constant T, and these two parameters can be obtained immediately if a step response test can be performed, so it is easy to actually obtain them. It is.

In the optimal control device of this embodiment, the phase of the time series data of the manipulated variable is delayed by multiplying the time series data of the manipulated variable by the transfer function of equations (6) and (7) above. The phase difference between the time series data of the phase delay manipulated variable incorporating the phase delay information and the time series data of the evaluation value is eliminated.

Note that equation (7) is a special system of a typical model used for adjusting PID control. When adjusting PID control, it is common to use a model called Kexp(-Ls)/(Ts+1) instead of exp(-Ls)/(Ts+1), and also identify another parameter K called process gain. The reason why K is not included in equation (7) is because static information of the plant is not related to the phase delay, as will be described later.

Further, when the phase delay parameter estimating section 300 calculates the estimated values of the delay time L and the time constant T online, the concept of the adaptive observer described above may be used.
When calculating the estimated values of the two parameters, lag time L and time constant T, the phase lag parameter estimator 300 can use information on the dither signal. For example, when a rectangular wave is used instead of a sine wave as a dither signal, the period before and after the change in the rectangular wave can be considered as one step, so the phase lag parameter estimating unit 300 directly utilizes the concept of step response. Then, the delay time L and time constant T can be identified.

For example, when a sine wave is used as the dither signal, the identifiability deteriorates and it may be difficult to directly obtain it, but the phase lag parameter estimator 300 The delay time L and time constant T can be calculated backward by changing the phase difference by two steps (or two or more steps) and directly estimating the phase difference between the two time series data corresponding to U(s) and Y(s). can be found.

Since the phase of the first-order delay dead time is expressed by a function of the delay time L and the time constant T, if two pieces of phase information are obtained, the delay time L and the time constant T can be obtained by back calculation. For example, when three or more frequencies are used, the delay time L and time constant T can be determined by setting an error minimization criterion and optimizing the parameters.

On the other hand, if you already have a process model for the controlled process (controlled process model) or a process simulator, it is not necessary to use a linear transfer function such as equation (6), or a special example of the linear transfer function such as equation (7). It is not necessary to identify a first-order delay + dead time model, and a process simulator can be used directly.

In the optimal control device of this embodiment, if there is a flocculation process model or flocculation process simulator in which the moving speed or the turbidity at the outlet of the sedimentation tank or water purification tank changes depending on the PAC injection and the PAC injection at the water treatment plant, the process Identifying various parameters included in the simulator corresponds to phase lag parameter estimation processing.

Various methods can be considered for estimating various parameters included in the process simulator. If the process simulator is a nonlinear model, in order to match the input and output of the actual process and the input and output of the process simulator as much as possible, for example, for the error between the actual process output and the simulator output for the same input, , squared error minimization, etc. may be used as an evaluation criterion to solve an optimization problem that minimizes the error, or a method called data assimilation may be applied. This allows the phase lag parameter estimator 300 to identify the parameters of the controlled process model.

As described above, the phase lag parameter estimator 300 can identify the phase lag parameter offline or online. The phase lag parameter estimation section 300 supplies the estimated value of the identified parameter to the phase lag estimation section 1100.

The phase lag estimating unit 1100 can use the estimated values of the parameters supplied from the phase lag parameter estimating unit 300 to determine the phase lag from the manipulated variable to the evaluation variable.
In the optimal control device of this embodiment, the phase lag estimation unit 1100 uses the transfer function model of equation (6) or equation (7), for example, to incorporate phase lag information into the demodulation dither signal. Calculate the phase delay. The phase lag estimation unit 1100 can calculate the phase lag of a transfer function model by directly applying a widely known phase calculation method used for creating a Bode diagram of a linear transfer function model. Here, for the sake of simplicity, a specific equation for calculating the phase delay in the case of the transfer function model of equation (7) is shown in equation (8).
∠G(s)=-ωL-tan ^-1 (ωT)…………………………………………(8)
Here, ω is the frequency. In the optimal control device of this embodiment, the manipulated variable is forcibly moved using a dither signal, so when a sine wave is used as the dither signal, for example, the frequency of the dither signal can be set to the value of ω.

Note that when the phase delay estimator 1100 calculates the phase delay using the transfer function model of equation (6), more complicated calculations are required than when using the model of equation (7), but the frequency ω It is possible to calculate the phase delay from and the parameter values of the linear transfer function model.
On the other hand, if it is not appropriate to express it using a linear transfer function model, the phase can be estimated in advance directly using a process simulator (a simulation model written using a physicochemical formula that simulates the process to be controlled). At this time, in order to apply extreme value control to phase estimation, some kind of evaluation function is required as an output, so a simulation model that combines a so-called process simulator (simulation model of the controlled process) and an evaluation function model is used. ,is necessary.
For example, in a water purification process or a sewage treatment process, it is possible to create a process simulator that describes the process, and the output is, for example, water quality. When applying extreme value control, it is necessary to look at not only water quality but also the overall balance between good water quality and the cost involved. Therefore, in this embodiment, as an evaluation function model for optimality, for example, Define overall costs, etc. In this way, by directly using the model composed of the process simulator and the evaluation function model, it is possible to perform phase estimation by extreme value control.

Furthermore, when using a process simulator, it is generally not possible to explicitly express the relationship between frequency and phase as in equation (8) using a mathematical equation. For this reason, we input many sine waves of different frequencies into a model consisting of a process simulator and an evaluation function model, and evaluate the waveform deviation between the simulation input (operated amount) and the output (evaluation function value). , evaluate the phase delay. It is sufficient to measure the phase delay of the output waveform with respect to each input sine wave, and create an approximate function representing the phase as a function of frequency ω corresponding to equation (8) in the phase delay estimating section 1100.
In other words, the concept of phase is defined with respect to frequency ω, so by varying the input frequency ω and evaluating by simulation how much the output phase lags at each frequency ω, the frequency ω can be calculated by A table can be created in advance that represents a function when the horizontal axis is plotted and the phase delay is plotted as the vertical axis. The phase lag estimation unit 1100 can calculate the estimated value of the phase lag using the above function (table).

Alternatively, the phase delay estimation unit 1100 analytically derives a linear approximation model near the actual manipulated variable from the controlled target process model, and calculates the phase calculation formula of the linear transfer function model in which the linear approximation model is expressed by a linear transfer function. The phase delay may be calculated by the applied simulation.
For example, sewage treatment process simulators are described using nonlinear differential equations. From this nonlinear differential equation, a linear approximation model can be derived in the vicinity of the operating point (the state of water quality or the state of the manipulated variable at the time of interest). The linear approximation model is a linear differential equation that approximates the original nonlinear differential equation, and in this embodiment, the linear approximation model means a linear differential equation. A linear differential equation can also be described in the form of a linear transfer function of equation (6), for example. If the linear approximation model can be described in the form of a linear transfer function, the phase lag can be analytically determined by a phase lag calculation formula corresponding to equation (8). The phase lag estimation unit 1100 can calculate the estimated value of the phase lag using the phase lag calculation formula obtained by expressing the linear approximation model in the form of a linear transfer function as described above.
The phase lag estimation section 1100 supplies the calculated phase lag value to the demodulation dither signal generation section 130.

The demodulation dither signal generation unit 130 generates a first periodic signal (demodulation dither signal) necessary for estimating the gradient (first derivative, Jacobian) of the evaluation function using extreme value control.
Although a sine wave is typically used as the dither signal, it does not necessarily have to be a sine wave, and may be a rectangular wave or a triangular wave as long as it is a periodic signal. The demodulation dither signal has a waveform with the same period and the same shape as the modulation dither signal.
Equation (9) shows a demodulation dither signal when a sine wave is used as the dither signal.
D(t)=sin(ωt+φ)………………………………………………(9)

Here, ω is the frequency of the dither signal, and φ is the estimated value of the phase lag calculated by the phase lag estimator 1100. That is, when the model of equation (7) is used in the phase lag parameter estimation unit 300, it corresponds to that of equation (8), and the phase lag φ can be expressed by equation (10) below.
φ:=-ωL-tan ^-1 (ωT)………………………………………………(10)
In the optimal control device of this embodiment, phase delay information is incorporated into the demodulation dither signal generated by the demodulation dither signal generation section 130 as described above.
The demodulation dither signal generation unit 130 supplies the demodulation dither signal including the phase delay φ to the evaluation function gradient estimation unit 140.

FIG. 6 is a diagram schematically showing an example of time series data of manipulated variables and time series data of evaluation values before and after phase lag compensation.
As described above, by calculating the slope of the evaluation function in the evaluation function gradient estimating unit 140 using the demodulation dither signal including the phase delay φ, the time series data u'(t) of the phase delay manipulated variable is evaluated. The phase difference that occurs between the value and the time series data y'(t) can be made approximately zero.

Note that in the optimal control device of this embodiment, a sine wave is used as an example of the dither signal, but it is also possible to use a periodic signal such as a rectangular wave or a triangular wave instead of a sine wave as the dither signal. When using a periodic signal such as a rectangular wave or a triangular wave as a dither signal, the value obtained by equation (10) cannot be directly used as the phase delay φ. The phase delay compensation according to equation (9) is a correction process to make the phase difference between the demodulation dither signal of equation (9) and the modulation dither signal of equation (5) zero. , the ratio of the phase delay φ to the period of the dither signal may be determined, and a signal obtained by correcting the period of the modulation dither signal by that amount may be used as the demodulation dither signal.
In order to mechanically perform such phase lag compensation, the demodulation dither signal generation unit 130 expands a periodic signal such as a rectangular wave into a Fourier series, expresses it as a sine wave series, and expresses it as a sine wave series. A method of substituting a phase delay corresponding to equation (10) may also be adopted.

Next, the effects obtained by performing the phase delay compensation in the optimal control device of this embodiment will be explained.
Conventionally, regarding the relationship between the stability of extreme value control and control performance, for example, by performing stability analysis of extreme value control, it has been pointed out that it is necessary to appropriately set parameters to guarantee stability. Qualitatively, if the values of the control parameters (dither signal amplitude, dither signal frequency, integrator gain (integral gain), low-pass filter cutoff frequency, high-pass filter cutoff frequency, etc.) are not made sufficiently small, It has been shown that stability is compromised.

Setting the control parameter to be "sufficiently small" means that the control operation is performed "slowly" and "weakly". In other words, there is a trade-off relationship between "stability" and "control performance"; in order to maintain stability, it is essentially necessary to reduce control performance, and conversely, it is necessary to improve control performance. This shows that if the parameters are set to a large value, the stability will collapse and the risk of control failure will increase as a result.

In the extreme value control by the optimal control device of this embodiment, there is a phase difference φ between the modulation dither signal in equation (5) and the demodulation dither signal in equation (9), and evaluation of the manipulated variable time series data is performed. The phase delay of the time series data is compensated. This phase lag compensation makes it possible to set the frequency ω of the dither signal large (that is, reduce the period of the dither signal) and to set the integral gain K large, which improves the control performance of extreme value search (convergence speed ) can be improved.

7 to 10 are diagrams for explaining an example of the relationship between control stability and control performance when changing the frequency of a periodic signal used for extreme value control.
In the flocculant injection process, there is a response time delay of several hours from the target process to the turbidity at the outlet of the sedimentation tank or the outlet of the water purification tank. Here, for a target process with a response delay on the order of several hours, the target value when the manipulated variable is driven with a dither signal with a period of 1500 minutes, 1000 minutes, and 500 minutes without performing the above phase delay compensation. , shows the simulation results of the time variation of sedimentation basin turbidity, PAC injection amount, and total cost.

According to FIGS. 7 to 10, when the period of the dither signal was 1500 minutes, all evaluation values converged, but it took a long time for the numerical values to converge near the optimum value. When the dither signal period was 1000 minutes, the time to converge near the optimal value was shorter than when the dither signal period was 1500 minutes, but when the dither signal period was 500 minutes, the evaluation value became the optimal value. It diverged without converging.

FIGS. 11 to 13 are diagrams for explaining an example of the effects of the optimal control device of the embodiment.
Here, in the optimal control device of this embodiment, the plant time constant T = 0.67 s, the plant dead time L = 0.67 s, the optimal value of the manipulated variable U ^* = 2, and the period of the dither signal is set to the plant time constant. An example of the results of simulating the time conversion of the manipulated variable is shown, with the value being 2 to 5 times the sum of the constant and the plant dead time (T+L).
Further, FIGS. 11 to 13 also show examples of simulation results of temporal changes in the manipulated variable by the optimal control device of the first comparative example. The optimal control device of the first comparative example has the same conditions as the optimal control device of the present embodiment except that phase delay compensation is not applied.

According to the simulation results, according to the optimal control device of this embodiment, the shorter the period of the dither signal, the shorter the time it takes for the manipulated variable to converge to the optimal value. On the other hand, regarding the optimal control device of the first comparative example, when the period of the dither signal is decreased, the time until the manipulated variable converges to the optimal value tends to become longer, and the period of the dither signal is set to 2.51 s ((T+L )×2), the manipulated variable diverged and did not converge to the optimal value.

As described above, the optimal control device of this embodiment improves the performance of extreme value control (increases the convergence speed) by introducing a function that compensates for the phase difference that exists between the manipulated variable and the process evaluation value. increase) becomes possible. This can improve the deterioration in extreme value search performance due to slow convergence speed.

FIGS. 14 to 17 are diagrams for explaining other examples of effects achieved by the optimal control device of the embodiment.
Here, an example of simulation results when there is a disturbance (daily fluctuation in inflow amount) is shown for the optimal control device of the third comparative example and the optimal control device of the present embodiment.
FIGS. 14 and 15 show an example of a simulation result of extreme value control by the optimal control device of the third comparative example, and an example of a simulation result when the target value SV is kept constant. Note that the optimal control device of the third comparative example sets the period of the dither signal to 1500 minutes (equivalent to 1440 minutes (one day)), and calculates the target value by extreme value control without applying phase lag compensation. There is.
According to the simulation results by the optimal control device of the third comparative example, it is found that the control is unstable due to the influence of daily fluctuations in the inflow amount.

16 and 17 show an example of a simulation result of extreme value control by the optimal control device of this embodiment, and an example of a simulation result when the target value SV is kept constant. Note that the optimal control device of this embodiment sets the period of the dither signal to 900 minutes, and calculates the target value by extreme value control to which phase lag compensation is applied.
Compared to the optimal control device of the third comparative example, the simulation results of the optimal control device of this embodiment show that the influence of daily fluctuations in the inflow amount (disturbance) is suppressed and the control is stable.

As described above, according to this embodiment, stability is maintained by overcoming the problem of achieving both "stability" and "control performance (convergence speed)" in extreme value control through an operation called phase compensation. At the same time, it is possible to provide an optimal control device, an optimal control method, and a computer program that realize a search for a (local) optimal value that maximizes control performance.

Further, according to the optimal control device of this embodiment, a means for performing phase lag compensation can be achieved by simply performing a simple response test such as a step response test used when adjusting PID control that is widely used in industry. It is possible to construct an extreme value control system that can be easily used in practice.
Furthermore, by providing the optimal control device of this embodiment with a configuration for normalizing the gradient estimation amount, it becomes possible to configure an extreme value control system with a faster convergence speed.
Furthermore, by calculating parameter estimates online in the phase lag parameter estimating section 300, and by performing slight processing on the application of the dither signal, extreme value control with fast convergence speed that eliminates the need for tests such as step response etc. It becomes possible to configure the system.

FIG. 18 is a diagram schematically showing another configuration example of a part of the optimal control device of the first embodiment.
As shown in FIG. 18, the normalized signal generation section 150 and the gradient estimation amount normalization section 160 are not essential components and may be omitted. In that case, the output value of the evaluation function gradient estimation section 140 is supplied to the extreme value search section 170.
In this way, even in a configuration in which the normalized signal generation section 150 and the gradient estimator normalization section 160 are omitted, it is possible to realize a search for a (local) optimal value that maximizes control performance while maintaining stability. An optimal control device, an optimal control method, and a computer program can be provided.

Next, an optimal control device, an optimal control method, and a computer program according to a second embodiment will be described in detail with reference to the drawings.
In the following description, the same components as those in the first embodiment described above will be denoted by the same reference numerals, and the description will be omitted.
FIG. 19 is a diagram schematically showing a configuration example of the optimal control device according to the second embodiment.
The optimal control device of this embodiment differs from the above-described first embodiment in that phase delay information is incorporated during gradient estimation without using a demodulation dither signal.

The optimal control device of this embodiment includes a process measurement value acquisition section 110, a process evaluation value calculation section 120, an evaluation function gradient estimation section 140, an extreme value search section (optimum operation amount adaptive adjustment section) 170, and an operation amount It includes an output section 190 and a process phase delay estimation section 400. Process phase lag estimating section 400 includes a phase lag parameter estimating section 300 and a phase lag estimating section 1100.

FIG. 20 is a diagram schematically showing a partial configuration example of the optimal control device according to the second embodiment.
The phase lag estimating unit 1100 does not explicitly estimate the phase lag in the form of equation (8), but inputs manipulated variable data in real time to the models of equations (6) and (7). The time series data of the output (hereinafter referred to as phase lag correction operation amount data) is set as the output of the phase lag estimating section 1100.
Similarly, when using a process simulator, the phase delay estimating unit 1100 can output time-series data obtained by inputting manipulated variable data into the process simulator in real time.

Thereby, the phase shift between the manipulated variable and the evaluation value can be corrected. That is, there is always a phase delay between the manipulated variable and the evaluation value due to the passage of the controlled object 200 having dynamics. In the optimal control device of this embodiment, the phase lag correction operation amount is used instead of the operation amount, thereby making the phase lag between the phase lag correction operation amount and the evaluation value extremely small. In principle, if there is no error between the linear transfer function model and the actual controlled object, it is possible to eliminate the phase lag between the phase lag correction manipulated variable and the evaluation value.

In this embodiment, the evaluation function gradient estimation unit 140 directly calculates the slope (gradient) of the evaluation function at the operating point without using the demodulation dither signal.
As mentioned earlier, for example, the overall shape of the evaluation function shown in Figure 2 is unknown when extreme value control is actually implemented, but the values near the operating point can be determined in real time during control. I have been able to obtain it. Therefore, by using data near the operating point, that is, time-series data of the most recent manipulated variables and evaluation variables during control execution, it is possible to estimate the slope of the straight line representing the slope of the evaluation function as shown in Figure 2. is possible.

For example, when implementing control, past manipulated variable time-series data and evaluation amount time-series data are accumulated for a predetermined period of time, and the accumulated time-series data is used to perform a regression algorithm such as the least squares method or the sequential least squares method. The coefficients of the straight line in the following equation (11) can be found.
J=cU+d…………………………………………………………………………(11)
Here, J is the evaluation amount (process evaluation value) in FIG. 2, and U is the operation amount in FIG. c and d are linear regression coefficients. These coefficients c and d can be determined using an algorithm such as the method of least squares.
Once the coefficient of the straight line in equation (11) is determined, the slope of the straight line can be regarded as the slope of the evaluation function, so the coefficient c corresponding to the slope can be used as slope information (estimated slope value) of the evaluation function.

In the present embodiment, the evaluation function gradient estimating unit 140 includes a data storage unit (not shown) that stores past manipulated variable time series data and evaluation amount time series data for a predetermined period when performing control. Using the obtained data, the coefficient c is calculated as the gradient estimated value using the following equation (12) instead of equation (11).
J=cU _f +d……………………………………………………………………(12)
Here, U _f is not the actual manipulated variable, but the phase lag manipulated variable that adds information on the estimated value of the phase lag to the manipulated variable, and the manipulated variable is expressed in equations (6) and (7). It is passed through a linear transfer function. That is, it is the value of the output signal of the linear transfer function model of the phase delay estimation section 1100.

As mentioned above, by using the value passed through the linear transfer function model for the manipulated variable rather than the manipulated variable itself, the slope of the evaluation function is estimated without a phase lag of the manipulated variable, as shown in Figure 2. This can be expected to improve the accuracy of gradient estimation.
Ideally, if the phase delay in the controlled object 200 is accurately estimated using the linear transfer function model of equations (6) and (7), it is possible to reduce the phase delay of the manipulated variable to zero. can. Therefore, since the frequency of the dither signal can be made as fast as possible, the trade-off problem between "stability" and "control performance" can be resolved, and control performance can be improved while maintaining control stability. (increase the convergence speed).

Note that it is actually impossible to accurately identify the phase delay in the controlled object using a linear transfer function model, so although there is a limit to control performance (convergence speed), time series data of manipulated variables and evaluation values By eliminating the phase difference with the time series data, control performance can be significantly improved without sacrificing stability.

In this embodiment, the method of calculating the gradient estimated value is not limited to the above. For example, a "correlation coefficient" between the manipulated variable and the evaluation value may be regarded as the estimated gradient value. This means that the positive or negative slope of the slope of the function representing the manipulated variable and the evaluation matches the positive or negative sign of the correlation coefficient between the manipulated variable and the evaluated value, and the correlation coefficient does not depend on the magnitude of the slope. This is because it serves as a substitute. In other words, between the regression coefficient of linear regression of variable X and variable Y and the correlation coefficient of variable X and variable Y, there is a difference between , the coefficient c of the regression line matches the correlation coefficient r, and the bias b=0. Therefore, using the correlation coefficient r instead of the regression coefficient c corresponds to determining the regression coefficient after normalizing the manipulated variable time series data and the evaluation value time series data, respectively.

Also in the optimal control device of this embodiment, the evaluation function gradient estimating unit 140 can calculate the estimated gradient value using the correlation coefficient between the evaluation amount J and the phase delay operation amount U _f . By using the correlation coefficient as gradient information instead of the regression coefficient in the optimal control device of this embodiment, it is possible to obtain the same effect as the normalization operation in the optimal control device of the first embodiment.

Note that the configuration of extreme value control in which the slope is estimated by linear approximation in FIG. It has a structure similar to that of internal model control (a type of internal model control).
In target value tracking type feedback control, internal model control arranges the controlled object and the controlled object process model representing the controlled object in parallel, and feeds back the error between the output of the controlled object and the output of the process model representing the controlled object. This is a control that improves control performance, and can be considered a type of predictive control, and is widely known to be particularly effective for processes with long dead times.

The optimal control device of this embodiment can be considered to have introduced a concept similar to that of internal model control in extreme value search type feedback control. In other words, when the delay time is long, a model that predicts it is arranged in parallel with the controlled object, a type of prediction is made, and optimization (gradient estimation) is performed based on the manipulated variable that compensates for the delay in the controlled object. .
The similarity between the extreme value search type feedback control in the optimal control device of this embodiment and the internal model control and Smith compensation control is that prediction is performed using a model of the controlled object.

On the other hand, the difference is (1) Internal model control uses a controlled object process model, but in this embodiment, static characteristics of the controlled object are not required. (2) In internal model control, the output of the controlled object can be In contrast to feeding back the error between the value and the predicted value of the controlled target process model, in this embodiment, the gradient representing the relationship between the predicted value of the phase characteristic of the process and the output value of the controlled target (= evaluation function value) is fed back. This is the point at which the gradient is estimated.
The effects of internal model control have already been clarified, and the effects of this embodiment can also be inferred from the viewpoint of the similarity between feedback control in the optimal control device of this embodiment as described above and internal model control. be able to.

FIGS. 11 to 13 show examples of simulation results of temporal changes in the manipulated variable by the optimal control devices of the second embodiment and the second comparative example.
Here, in the optimal control device of this embodiment, the plant time constant T = 0.67 s, the plant dead time L = 0.67 s, the optimal value of the manipulated variable U ^* = 2, and the period of the dither signal is set to the plant time constant. An example of the results of simulating the time conversion of the manipulated variable is shown, with the value being 2 to 5 times the sum of the constant and the plant dead time (T+L).
The optimal control device of the second comparative example has the same conditions as the optimal control device of the present embodiment except that phase delay compensation is not applied.

According to the simulation results, according to the optimal control device of this embodiment and the optimal control device of the second comparative example, there is a tendency that the shorter the period of the dither signal, the shorter the time it takes for the manipulated variable to converge to the optimal value. It was done. Furthermore, regardless of the period of the dither signal, the time required for the manipulated variable to converge to the optimal value was shorter with the optimal control device of this embodiment than with the optimal control device of the second comparative example.

As described above, the optimal control device of this embodiment improves the performance of extreme value control (fastens convergence) by introducing a function that compensates for the phase difference that exists between the manipulated variable and the process evaluation value. ) becomes possible. This can improve the deterioration in extreme value search performance due to slow convergence speed.

That is, according to the present embodiment, an optimal control device, an optimal control method, which realizes a search for a (local) optimal value that maximizes control performance while maintaining stability without applying a forced dither signal. And, a computer program can be provided.

FIG. 21 is a diagram schematically showing a modification of the configuration of the optimal control device according to the second embodiment.
This example differs from the examples shown in FIGS. 17 and 18 in that the optimal control device includes a modulation dither signal generation section 180.
FIG. 22 is a diagram schematically showing a modification of a part of the configuration of the optimal control device of the second embodiment.
In this embodiment, by incorporating a modulating dither signal into the manipulated variable, data used for regression estimation and correlation estimation are forcibly varied. For example, if the controlled object 200 is not sufficiently excited (driven) due to disturbance etc., the reliability of estimated values of regression parameters and correlation coefficients may deteriorate, so the modulation dither signal may be It is preferable to add it to

However, in this case, the modulating dither signal does not necessarily need to be a periodic signal or a sine wave. For example, a method may be used in which a signal such as a random number is used as a modulating dither signal and is applied to the controlled object.
If the controlled object is sufficiently driven by a disturbance or the like, it is also possible to configure an extreme value control system that does not include the modulation dither signal generation section 180 as described above.

This modification has a configuration similar to that of the optimal control device in FIGS. 19 and 20 except for the configuration described above. Therefore, according to this modification, there is provided an optimal control device, an optimal control method, and an optimal control method that realize a search for a (local) optimal value that maximizes control performance while maintaining stability by applying the minimum necessary external input. , may provide computer programs.

Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.

DESCRIPTION OF SYMBOLS 100... Control system, 110... Process measurement value acquisition part, 120... Process evaluation value calculation part, 130... Dither signal generation part for demodulation, 140... Evaluation function gradient estimation part, 150... Normalization signal generation part, 160... Gradient estimation Quantity normalization section, 170...Extreme value search section, 171...Integrator, 172...Gain multiplication section, 180...Modulation dither signal generation section, 190...Manipulated amount output section, 200...Controlled object (target plant), 300... Phase lag parameter estimation unit, 400... Process phase lag estimation unit, 1100... Phase lag estimation unit

Claims (11)

Optimization that searches for the optimal value of the evaluation value by manipulating the manipulated variable in real time based on the manipulated variable of the controlled process and the evaluation value of an evaluation function based on the controlled variable that changes according to the manipulated variable. A control device,
a process evaluation value calculation unit that calculates the evaluation value of the evaluation function using the measurement value acquired in the controlled process;
a process phase lag estimation unit that uses the manipulated variable and the evaluation value to calculate an estimated value of a phase delay from the manipulated variable to the evaluated value;
an evaluation function gradient estimation unit that calculates an estimated value of a rate of change of the evaluation value with respect to the manipulated variable using information regarding the estimated value of the phase delay and information on the evaluation value;
an extreme value search unit that determines the direction and amount of movement of the manipulated variable by integrating the estimated value of the rate of change;
An optimal control device comprising: a manipulated variable output unit that outputs the manipulated variable to the controlled process based on information about the direction and amount of movement of the manipulated variable determined by the extreme value search unit. The process phase delay estimator expresses the transfer function from the manipulated variable of the controlled process to the evaluation value using a linear transfer function model excluding static nonlinear elements, and calculates the transfer function by phase calculation of the linear transfer function model. The optimal control device according to claim 1, which calculates an estimated value of phase delay. The optimal control device according to claim 2, wherein the process phase lag estimator includes a phase lag parameter estimator that identifies parameters of the linear transfer function model online. The process phase delay estimation unit configures a model from the evaluation amount of the controlled process to the evaluation value of the evaluation function from a simulation model of the controlled process and an evaluation function model, and by simulating the model, The optimal control device according to claim 1, wherein the estimated value of the phase delay is calculated. a demodulation dither signal generation unit that generates a first periodic signal having a predetermined period;
a modulation dither signal generation unit that generates a second periodic signal having the same period as the first periodic signal;
The manipulated variable output unit outputs a signal obtained by adding the second periodic signal to information about the direction and amount of movement of the manipulated variable determined by the extreme value search unit,
The evaluation function gradient estimating unit calculates an estimated value of the rate of change of the evaluation value with respect to the manipulated variable using the second periodic signal and information on the evaluation value,
The optimal control device according to claim 2 or 4, wherein the first periodic signal is a signal obtained by adding the estimated value of the phase delay to the second periodic signal. further comprising a normalization unit that normalizes the estimated value of the rate of change and outputs information on the direction of the estimated value of the rate of change to the extreme value search unit,
6. The optimal control device according to claim 5, wherein the extreme value search unit determines the direction and amount of movement of the manipulated variable by integrating the sign of the estimated value of the rate of change. The process phase lag estimator expresses a linear approximation model in the vicinity of the manipulated variable of the controlled process model using a linear transfer function model, and calculates the estimated value of the phase lag by phase calculation of the linear transfer function model. The optimal control device according to claim 4, which calculates the optimal control device. The process phase lag estimator outputs a phase lag manipulated variable passed through the linear transfer function model with respect to the manipulated variable as an estimated value of the phase lag,
The evaluation function gradient estimator uses time series data of the value of the phase delay operation amount and the evaluation value for a predetermined period to calculate a straight line with the phase delay operation amount as input and the evaluation value as output. The optimal control device according to claim 2 or 7, wherein the coefficient of the first-order term of the straight line is calculated by regression, and the coefficient of the first-order term of the straight line is used as the estimated value of the rate of change. The process phase lag estimator outputs a phase lag manipulated variable passed through the linear transfer function model with respect to the manipulated variable as an estimated value of the phase lag,
The optimal control device according to claim 2 or 7, wherein the evaluation function gradient estimator calculates a correlation coefficient between the value of the phase delay manipulated variable and the evaluation value as the estimated value of the rate of change. Optimization that searches for the optimal value of the evaluation value by manipulating the manipulated variable in real time based on the manipulated variable of the controlled process and the evaluation value of an evaluation function based on the controlled variable that changes according to the manipulated variable. A control method,
Calculating the evaluation value of the evaluation function using the measurement value acquired in the controlled process,
Using the manipulated variable and the evaluation value, calculate an estimated value of a phase delay from the manipulated variable to the evaluated value,
Using information on the estimated value of the phase lag and information on the evaluation value, calculate an estimated value of the rate of change of the evaluation value with respect to the manipulated variable,
determining the direction and amount of movement of the manipulated variable by integrating the estimated value of the rate of change;
An optimal control method that outputs the manipulated variable to the controlled process based on information about the determined direction and amount of movement of the manipulated variable. A computer program for causing a computer to perform the method according to claim 10.